A data processing method, apparatus and medium

Abstract

This invention discloses a data processing method, apparatus, and medium. The method includes: acquiring multiple service usage datasets sent by multiple clients at a target reporting time point, where each service usage dataset represents the client's call information to a service object during a target time period; determining data writing rules based on the identification information carried by the multiple service usage datasets; writing the multiple service usage datasets into a message queue according to the data writing rules; reading the multiple service usage datasets from the message queue according to a preset reading rule; clustering the multiple service usage datasets using the service object identifier of the service object as the center point to obtain clustering results; and generating billing data based on the clustering results. This invention periodically acquires the call information of each client to a service object during a target time period and introduces a message queue for orderly storage of multiple service usage datasets, thereby improving the efficiency and accuracy of generating billing data.

Description

Technical Field

[0001] This invention relates to the field of Internet communication technology, and in particular to a data processing method, apparatus and medium. Background Technology

[0002] Cloud services are internet-based models for enabling the provision of dynamically scalable and often virtualized resources. With the development of enterprise cloud services, the demand for cloud services among enterprises is constantly increasing, and the types of cloud services used by enterprises are also becoming more diverse.

[0003] In related technologies, cloud service providers often rely on log files for billing enterprise users' cloud service usage. Based on enterprise users' cloud service usage, the cloud service provider's business modules generate a large amount of log data. A data collection module periodically downloads this log data to obtain log files, and a statistics module uses these log files to determine the enterprise user's service usage and generate invoices based on billing rules. In this log file-dependent billing method, log data generated by one business module often points to a corresponding cloud service. When an enterprise user uses at least two cloud services simultaneously, the corresponding log data may be scattered across different business modules. Consequently, the cloud service provider's efficiency in determining the enterprise user's service usage is relatively low. Therefore, a more efficient solution for determining service usage is needed. Summary of the Invention

[0004] To address the inefficiencies of existing technologies in determining service usage by enterprise users, this invention provides a data processing method, apparatus, and medium:

[0005] On one hand, the present invention provides a data processing method, the method comprising:

[0006] At the target reporting time point, obtain multiple service usage datasets sent by multiple clients, each of which represents the call information of the corresponding client to the service object during the target time period;

[0007] Data writing rules are determined based on the identification information carried by the datasets used by the multiple services;

[0008] According to the data writing rules, the multiple services use datasets to write data into the message queue;

[0009] The multiple service usage datasets are read from the message queue according to preset reading rules;

[0010] Cluster the multiple services using the service object identifier of the service object as the center point to obtain the clustering result;

[0011] Billing data is generated based on the clustering results.

[0012] On the other hand, a data processing apparatus is provided, the apparatus comprising:

[0013] Acquisition module: used to acquire multiple service usage datasets sent by multiple clients at the target reporting time point, each of the service usage datasets representing the call information of the corresponding client to the service object in the target time period;

[0014] Determining module: used to determine data writing rules based on the identification information carried by the service dataset;

[0015] Write module: used to write the datasets of the multiple services into the message queue according to the data writing rules;

[0016] Reading module: used to read the multiple service usage datasets from the message queue according to preset reading rules;

[0017] Clustering module: used to cluster the multiple services using the service object identifier of the service object as the center point, and obtain the clustering result;

[0018] Billing module: Used to generate billing data based on the clustering results.

[0019] On the other hand, an electronic device is provided, comprising a processor and a memory, wherein the memory stores at least one instruction or at least one program, the at least one instruction or the at least one program being loaded and executed by the processor to implement the data processing method described above.

[0020] On the other hand, a computer-readable storage medium is provided, wherein at least one instruction or at least one program is stored therein, the at least one instruction or the at least one program being loaded and executed by a processor to implement the data processing method as described above.

[0021] The data processing method, apparatus, and medium provided by this invention have the following technical effects:

[0022] This invention acquires multiple service usage datasets sent by multiple clients at a target reporting time point, and periodically retrieves service object call information from each client within a target time period. Introducing a message queue allows for ordered storage of multiple service usage datasets and ensures a reliable and stable data source for clustering these datasets. This invention improves the efficiency and accuracy of generating billing data. Attached Figure Description

[0023] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This is a schematic diagram of an application environment provided by an embodiment of the present invention;

[0025] Figure 2 This is a flowchart illustrating a data processing method provided in an embodiment of the present invention;

[0026] Figure 3 This is a flowchart illustrating the process of determining the target reporting time point and the target time period provided in an embodiment of the present invention;

[0027] Figure 4 This is a flowchart illustrating a method for determining data writing rules based on the identification information carried by multiple service usage datasets, as provided in an embodiment of the present invention.

[0028] Figure 5 This is a schematic diagram of a process for reading multiple service usage datasets from a message queue according to preset reading rules, provided by an embodiment of the present invention;

[0029] Figure 6 This is also a flowchart illustrating a data processing method provided in an embodiment of the present invention;

[0030] Figure 7 This is also a schematic diagram of a process for reading multiple service usage datasets from a message queue according to preset reading rules, provided by an embodiment of the present invention;

[0031] Figure 8 This is a schematic diagram of a process for cyclically detecting whether there are currently any fragments that need to be processed, provided in an embodiment of the present invention;

[0032] Figure 9 This is a schematic diagram of a process for clustering and organizing multiple service datasets provided in an embodiment of the present invention;

[0033] Figure 10 This is a schematic diagram of a settlement process based on billing data provided in an embodiment of the present invention;

[0034] Figure 11 This is a block diagram of a data processing device provided in an embodiment of the present invention;

[0035] Figure 12 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] It should be noted that the terms "comprising" and "having" and any variations thereof in the specification, claims, and accompanying drawings of this invention are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices.

[0038] Please see Figure 1 , Figure 1 This is a schematic diagram of an application environment provided by an embodiment of the present invention, which may include a client 01 and a server 02. The client and the server can be directly or indirectly connected via wired or wireless communication. Multiple clients send multiple service usage datasets to the server, and each service usage dataset represents the call information of the corresponding client to the service object within a time period. The server performs corresponding storage, clustering, billing, and other processing on the received multiple service usage datasets to obtain the corresponding billing data. It should be noted that... Figure 1 This is just one example.

[0039] Specifically, client 01 can include physical devices such as smartphones, desktop computers, tablets, laptops, augmented reality (AR) / virtual reality (VR) devices, digital assistants, smart speakers, and smart wearable devices. It can also include software running on physical devices, such as computer programs. The operating system running on client 01 can include, but is not limited to, Android, iOS (a mobile operating system developed by Apple), Linux, and Microsoft Windows.

[0040] Specifically, server 02 can be a standalone physical server, a server cluster or distributed system consisting of multiple physical servers, or a cloud server providing cloud computing services. Server 02 may include network communication units, processors, and memory, etc. Server 02 can provide backend services for the aforementioned clients.

[0041] In practical applications, enterprise users purchase cloud service products from cloud service providers. The cloud service providers then offer these services to enterprise users. The server corresponds to the cloud service provider's backend, and the client corresponds to a component that reports service usage data to this backend. Enterprise users can integrate these components into their developed products. Similarly, individual users utilize products provided by enterprise users. During use, these products invoke cloud services to provide relevant service experiences, and the components integrated into the products report service usage data to the backend. These components can be SaaS (Software as a Service) components, PaaS (Platform as a Service) components, etc. Furthermore, cloud service providers can offer enterprise users products that integrate multiple cloud services.

[0042] Cloud service providers offer enterprise users services including private cloud, hybrid cloud, and cloud conferencing services. Hybrid cloud, which integrates public and private clouds, is a major model and development direction of cloud computing in recent years. Private clouds are primarily geared towards enterprise users. For security reasons, enterprises prefer to store data in private clouds but also want access to public cloud computing resources. Hybrid clouds are increasingly adopted in this context, combining and matching public and private clouds to achieve optimal results. This personalized solution achieves both cost-effectiveness and security. Cloud conferencing is an efficient, convenient, and low-cost conferencing format based on cloud computing technology. Users only need to use an internet interface for simple and easy-to-use operations to quickly and efficiently share voice, data files, and video with teams and clients around the world. The complex technologies involved in data transmission and processing during the meeting are handled by the cloud conferencing service provider.

[0043] The following describes a specific embodiment of a data processing method according to the present invention. Figure 2 This is a flowchart illustrating a data processing method provided by an embodiment of the present invention. This specification provides the operational steps of the method described in the embodiments or flowcharts, but based on conventional or non-inventive methods, more or fewer operational steps may be included. The order of steps listed in the embodiments is merely one possible execution order among many and does not represent the only possible execution order. In actual system or server product execution, the method can be executed sequentially according to the embodiments or drawings, or in parallel (e.g., in a parallel processor or multi-threaded processing environment). Specifically, as shown in the figures... Figure 2 As shown, the method may include:

[0044] S201: At the target reporting time point, obtain multiple service usage datasets sent by multiple clients, each of the service usage datasets representing the call information of the corresponding client to the service object in the target time period;

[0045] In this embodiment of the invention, the server acquires multiple service usage datasets sent by multiple clients at the target reporting time point. The server may correspond to the backend of the cloud service provider, and the client may be a software component that reports the service usage dataset to the server, software developed by an enterprise user that includes the software component, or a physical device running the software. Based on the cloud service products purchased by the enterprise user from the cloud service provider (the specific type of cloud service product corresponds to a service object), the server and client point to the same cloud service provider. Each service usage dataset represents the client's call information to the service object (corresponding cloud service) during the target time period. The cloud service provider can provide enterprise users with products that integrate multiple cloud services. When the corresponding client is software developed by the enterprise user, individual users use the software. During use, the software calls cloud services (corresponding service objects) based on software components to provide relevant service experiences to individual users. The service usage dataset points to these called cloud services.

[0046] From the perspective of software components, a client sends a service usage dataset, which indicates its invocation information for at least one service object (corresponding cloud service) of a corresponding cloud service product. From the perspective of software or physical devices, a client can send at least one service usage dataset, each indicating its invocation information for at least one service object (corresponding cloud service) of a corresponding cloud service product. At least one service object of the cloud service product corresponds to a cloud service purchased by the enterprise user's software from the cloud service provider. The invocation information may include the usage duration and data usage associated with the service object.

[0047] The target reporting time point indicates the time point at which the server obtains multiple service usage datasets, and the target time period determines the client's call information to the service object within this time range as the service usage dataset.

[0048] In a specific embodiment, such as Figure 3 As shown, the method further includes determining the target reporting time point and the target time period:

[0049] S301: Configure the heartbeat reporting interface and heartbeat interval;

[0050] S302: Get the time point of the last report;

[0051] S303: Determine the target time period and the target reporting time point based on the previous reporting time point and the heartbeat interval;

[0052] The server provides a heartbeat reporting interface, which supports simultaneous and differentiated reporting from multiple cloud services and modules. It also supports simultaneous reporting from multiple clients. Furthermore, the heartbeat reporting interface allows cloud services to customize their reported usage data. Additionally, the interface enables the server to forward (transparently transmit) usage data reported by cloud service A to cloud service B for its related business logic. The server sets the heartbeat interval, which can be a fixed or variable value. It's important to note that the heartbeat interval can be set by the cloud service provider for a specific cloud service product purchased by the enterprise user; furthermore, it can be set by the cloud service provider for a specific cloud service product purchased by a particular enterprise user.

[0053] By obtaining the previous reporting time point and combining it with the heartbeat interval, the target time period and target reporting time point can be determined. The target time period is consistent with the duration indicated by the heartbeat interval, and its starting point is the previous reporting time point. The target reporting time point can be regarded as the subsequent reporting time point of the previous reporting time point.

[0054] Accordingly, the server uses the heartbeat reporting interface to obtain the heartbeat messages reported by the multiple clients at the target reporting time point, and uses these heartbeat messages as the dataset for the use of multiple services. The server can treat all messages obtained at the target reporting time point as a single message data, which effectively records the usage data of multiple cloud services. In practical applications, even when integrated into the same cloud service product, the usage times of various cloud services differ. Based on the relevant settings of the heartbeat reporting interface, mixed heartbeat reporting of multiple cloud services can be achieved, facilitating the server's unified management of cloud service usage data for enterprise users. When enterprise users' software includes components pointing to products integrating multiple cloud services, and different enterprise users have different billing rules, the server can facilitate unified and flexible billing processing for enterprise users of related cloud service products (for example, a unified bill may include independent billing items for multiple cloud services integrated into the cloud service product), providing accurate bills for enterprise users.

[0055] S202: Determine data writing rules based on the identification information carried by the dataset used by the multiple services;

[0056] In this embodiment of the invention, the identification information carried by each service dataset may include a corresponding set of service object identifiers, a business object identifier, and a user identifier. The service object identifier (represented by `BusinessType`) corresponds to a service object, that is, to a specific type of cloud service involved in a cloud service product. The business object identifier (represented by `sdkappid`) corresponds to an enterprise user, specifically the cloud service product purchased by the enterprise user, which may integrate multiple cloud services. The user identifier (represented by `userid`) corresponds to an individual user, specifically a registered user's account or a client identifier corresponding to a guest user. Registered users and guest users are users of the software developed by the enterprise user.

[0057] In a specific embodiment, such as Figure 4 As shown, the step of determining data writing rules based on the identification information carried by the dataset used by the multiple services includes:

[0058] S401: Parse each of the service usage datasets to obtain the service object identifier set, business object identifier, and user identifier corresponding to each service usage dataset;

[0059] From the perspective of software components, a client sends one service usage dataset. From the perspective of software or physical devices, a client can send at least one service usage dataset, meaning it can support mixed reporting of multiple services. Service usage dataset A can represent the call information of individual user A to cloud services in cloud service product A (purchased by enterprise user A from the cloud service provider) when using software A developed by enterprise user A within a target time period. After parsing, the service usage dataset A can be obtained as follows: service object identifier set (corresponding to cloud services in cloud service product A), business object identifier (corresponding to enterprise user A, specifically the cloud service product A purchased by enterprise user A), and user identifier (corresponding to individual user A).

[0060] S402: Based on the set of service object identifiers corresponding to the service usage dataset, extract the service object data corresponding to each service object identifier from the service usage dataset;

[0061] The cloud services integrated by cloud service product A may include cloud service a1, cloud service a2, and cloud service a3. Accordingly, based on the dimension of specific cloud services, data corresponding to the service object identifier of cloud service a1 can be extracted from service usage dataset A as service object data a1, data corresponding to the service object identifier of cloud service a2 can be extracted as service object data a2, and data corresponding to the service object identifier of cloud service a3 can be extracted as service object data a3.

[0062] S403: Obtain the service object registry corresponding to the message queue;

[0063] The service object registry is pre-configured based on the message queue and records the service objects corresponding to the data that the message queue is allowed to store. Of course, the service object registry can be updated flexibly. For example, the current service object registry records the service objects corresponding to cloud service a1 and cloud service a2.

[0064] S404: Compare the service object identifier corresponding to the service object data with the service object registry;

[0065] The service object identifier corresponding to service object data a1 is compared with the service object registry, the service object identifier corresponding to service object data a2 is compared with the service object registry, and the service object identifier corresponding to service object data a3 is compared with the service object registry.

[0066] S405: When the service object registry contains a service object identifier corresponding to the service object data, the service object data is used as candidate data, and the subscription topic in the message queue is determined based on the service object identifier corresponding to the service object data as the candidate data;

[0067] Based on the current service object registry entries for cloud service a1 and cloud service a2, service object data a1 and a2 are selected as candidate data, while service object data a3 is excluded. Then, subscription topics in the message queue are determined based on the service object identifier of service object data a1, and subscription topics in the message queue are determined based on the service object identifier of service object data a2. For service object data a3, an entry for service object data a3 can be added to the service object registry, and then a subscription topic in the message queue can be determined based on the service object identifier of service object data a3. Otherwise, an error log is printed, and the data is discarded. Subscription topics allow for data isolation of data written to the message queue.

[0068] S406: Determine the writing area of ​​the candidate data in the message queue based on the business object identifier and user identifier corresponding to the candidate data;

[0069] The write area can correspond to different partitions configured for each message queue. Data written to the message queue needs to be stored in the corresponding partition. The data in each partition is ordered, which ensures that the data source for subsequent data reading for clustering and billing processing is ordered, thereby avoiding errors in clustering and billing processing. Determining the write area of ​​candidate data in the message queue based on the business object identifier and user identifier corresponding to the candidate data is to ensure that the candidate data is stored according to the dimension of individual user groups. That is, each write area stores the candidate data corresponding to one individual user group, where each individual user group corresponds to multiple user identifiers.

[0070] To determine the write area, the following steps can be performed: First, generate a key value for the candidate data based on the business object identifier and user identifier corresponding to the candidate data. Then, obtain the target hash function corresponding to the message queue. Next, determine the write area based on the key value of the candidate data and the target hash function. The message queue can correspond to a hash table (including the target hash function) that records the storage location. Here, a key value is obtained by performing a modulo operation on the business object identifier (e.g., sdkappid) and user identifier (e.g., userid), which serve as the hash partition key. Then, the key value is substituted into the target hash function to obtain the aforementioned write area.

[0071] S407: The data writing rules are obtained based on the subscription topic and writing area corresponding to each candidate data.

[0072] Accordingly, data writing rules are obtained based on the subscription topic and writing area corresponding to each candidate data determined in the above steps.

[0073] S203: Write the datasets used by the multiple services into the message queue according to the data writing rules;

[0074] In this embodiment of the invention, the message queue can correspond to a high-throughput distributed messaging system using a publish-subscribe pattern, such as Kafka. Alternatively, the message queue can correspond to a high-performance, highly available, and high-concurrency message middleware that has been extended and optimized based on Kafka.

[0075] Based on the established data writing rules, the server can utilize message queue components to directly process multiple service usage datasets. These datasets can be accurately and completely recorded in the message queue's message log. Combining the relevant steps S401-S407 above, a message queue instance can be created, and the number of partitions configured. The message queue components are then started, and the asynchronous publisher within it sends multiple service usage datasets to the message queue instance for orderly storage according to the data writing rules. In case of sending failure, a sending failure error message will be printed. Leveraging the advantages of message queues, such as decoupling of sending and receiving, peak shaving and valley filling, and support for horizontal scaling, multiple service usage datasets can be stored in an orderly manner, reducing the probability of data loss and ensuring a reliable and stable data source for clustering and billing processing of multiple service usage datasets. When billing data is abnormal, the accuracy of the data from the message queue can support data verification. Accordingly, introducing a message queue can provide underlying support for billing processing.

[0076] S204: Read the multiple service usage datasets from the message queue according to the preset reading rules;

[0077] In this embodiment of the invention, a preset read rule is used to define the time point at which data reading begins from the message queue, whether to introduce the concept of sharding to organize the read data, and the time interval for creating two adjacent shards when sharding is introduced. The preset read rule can be generated according to the following steps: 1) Determine the start time point for writing data to the message queue; 2) Set a target time interval for aggregating the data extracted from the message queue; 3) Generate the preset read rule based on the start time point and the target time interval.

[0078] like Figure 5 , 7 As shown, the step of reading the multiple service usage datasets from the message queue according to preset reading rules includes:

[0079] S501: Based on the start time point, data is sequentially extracted from the message queue, and the data extracted from the message queue is aggregated every target time interval to obtain the corresponding data shards;

[0080] S502: Obtain the clustering processing trigger condition, and trigger the clustering processing of the corresponding data shards according to the clustering processing trigger condition;

[0081] S503: Repeat the steps described above, from extracting data from the message queue to triggering clustering processing of the corresponding data shards, until the multiple service datasets have been extracted from the message queue.

[0082] This can be viewed as sharding and organizing a large amount of data in the message queue, which can improve the efficiency of subsequent clustering processing. Based on the start time, the data extracted from the message queue corresponding to each target time interval is taken as a shard. The aggregated shards can trigger subsequent clustering processing steps in real time, or wait for a triggering opportunity to initiate subsequent clustering processing steps. The aggregated shards can be cached in Redis (a key-value database) and stored in the form of a list.

[0083] Specifically, a shard can be created based on the start time (at which point no data has been extracted from the message queue). Then, data is extracted from the message queue one by one according to the first-in, first-out (FIFO) principle. The data extracted from the message queue corresponding to the target time interval is then inserted into this shard. For example, if a shard is created at the start time and the target time interval is 5 minutes, if the time interval between the extraction time corresponding to the extracted data and the start time is greater than 5 minutes, this data will be inserted into the newly created shard; otherwise, it will be inserted into the shard created at the start time.

[0084] like Figure 8 As shown, the clustering trigger condition determines when to perform clustering processing on shards. It can iteratively check if any shards need processing. If there are two or more shards in the current shard pool (corresponding to the location of cached shards that have already been aggregated, such as in Redis mentioned above), then the shard that was aggregated earlier is considered to need processing; otherwise, the check continues. The `sleep` function (which allows a computer program (process, task, or thread) to enter a sleep state, making it inactive for a period of time; execution resumes when the timer expires, a signal is received, or the program is interrupted) can be used for iterative processing.

[0085] Furthermore, when extracting data from the message queue, a validity check needs to be performed on the data before it is pushed into a shard. For example, if the data reveals that a registered user's age corresponds to 10,000 years old, an error log can be printed and the data discarded. Data that meets the validity check requirements is then pushed into the shard.

[0086] S205: Cluster the multiple service usage datasets using the service object identifier of the service object as the center point to obtain the clustering result;

[0087] In this embodiment of the invention, based on multiple service usage datasets read from the message queue, a set of service object identifiers corresponding to each service usage dataset can be obtained. These service usage datasets are then clustered using the service object identifiers as the centroids to obtain the clustering results. In the clustering results, all service object data corresponding to the same service object identifier (e.g., cloud service a1, specifically cloud service a1 in cloud service product A) constitute a set. The user identifiers corresponding to the data in this set can include the user identifiers corresponding to individual user A, individual user B, and individual user C, etc.

[0088] In a specific implementation, multiple service usage datasets can first be clustered using business object identifiers (corresponding to enterprise users, specifically cloud service products purchased by enterprise users) as the central point, resulting in multiple large groups along the enterprise user dimension or even the cloud service product dimension purchased by enterprise users. Then, the data within each large group is clustered using service object identifiers (corresponding to cloud services within the cloud service products purchased by enterprise users) as the central point, resulting in multiple medium groups along the cloud service dimension. Finally, the data within each small group is clustered using user identifiers (corresponding to individual users) as the central point, resulting in multiple small groups along the individual user dimension.

[0089] Combining the relevant content in steps S501-S503 above, based on the task processing trigger conditions, the service usage data in the processing tasks that need to be executed using clustering processing is traversed, or in other words, the service usage data in the shards that need to be clustered is traversed. This data is grouped according to the enterprise user dimension, cloud service product dimension, cloud service dimension (including sub-cloud service dimension), and user dimension, and the usage data of individual users under each enterprise user for each cloud service is aggregated and organized. This method of sharding and aggregating large amounts of data separates and records the usage of all users for each cloud service. Combined with the introduced message queue, multiple service usage datasets can be processed quickly and efficiently, accurately recording the service usage data of each user, providing reliable assurance for subsequent billing data generation and facilitating billing detail queries. It should be noted that multiple service usage datasets can be treated as a whole, replicated based on different dimensions, and individually analyzed and saved. Alternatively, this whole can be individually analyzed and saved based on different dimensional hierarchical settings (see [link to relevant documentation]). Figure 9 The two statistical methods mentioned above can coexist, and of course, the statistical methods can be flexibly set.

[0090] S206: Generate billing data based on the clustering results.

[0091] In this embodiment of the invention, the clustering results are statistically analyzed based on pre-designed fee rules to generate billing data. Different enterprise users have different billing rules. The pre-designed fee rules point to the target business object and correspond to a specific enterprise user, which can further correspond to the specific cloud service product purchased by that specific enterprise user.

[0092] In one specific embodiment, before generating billing data based on the clustering results, corresponding billing information can be configured for multiple service objects corresponding to the target business object. This corresponding billing information includes at least one of the following: billing unit (e.g., duration calculation unit, traffic unit), billing price, billing method (e.g., prepayment), and billing period. Different cloud services integrated into cloud service products can correspond to different billing information. Pre-designed billing rules can be determined based on the corresponding billing information.

[0093] Accordingly, generating billing data based on the clustering results includes: 1) obtaining the corresponding billing information; 2) obtaining a first statistical data point corresponding to the service object dimension of the target business object based on the clustering results, and obtaining first billing data corresponding to the service object dimension of the target business object based on the corresponding billing information and the first statistical data. The first statistical data can characterize the usage of each cloud service in a certain cloud service product by all individual users. For example, the first statistical data can include service usage data of all individual users for cloud service a1, service usage data of all individual users for cloud service a2, etc. The first billing data can characterize the usage and consumption information of each cloud service in a certain cloud service product by all individual users. For example, the first billing data can include consumption amount data of all individual users for cloud service a1, consumption amount data of all individual users for cloud service a2, etc. 3) obtaining a second statistical data point corresponding to the user dimension of the target business object based on the clustering results, and obtaining second billing data corresponding to the user dimension of the target business object based on the corresponding billing information and the second statistical data. The second set of statistical data can represent the usage of each individual user across all cloud services (e.g., cloud service a1 + cloud service a2 + cloud service a3) within a specific cloud service product. For example, the second set of statistical data could include user A's usage data for cloud services a1 + a2 + a3, user B's usage data for cloud services a1 + a2 + a3, and so on. The second set of billing data can represent each individual user's consumption information across all cloud services within a specific cloud service product. For example, the second set of billing data could include user A's consumption amount for cloud services a1 + a2 + a3, user B's consumption amount for cloud services a1 + a2 + a3, and so on.

[0094] In another specific embodiment, such as Figure 10 As shown, after generating billing data based on the billing rules and the clustering results, a first unsettled value can be determined based on the billing data. The first unsettled value can indicate the unsettled value (such as monetary amount, duration, etc.) generated by a specific enterprise user's use of the corresponding cloud service product. Then, the target value corresponding to the target business object is obtained. The target value can correspond to the value (such as monetary amount, duration, etc.) that the specific enterprise user has pre-purchased from the cloud service provider but has not yet used. The target value can be specific to the cloud service product. Furthermore, 1) when the target value is greater than or equal to the first unsettled value, the target value is updated based on the first unsettled value. That is, when the pre-stored value is greater than or equal to the unsettled value, the pre-stored value is sufficient to offset the unsettled value generated by the specific enterprise user's use of the corresponding cloud service product. The target value is updated based on the first unsettled value, and the value obtained by subtracting the first unsettled value from the current target value is used as the next target value. 2) when the target value is less than the first unsettled value, a second unsettled value is determined based on the first unsettled value and the target value, and an instruction to stop calling the service object corresponding to the target business object is generated based on the second unsettled value. In other words, when the pre-stored value is less than the value to be settled, the pre-stored value is insufficient to offset the value to be settled incurred by a specific enterprise user when using the corresponding cloud service product. The second unsettled value represents the value to be settled that has not yet been offset after the target value has been used. At this point, an instruction can be generated to stop calling the service object corresponding to the target business object, suspending the provision of the corresponding cloud service product to the specific enterprise user.

[0095] In practical applications, the "sharding" described in steps S204 and S205 can be used as a framework. First, service usage data for individual users within each shard is read to calculate the service usage of each individual user. Then, all individual users corresponding to the same enterprise user are traversed to calculate the total service usage of the enterprise within the statistical time period. Next, the prepaid service packages purchased by the enterprise users are retrieved, usage is settled, and the remaining capacity of the packages is deducted. Enterprise users with insufficient remaining capacity are suspended from service. This provides a reliable guarantee for subsequent billing data generation and facilitates billing detail queries. This improves the accuracy and timeliness of billing settlement and supports subsequent multi-dimensional billing detail queries (e.g., providing usage data for individual users, summarizing usage data for all individual users corresponding to the same enterprise user, and storing it in the database). Furthermore, the prepaid service packages purchased by enterprise users can be flexibly allocated based on previous usage data and actual needs, including the prepaid cloud service usage corresponding to the prepaid service packages.

[0096] As can be seen from the technical solutions provided in the embodiments of this specification above, these embodiments acquire multiple service usage datasets sent by multiple clients at the target reporting time point, and periodically acquire unified information on each client's call to service objects within the target time period. Introducing a message queue allows for ordered storage of multiple service usage datasets and ensures a reliable and stable data source for clustering processing of these datasets. This invention can improve the efficiency and accuracy of generating billing data and enhance the commercialization capabilities of cloud service providers.

[0097] This invention also provides a data processing apparatus, such as... Figure 11 As shown, the device includes:

[0098] Acquisition module 1110: used to acquire multiple service usage datasets sent by multiple clients at the target reporting time point, each of the service usage datasets representing the call information of the corresponding client to the service object in the target time period;

[0099] Determining module 1120: used to determine data writing rules based on the identification information carried by the service usage dataset;

[0100] Writing module 1130: used to write the multiple service datasets into a message queue according to the data writing rules;

[0101] Reading module 1140: used to read the multiple service usage datasets from the message queue according to preset reading rules;

[0102] Clustering module 1150: used to cluster the multiple services using the service object identifier of the service object as the center point, and obtain the clustering result;

[0103] Billing module 1160: Used to generate billing data based on the clustering results.

[0104] Specifically, such as Figure 6 As shown, the acquisition module, determination module, and writing module can correspond to each other. Figure 6 The heartbeat reporting module shown can be read by the corresponding module. Figure 6 The usage collection module, clustering module, and billing module shown can be corresponding to... Figure 6 The metering and billing module is shown. The clustering module categorizes and reports the data to the billing module, which then persistently stores each usage record for reconciliation.

[0105] It should be noted that the apparatus and method embodiments described in the device embodiments are based on the same inventive concept.

[0106] This invention provides an electronic device including a processor and a memory. The memory stores at least one instruction or at least one program segment, which is loaded and executed by the processor to implement the data processing method provided in the above method embodiments.

[0107] Furthermore, Figure 12 A schematic diagram of the hardware structure of an electronic device for implementing the data processing method provided in the embodiments of the present invention is shown. The electronic device may participate in or include the data processing apparatus provided in the embodiments of the present invention. Figure 12 As shown, the electronic device 120 may include one or more processors 1202 (shown as 1202a, 1202b, ..., 1202n in the figure) 1202 (processor 1202 may include, but is not limited to, a microprocessor MCU or a programmable logic device FPGA, etc.), a memory 1204 for storing data, and a transmission device 1206 for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply, and / or a camera. Those skilled in the art will understand that... Figure 12 The structure shown is for illustrative purposes only and does not limit the structure of the electronic device described above. For example, the electronic device 120 may also include... Figure 12 The more or fewer components shown, or having the same Figure 12 The different configurations shown.

[0108] It should be noted that the aforementioned one or more processors 1202 and / or other data processing circuits are generally referred to herein as "data processing circuits". These data processing circuits may be embodied, in whole or in part, in software, hardware, firmware, or any other combination thereof. Furthermore, the data processing circuits may be a single, independent processing module, or may be integrated, in whole or in part, into any other element within the electronic device 120 (or mobile device). As involved in the embodiments of this application, the data processing circuits serve as a processor control mechanism (e.g., selection of a variable resistor termination path connected to an interface).

[0109] The memory 1204 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the method described in the embodiments of the present invention. The processor 1202 executes various functional applications and data processing by running the software programs and modules stored in the memory 124, thereby realizing the above-described data processing method. The memory 1204 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 1204 may further include memory remotely located relative to the processor 1202, and these remote memories can be connected to the electronic device 120 via a network. Examples of such networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

[0110] The transmission device 1206 is used to receive or send data via a network. Specific examples of the network described above may include a wireless network provided by the communication provider of the electronic device 120. In one example, the transmission device 1206 includes a network interface controller (NIC), which can connect to other network devices via a base station to communicate with the Internet. In one embodiment, the transmission device 1206 may be a radio frequency (RF) module for wireless communication with the Internet.

[0111] The display can be, for example, a touchscreen liquid crystal display (LCD) that allows a user to interact with the user interface of the electronic device 120 (or mobile device).

[0112] Embodiments of the present invention also provide a storage medium, which can be disposed in an electronic device to store at least one instruction or at least one program related to implementing a data processing method in the method embodiments. The at least one instruction or the at least one program is loaded and executed by the processor to implement the data processing method provided in the above method embodiments.

[0113] Optionally, in this embodiment, the storage medium may be located at at least one of the multiple network servers in a computer network. Optionally, in this embodiment, the storage medium may include, but is not limited to, various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0114] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0115] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the device and electronic device embodiments are basically similar to the method embodiments, so the descriptions are relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0116] Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by hardware or by a program instructing related hardware. The program can be stored in a computer-readable storage medium, such as a read-only memory, a disk, or an optical disk.

[0117] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A data processing method, characterized in that, The method includes: At the target reporting time point, obtain multiple service usage datasets sent by multiple clients, each of which represents the call information of the corresponding client to the service object during the target time period; The data writing rules are determined based on the identification information carried by the multiple service usage datasets: Each service usage dataset is parsed to obtain a set of service object identifiers, a business object identifier, and a user identifier corresponding to each dataset; based on the set of service object identifiers, service object data corresponding to each service object identifier is extracted from the dataset; a service object registry corresponding to the message queue is obtained; the service object identifier corresponding to the service object data is compared with the service object registry; when the service object registry records a service object identifier corresponding to the service object data, the service object data is used as candidate data, and a subscription topic in the message queue is determined based on the service object identifier corresponding to the service object data as the candidate data; the writing area of ​​the candidate data in the message queue is determined based on the business object identifier and user identifier corresponding to the candidate data; and the data writing rules are obtained based on the subscription topic and writing area corresponding to each candidate data. According to the data writing rules, the multiple services use datasets to write data into the message queue; The multiple service usage datasets are read from the message queue according to preset reading rules; Cluster the multiple services using the service object identifier of the service object as the center point to obtain the clustering result; Billing data is generated based on the clustering results.

2. The method according to claim 1, characterized in that, The method further includes determining the target reporting time point and the target time period: Configure the heartbeat reporting interface and heartbeat interval; Get the time of the last report; The target time period and the target reporting time point are determined based on the previous reporting time point and the heartbeat interval; Accordingly, obtaining multiple service usage datasets sent by multiple clients at the target reporting time point includes: The heartbeat reporting interface is used to obtain the heartbeat messages reported by the multiple clients at the target reporting time point, and the heartbeat messages reported by the multiple clients are used as the dataset for the multiple services.

3. The method according to claim 1, characterized in that, The step of determining the writing area of ​​the candidate data in the message queue based on the business object identifier and user identifier corresponding to the candidate data includes: Generate key values ​​for the candidate data based on the business object identifier and user identifier corresponding to the candidate data; Obtain the target hash function corresponding to the message queue; The write area is determined based on the key values ​​of the candidate data and the target hash function.

4. The method according to claim 1, characterized in that, Before reading the multiple service usage datasets from the message queue according to preset reading rules, the method further includes: Determine the start time point for writing data to the message queue; Set the target time interval for aggregating data extracted from the message queue; The preset reading rules are generated based on the start time and the target time interval.

5. The method according to claim 4, characterized in that, The step of reading the multiple service usage datasets from the message queue according to preset reading rules includes: Based on the start time point, data is sequentially extracted from the message queue, and the data extracted from the message queue is aggregated at each target time interval to obtain the corresponding data shards; The steps include obtaining clustering processing trigger conditions and triggering clustering processing on the corresponding data shards based on the clustering processing trigger conditions. Repeat the above steps of extracting data from the message queue until clustering processing of the corresponding data shards is triggered, until the datasets of the multiple services are completely extracted from the message queue.

6. The method according to claim 1, characterized in that, Before generating billing data based on the clustering results, the method further includes: Configure the corresponding billing information for each of the multiple service objects corresponding to the target business object; Accordingly, generating billing data based on the clustering results includes: Obtain the corresponding billing information; Based on the clustering results, a first statistical data point corresponding to the service object dimension of the target business object is obtained, and based on the corresponding billing information and the first statistical data point, a first billing data point corresponding to the service object dimension of the target business object is obtained; Based on the clustering results, a second statistical data point corresponding to the user dimension of the target business object is obtained, and based on the corresponding billing information and the second statistical data point, a second billing data point corresponding to the user dimension of the target business object is obtained.

7. The method according to claim 1, characterized in that, After generating billing data based on the clustering results, the method further includes: The first unsettled value is determined based on the billing data; Obtain the target value corresponding to the target business object; When the target value is greater than or equal to the first unsettled value, the target value is updated based on the first unsettled value; When the target value is less than the first unsettled value, a second unsettled value is determined based on the first unsettled value and the target value, and an instruction to stop calling the service object corresponding to the target business object is generated based on the second unsettled value.

8. A data processing apparatus, characterized in that, The device includes: Acquisition module: used to acquire multiple service usage datasets sent by multiple clients at the target reporting time point, each of the service usage datasets representing the call information of the corresponding client to the service object in the target time period; Determining module: used to determine data writing rules based on the identification information carried by the service dataset; Write module: used to write the datasets of the multiple services into the message queue according to the data writing rules; Reading module: used to read the multiple service usage datasets from the message queue according to preset reading rules; Clustering module: used to cluster the multiple services using the service object identifier of the service object as the center point, and obtain the clustering result; Billing module: used to generate billing data based on the clustering results; The step of determining the data writing rules based on the identification information carried by the multiple service usage datasets includes: parsing each service usage dataset to obtain a set of service object identifiers, a business object identifier, and a user identifier corresponding to each service usage dataset; extracting service object data corresponding to each service object identifier from the service usage dataset based on the set of service object identifiers corresponding to the service usage dataset; obtaining the service object registry corresponding to the message queue; comparing the service object identifier corresponding to the service object data with the service object registry; when the service object registry records the service object identifier corresponding to the service object data, using the service object data as candidate data, and determining the subscription topic in the message queue based on the service object identifier corresponding to the service object data as the candidate data; determining the writing area of ​​the candidate data in the message queue based on the business object identifier and user identifier corresponding to the candidate data; and obtaining the data writing rules based on the subscription topic and writing area corresponding to each candidate data.

9. The apparatus according to claim 8, characterized in that, The device is also used to determine the target reporting time point and the target time period: Configure the heartbeat reporting interface and heartbeat interval; Get the time of the last report; The target time period and the target reporting time point are determined based on the previous reporting time point and the heartbeat interval; Accordingly, the acquisition module is also used to: acquire the heartbeat messages reported by the multiple clients at the target reporting time point using the heartbeat reporting interface, and use the heartbeat messages reported by the multiple clients as the dataset used by the multiple services.

10. The apparatus according to claim 8, characterized in that, The step of determining the writing area of ​​the candidate data in the message queue based on the business object identifier and user identifier corresponding to the candidate data includes: Generate key values ​​for the candidate data based on the business object identifier and user identifier corresponding to the candidate data; Obtain the target hash function corresponding to the message queue; The write area is determined based on the key values ​​of the candidate data and the target hash function.

11. The apparatus according to claim 8, characterized in that, The device is also used for: Determine the start time point for writing data to the message queue; Set the target time interval for aggregating data extracted from the message queue; The preset reading rules are generated based on the start time and the target time interval.

12. The apparatus according to claim 11, characterized in that, The reading module is also used for: Based on the start time point, data is sequentially extracted from the message queue, and the data extracted from the message queue is aggregated at each target time interval to obtain the corresponding data shards; The steps include obtaining clustering processing trigger conditions and triggering clustering processing on the corresponding data shards based on the clustering processing trigger conditions. Repeat the above steps of extracting data from the message queue until clustering processing of the corresponding data shards is triggered, until the datasets of the multiple services are completely extracted from the message queue.

13. The apparatus according to claim 8, characterized in that, The device is also used for: Configure the corresponding billing information for each of the multiple service objects corresponding to the target business object; Accordingly, the billing module is also used for: Obtain the corresponding billing information; Based on the clustering results, a first statistical data point corresponding to the service object dimension of the target business object is obtained, and based on the corresponding billing information and the first statistical data point, a first billing data point corresponding to the service object dimension of the target business object is obtained; Based on the clustering results, a second statistical data point corresponding to the user dimension of the target business object is obtained, and based on the corresponding billing information and the second statistical data point, a second billing data point corresponding to the user dimension of the target business object is obtained.

14. The apparatus according to claim 8, characterized in that, The device is also used for: The first unsettled value is determined based on the billing data; Obtain the target value corresponding to the target business object; When the target value is greater than or equal to the first unsettled value, the target value is updated based on the first unsettled value; When the target value is less than the first unsettled value, a second unsettled value is determined based on the first unsettled value and the target value, and an instruction to stop calling the service object corresponding to the target business object is generated based on the second unsettled value.

15. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction or at least one program, which is loaded and executed by a processor to implement the data processing method as described in any one of claims 1-7.

16. An electronic device, characterized in that, The electronic device includes a processor and a memory, the memory storing at least one instruction or at least one program, the at least one instruction or at least one program being loaded and executed by the processor to implement the data processing method as described in any one of claims 1-7.

Images