Data processing method and apparatus

Abstract

This specification provides a data processing method and apparatus. The method includes: acquiring user traffic data based on level tags, wherein the user traffic data carries category tags and level tags; storing the user traffic data into a pre-established traffic data list based on the category tags, wherein the traffic data list includes a task table for the category tags, and the task table manages the acquired user traffic data; acquiring target traffic data from the task table, and performing data processing based on the target traffic data. Acquiring user traffic data through level tags, obtaining user data traffic corresponding to each level tag, and storing it in the traffic data list makes the traffic data obtained from the traffic data list more comprehensive. Furthermore, storing the traffic data in the task table by category in the data traffic list ensures the stability of the output of each type of user traffic data and facilitates the detection of abnormalities in user traffic data processing.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of data processing technology, and in particular to a data processing method. Background Technology

[0002] Risk control is the core underlying mechanism for online micro-finance. Numerous production changes occur daily, including offline data switching and strategy model deployment. To safeguard production security and mitigate the impact of changes not meeting expectations, automated production log reflow is implemented. During the pre-implementation phase of changes, both the production and pre-implementation environments are called in real-time, and the results are compared to verify the correctness of the changes in advance—a process we call "traffic simulation."

[0003] The current approach is to simply randomly recycle production logs, which inevitably leads to uneven traffic distribution. Popular products receive the majority of the traffic, while most niche products receive very little, making comprehensive verification impossible. Summary of the Invention

[0004] In view of the above, embodiments of this specification provide a data processing method. One or more embodiments of this specification also relate to a data processing apparatus, a computing device, a computer-readable storage medium, and a computer program, to address the technical deficiencies existing in the prior art.

[0005] According to a first aspect of the embodiments of this specification, a data processing method is provided, comprising:

[0006] User traffic data is obtained based on the level label, wherein the user traffic data carries a category label and the level label;

[0007] According to the category label, the user traffic data is stored in a pre-established traffic data list, wherein the traffic data list includes a task table for the category label, and the task table manages the acquired user traffic data;

[0008] Obtain target traffic data from the task table and perform data processing based on the target traffic data.

[0009] According to a second aspect of the embodiments of this specification, a data processing apparatus is provided, comprising:

[0010] The data acquisition module is configured to acquire user traffic data based on level labels, wherein the user traffic data carries category labels and level labels;

[0011] The data update module is configured to store the user traffic data into a pre-established traffic data list according to the category label, wherein the traffic data list includes a task table for the category label, and the task table manages the acquired user traffic data;

[0012] The data processing module is configured to obtain target traffic data from the task table and perform data processing based on the target traffic data.

[0013] According to a third aspect of the embodiments of this specification, a computing device is provided, comprising:

[0014] Memory and processor;

[0015] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the above-described data processing method.

[0016] According to a fourth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions, which, when executed by a processor, implement the steps of the data processing method described above.

[0017] According to a fifth aspect of the embodiments of this specification, a computer program is provided, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the above-described data processing method.

[0018] This specification provides a data processing method and apparatus. The data processing method includes: acquiring user traffic data based on level tags, wherein the user traffic data carries category tags and level tags; storing the user traffic data into a pre-established traffic data list based on the category tags, wherein the traffic data list includes a task table for the category tags, and the task table manages the acquired user traffic data; acquiring target traffic data from the task table, and performing data processing based on the target traffic data. Acquiring user traffic data through level tags allows for the acquisition of user data traffic corresponding to each level tag, which is then stored in the traffic data list. This makes the traffic data acquired from the traffic data list more comprehensive and balanced. Furthermore, because the traffic data is stored in the task table by category in the traffic list, it ensures that each type of user traffic data can be output, resulting in a large and stable output traffic volume, and facilitating the detection of abnormalities in user traffic data processing. Attached Figure Description

[0019] Figure 1 This is a schematic diagram illustrating a data processing method provided in one embodiment of this specification.

[0020] Figure 2This is a flowchart illustrating a data processing method provided in one embodiment of this specification;

[0021] Figure 3 This is a flowchart illustrating the processing procedure of a data processing method provided in one embodiment of this specification.

[0022] Figure 4 This is a schematic diagram of the structure of a data processing device provided in one embodiment of this specification;

[0023] Figure 5 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0024] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0025] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0026] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0027] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0028] Traffic simulation: Automated backflow of production logs, during the pre-effective phase of changes, calls the production environment and the pre-effective environment in real time, and compares the results returned by the two to achieve the purpose of verifying the correctness of the changes in advance.

[0029] Traffic pool: A device that categorizes online product traffic into three types based on high-frequency traffic, low-frequency traffic, and new business traffic, and then handles each type of traffic separately.

[0030] A list is a finite sequence of data items, that is, a collection of data items arranged in a certain linear order. The basic operations performed on this data structure include searching, inserting, and deleting elements.

[0031] This specification provides a data processing method, and also relates to a data processing apparatus, a computing device, and a computer-readable storage medium, which will be described in detail in the following embodiments.

[0032] See Figure 1 , Figure 1 The illustration shows a scenario diagram of a data processing method provided according to an embodiment of this specification, specifically including a server 110, a client 120, a traffic data list 130, and a task table 140.

[0033] Specifically, server 110 receives user traffic data sent by different clients 120, such as requests to view balance, requests to view credit limit, etc. This user traffic data has a level label and a category label. The level label is used to represent the level corresponding to the user data traffic, and the category label is used to represent the business category to which the user traffic data belongs.

[0034] After receiving these traffic data, the server 110 extracts a portion of the user traffic data from all the user traffic data according to the level label and stores it in the traffic pool 130. The traffic pool 130 includes multiple task tables 140, each of which corresponds to traffic data for a business category.

[0035] By obtaining user traffic data through level tags, the user data traffic corresponding to each level tag can be obtained and stored in the traffic data list, making the traffic data obtained from the traffic data list more comprehensive. At the same time, because the data traffic is stored in the task table by category in the data traffic list, the stability of the output of each type of user traffic data is guaranteed, and it is also easy to detect abnormal situations in user traffic data processing.

[0036] See Figure 2 , Figure 2 A flowchart of a data processing method according to an embodiment of this specification is shown, which specifically includes the following steps.

[0037] Step 202: Obtain user traffic data based on the level label, wherein the user traffic data carries a category label and the level label.

[0038] The grade label can be a label representing the grade corresponding to the user traffic data. This grade label is determined based on the frequency of occurrence of the user traffic data. In practical applications, the higher the frequency of occurrence of user traffic data, the higher the grade label of the user traffic data; the lower the frequency of occurrence of user traffic data, the lower the grade label of the user traffic data. For example, if a user requests to view a loan limit and user traffic data A occurs 100 times per minute, then the grade label of user traffic data A is high-frequency traffic. User traffic data can be the request data sent by the user through the client to the server, such as a user requesting to view a loan limit. The category label can be a label representing the business type to which the user traffic data belongs. For example, if the user traffic data is traffic data from a user requesting to view a loan limit, then the corresponding type label for the user traffic data could be a loan business label.

[0039] In practical applications, user traffic data includes traffic data with high frequency and traffic data with low frequency. If the frequency of user traffic data is not considered and random sampling is performed, it may result in only high-frequency traffic being obtained. This may lead to inaccurate processing results when processing user traffic data later because it is impossible to obtain accurate processing results from comprehensive user traffic data. Therefore, level labels are added to user traffic data to extract user traffic data with different level labels.

[0040] For example, the level labels include high-frequency traffic level labels and low-frequency traffic level labels, and 100 user traffic data are extracted from the high-frequency traffic level labels and low-frequency traffic level labels respectively.

[0041] The embodiments in this specification obtain user traffic data based on level labels, thereby obtaining user traffic data corresponding to different level labels and improving the comprehensiveness of user traffic data.

[0042] In one possible implementation, obtaining user traffic data based on grade tags includes:

[0043] User traffic data is obtained based on level tags at preset time intervals.

[0044] The preset time interval can be understood as a pre-set time period. For example, if the preset time interval is ten minutes, it means that the operation of obtaining user traffic data based on the level label is performed once every ten minutes.

[0045] In practical applications, it is necessary to continuously collect new user traffic data to make the user traffic data more closely reflect the actual user traffic data in recent times. Therefore, it is necessary to acquire user data traffic at preset time intervals.

[0046] For example, if the preset time interval is ten minutes, then user data traffic will be collected once every ten minutes. The level tags include high-frequency traffic level tags and low-frequency traffic level tags. Every ten minutes, 100 user traffic data points will be extracted from the high-frequency traffic level tags and the low-frequency traffic level tags respectively.

[0047] The embodiments in this specification obtain user data traffic at preset time intervals to ensure that user data traffic is more consistent with the current time, thereby improving the authenticity of user data traffic.

[0048] In one feasible approach, the rating labels are at least two;

[0049] Accordingly, obtaining user traffic data based on level tags includes:

[0050] A preset number of user traffic data are obtained from the user traffic data corresponding to the at least two level labels.

[0051] The preset quantity is the preset number of user traffic data entries. For example, if the preset quantity is 100 entries, then 100 user traffic data entries will be retrieved.

[0052] In practical applications, by setting the amount of user data traffic acquired, the speed of user data traffic updates can be controlled, thereby improving the stability of user traffic data.

[0053] For example, if the preset quantity is 100 and the preset time interval is ten minutes, then user data traffic will be collected once every ten minutes. The level tags include high-frequency traffic level tags and low-frequency traffic level tags. Every ten minutes, 100 user traffic data will be extracted from the high-frequency traffic level tags and the low-frequency traffic level tags respectively.

[0054] The embodiments in this specification can control the speed of user data traffic updates by setting the amount of user data traffic acquired, thereby improving the stability of user traffic data.

[0055] In one possible implementation, the level label includes a first level label, wherein the frequency of user traffic data corresponding to the first level label is greater than a preset frequency threshold.

[0056] Accordingly, obtaining user traffic data based on level tags includes:

[0057] Obtain the preset number of user traffic data from the user traffic data corresponding to the first level label.

[0058] The first-level label can be a high-frequency label. For example, "1" can represent the first-level label. If the user traffic data corresponding to the user's balance acquisition is a high-frequency label, then "1" is marked on the user traffic data corresponding to the user's balance acquisition to indicate that the user traffic data corresponding to the user's balance acquisition is high-frequency traffic data. The frequency of user traffic data can be understood as the number of times user data traffic occurs within a certain period of time. For example, if user traffic data A occurs 100 times in one hour, then the frequency of user traffic data A is 100 times per hour. The preset frequency threshold can be a threshold for determining user traffic data. For example, a frequency of more than 100 times per hour is defined as high-frequency traffic data, and a frequency of one hour or less is defined as low-frequency traffic data.

[0059] In practical applications, user traffic data can be divided into multiple levels based on the frequency of occurrence, and a traffic label can be set for each level. In this way, user data traffic of different frequencies can be collected, and there will be no situation where some user traffic data cannot be collected because the frequency of occurrence of some user traffic data is low.

[0060] For example, the preset frequency thresholds are: more than 100 times per hour is defined as high-frequency traffic data, and a frequency of one hour or less is defined as low-frequency traffic data. If, within a certain period, the user traffic data corresponding to the user's balance acquisition appears more than 100 times per hour, and the user traffic data corresponding to the user's credit acquisition appears more than 100 times per hour, then the user traffic data corresponding to the user's balance acquisition and credit acquisition is considered high-frequency traffic data. Because the user traffic data corresponding to the user's balance acquisition and credit acquisition is large—for example, 1000 user traffic data entries every ten minutes for both—1000 user traffic data entries can be randomly selected from these two categories.

[0061] For example, if, over a period of time, the user traffic data corresponding to the user acquiring balance appears more than 100 times per hour, and the user traffic data corresponding to the user acquiring credit limit also appears more than 100 times per hour, then the user traffic data corresponding to the user acquiring balance and the user acquiring credit limit are high-frequency traffic data. Because the data volume of these two types of user traffic data is large—for example, if there are 1000 user traffic data entries for each type of user acquiring balance and credit limit every ten minutes—then 10 entries can be extracted in the first minute, 10 in the second minute, and so on, up to the tenth minute, meaning an average of 100 entries are extracted over 10 minutes.

[0062] It should be noted that the embodiments in this specification do not limit the extraction method for high-frequency traffic data; any specific extraction method can be used.

[0063] The embodiments in this specification randomly acquire user traffic data corresponding to high-frequency level tags to obtain representative high-frequency traffic data, thereby improving the authenticity of the extracted user traffic data.

[0064] In one possible implementation, the level label includes a second level label, wherein the frequency of the user traffic data corresponding to the second level label is less than or equal to the preset frequency threshold.

[0065] Accordingly, obtaining user traffic data based on level tags includes:

[0066] Monitor the user traffic data corresponding to the second-level label, and if the number of user traffic data corresponding to the second-level label meets the preset number, obtain the preset number of user traffic data corresponding to the second-level label.

[0067] The second-level label can be a low-frequency label. For example, "2" can represent the second-level label. If the user traffic data corresponding to the user's credit limit is a low-frequency label, then "2" can be added to the user traffic data corresponding to the user's credit limit to indicate that the user traffic data corresponding to the user's credit limit is low-frequency traffic data.

[0068] In practical applications, for low-frequency services, that is, user traffic data corresponding to low-frequency level tags, since online traffic data is scarce, an aggregation method is adopted to store all the traffic within a certain period of time, while also replacing old data with new data.

[0069] For example, the preset frequency thresholds are: more than 100 times per hour is defined as high-frequency traffic data, and a frequency of one hour or less is defined as low-frequency traffic data. If, within a certain period, the frequency of user traffic data related to the repayment date is less than 100 times per hour, and the frequency of user traffic data related to the configuration item is less than 100 times per hour, then the user traffic data related to the repayment date and the configuration item is considered low-frequency traffic data. Since the amount of user traffic data related to the repayment date and the configuration item is small—for example, 50 instances of user traffic data related to the repayment date every ten minutes, and 10 instances of user traffic data related to the configuration item every ten minutes—it can wait until there are at least 100 instances of user traffic data related to the repayment date and the configuration item before retrieving them.

[0070] For example, if, within a certain period, the frequency of user traffic data related to the repayment date appears less than 100 times per hour, and the frequency of user traffic data related to the configuration item appears less than 100 times per hour, then the user traffic data related to the repayment date and the configuration item are considered low-frequency traffic data. Since the data volume for the user traffic data related to the repayment date and the configuration item is small—for example, 50 user traffic data points related to the repayment date every ten minutes, and 10 user traffic data points related to the configuration item every ten minutes—then we can directly obtain the user traffic data related to the repayment date and the configuration item, i.e., 60 user traffic data points.

[0071] The embodiments in this specification collect user traffic data corresponding to low-frequency level tags, so that infrequently occurring user data traffic is also collected, thereby improving the comprehensiveness of user traffic data.

[0072] In one possible implementation, the level label includes a third level label, wherein the third level label is created according to a preset business scenario;

[0073] Accordingly, obtaining user traffic data based on level tags includes:

[0074] Based on the third-level label, receive the preset number of user traffic data corresponding to the third-level label.

[0075] The third-level label can be an unpublished business level label, i.e. a new business level label. For example, if a new business N has not been published to customers, the user traffic data corresponding to the new business N can be labeled with a third-level label. The preset business scenario can be a newly established, unpublished business scenario.

[0076] In practical applications, for new businesses and new scenarios, there is no traffic data online, so it is necessary to construct and store it manually. Here, online can be understood as the form in which users send data to the server via the network.

[0077] For example, for business scenarios that have not yet been launched, there will be no real traffic data. Therefore, testers need to manually construct traffic data. For instance, testers can construct 50 user traffic data entries corresponding to the new business N and send these 50 user traffic data entries corresponding to the new business N to the collection module, which is to receive the preset number of user traffic data entries corresponding to the third-level tags.

[0078] This specification's embodiments collect data from unlaunched business scenarios, meaning it can also acquire real user traffic data to provide traffic data for testing new services.

[0079] Step 204: Based on the category label, store the user traffic data into a pre-established traffic data list, wherein the traffic data list includes a task table for the category label, and the task table manages the acquired user traffic data.

[0080] The traffic data list can be the traffic pool in the above embodiments, used to store user traffic data, and can be implemented using a list-based data structure.

[0081] In practical applications, core high-frequency traffic, low-frequency traffic, and traffic artificially constructed in new scenarios are all stored in a unified traffic pool. Ultimately, after various products pass through the traffic pool, the output verification traffic remains stable.

[0082] For example, with a preset quantity of 100 entries and a preset time interval of ten minutes, user data traffic is collected every ten minutes. The data is categorized into high-frequency, low-frequency, and new service categories. Every ten minutes, 100 user traffic data entries are extracted from each of these categories. Within a given ten-minute period, the preset frequency thresholds are: data with a frequency of more than 100 entries per hour is defined as high-frequency traffic data; data with a frequency of one entry per hour or less is defined as low-frequency traffic data. If, within a given period, user traffic data corresponding to balance acquisition occurs more than 100 times per hour, and user traffic data corresponding to credit acquisition occurs more than 100 times per hour, then the user traffic data corresponding to balance acquisition and credit acquisition are considered high-frequency traffic data. Because the user traffic data related to balance retrieval and credit limit retrieval is large (e.g., 1000 data points every ten minutes for both), 100 data points can be randomly selected from these two categories. The user traffic data related to repayment date retrieval and configuration item retrieval occurs less than 100 times per hour; these are considered low-frequency traffic data. Conversely, because the data volume for repayment date retrieval and configuration item retrieval is small (e.g., 50 data points every ten minutes for both), retrieval can be delayed until there are at least 100 data points before proceeding with the retrieval. The test staff constructs 50 user traffic data entries corresponding to the new service N and sends these 50 user traffic data entries corresponding to the new service N to the acquisition module, which is to receive the preset number of user traffic data corresponding to the third-level tag.

[0083] If 53 user traffic data entries are obtained corresponding to users obtaining balances, 47 user traffic data entries corresponding to users obtaining credit limits, 71 user traffic data entries corresponding to users obtaining repayment dates, 29 user traffic data entries corresponding to users obtaining configuration items, and 50 user traffic data entries corresponding to new business N, then these 150 data entries are added to the traffic data list.

[0084] The embodiments in this specification store user traffic data into a pre-established traffic data list based on category tags to ensure that the traffic data list contains user traffic data corresponding to all category tags, thereby improving the comprehensiveness of the data.

[0085] In one possible implementation, storing the user traffic data into a pre-established traffic data list according to the category label includes:

[0086] Determine the target task table for the traffic data list corresponding to the category label, and store the user traffic data corresponding to the category label into the target task table.

[0087] The target task table can be a task table corresponding to a category of user traffic data, such as a task table corresponding to users obtaining balances.

[0088] In practical applications, the traffic data list can be a queue list, with a corresponding task table in the queue list for each type of business's user traffic data. Each task table always stores 100 online request records in the queue list. This way, records for the corresponding tasks are retrieved from this queue list every minute to construct a verification request, maintaining stable verification.

[0089] For example, if 53 user traffic data entries correspond to users acquiring balance, 47 user traffic data entries correspond to users acquiring credit limits, 71 user traffic data entries correspond to users acquiring repayment dates, 29 user traffic data entries correspond to users acquiring configuration items, and 50 user traffic data entries correspond to new service N, then these 150 data entries are placed into the traffic data list. Then, a task table corresponding to users acquiring balance is determined, and the 53 user traffic data entries corresponding to users acquiring balance are stored in this task table. Similarly, a task table corresponding to users acquiring credit limits is determined, and the 47 user traffic data entries corresponding to users acquiring credit limits are stored in this task table. A task table corresponding to users acquiring repayment dates is determined, and the 71 user traffic data entries corresponding to users acquiring repayment dates are stored in this task table. A task table corresponding to users acquiring configuration items is determined, and the 29 user traffic data entries corresponding to user acquiring configuration items are stored in this task table. Finally, a task table corresponding to new service N is determined, and the 50 user traffic data entries corresponding to new service N are stored in this task table.

[0090] For example, after obtaining the user traffic data corresponding to the user's balance acquisition, the task table corresponding to the user's balance acquisition is determined. However, since this is the first time obtaining the user traffic data corresponding to the user's balance acquisition, there is no corresponding task table. Therefore, a task table corresponding to the user's balance acquisition is generated first, and then the user traffic data corresponding to the user's balance acquisition is put into the task table corresponding to the user's balance acquisition.

[0091] The embodiments in this specification use a task table to ensure that multiple categories of user traffic data can be stored, and user traffic data can be obtained from this task table, thereby improving data stability.

[0092] In one possible implementation, after storing the user traffic data corresponding to the category labels into the target task table, the method further includes:

[0093] Determine the number of user traffic data entries to be stored in the target task table, and delete the specified number of historical user traffic data entries from the target task table.

[0094] The number of data items can be understood as the number of user traffic data entries, for example, 40 data entries; historical user traffic data can be understood as the user traffic data previously stored in the task table, for example, if there were 100 user traffic data entries in the previous task table, then there are 100 historical user traffic data entries.

[0095] In practical applications, in order to ensure that the user traffic data is more consistent with reality and to ensure the rational use of resources, it is necessary to delete the old traffic data.

[0096] For example, if there are 53 user traffic data entries corresponding to a user obtaining a balance, then the task table corresponding to the user obtaining a balance is determined, and the 53 user traffic data entries corresponding to the user obtaining a balance are stored in the task table corresponding to the user obtaining a balance. At the same time, the 53 old user traffic data entries are deleted from the task table corresponding to the user obtaining a balance.

[0097] The embodiments in this specification improve resource utilization by deleting the same amount of old user traffic data after adding new user traffic data.

[0098] Step 206: Obtain target traffic data from the task table and perform data processing based on the target traffic data.

[0099] The target traffic data can be a portion of the data from the task table, for example, extracting 10 data entries from each task table.

[0100] In practical applications, user data traffic obtained from the task table can be used for traffic simulation, which means calling the production environment and the pre-effective environment in real time and comparing the results returned by the two to verify the correctness of the changes in advance.

[0101] For example, if the task table includes a task table for users to obtain credit limits and a task table for users to obtain repayment dates, then 50 user data entries can be obtained from each of the task tables for users to obtain credit limits and the task tables for users to obtain repayment dates.

[0102] The embodiments in this specification obtain user data traffic from a task table. Since the task table includes user traffic data with multiple levels of tags, the data is highly comprehensive, and the accuracy of the data obtained by processing the target traffic data is high.

[0103] In one possible implementation, the data processing based on the target traffic data includes:

[0104] The target traffic data is identified according to a preset identification rule, and the identified target traffic data is forwarded to the first type of processing device and the second type of processing device according to a preset routing rule.

[0105] Among them, the preset identification rule can be a preset rule for identifying and identifying the target traffic data, for example, determining whether user data traffic is target traffic based on the port number and identifying the target traffic; the preset routing rule can be a rule for forwarding traffic data; the first type of processing device can be understood as a production machine, that is, a user-facing server, and the configuration and policy of the production machine are user-facing configurations and policies; the second type of processing device can be understood as a test machine, and the configuration and policy of the test machine are the configurations and policies to be tested.

[0106] In practical applications, RPC interception is used to intercept traffic when the application is called by other applications as a server or when it calls other applications as a client. When a specified interface, method, and parameters are met, a business tag is added to the context, i.e., coloring. Simultaneously, routing rules are sent to the client, and successfully colored traffic is routed to the specified machine.

[0107] For example, if the preset identification rule is that user traffic data with port number 5555 will be identified as target traffic data, then the target traffic data will be identified. Then, the network address of the first type of processing device and the network address of the second type of processing device corresponding to the target traffic data with port number 5555 can be determined through the routing rules. The target traffic data will be forwarded to the first type of processing device according to the network address of the first type of processing device, and the target traffic data will be forwarded to the second type of processing device according to the network address of the second type of processing device.

[0108] This specification provides a data processing method and apparatus. The data processing method includes: acquiring user traffic data based on level tags, wherein the user traffic data carries category tags and level tags; storing the user traffic data into a pre-established traffic data list based on the category tags, wherein the traffic data list includes a task table for the category tags, and the task table manages the acquired user traffic data; acquiring target traffic data from the task table, and performing data processing based on the target traffic data. Acquiring user traffic data through level tags allows for the acquisition of user data traffic corresponding to each level tag, which is then stored in the traffic data list. This makes the traffic data obtained from the traffic data list more comprehensive. Furthermore, storing the traffic data in the task table by category ensures the stability of the output of each type of user traffic data and facilitates the detection of abnormalities in user traffic data processing.

[0109] The following is in conjunction with the appendix Figure 3 Taking the application of the data processing method provided in this specification on a server as an example, the data processing method will be further explained. Figure 3 A flowchart illustrating the processing procedure of a data processing method according to an embodiment of this specification is shown, specifically including the following steps.

[0110] Step 302: Obtain the preset number of user traffic data from the user traffic data corresponding to the first level label.

[0111] The first-level label can be a high-frequency label. For example, "1" can represent the first-level label. If the user traffic data corresponding to the user's balance acquisition is a high-frequency label, then "1" is marked on the user traffic data corresponding to the user's balance acquisition to indicate that the user traffic data corresponding to the user's balance acquisition is high-frequency traffic data. The frequency of user traffic data can be understood as the number of times user data traffic occurs within a certain period of time. For example, if user traffic data A occurs 100 times in one hour, then the frequency of user traffic data A is 100 times per hour. The preset frequency threshold can be a threshold for determining user traffic data. For example, a frequency of more than 100 times per hour is defined as high-frequency traffic data, and a frequency of one hour or less is defined as low-frequency traffic data.

[0112] In practical applications, user traffic data can be divided into multiple levels based on the frequency of occurrence, and a traffic label can be set for each level. In this way, user data traffic of different frequencies can be collected, and there will be no situation where some user traffic data cannot be collected because the frequency of occurrence of some user traffic data is low.

[0113] For example, the preset frequency thresholds are: more than 100 times per hour is defined as high-frequency traffic data, and a frequency of one hour or less is defined as low-frequency traffic data. If, within a certain period, the user traffic data corresponding to the user's balance acquisition appears more than 100 times per hour, and the user traffic data corresponding to the user's credit acquisition appears more than 100 times per hour, then the user traffic data corresponding to the user's balance acquisition and credit acquisition is considered high-frequency traffic data. Because the user traffic data corresponding to the user's balance acquisition and credit acquisition is large—for example, 1000 user traffic data entries every ten minutes for both—1000 user traffic data entries can be randomly selected from these two categories.

[0114] Step 304: Monitor the user traffic data corresponding to the second level label. If the number of user traffic data corresponding to the second level label meets the preset number, obtain the preset number of user traffic data corresponding to the second level label.

[0115] The second-level label can be a low-frequency label. For example, "2" can represent the second-level label. If the user traffic data corresponding to the user's credit limit is a low-frequency label, then "2" can be added to the user traffic data corresponding to the user's credit limit to indicate that the user traffic data corresponding to the user's credit limit is low-frequency traffic data.

[0116] In practical applications, for low-frequency services, that is, user traffic data corresponding to low-frequency level tags, since online traffic data is scarce, an aggregation method is adopted to store all the traffic within a certain period of time, while also replacing old data with new data.

[0117] For example, the preset frequency thresholds are: more than 100 times per hour is defined as high-frequency traffic data, and a frequency of one hour or less is defined as low-frequency traffic data. If, within a certain period, the frequency of user traffic data related to the repayment date is less than 100 times per hour, and the frequency of user traffic data related to the configuration item is less than 100 times per hour, then the user traffic data related to the repayment date and the configuration item is considered low-frequency traffic data. Since the amount of user traffic data related to the repayment date and the configuration item is small—for example, 50 instances of user traffic data related to the repayment date every ten minutes, and 10 instances of user traffic data related to the configuration item every ten minutes—it can wait until there are at least 100 instances of user traffic data related to the repayment date and the configuration item before retrieving them.

[0118] Step 306: Receive the preset number of user traffic data corresponding to the third-level tags based on the third-level tags.

[0119] The third-level label can be an unpublished business level label, i.e. a new business level label. For example, if a new business N has not been published to customers, the user traffic data corresponding to the new business N can be labeled with a third-level label. The preset business scenario can be a newly established, unpublished business scenario.

[0120] In practical applications, for new businesses and new scenarios, there is no traffic data online, so it is necessary to construct and store it manually. Here, online can be understood as the form in which users send data to the server via the network.

[0121] For example, for business scenarios that have not yet been launched, there will be no real traffic data. Therefore, testers need to manually construct traffic data. For instance, testers can construct 50 user traffic data entries corresponding to the new business N and send these 50 user traffic data entries corresponding to the new business N to the collection module, which is to receive the preset number of user traffic data entries corresponding to the third-level tags.

[0122] Step 308: Determine the target task table for the traffic data list corresponding to the category label, and store the user traffic data corresponding to the category label into the target task table.

[0123] The target task table can be a task table corresponding to a category of user traffic data, such as a task table corresponding to users obtaining balances.

[0124] In practical applications, the traffic data list can be a queue list, with a corresponding task table in the queue list for each type of business's user traffic data. Each task table always stores 100 online request records in the queue list. This way, records for the corresponding tasks are retrieved from this queue list every minute to construct a verification request, maintaining stable verification.

[0125] For example, if 53 user traffic data entries correspond to users acquiring balance, 47 user traffic data entries correspond to users acquiring credit limits, 71 user traffic data entries correspond to users acquiring repayment dates, 29 user traffic data entries correspond to users acquiring configuration items, and 50 user traffic data entries correspond to new service N, then these 150 data entries are placed into the traffic data list. Then, a task table corresponding to users acquiring balance is determined, and the 53 user traffic data entries corresponding to users acquiring balance are stored in this task table. Similarly, a task table corresponding to users acquiring credit limits is determined, and the 47 user traffic data entries corresponding to users acquiring credit limits are stored in this task table. A task table corresponding to users acquiring repayment dates is determined, and the 71 user traffic data entries corresponding to users acquiring repayment dates are stored in this task table. A task table corresponding to users acquiring configuration items is determined, and the 29 user traffic data entries corresponding to user acquiring configuration items are stored in this task table. Finally, a task table corresponding to new service N is determined, and the 50 user traffic data entries corresponding to new service N are stored in this task table.

[0126] Step 310: Determine the number of user traffic data entries stored in the target task table, and delete the specified number of historical user traffic data entries from the target task table.

[0127] The number of data items can be understood as the number of user traffic data entries, for example, 40 data entries; historical user traffic data can be understood as the user traffic data previously stored in the task table, for example, if there were 100 user traffic data entries in the previous task table, then there are 100 historical user traffic data entries.

[0128] In practical applications, in order to ensure that the user traffic data is more consistent with reality and to ensure the rational use of resources, it is necessary to delete the old traffic data.

[0129] For example, if there are 53 user traffic data entries corresponding to a user obtaining a balance, then the task table corresponding to the user obtaining a balance is determined, and the 53 user traffic data entries corresponding to the user obtaining a balance are stored in the task table corresponding to the user obtaining a balance. At the same time, the 53 old user traffic data entries are deleted from the task table corresponding to the user obtaining a balance.

[0130] Step 312: Identify the target traffic data according to the preset identification rules, and forward the identified target traffic data to the first type of processing device and the second type of processing device according to the preset routing rules.

[0131] Among them, the preset identification rule can be a preset rule for identifying and identifying the target traffic data, for example, determining whether user data traffic is target traffic based on the port number and identifying the target traffic; the preset routing rule can be a rule for forwarding traffic data; the first type of processing device can be understood as a production machine, that is, a user-facing server, and the configuration and policy of the production machine are user-facing configurations and policies; the second type of processing device can be understood as a test machine, and the configuration and policy of the test machine are the configurations and policies to be tested.

[0132] In practical applications, RPC interception is used to intercept traffic when the application is called by other applications as a server or when it calls other applications as a client. When a specified interface, method, and parameters are met, a business tag is added to the context, i.e., coloring. Simultaneously, routing rules are sent to the client, and successfully colored traffic is routed to the specified machine.

[0133] For example, if the preset identification rule is that user traffic data with port number 5555 will be identified as target traffic data, then the target traffic data will be identified. Then, the network address of the first type of processing device and the network address of the second type of processing device corresponding to the target traffic data with port number 5555 can be determined through the routing rules. The target traffic data will be forwarded to the first type of processing device according to the network address of the first type of processing device, and the target traffic data will be forwarded to the second type of processing device according to the network address of the second type of processing device.

[0134] By obtaining user traffic data through level tags, the user data traffic corresponding to each level tag can be obtained and stored in the traffic data list, making the traffic data obtained from the traffic data list more comprehensive. At the same time, because the data traffic is stored in the task table by category in the data traffic list, the stability of the output of each type of user traffic data is guaranteed, and it is also easy to detect abnormal situations in user traffic data processing.

[0135] Corresponding to the above method embodiments, this specification also provides data processing apparatus embodiments. Figure 4 A schematic diagram of the structure of a data processing apparatus according to one embodiment of this specification is shown. Figure 4 As shown, the device includes:

[0136] The data acquisition module 402 is configured to acquire user traffic data based on the level label, wherein the user traffic data carries a category label and the level label;

[0137] The data update module 404 is configured to store the user traffic data into a pre-established traffic data list according to the category label, wherein the traffic data list includes a task table for the category label, and the task table manages the acquired user traffic data.

[0138] The data processing module 406 is configured to obtain target traffic data from the task table and perform data processing based on the target traffic data.

[0139] In one possible implementation, the data acquisition module 402 is configured as follows:

[0140] User traffic data is obtained based on level tags at preset time intervals.

[0141] In one possible implementation, the data acquisition module 402 is configured as follows:

[0142] The grade label shall be at least two;

[0143] Accordingly, obtaining user traffic data based on level tags includes:

[0144] A preset number of user traffic data are obtained from the user traffic data corresponding to the at least two level labels.

[0145] In one possible implementation, the data acquisition module 402 is configured as follows:

[0146] The level label includes a first level label, wherein the frequency of user traffic data corresponding to the first level label is greater than a preset frequency threshold.

[0147] Accordingly, obtaining user traffic data based on level tags includes:

[0148] Obtain the preset number of user traffic data from the user traffic data corresponding to the first level label.

[0149] In one possible implementation, the data acquisition module 402 is configured as follows:

[0150] The level label includes a second level label, wherein the frequency of user traffic data corresponding to the second level label is less than or equal to the preset frequency threshold.

[0151] Accordingly, obtaining user traffic data based on level tags includes:

[0152] Monitor the user traffic data corresponding to the second-level label, and if the number of user traffic data corresponding to the second-level label meets the preset number, obtain the preset number of user traffic data corresponding to the second-level label.

[0153] In one possible implementation, the data acquisition module 402 is configured as follows:

[0154] The level label includes a third level label, wherein the third level label is created according to a preset business scenario;

[0155] Accordingly, obtaining user traffic data based on level tags includes:

[0156] Based on the third-level label, receive the preset number of user traffic data corresponding to the third-level label.

[0157] In one possible implementation, the data update module 404 is configured as follows:

[0158] The step of storing the user traffic data into a pre-established traffic data list according to the category label includes:

[0159] Determine the target task table for the traffic data list corresponding to the category label, and store the user traffic data corresponding to the category label into the target task table.

[0160] In one possible implementation, the data update module 404 is configured as follows:

[0161] After storing the user traffic data corresponding to the category tags into the target task table, the method further includes:

[0162] Determine the number of user traffic data entries to be stored in the target task table, and delete the specified number of historical user traffic data entries from the target task table.

[0163] In one possible implementation, the data processing module 406 is configured as follows:

[0164] The target traffic data is identified according to a preset identification rule, and the identified target traffic data is forwarded to the first type of processing device and the second type of processing device according to a preset routing rule.

[0165] This specification provides a data processing method and apparatus, wherein the data processing apparatus includes: a data acquisition module configured to acquire user traffic data based on level tags, wherein the user traffic data carries category tags and the level tags; a data update module configured to store the user traffic data into a pre-established traffic data list based on the category tags, wherein the traffic data list includes a task table for the category tags, and the task table manages the acquired user traffic data; and a data processing module configured to acquire target traffic data from the task table and perform data processing based on the target traffic data. Acquiring user traffic data through level tags allows for the acquisition of user traffic data corresponding to each level tag and its storage in the traffic data list, making the traffic data obtained from the traffic data list more comprehensive. Furthermore, storing the traffic data in the task table by category in the traffic list ensures the stability of the output of each type of user traffic data and facilitates the detection of abnormalities in user traffic data processing.

[0166] The above is an illustrative scheme of a data processing apparatus according to this embodiment. It should be noted that the technical solution of this data processing apparatus and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the data processing apparatus, please refer to the description of the technical solution of the data processing method described above.

[0167] Figure 5 A structural block diagram of a computing device 500 according to one embodiment of this specification is shown. The components of the computing device 500 include, but are not limited to, a memory 510 and a processor 520. The processor 520 is connected to the memory 510 via a bus 530, and a database 540 is used to store data.

[0168] The computing device 500 also includes an access device 540, which enables the computing device 500 to communicate via one or more networks 560. Examples of these networks include a Public Switched Telephone Network (PSTN), a Local Area Network (LAN), a Wide Area Network (WAN), a Personal Area Network (PAN), or a combination of communication networks such as the Internet. The access device 550 may include one or more of any type of wired or wireless network interface (e.g., a Network Interface Card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) interface, a Wi-MAX interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0169] In one embodiment of this specification, the above-described components of the computing device 500 and Figure 5 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 5 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0170] The computing device 500 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or PCs. The computing device 500 can also be a mobile or stationary server.

[0171] The processor 520 is configured to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the above-described data processing method.

[0172] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the data processing method described above.

[0173] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the above-described data processing method.

[0174] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the data processing method described above.

[0175] An embodiment of this specification also provides a computer program, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the above-described data processing method.

[0176] The above is an illustrative example of a computer program according to this embodiment. It should be noted that the technical solution of this computer program and the technical solution of the data processing method described above belong to the same concept. Details not described in detail in the technical solution of the computer program can be found in the description of the technical solution of the data processing method described above.

[0177] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0178] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added to or subtracted according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0179] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0180] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0181] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A data processing method, comprising: User traffic data is obtained according to the level label at a preset time interval. The user traffic data carries a category label and the level label. There are at least two level labels. The step of obtaining user traffic data according to the level label includes obtaining a preset number of user traffic data from the user traffic data corresponding to the at least two level labels respectively. Determine the target task table for the traffic data list corresponding to the category label, and store the user traffic data corresponding to the category label into the target task table. The traffic data list includes a task table for the category label, the task table manages the acquired user traffic data, and the target task table is the task table corresponding to the user traffic data determined according to the business type. Determine the number of user traffic data entries to be stored in the target task table, and delete the specified number of historical user traffic data entries from the target task table. Obtain target traffic data from the task table and perform data processing based on the target traffic data.

2. The method according to claim 1, wherein the level label includes a first level label, wherein, The frequency of user traffic data corresponding to the first-level label is greater than a preset frequency threshold. Accordingly, obtaining user traffic data based on level tags includes: Obtain the preset number of user traffic data from the user traffic data corresponding to the first level label.

3. The method according to claim 2, wherein the level label includes a second level label, wherein, The frequency of user traffic data corresponding to the second-level label is less than or equal to the preset frequency threshold. Accordingly, obtaining user traffic data based on level tags includes: Monitor the user traffic data corresponding to the second-level label, and if the number of user traffic data corresponding to the second-level label meets the preset number, obtain the preset number of user traffic data corresponding to the second-level label.

4. The method according to claim 3, wherein the level label includes a third level label, wherein, The third-level label is created based on a preset business scenario; Accordingly, obtaining user traffic data based on level tags includes: Based on the third-level label, receive the preset number of user traffic data corresponding to the third-level label.

5. The method according to claim 1, wherein the data processing based on the target traffic data includes: The target traffic data is identified according to a preset identification rule, and the identified target traffic data is forwarded to the first type of processing device and the second type of processing device according to a preset routing rule.

6. A data processing apparatus, comprising: The data acquisition module is configured to acquire user traffic data based on grade tags at preset time intervals, wherein the user traffic data carries category tags and grade tags, and there are at least two grade tags. The acquisition of user traffic data based on grade tags includes acquiring a preset number of user traffic data from the user traffic data corresponding to the at least two grade tags respectively. The data update module is configured to determine the target task table of the traffic data list corresponding to the category label, store the user traffic data corresponding to the category label into the target task table, wherein the traffic data list includes a task table for the category label, the task table manages the acquired user traffic data, the target task table is a task table corresponding to the user traffic data determined according to the business type, determine the number of user traffic data to be stored in the target task table, and delete the historical user traffic data of the specified number of data in the target task table; The data processing module is configured to obtain target traffic data from the task table and perform data processing based on the target traffic data.

7. A computing device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the data processing method according to any one of claims 1 to 5.

8. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the data processing method according to any one of claims 1 to 5.

Images