Operational data processing methods, devices, computer equipment and storage media

Abstract

This disclosure relates to the field of data processing technology, specifically disclosing an operational data processing method, apparatus, computer equipment, storage medium, and computer program product. It acquires a scheduled task for processing operational data, and obtains a scheduled task matching the current time based on the task's start time as a task to be executed. Then, based on the task type of at least one subtask of the task to be executed, it executes each subtask of the task to obtain the target data after task execution. Since the scheduled task can perform marking and statistical processing on the operational data, it eliminates the need to add object marking code at key transformation points, thus decoupling the marking and statistical code and accelerating response speed. Furthermore, since the scheduled task can flexibly configure its execution time, it can be configured to execute during idle times, not only avoiding the risk of task failure but also further improving response speed, thereby increasing data processing efficiency.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to an operational data processing method, apparatus, computer equipment, storage medium, and computer program product. Background Technology

[0002] With the development of internet technology, operational data has been widely applied. Typically, operational data processing includes data aggregation and log-based data processing. For example, aggregation involves obtaining statistical data such as the number of enterprises and institutions, physical canteens, and customers. Log-based data processing involves recording user behavior and user status through event tracking interfaces.

[0003] In traditional technologies, data owners can analyze and mine operational data to publish or use important information hidden in massive amounts of data in a compliant manner. Currently, common methods for processing operational data generally involve adding object tagging code to key conversion points such as products and services, formulating tagging rules based on business needs, marking objects that meet the tagging rules, and performing real-time data statistics and aggregation processing when querying data in the backend.

[0004] However, if the tagging rules are complex and the database stores a large amount of object data, it increases the complexity of the business interface, the execution time of the business interface, and the maintenance cost, resulting in lower efficiency in processing operational data. Summary of the Invention

[0005] Therefore, it is necessary to provide an operational data processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product that can improve the efficiency of operational data processing, addressing the aforementioned technical problem of low operational data processing efficiency.

[0006] Firstly, this application provides an operational data processing method. The method includes:

[0007] Obtain a scheduled task to process operational data. The scheduled task includes a task start time, at least one subtask, and a task type for the at least one subtask. The task type includes a marking task type that marks the operational data or a statistical task type that performs statistics on the operational data.

[0008] Based on the task start time, obtain the timed task that matches the current time as the task to be executed;

[0009] Based on the task type of at least one subtask of the task to be executed, each subtask of the task to be executed is executed to perform labeling or statistical processing on the operational data to obtain the target data after task execution.

[0010] In one embodiment, the scheduled task further includes an execution order of at least one subtask; the step of executing each subtask of the task to be executed according to the task type of the at least one subtask of the task to be executed, in order to perform marking or statistical processing on the operational data to obtain the target data after task execution, includes: sorting the at least one subtask according to the execution order of the at least one subtask to obtain a sorted task queue; and sequentially executing the corresponding subtask on the operational data according to the task type of the at least one subtask in the task queue, in order to perform marking or statistical processing on the operational data, until the last subtask in the task queue is executed to obtain the target data after task execution.

[0011] In one embodiment, the task start time includes at least a first start time and a second start time of the scheduled task, wherein the second start time is later than the first start time; the method further includes: when the first start time arrives, determining the scheduled task as a first task to be executed, and executing each subtask of the first task to be executed on the operational data using a first thread according to the task type of at least one subtask of the first task to be executed; when the second start time arrives and the first task to be executed has not been completed, determining the scheduled task as a second task to be executed, and executing each subtask of the second task to be executed on the operational data using a second thread according to the task type of at least one subtask of the second task to be executed.

[0012] In one embodiment, after obtaining the timed task matching the current time based on the task start time as the task to be executed, the method further includes: when there are multiple tasks to be executed, executing multiple tasks to be executed asynchronously based on multiple threads.

[0013] In one embodiment, before obtaining the scheduled task for processing operational data, the method further includes: obtaining task configuration information, the task configuration information including a configured task start time and the execution order and processing logic of at least one subtask; generating a task type for the corresponding subtask according to the processing logic; and generating a scheduled task according to the task start time, the execution order of at least one subtask, the task type, and the processing logic.

[0014] In one embodiment, generating the task type of the corresponding subtask according to the processing logic includes: when the processing logic is a marking rule for marking the operational data, the task type of the corresponding subtask is a marking task type; or, when the processing logic is a statistical rule for statistical processing the operational data, the task type of the corresponding subtask is a statistical task type.

[0015] In one embodiment, the step of executing each subtask of the task to be executed according to the task type of at least one subtask of the task to be executed includes: when the task type of the subtask is a marking task type, marking the operational data according to the corresponding marking rules to obtain the marked operational data; when the task type of the subtask is a statistics task type, extracting corresponding data from the operational data according to the corresponding statistics rules and performing data cleaning to obtain the corresponding statistical data.

[0016] In one embodiment, the statistical rules include at least one statistical dimension; the step of extracting corresponding data from the operational data according to the corresponding statistical rules and performing data cleaning to obtain corresponding statistical data includes: extracting data of the corresponding statistical dimension from the operational data according to at least one statistical dimension and performing data cleaning to obtain statistical data of at least one statistical dimension.

[0017] In one embodiment, after obtaining statistical data for at least one statistical dimension, the method further includes: generating and displaying a visualization chart based on the statistical data for the at least one statistical dimension.

[0018] Secondly, this application also provides an operational data processing apparatus. The apparatus includes:

[0019] The scheduled task acquisition module is used to acquire scheduled tasks that process operational data. The scheduled task includes a task start time, at least one subtask, and a task type for the at least one subtask. The task type includes a marking task type that marks the operational data or a statistical task type that performs statistics on the operational data.

[0020] The task to be executed determination module is used to obtain a timed task that matches the current time as the task to be executed based on the task start time.

[0021] The task execution module is used to execute each subtask of the task to be executed according to the task type of at least one subtask of the task to be executed, so as to perform labeling or statistical processing on the operational data and obtain the target data after task execution.

[0022] Thirdly, this application also provides a computer device. The computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the method described in the first aspect above.

[0023] Fourthly, this application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program thereon, which, when executed by a processor, implements the steps of the method described in the first aspect above.

[0024] Fifthly, this application also provides a computer program product. The computer program product includes a computer program that, when executed by a processor, implements the steps of the method described in the first aspect above.

[0025] The aforementioned operational data processing method, apparatus, computer equipment, storage medium, and computer program product acquire a scheduled task for processing operational data, and obtains a scheduled task matching the current time as a task to be executed based on the task start time. Then, based on the task type of at least one subtask of the task to be executed, each subtask of the task to be executed is executed to achieve marking or statistical processing of the operational data, finally obtaining the target data after task execution. This embodiment achieves marking and statistical processing of operational data through scheduled tasks, eliminating the need to add object marking code at key transformation points, thus decoupling the marking and statistical code and accelerating the response speed of operational data processing. Furthermore, since scheduled tasks can be flexibly configured for execution time, they can be configured to execute during idle periods, not only avoiding the risk of task execution failure but also further improving the response speed of operational data processing, thereby improving the overall efficiency of operational data processing. Attached Figure Description

[0026] Figure 1 This is an application environment diagram of an operational data processing method in one embodiment;

[0027] Figure 2 This is a flowchart illustrating an operational data processing method in one embodiment;

[0028] Figure 3 This is a flowchart illustrating the task execution process in one embodiment;

[0029] Figure 4 This is a flowchart illustrating the task execution process in another embodiment;

[0030] Figure 5 This is a flowchart illustrating the operational data processing method in another embodiment;

[0031] Figure 6 This is a structural block diagram of an operational data processing device in one embodiment;

[0032] Figure 7 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0033] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0034] The acquisition, storage, use, and processing of data in this application all comply with the relevant provisions of national laws and regulations.

[0035] The operational data processing method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, the operation server 102 connects to various channel servers 104 (e.g., via a network) Figure 1 The terminals 106 (e.g., the first channel server and the second channel server in the system) communicate with each other and interact with different channel servers 104. Figure 1 The first and second terminals in the system can also access the operation server 102 via the corresponding channel server 104. Specifically, the operation server 102 can be implemented using a standalone server or a server cluster consisting of multiple servers. The terminal 106 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices.

[0036] In one implementation, the operation server 102 may be a server providing a first service, such as a shopping service, a payment service, or a live streaming service. The channel server 104 may be a server providing a second service and authorized to access the first service. For example, the second service may be an instant messaging service or a payment service. It is understood that in this embodiment, the first service provided by the operation server 102 is different from the second service provided by the channel server 104; that is, they are not servers providing the same service. For example, if the operation server 102 is a shopping server, then the channel server 104 may be an instant messaging server. If the operation server 102 is a first payment server, then the channel server 104 may be a second payment server, providing a different payment service than the first payment server.

[0037] In a scenario, such as Figure 1As shown, taking the operation server 102 as the shopping server and the channel server 104 as the instant messaging server as an example, when user A obtains the first instant messaging service through the first terminal, they can also access the shopping server (i.e., the operation server) based on the corresponding first instant messaging server (i.e., the first channel server) to obtain the services provided by the shopping server; similarly, when user B obtains the second instant messaging service through the second terminal, they can also access the shopping server (i.e., the operation server) based on the corresponding second instant messaging server (i.e., the second channel server) to obtain the services provided by the shopping server. It should be noted that the first instant messaging service and the second instant messaging service can be different instant messaging services; for example, the first instant messaging service can be WeChat, and the second instant messaging service can be QQ.

[0038] The data generated during the interaction between various channel servers and terminals and the shopping server constitutes operational data. Specifically, operational data includes, but is not limited to, user access data to the shopping server, corresponding channel data, registration data, and specific shopping-related data. In practical applications, with data authorization, operators can further analyze, mine, and statistically analyze operational data, thereby releasing or using important information hidden within the data in a compliant manner. For example, operational data can be used to calculate the conversion rate of a particular product relative to various channels, or to calculate the total conversion rate of a particular product; it can also be used to calculate user conversion rates, user retention rates, and the number of active users across various channels, and then operational strategies can be adjusted based on the mined information.

[0039] However, traditional methods for processing operational data involve adding object tagging code to key conversion points such as products and services, defining tagging rules based on business needs, and marking objects that conform to these rules. Backend data queries then perform real-time statistical aggregation. However, if the tagging rules are complex or the amount of stored object data is large, it increases the complexity of business interfaces, their execution time, and maintenance costs, resulting in low efficiency in processing operational data.

[0040] Based on this, such as Figure 2 As shown, this disclosure provides an operational data processing method, which is applied to... Figure 1 Taking the operating server in the example, the specific steps are 202 to 206.

[0041] Step 202: Obtain the scheduled task for processing operational data.

[0042] Operational data can be data generated by the operations server during its interaction with external systems. This includes, but is not limited to, query data, registration data, and transaction data. Scheduled tasks can be tasks that are triggered at regular intervals to achieve a specific task objective. Specifically, scheduled tasks can be implemented using Quartz (a job scheduling framework).

[0043] In one scenario, a scheduled task can include a task start time, at least one subtask, and the task type of the at least one subtask. The task start time refers to the set start time of task execution. A subtask is a specific task that needs to be executed within the job scheduling framework to achieve the corresponding task objective. It's understandable that when the task objective of a scheduled task is relatively simple, if it can be achieved through a single subtask, then the scheduled task can include only one subtask. However, when the process of achieving the task objective is more complex, the task objective can be decomposed into multiple decomposed objectives. The task that can achieve one of these decomposed objectives can be a subtask of the scheduled task; therefore, a scheduled task can also include multiple subtasks.

[0044] Task types can be processing types that involve specific manipulation of operational data. Specifically, task types can include labeling task types (marking operational data) or statistical task types (summarizing operational data). For example, when a subtask in a scheduled task is used to label operational data, the corresponding task type is a labeling task type; when a subtask in a scheduled task is used to summarize operational data, the corresponding task type is a statistical task type. It is understood that a scheduled task can include only subtasks of the labeling task type or subtasks of the statistical task type, or it can include subtasks of both types.

[0045] In this embodiment, when the operation server needs to process operation data, it can first obtain a scheduled task for processing the operation data, and then process it through subsequent steps.

[0046] Step 204: Obtain the scheduled task that matches the current time based on the task start time as the task to be executed.

[0047] Here, "current time" refers to any moment when the operations server checks scheduled tasks for processing. Typically, to reduce the processing load on the operations server, it can be configured to periodically check scheduled tasks to identify tasks awaiting execution. These tasks can be tasks that are currently scheduled to be executed. Scheduled tasks matching the current time are those whose start times match the current time. For example, the operations server can be configured to check scheduled tasks every hour to identify those whose start times fall within that hour and designate them as tasks awaiting execution.

[0048] Step 206: Execute each subtask of the task to be executed according to the task type of at least one subtask of the task to be executed.

[0049] In this embodiment, the operations server can process the operations data according to the tasks to be executed as determined above. Specifically, the operations server can execute each subtask of the task to be executed according to the task type of at least one subtask of the task to be executed, so as to perform marking or statistical processing on the operations data and obtain the target data after task execution.

[0050] For example, when there is only one subtask in the task to be executed, the operational data can be labeled or statistically processed according to the task type of that subtask to obtain the target data after task execution. When there are multiple subtasks in the task to be executed, each subtask can be executed separately according to its task type to label or statistically process the operational data, thereby obtaining the target data after each subtask is executed.

[0051] In the above-described operational data processing method, the operational server obtains scheduled tasks for processing operational data, and selects a scheduled task matching the current time as a task to be executed based on the task start time. Then, according to the task type of at least one subtask of the task to be executed, it executes each subtask to perform marking or statistical processing on the operational data, ultimately obtaining the target data after task execution. This embodiment achieves marking and statistical processing of operational data through scheduled tasks, eliminating the need to add object marking code at key transformation points. This decouples the marking and statistical code, thereby accelerating the response speed of operational data processing. Furthermore, since scheduled tasks can be flexibly configured for execution time, they can be configured to execute during idle periods, not only avoiding the risk of task execution failure but also further improving the response speed of operational data processing, thus improving the overall efficiency of operational data processing.

[0052] In one embodiment, the scheduled task may further include the execution order of at least one subtask. For example... Figure 3As shown, the above-mentioned execution of each subtask of the task to be executed, based on the task type of at least one subtask of the task to be executed, to perform labeling or statistical processing on the operational data, and to obtain the target data after task execution, may further include:

[0053] Step 302: Sort at least one subtask according to the execution order of at least one subtask to obtain a sorted task queue.

[0054] The execution order refers to the execution priority of each subtask under the same scheduled task. Specifically, when there are multiple subtasks under a scheduled task, each subtask has a corresponding execution priority, i.e., execution order, based on the logical relationship between them. Therefore, the operation server can sort the subtasks according to their execution priorities. For example, the subtasks can be sorted from high to low execution priority to obtain a task queue composed of the sorted subtasks.

[0055] Step 304: Based on the task type of at least one subtask in the task queue, execute the corresponding subtasks for the operational data in sequence.

[0056] In this embodiment, the operations server can sequentially execute corresponding subtasks according to the task queue. Specifically, the operations server sequentially obtains the corresponding subtasks according to the task queue, and performs corresponding marking or statistical processing on the operations data according to the task type of the subtask, until the last subtask in the task queue is executed, thereby obtaining the target data after task execution.

[0057] In the above embodiments, the operation server can sort at least one subtask according to its execution order to obtain a sorted task queue. Then, based on the task type of at least one subtask in the task queue, the operation server sequentially executes the corresponding subtasks on the operation data until the last subtask in the task queue is executed. Since the operation server can execute each subtask sequentially according to its execution order, it can follow the task execution logic, ensure the correctness of task execution, and thus improve the accuracy of the target data after task execution.

[0058] In one embodiment, the task start time includes at least a first start time and a second start time of the timed task, wherein the second start time is later than the first start time. For example... Figure 4 As shown, the above method may further include:

[0059] Step 402: When the first start time arrives, determine the timed task as the first task to be executed, and use the first thread to execute the task.

[0060] Specifically, in the process of processing operational data, some processing methods may be executed repeatedly. For example, counting the number of registrations made by accessing the operational server through different channels each day would be executed once a day. Similarly, counting the number of user visits to the operational server through different channels each hour would be executed once an hour. Therefore, the start time of scheduled tasks can be implemented based on cron (timer) expressions, meaning that the start time of each scheduled task execution can be specified through cron expressions.

[0061] In one scenario, let's take a scheduled task as an example to count the number of users accessing the operations server through different channels every hour. If the cron expression specifies that statistics are collected every hour, such as counting at XX:00:00, then the start time corresponding to this scheduled task includes every hour. For ease of explanation, this embodiment uses two hourly times as examples, defining them as the first start time and the second start time, respectively, with the second start time being later than the first start time. For example, if the first start time is 9:00, then the second start time is 10:00; if the first start time is 12:00, then the second start time is 13:00.

[0062] Specifically, when the first start time arrives, the scheduled task is triggered. Therefore, this scheduled task can be identified as a task to be executed, and the first thread is used to execute it. Since this scheduled task is triggered every hour on the hour, to distinguish the same task triggered at different hours, this embodiment identifies the scheduled task triggered at the first start time as the first task to be executed, and the first thread is used to execute this first task to be executed. That is, based on the task type of at least one subtask of the first task to be executed, the first thread executes each subtask of the first task to be executed on the operational data.

[0063] Step 404: When the second start time arrives and the first task to be executed has not been completed, the second thread is used to execute the second task to be executed.

[0064] Specifically, when the second start time arrives and the first task to be executed has not yet been completed, the corresponding scheduled task can be determined as the second task to be executed, and the second thread is used to execute the second task to be executed. That is, based on the task type of at least one subtask of the second task to be executed, the second thread is used to execute each subtask of the second task to be executed on the operational data.

[0065] In the above embodiments, when the first start time arrives, the operation server can determine the scheduled task as the first task to be executed and use the first thread to execute the task. When the second start time arrives and the first task to be executed has not been completed, the operation server can also use the second thread to execute the second task to be executed. This allows the same scheduled task to start a new schedule with a new thread when the previous schedule is not completed and the next schedule time arrives, ensuring that multiple scheduled tasks do not interfere with each other and further improving task processing efficiency.

[0066] In one embodiment, in step 204 above, when there are multiple scheduled tasks matching the current time obtained from the task start time, the operations server can process the operations data by treating these multiple scheduled tasks as tasks to be executed. Specifically, when executing these multiple tasks to be executed, the operations server can execute these multiple tasks asynchronously based on multiple threads.

[0067] For example, if the operations server determines that the tasks to be executed at the start time of a certain task include: task A, task B, and task C, that is, tasks A, B, and C start at the same time, then the operations server can execute these multiple tasks asynchronously using thread one to execute task A, thread two to execute task B, and thread three to execute task C.

[0068] In the above embodiments, when there are multiple tasks to be executed, the operation server can asynchronously execute multiple tasks based on multiple threads, so that the multiple tasks do not affect each other, thereby further improving task processing efficiency.

[0069] In one embodiment, such as Figure 5 As shown, before obtaining the scheduled task for processing operational data, the above method may further include the following steps:

[0070] Step 502: Obtain task configuration information.

[0071] The task configuration information can include information related to task configuration, including but not limited to the configured task start time and the execution order and processing logic of at least one subtask. Specifically, the processing logic refers to the processing flow or content of the corresponding subtask. The execution order refers to the execution priority of the corresponding subtask.

[0072] Specifically, in this embodiment, when there is a need to process operational data, corresponding scheduled tasks can be configured through the operations server. The operations server can then obtain the relevant task configuration information for the scheduled tasks.

[0073] Step 504: Generate the task type of the corresponding subtask according to the processing logic.

[0074] Specifically, when the processing logic involves marking rules for operational data, the operations server can generate a corresponding subtask with the task type of marking. Conversely, when the processing logic involves statistical rules for operational data, the operations server can generate a corresponding subtask with the task type of statistical. This allows the operations server to automatically generate the corresponding subtask task type based on the processing logic, thereby improving the configuration efficiency of scheduled tasks.

[0075] The tagging rules can be rules set based on the corresponding task processing requirements for tagging data. For example, when the task processing requirement is to calculate the conversion rate of users accessing the server through the first channel on a certain day, the configuration information of the scheduled task based on this task requirement will at least include tagging rules for tagging user access channels. Specifically, this tagging rule can be to tag users who access the operations server through the first channel on a certain day; that is, any user who accesses the operations server through the first channel on that day will be tagged. Therefore, based on this tagging rule, the task type of the corresponding subtask can be generated as the tagging task type.

[0076] Statistical rules are rules set based on the corresponding task processing requirements to perform statistical analysis on the data. Taking the above task processing requirement as an example, since the user conversion rate is the percentage between the number of users accessing the operations server through the first channel server on that day and the number of users who accessed and registered through the first channel server on that day, the configuration information of the scheduled task based on this task requirement must include at least the rules for statistically analyzing the number of users accessing the operations server through the first channel server on that day, and the rules for statistically analyzing the number of users who accessed and registered through the first channel server on that day. Therefore, based on the statistical rules, the task type of the corresponding sub-task can be generated as a statistical task type.

[0077] Step 506: Generate a scheduled task based on the task start time, the execution order of at least one subtask, the task type, and the processing logic.

[0078] In this embodiment, the operation server can construct a job task scheduling framework based on the configured task start time, the execution order of at least one subtask, the task type, and the processing logic, that is, generate a scheduled task.

[0079] In the above embodiments, the operation server obtains task configuration information and generates corresponding sub-task task types according to processing logic. Then, it generates scheduled tasks based on the task start time, the execution order of at least one sub-task, the task type, and the processing logic. This enables flexible configuration of scheduled tasks. For example, the task start time of a scheduled task can be configured to be during network off-peak hours, thereby improving the response speed of task execution and thus improving data processing efficiency.

[0080] In one embodiment, the execution of each subtask of the task to be executed, based on the task type of at least one subtask of the task to be executed, may specifically include: when the operation server determines that the task type of the subtask is a marking task type, it can mark the operation data according to the corresponding marking rules to obtain the marked operation data. When the operation server determines that the task type of the subtask is a statistical task type, it can extract the corresponding data from the operation data according to the corresponding statistical rules and perform data cleaning to obtain the corresponding statistical data. Data cleaning includes, but is not limited to, removing null values, abnormal data, duplicate values, and processing related to union and intersection.

[0081] In this embodiment, when the operations server executes each subtask of the task to be executed, if it determines that the task type of the current subtask is a marking task type, it can mark the operations data according to the marking rules corresponding to the current subtask, thereby obtaining the marked operations data. When the operations server executes each subtask of the task to be executed, if it determines that the task type of the current subtask is a statistical task type, it can statistically process the operations data according to the statistical rules corresponding to the current subtask, thereby obtaining the statistical data. This process continues until the last subtask of the task to be executed is completed, resulting in the target data after task execution. Because the operations server can process the operations data according to its corresponding task type when executing each subtask of the task to be executed, it helps to improve the task processing efficiency.

[0082] In one embodiment, the statistical rules may include at least one statistical dimension. Then, according to the corresponding statistical rules, corresponding data is extracted from the operational data and cleaned to obtain the corresponding statistical data. Specifically, this may include: extracting data corresponding to at least one statistical dimension from the operational data based on at least one statistical dimension and cleaning the data to obtain statistical data for at least one statistical dimension.

[0083] The statistical dimension can be a time dimension based on different times or a spatial dimension based on different spaces. For example, taking the time dimension, statistics can be compiled on data for one hour, one day, one week, or one month. Taking the spatial dimension, statistics can be compiled only on data related to a specific channel, or total data from all channels can be compiled. It is understood that the above examples are only used to illustrate the principles of this disclosure and are not intended to limit the scope of protection of this disclosure. In practical applications, any dimension can be set according to actual needs.

[0084] Specifically, in this embodiment, during the execution of a corresponding task, the operation server can extract data of the corresponding statistical dimension from the operation data according to at least one statistical dimension in the statistical rules of the task and perform data cleaning, thereby obtaining statistical data of at least one statistical dimension. This hierarchical and dimensional statistical approach improves the efficiency of subsequent queries on the operation data. Furthermore, since this disclosure performs statistical processing on the operation data through scheduled tasks, it reduces the processing pressure on the operation server compared to traditional real-time statistics.

[0085] In one embodiment, after obtaining statistical data for at least one statistical dimension, the method may further include: generating and displaying a visualization chart based on the statistical data for at least one statistical dimension. The visualization chart includes, but is not limited to, data reports and statistical charts displayed using bar charts, line charts, scatter plots, bubble charts, etc. In this embodiment, the operations server can generate and display a corresponding visualization chart based on the statistical data for at least one statistical dimension, thereby improving the visualization effect of operational data. This provides dynamic information support for the operations team and offers a basis and direction for further service quality improvement.

[0086] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0087] Based on the same inventive concept, this application also provides an operational data processing apparatus for implementing the operational data processing method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more operational data processing apparatus embodiments provided below can be found in the limitations of the operational data processing method described above, and will not be repeated here.

[0088] In one embodiment, such as Figure 6 As shown, an operational data processing device is provided, including: a scheduled task acquisition module 602, a task to be executed determination module 604, and a task execution module 608, wherein:

[0089] The scheduled task acquisition module 602 is used to acquire scheduled tasks that process operational data. The scheduled task includes a task start time, at least one subtask, and a task type for the at least one subtask. The task type includes a marking task type that marks the operational data or a statistical task type that performs statistics on the operational data.

[0090] The task to be executed module 604 is used to obtain a timed task that matches the current time as the task to be executed based on the task start time.

[0091] The task execution module 608 is used to execute each subtask of the task to be executed according to the task type of at least one subtask of the task to be executed, so as to perform marking or statistical processing on the operational data and obtain the target data after task execution.

[0092] In one embodiment, the scheduled task further includes the execution order of at least one subtask; the task execution module is further configured to: sort the at least one subtask according to the execution order of the at least one subtask to obtain a sorted task queue; and execute the corresponding subtasks on the operational data sequentially according to the task type of at least one subtask in the task queue to perform marking or statistical processing on the operational data until the last subtask in the task queue is executed to obtain the target data after task execution.

[0093] In one embodiment, the task start time includes at least a first start time and a second start time of the scheduled task, wherein the second start time is later than the first start time; the task execution module is further configured to: when the first start time arrives, determine the scheduled task as a first task to be executed, and execute each subtask of the first task to be executed on the operational data using a first thread according to the task type of at least one subtask of the first task to be executed; when the second start time arrives and the first task to be executed has not been completed, determine the scheduled task as a second task to be executed, and execute each subtask of the second task to be executed on the operational data using a second thread according to the task type of at least one subtask of the second task to be executed.

[0094] In one embodiment, the task execution module is further configured to: when there are multiple tasks to be executed, execute multiple tasks to be executed asynchronously based on multiple threads.

[0095] In one embodiment, the scheduled task acquisition module is specifically used to: acquire task configuration information, the task configuration information including the configured task start time and the execution order and processing logic of at least one subtask; generate the task type of the corresponding subtask according to the processing logic; and generate a scheduled task according to the task start time, the execution order of at least one subtask, the task type, and the processing logic.

[0096] In one embodiment, the scheduled task acquisition module is further configured to: when the processing logic is a marking rule for marking the operational data, generate a corresponding sub-task with the task type of marking task; or, when the processing logic is a statistical rule for statistical processing the operational data, generate a corresponding sub-task with the task type of statistical task.

[0097] In one embodiment, the task execution module is further configured to: when the task type of the subtask is a marking task type, mark the operational data according to the corresponding marking rules to obtain the marked operational data; when the task type of the subtask is a statistics task type, extract the corresponding data from the operational data according to the corresponding statistics rules and perform data cleaning to obtain the corresponding statistical data.

[0098] In one embodiment, the statistical rule includes at least one statistical dimension; the task execution module is further configured to: extract data corresponding to the statistical dimension from the operational data according to the at least one statistical dimension and perform data cleaning to obtain statistical data of at least one statistical dimension.

[0099] In one embodiment, the apparatus further includes a display module for generating and displaying visual charts based on statistical data from the at least one statistical dimension.

[0100] Each module in the aforementioned operational data processing device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the operations corresponding to each module.

[0101] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 7 As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The database stores operational data. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements an operational data processing method.

[0102] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0103] In one embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0104] Obtain a scheduled task to process operational data. The scheduled task includes a task start time, at least one subtask, and a task type for the at least one subtask. The task type includes a marking task type that marks the operational data or a statistical task type that performs statistics on the operational data.

[0105] Based on the task start time, obtain the timed task that matches the current time as the task to be executed;

[0106] Based on the task type of at least one subtask of the task to be executed, each subtask of the task to be executed is executed to perform labeling or statistical processing on the operational data to obtain the target data after task execution.

[0107] In one embodiment, the timed task further includes the execution order of at least one subtask; when the processor executes the computer program, it also performs the following steps: sorting the at least one subtask according to the execution order of the at least one subtask to obtain a sorted task queue; according to the task type of the at least one subtask in the task queue, sequentially executing the corresponding subtask on the operational data to perform marking or statistical processing on the operational data, until the last subtask in the task queue is executed to obtain the target data after task execution.

[0108] In one embodiment, the task start time includes at least a first start time and a second start time of the scheduled task, wherein the second start time is later than the first start time; when the processor executes the computer program, it further implements the following steps: when the first start time arrives, the scheduled task is determined to be a first task to be executed, and according to the task type of at least one subtask of the first task to be executed, the operational data is executed using a first thread for each subtask of the first task to be executed; when the second start time arrives and the first task to be executed has not been completed, the scheduled task is determined to be a second task to be executed, and according to the task type of at least one subtask of the second task to be executed, the operational data is executed using a second thread for each subtask of the second task to be executed.

[0109] In one embodiment, when the processor executes a computer program, it further performs the following steps: when there are multiple tasks to be executed, it asynchronously executes multiple tasks to be executed based on multiple threads.

[0110] In one embodiment, when the processor executes the computer program, it further performs the following steps: obtaining task configuration information, the task configuration information including a configured task start time and the execution order and processing logic of at least one subtask; generating a task type for the corresponding subtask according to the processing logic; and generating a timed task according to the task start time, the execution order of at least one subtask, the task type, and the processing logic.

[0111] In one embodiment, when the processor executes the computer program, it further implements the following steps: when the processing logic is a marking rule for marking the operational data, the task type of the corresponding subtask is a marking task type; or, when the processing logic is a statistical rule for statistical processing the operational data, the task type of the corresponding subtask is a statistical task type.

[0112] In one embodiment, when the processor executes the computer program, it further implements the following steps: when the task type of the subtask is a marking task type, the operational data is marked according to the corresponding marking rules to obtain the marked operational data; when the task type of the subtask is a statistics task type, corresponding data is extracted from the operational data according to the corresponding statistics rules and the data is cleaned to obtain the corresponding statistical data.

[0113] In one embodiment, the statistical rule includes at least one statistical dimension; when the processor executes the computer program, it further implements the following steps: extracting data corresponding to the statistical dimension from the operational data according to the at least one statistical dimension and performing data cleaning to obtain statistical data of at least one statistical dimension.

[0114] In one embodiment, after obtaining statistical data for at least one statistical dimension, the processor, when executing a computer program, further performs the following steps: generating and displaying a visualization chart based on the statistical data for the at least one statistical dimension.

[0115] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, the computer program performing the following steps when executed by a processor:

[0116] Obtain a scheduled task to process operational data. The scheduled task includes a task start time, at least one subtask, and a task type for the at least one subtask. The task type includes a marking task type that marks the operational data or a statistical task type that performs statistics on the operational data.

[0117] Based on the task start time, obtain the timed task that matches the current time as the task to be executed;

[0118] Based on the task type of at least one subtask of the task to be executed, each subtask of the task to be executed is executed to perform labeling or statistical processing on the operational data to obtain the target data after task execution.

[0119] In one embodiment, the scheduled task further includes the execution order of at least one subtask; when the computer program is executed by the processor, it further implements the following steps: sorting the at least one subtask according to the execution order of the at least one subtask to obtain a sorted task queue; according to the task type of at least one subtask in the task queue, sequentially executing the corresponding subtask on the operational data to perform marking or statistical processing on the operational data, until the last subtask in the task queue is executed to obtain the target data after task execution.

[0120] In one embodiment, the task start time includes at least a first start time and a second start time of the scheduled task, wherein the second start time is later than the first start time; when the computer program is executed by the processor, it further implements the following steps: when the first start time arrives, the scheduled task is determined to be a first task to be executed, and each subtask of the first task to be executed is executed on the operational data using a first thread according to the task type of at least one subtask of the first task to be executed; when the second start time arrives and the first task to be executed has not been completed, the scheduled task is determined to be a second task to be executed, and each subtask of the second task to be executed is executed on the operational data using a second thread according to the task type of at least one subtask of the second task to be executed.

[0121] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: when there are multiple tasks to be executed, the multiple tasks to be executed are executed asynchronously based on multiple threads.

[0122] In one embodiment, when the computer program is executed by a processor, it further performs the following steps: obtaining task configuration information, the task configuration information including a configured task start time and the execution order and processing logic of at least one subtask; generating a task type for the corresponding subtask according to the processing logic; and generating a timed task according to the task start time, the execution order of at least one subtask, the task type, and the processing logic.

[0123] In one embodiment, when the computer program is executed by the processor, it further implements the following steps: when the processing logic is a marking rule for marking the operational data, the task type of the corresponding subtask is a marking task type; or, when the processing logic is a statistical rule for statistical processing the operational data, the task type of the corresponding subtask is a statistical task type.

[0124] In one embodiment, when the computer program is executed by the processor, it further implements the following steps: when the task type of the subtask is a marking task type, the operational data is marked according to the corresponding marking rules to obtain the marked operational data; when the task type of the subtask is a statistics task type, corresponding data is extracted from the operational data according to the corresponding statistics rules and data cleaning is performed to obtain the corresponding statistical data.

[0125] In one embodiment, the statistical rule includes at least one statistical dimension; when the computer program is executed by the processor, it further performs the following steps: extracting data corresponding to the statistical dimension from the operational data according to the at least one statistical dimension and performing data cleaning to obtain statistical data of at least one statistical dimension.

[0126] In one embodiment, after obtaining statistical data for at least one statistical dimension, the computer program, when executed by a processor, further performs the following steps: generating and displaying a visualization chart based on the statistical data for the at least one statistical dimension.

[0127] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, performs the following steps:

[0128] Obtain a scheduled task to process operational data. The scheduled task includes a task start time, at least one subtask, and a task type for the at least one subtask. The task type includes a marking task type that marks the operational data or a statistical task type that performs statistics on the operational data.

[0129] Based on the task start time, obtain the timed task that matches the current time as the task to be executed;

[0130] Based on the task type of at least one subtask of the task to be executed, each subtask of the task to be executed is executed to perform labeling or statistical processing on the operational data to obtain the target data after task execution.

[0131] In one embodiment, the scheduled task further includes the execution order of at least one subtask; when the computer program is executed by the processor, it further implements the following steps: sorting the at least one subtask according to the execution order of the at least one subtask to obtain a sorted task queue; according to the task type of at least one subtask in the task queue, sequentially executing the corresponding subtask on the operational data to perform marking or statistical processing on the operational data, until the last subtask in the task queue is executed to obtain the target data after task execution.

[0132] In one embodiment, the task start time includes at least a first start time and a second start time of the scheduled task, wherein the second start time is later than the first start time; when the computer program is executed by the processor, it further implements the following steps: when the first start time arrives, the scheduled task is determined to be a first task to be executed, and each subtask of the first task to be executed is executed on the operational data using a first thread according to the task type of at least one subtask of the first task to be executed; when the second start time arrives and the first task to be executed has not been completed, the scheduled task is determined to be a second task to be executed, and each subtask of the second task to be executed is executed on the operational data using a second thread according to the task type of at least one subtask of the second task to be executed.

[0133] In one embodiment, when the computer program is executed by the processor, it further performs the following steps: when there are multiple tasks to be executed, the multiple tasks to be executed are executed asynchronously based on multiple threads.

[0134] In one embodiment, when the computer program is executed by a processor, it further performs the following steps: obtaining task configuration information, the task configuration information including a configured task start time and the execution order and processing logic of at least one subtask; generating a task type for the corresponding subtask according to the processing logic; and generating a timed task according to the task start time, the execution order of at least one subtask, the task type, and the processing logic.

[0135] In one embodiment, when the computer program is executed by the processor, it further implements the following steps: when the processing logic is a marking rule for marking the operational data, the task type of the corresponding subtask is a marking task type; or, when the processing logic is a statistical rule for statistical processing the operational data, the task type of the corresponding subtask is a statistical task type.

[0136] In one embodiment, when the computer program is executed by the processor, it further implements the following steps: when the task type of the subtask is a marking task type, the operational data is marked according to the corresponding marking rules to obtain the marked operational data; when the task type of the subtask is a statistics task type, corresponding data is extracted from the operational data according to the corresponding statistics rules and data cleaning is performed to obtain the corresponding statistical data.

[0137] In one embodiment, the statistical rule includes at least one statistical dimension; when the computer program is executed by the processor, it further performs the following steps: extracting data corresponding to the statistical dimension from the operational data according to the at least one statistical dimension and performing data cleaning to obtain statistical data of at least one statistical dimension.

[0138] In one embodiment, after obtaining statistical data for at least one statistical dimension, the computer program, when executed by a processor, further performs the following steps: generating and displaying a visualization chart based on the statistical data for the at least one statistical dimension.

[0139] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0140] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0141] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0142] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An operational data processing method, characterized in that, Applied to an operational server, the method includes: Obtain task configuration information, which includes the task start time obtained by configuring the scheduled task through the operation server, as well as the execution order and processing logic of at least one subtask. When the processing logic is a tagging rule for marking operational data, the task type for generating the corresponding subtask is a tagging task type; or, when the processing logic is a statistical rule for statistical processing of operational data, the task type for generating the corresponding subtask is a statistical task type; a scheduled task is generated based on the task start time, the execution order of at least one subtask, the task type, and the processing logic; the operational data includes data generated during the interaction between multiple channel servers and the operational server, and the scheduled task includes the task start time, at least one subtask, and the task type of the at least one subtask; for a scheduled task, all subtasks in the at least one subtask are tagging task types; Alternatively, all subtasks in the at least one subtask may be statistical task types; or, some subtasks in the at least one subtask may be tagging task types, and others may be statistical task types; the tagging rules include tagging rules for tagging access channels accessed by users through the channel server; the statistical rules include rules for statistically analyzing the number of users accessing the operation server through the channel server, and rules for statistically analyzing the number of users accessing and registering through the channel server, the statistical rules including a time dimension for statistically analyzing operation data within a specified time range, and a spatial dimension for statistically analyzing one or more specified channel servers; Based on the task start time, a scheduled task matching the current time is obtained as a task to be executed; the task start time includes at least a first start time and a second start time of the scheduled task, the second start time being later than the first start time; when the first start time arrives, the scheduled task is determined as the first task to be executed, and according to the task type of at least one subtask of the first task to be executed, the operational data is executed using a first thread for each subtask of the first task to be executed; when the second start time arrives, and the first task to be executed has not been completed, the scheduled task is determined as the second task to be executed, and according to the task type of at least one subtask of the second task to be executed, the operational data is executed using a second thread for each subtask of the second task to be executed; Based on the task type of at least one subtask of the task to be executed, each subtask of the task to be executed is executed to mark the operational data according to the marking rules or to perform statistical processing on the operational data according to the statistical rules, so as to obtain the target data after task execution.

2. The method according to claim 1, characterized in that, The scheduled task further includes the execution order of at least one subtask; the step of executing each subtask of the task to be executed according to the task type of at least one subtask of the task to be executed, to perform marking processing on the operational data according to the marking rules or statistical processing on the operational data according to the statistical rules, to obtain the target data after task execution, includes: The at least one subtask is sorted according to the execution order of the at least one subtask to obtain a sorted task queue; Based on the task type of at least one subtask in the task queue, the corresponding subtasks are executed sequentially on the operational data to perform marking or statistical processing on the operational data until the last subtask in the task queue is executed, thereby obtaining the target data after task execution.

3. The method according to claim 1, characterized in that, After obtaining the scheduled task matching the current time based on the task start time as the task to be executed, the method further includes: When there are multiple tasks to be executed, multiple tasks are executed asynchronously using multiple threads.

4. The method according to claim 1, characterized in that, The step of executing each subtask of the task to be executed according to the task type of at least one subtask of the task to be executed includes: When the task type of the subtask is a tagging task type, the operational data is tagged according to the corresponding tagging rules to obtain the tagged operational data. When the task type of the subtask is a statistical task, the corresponding data is extracted from the operational data according to the corresponding statistical rules and the data is cleaned to obtain the corresponding statistical data.

5. The method according to claim 4, characterized in that, The step of extracting corresponding data from the operational data according to the corresponding statistical rules and performing data cleaning to obtain the corresponding statistical data includes: Based on at least one statistical dimension, data corresponding to the statistical dimension is extracted from the operational data and the data is cleaned to obtain statistical data for at least one statistical dimension.

6. The method according to claim 5, characterized in that, After obtaining statistical data of at least one statistical dimension, the method further includes: Generate and display visualization charts based on statistical data from at least one statistical dimension.

7. An operational data processing device, characterized in that, The device, used in an operational server, includes: A scheduled task acquisition module is used to acquire task configuration information, which includes the task start time obtained through scheduled task configuration by the operation server, as well as the execution order and processing logic of at least one subtask. When the processing logic is a marking rule for marking operational data, the task type of the corresponding subtask is a marking task type; or, when the processing logic is a statistical rule for statistical processing of operational data, the task type of the corresponding subtask is a statistical task type. A scheduled task is generated based on the task start time, the execution order of at least one subtask, the task type, and the processing logic. The operational data includes data generated during the interaction between multiple channel servers and the operation server. The scheduled task includes the task start time, at least one subtask, and the execution order and processing logic of the at least one subtask. For a given timed task, all subtasks in the at least one subtask are classified as "marked task"; or, all subtasks in the at least one subtask are classified as "statistical task"; or, some subtasks in the at least one subtask are classified as "marked task" and others as "statistical task". The marking rules include marking rules for marking access channels accessed by users through the channel server; the statistical rules include rules for statistically analyzing the number of users accessing the operation server through the channel server, and rules for statistically analyzing the number of users accessing and registering through the channel server. The statistical rules include a time dimension for statistically analyzing operational data within a specified time range, and a spatial dimension for statistically analyzing one or more specified channel servers. The task-to-be-executed determination module is used to obtain a scheduled task matching the current time as a task-to-be-executed based on the task start time; the task start time includes at least a first start time and a second start time of the scheduled task, the second start time being later than the first start time; when the first start time arrives, the scheduled task is determined as a first task-to-be-executed, and according to the task type of at least one subtask of the first task-to-be-executed, each subtask of the first task-to-be-executed is executed on the operational data using a first thread; when the second start time arrives, and the first task-to-be-executed is not yet completed, the scheduled task is determined as a second task-to-be-executed, and according to the task type of at least one subtask of the second task-to-be-executed, each subtask of the second task-to-be-executed is executed on the operational data using a second thread; The task execution module is used to execute each subtask of the task to be executed according to the task type of at least one subtask of the task to be executed, so as to mark the operational data according to the marking rules or to perform statistical processing on the operational data according to the statistical rules, so as to obtain the target data after task execution.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images