Task processing method and device, computer device, and storage medium

Abstract

The embodiment of the application belongs to the field of artificial intelligence, and relates to a task processing method, comprising: judging whether a fault event corresponding to a first specified day's run batch task exists in a target application currently; wherein the first specified day refers to a day before the interval with the current day; if yes, switching a data source currently used by the target application to a preset target data source; obtaining target summary data corresponding to the first specified day from the target data source; generating a target run batch task corresponding to the target application based on the target summary data; and running the target run batch task to generate result data corresponding to the target run batch task. The application also provides a task processing device, a computer device and a storage medium. In addition, the application also relates to blockchain technology, and the result data can be stored in the blockchain. The application can quickly recover the run batch task processing of the target application, is conducive to improving the processing efficiency of the run batch task, and ensures the successful operation of the run batch task.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to task processing methods, apparatus, computer equipment and storage media. Background Technology

[0002] In the financial sector, batch processing, or big data processing, is an indispensable part of bank operations. Each insurance company uses its own batch processing program and process. The program generates monthly revenue and expenses through the company's corporate account at the bank, forming a general ledger in batches and conducting large-scale transactions, such as interest settlement, accrual and payment, and generating system reports in batches.

[0003] Currently, most insurance companies typically use Hadoop MapReduce for batch processing, with single task execution times generally exceeding 30 minutes. In practical applications, the process from importing source business data to processing final metrics involves multiple layers and a long chain, often requiring 2-3 hours. If a task fails, the entire chain needs to be rerun, which is time-consuming and unpredictable, with the risk that multiple reruns may not resolve the issue. Summary of the Invention

[0004] The purpose of this application is to provide a task processing method, apparatus, computer equipment, and storage medium to solve the technical problem that existing batch processing tasks, once a task failure occurs, require rerunning the entire process, which is time-consuming, has uncertainties, and may fail to recover even after multiple reruns.

[0005] To address the aforementioned technical problems, this application provides a task processing method, employing the following technical solution:

[0006] Determine whether the target application currently has a fault event corresponding to the batch run task on the first specified day; where the first specified day refers to the day before the current date.

[0007] If so, switch the data source currently used by the target application to the preset target data source;

[0008] Obtain the target summary data corresponding to the first specified day from the target data source;

[0009] Based on the target aggregated data, generate target batch processing tasks corresponding to the target application;

[0010] Run the target batch processing task to generate result data corresponding to the target batch processing task.

[0011] Furthermore, prior to the step of switching the data source currently used by the target application to a preset target data source, the method further includes:

[0012] Obtain real-time business data for the first specified day;

[0013] Within a preset time period, the indicators for the first specified day are processed based on the real-time business data to obtain the real-time summary data for the first specified day.

[0014] Obtain the first summary data corresponding to the second specified day; wherein, the second specified day refers to the day preceding the first specified day;

[0015] Based on the real-time aggregated data and the first aggregated data, generate second aggregated data corresponding to the first specified day;

[0016] The second summary data is used as the target summary data, and the target summary data is stored in the target data source.

[0017] Furthermore, the step of obtaining real-time business data for the first specified day specifically includes:

[0018] Retrieve archived logs corresponding to the target application from the preset source database;

[0019] Retrieve the target log data corresponding to the first specified day from the archived logs;

[0020] Extract the real-time business data for the first specified day from the target log data.

[0021] Furthermore, after the step of retrieving the target log data corresponding to the first specified day from the archived logs, the method further includes:

[0022] Invoke the preset anomaly prediction model;

[0023] The target log data is input into the anomaly prediction model, and the anomaly prediction model is used to perform anomaly analysis on the target log data to generate anomaly analysis results corresponding to the target log data.

[0024] Obtain the target user's communication information;

[0025] Based on the communication information, the anomaly analysis results are sent to the target user's communication device.

[0026] Furthermore, prior to the step of invoking the preset anomaly prediction model, the following steps are also included:

[0027] Obtain the preset sample data;

[0028] The sample data is divided into training sample data and test sample data according to a preset ratio; wherein, the training sample data includes several log sample data and category labels corresponding to the log sample data;

[0029] The log sample data in the training sample data is used as the model input, and the category label in the training sample data is used as the model output to train the preset machine learning model and obtain the trained machine learning model.

[0030] The trained machine learning model is tested based on the test sample data.

[0031] If the trained machine learning model passes the test, then the trained machine learning model will be used as the anomaly prediction model.

[0032] Furthermore, the step of processing the indicators for the first specified day based on the real-time business data to obtain the real-time summary data for the first specified day specifically includes:

[0033] Obtain the preset statistical rules for indicators;

[0034] Call the preset rule engine;

[0035] Using the rule engine, the real-time business data is processed according to the indicator statistics rules to obtain the corresponding indicator statistics results.

[0036] The statistical results of the aforementioned indicators are used as the real-time aggregated data.

[0037] Furthermore, the step of generating second summary data corresponding to the first specified day based on the real-time summary data and the first summary data specifically includes:

[0038] Call the preset indicator data calculation formula;

[0039] Based on the indicator data calculation formula, the real-time summary data and the first summary data are calculated and processed to obtain the corresponding calculated data;

[0040] The calculated data is used as the second summary data.

[0041] To address the aforementioned technical problems, this application also provides a task processing device, which employs the following technical solution:

[0042] The judgment module is used to determine whether the target application currently has a fault event corresponding to the batch task on the first specified day; wherein, the first specified day refers to the day before the current date.

[0043] The switching module is used to switch the data source currently used by the target application to a preset target data source if the condition is met.

[0044] The first acquisition module is used to acquire target summary data corresponding to the first specified day from the target data source;

[0045] The first generation module is used to generate target batch processing tasks corresponding to the target application based on the target aggregated data.

[0046] The second generation module is used to run the target batch processing task and generate result data corresponding to the target batch processing task.

[0047] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution:

[0048] Determine whether the target application currently has a fault event corresponding to the batch run task on the first specified day; where the first specified day refers to the day before the current date.

[0049] If so, switch the data source currently used by the target application to the preset target data source;

[0050] Obtain the target summary data corresponding to the first specified day from the target data source;

[0051] Based on the target aggregated data, generate target batch processing tasks corresponding to the target application;

[0052] Run the target batch processing task to generate result data corresponding to the target batch processing task.

[0053] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below:

[0054] Determine whether the target application currently has a fault event corresponding to the batch run task on the first specified day; where the first specified day refers to the day before the current date.

[0055] If so, switch the data source currently used by the target application to the preset target data source;

[0056] Obtain the target summary data corresponding to the first specified day from the target data source;

[0057] Based on the target aggregated data, generate target batch processing tasks corresponding to the target application;

[0058] Run the target batch processing task to generate result data corresponding to the target batch processing task.

[0059] Compared with the prior art, the embodiments of this application have the following main advantages:

[0060] In this embodiment, when a fault event corresponding to a batch processing task for a first specified day is detected in the target application, the data source currently used by the target application is switched to a preset target data source. Then, target summary data corresponding to the first specified day is obtained from the target data source. Subsequently, a target batch processing task corresponding to the target application is generated based on the target summary data. Finally, the target batch processing task is run to generate result data corresponding to the target batch processing task. In this embodiment, when the target application fails to execute a batch processing task, the data source currently used by the target application is switched to a pre-calculated target data source containing the target summary data for the batch processing task of that day. This allows for rapid recovery of the target application's batch processing task, improving processing efficiency and intelligence, and ensuring successful execution of the batch processing task. Attached Figure Description

[0061] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 This is an exemplary system architecture diagram to which this application can be applied;

[0063] Figure 2 A flowchart of an embodiment of the task processing method according to this application;

[0064] Figure 3 This is a schematic diagram of the structure of one embodiment of the task processing apparatus according to this application;

[0065] Figure 4 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation

[0066] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0067] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0068] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0069] like Figure 1 As shown, system architecture 100 may include terminal devices 101, 102, and 103, a network 104, and a server 105. Network 104 serves as the medium for providing communication links between terminal devices 101, 102, and 103 and server 105. Network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0070] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 101, 102, and 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social media platform software, etc.

[0071] Terminal devices 101, 102, and 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, e-book readers, MP3 players (Moving Picture Experts Group Audio Layer III), MP4 players (Moving Picture Experts Group Audio Layer IV), laptops, and desktop computers, etc.

[0072] Server 105 can be a server that provides various services, such as a backend server that supports the pages displayed on terminal devices 101, 102, and 103.

[0073] It should be noted that the task processing method provided in the embodiments of this application is generally executed by a server / terminal device, and correspondingly, the task processing device is generally set in the server / terminal device.

[0074] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0075] Continue to refer to Figure 2 A flowchart of an embodiment of the task processing method according to this application is shown. The task processing method includes the following steps:

[0076] Step S201: Determine whether the target application currently has a fault event corresponding to the batch running task of the first specified day.

[0077] In this embodiment, the task processing method runs on an electronic device (e.g., Figure 1 The server / terminal device shown can acquire fault events via wired or wireless connection. It should be noted that the aforementioned wireless connection methods may include, but are not limited to, 3G / 4G / 5G connections, Wi-Fi connections, Bluetooth connections, Wi-MAX connections, Zigbee connections, UWB (ultra-Width Band) connections, and other currently known or future-developed wireless connection methods. The aforementioned fault event refers to the failure of a batch processing task executed by the target application on a first specified day. Here, the first specified day refers to the day preceding the current date. Specifically, the aforementioned batch processing task may refer to a Hive batch processing task, which is a job execution method based on Hadoop MapReduce.

[0078] Step S202: If yes, switch the data source currently used by the target application to the preset target data source.

[0079] In this embodiment, the target data source is a pre-built data source that stores summary data for all dates prior to the stated date.

[0080] Step S203: Obtain the target summary data corresponding to the first specified day from the target data source.

[0081] In this embodiment, date information corresponding to the first specified date can be obtained, and then target summary data corresponding to the first specified date can be obtained from the target data source based on the date information.

[0082] Step S204: Generate a target batch task corresponding to the target application based on the target aggregated data.

[0083] In this embodiment, a pre-built batch processing task template can be obtained, and then the target aggregated data can be input into the batch processing task template to generate a target batch processing task corresponding to the target application. The batch processing task template can be a template pre-built based on actual business usage requirements and applied to batch processing tasks.

[0084] Step S205: Run the target batch processing task and generate result data corresponding to the target batch processing task.

[0085] In this embodiment, by running the target batch processing task, result data corresponding to the target batch processing task can be generated. Specifically, if the target batch processing task runs without failure, the corresponding result data is generated after the target batch processing task is completed. Furthermore, the generated result data can be stored in a blockchain node to ensure the privacy and security of the result data.

[0086] When this application detects whether a fault event corresponding to a batch processing task for a first specified day exists in the target application, it switches the data source currently used by the target application to a preset target data source. Then, it retrieves the target summary data corresponding to the first specified day from the target data source. Subsequently, it generates a target batch processing task corresponding to the target application based on the target summary data, and finally runs the target batch processing task to generate the result data corresponding to the target batch processing task. When the target application fails to execute a batch processing task, this application switches the data source currently used by the target application to a pre-calculated target summary data source required for the batch processing task of that day. This allows for rapid switching of the target application's current data source to store the target summary data of the first specified day when a task failure occurs, thereby quickly restoring the batch processing task of the target application. This improves the processing efficiency and intelligence of the batch processing task and ensures its successful execution.

[0087] In some alternative implementations, prior to step S202, the electronic device may also perform the following steps:

[0088] Obtain real-time business data for the first specified day.

[0089] In this embodiment, real-time business data for the first specified day can be further synchronized to the source layer. The real-time data import to the source layer can be achieved using flow technology, retrieving archived logs (Oracle: OGG / MySQL: Binlog) from the source database and writing the data to the data warehouse source layer. Furthermore, the specific implementation process for obtaining the real-time business data for the first specified day will be described in further detail in subsequent embodiments of this application, and will not be elaborated upon here.

[0090] Within a preset time period, the indicators for the first specified day are processed based on the real-time business data to obtain the real-time summary data for the first specified day.

[0091] In this embodiment, the aforementioned preset time period refers to after 24:00 on the first designated day. By performing real-time summary data calculations after 24:00 on the first designated day, it can be ensured that the real-time data for that day has been fully entered into the database, thereby guaranteeing the accuracy of the generated real-time summary data. The specific implementation process of processing the indicators of the first designated day based on the real-time business data to obtain the real-time summary data for the first designated day will be further described in detail in subsequent specific embodiments of this application, and will not be elaborated upon here. By using real-time business data to process the required daily indicators, since there is very little data on the first designated day, the processing task can be completed quickly, resulting in a short processing time.

[0092] Obtain the first summary data corresponding to the second specified day; wherein the second specified day refers to the day before the first specified day.

[0093] In this embodiment, the target data source can be queried using the date information of the second specified day to obtain the first summary data corresponding to the second specified day from the target data source.

[0094] Based on the real-time aggregated data and the first aggregated data, a second aggregated data corresponding to the first specified day is generated.

[0095] In this embodiment, the specific implementation process of generating the second summary data corresponding to the first specified day based on the real-time summary data and the first summary data will be further described in detail in subsequent specific embodiments of this application, and will not be elaborated on here.

[0096] The second summary data is used as the target summary data, and the target summary data is stored in the target data source.

[0097] This application obtains real-time business data for a first specified day, then processes the indicators for the first specified day based on the real-time business data within a preset time period to obtain real-time summary data for the first specified day. Next, it obtains first summary data corresponding to a second specified day, where the second specified day refers to the day preceding the first specified day. Subsequently, it generates second summary data corresponding to the first specified day based on the real-time summary data and the first summary data. Finally, it uses the second summary data as the target summary data and stores the target summary data in the target data source. This application calculates real-time summary data using real-time business data from the first specified day, thereby generating target summary data corresponding to the first specified day based on the real-time summary data and the first summary data. This allows for pre-calculation of the data required for batch processing tasks on that day. Therefore, in the event of a failure in the target application, the application can quickly switch its current data source to store the target summary data for the first specified day, enabling rapid recovery of batch processing tasks and improving processing efficiency and intelligence.

[0098] In some optional implementations of this embodiment, obtaining the real-time business data for the first specified day includes the following steps:

[0099] Retrieve archived logs corresponding to the target application from the preset source database.

[0100] In this embodiment, the aforementioned source database is a pre-built database that stores logs from various applications.

[0101] Retrieve the target log data corresponding to the first specified day from the archived logs.

[0102] In this embodiment, the archived log contains log data for each working day. Based on the date information of the first specified day, the target log data corresponding to the first specified day can be queried from all the archived logs.

[0103] Extract the real-time business data for the first specified day from the target log data.

[0104] In this embodiment, the target log data includes at least the real-time business data of the first specified day.

[0105] This application retrieves archived logs corresponding to the target application from a pre-defined source database, then queries the target log data corresponding to the first specified day from the archived logs, and finally extracts the real-time business data for the first specified day from the target log data. Based on query processing of the source database, this application can quickly and accurately obtain the real-time business data for the first specified day, improving the efficiency of real-time business data acquisition.

[0106] In some alternative implementations, after the step of retrieving the target log data corresponding to the first specified day from the archived logs, the electronic device may further perform the following steps:

[0107] Invoke the preset anomaly prediction model.

[0108] In this embodiment, the training and generation process of the above-mentioned anomaly prediction model will be described in more detail in subsequent specific embodiments, and will not be elaborated on here.

[0109] The target log data is input into the anomaly prediction model, and the anomaly prediction model is used to perform anomaly analysis on the target log data to generate anomaly analysis results corresponding to the target log data.

[0110] In this embodiment, the anomaly analysis results include whether the log data is normal or abnormal.

[0111] Obtain the target user's communication information.

[0112] In this embodiment, the target user may refer to staff members related to log data operation and maintenance, and the communication information may include mobile phone numbers or email addresses.

[0113] Based on the communication information, the anomaly analysis results are sent to the target user's communication device.

[0114] After retrieving the target log data corresponding to the first specified day from the archived logs, this application intelligently inputs the target log data into a preset anomaly prediction model. The anomaly prediction model performs anomaly analysis on the target log data, generating anomaly analysis results corresponding to the target log data. Then, it obtains the target user's communication information and, based on this information, sends the anomaly analysis results to the target user's communication device. By using anomaly prediction models to perform anomaly analysis on the target log data, this application can quickly and accurately generate corresponding anomaly analysis results and send them to the target user. This allows the user to analyze the reasons for the batch processing task failures of the day based on the received anomaly analysis results and perform corresponding repairs, thus improving the user experience.

[0115] In some alternative implementations, prior to the step of invoking the preset anomaly prediction model, the electronic device may also perform the following steps:

[0116] Obtain the preset sample data.

[0117] In this embodiment, the sample data may include pre-collected log data within a historical time period, and this log data has been annotated with anomaly tags. The tagging method may employ machine annotation, manual annotation, or a combination of both.

[0118] The sample data is divided into training sample data and test sample data according to a preset ratio; wherein, the training sample data includes several log sample data and category labels corresponding to the log sample data.

[0119] In this embodiment, the value of the preset ratio is not specifically limited and can be set according to actual needs, for example, it can be set to 30%. The test sample data includes multiple test log sample data and category labels corresponding to each test log sample data.

[0120] The log sample data in the training sample data is used as the model input, and the category labels in the training sample data are used as the model output to train the preset machine learning model, thereby obtaining the trained machine learning model.

[0121] In this embodiment, the machine learning model may include any one of the following: Naive Bayes model, Random Forest model, and Logistic Regression model. Furthermore, the training process for various machine learning models can refer to existing model training processes, and will not be elaborated upon here.

[0122] The trained machine learning model is tested based on the test sample data.

[0123] In this embodiment, testing the trained machine learning model based on the test sample dataset means: whether the prediction accuracy obtained by the trained machine learning model after performing anomaly prediction processing on the test sample dataset is greater than a preset accuracy threshold. If the prediction accuracy is greater than the accuracy threshold, the trained machine learning model is determined to have converged, thereby ending the training of the trained machine learning model, and the terminated machine learning model is taken as the aforementioned anomaly prediction model. Alternatively, if the prediction accuracy is less than the accuracy threshold, it indicates that the training of the trained machine learning model has not yet reached the preset standard, possibly due to insufficient training sample data or insufficient test sample data. Therefore, in this case, the number of training sample data can be further increased, for example, by increasing a fixed number or a random number each time. Then, the above training and testing processes are re-executed on this basis, and this process is repeated until the prediction accuracy of the trained machine learning model meets the requirement of being greater than the accuracy threshold, at which point the model training ends.

[0124] If the trained machine learning model passes the test, then the trained machine learning model will be used as the anomaly prediction model.

[0125] This application uses training and testing sample data, including several log sample data and category labels corresponding to the log sample data, to train and test a preset machine learning model. This enables the intelligent and rapid generation of anomaly prediction models that meet actual usage requirements. Subsequently, anomaly predictions can be performed on the target log data based on the trained anomaly prediction model, thereby quickly and accurately generating anomaly analysis results corresponding to the target log data. This improves the intelligence and efficiency of anomaly analysis result generation.

[0126] In some optional implementations of this embodiment, the step of processing the indicators for the first specified day based on the real-time business data to obtain the real-time summary data for the first specified day includes the following steps:

[0127] Obtain the preset statistical rules for indicators.

[0128] In this embodiment, the above-mentioned indicator statistics rules can be pre-written and generated according to actual business needs.

[0129] Call the preset rule engine.

[0130] In this embodiment, the rule engine mentioned above can specifically be Drools.

[0131] Using the rule engine, the real-time business data is processed according to the indicator statistics rules to obtain the corresponding indicator statistics results.

[0132] In this embodiment, the rule engine can perform time-window-based classification and aggregation operations on real-time business data according to time dimension information, thereby obtaining indicator data corresponding to the real-time business data.

[0133] The statistical results of the aforementioned indicators are used as the real-time aggregated data.

[0134] This application obtains preset indicator statistical rules, then calls a preset rule engine, and uses the rule engine to perform indicator statistical processing on the real-time business data according to the indicator statistical rules, obtaining the corresponding indicator statistical results, and using the indicator statistical results as the real-time summary data. Based on the application of indicator statistical rules and a rule engine, this application can quickly and accurately generate real-time summary data for a first specified day, which is beneficial for subsequently generating target summary data corresponding to the first specified day based on the obtained first summary data.

[0135] In some optional implementations of this embodiment, generating second summary data corresponding to the first specified day based on the real-time summary data and the first summary data includes the following steps:

[0136] Call the preset indicator data calculation formula.

[0137] In this embodiment, the above-mentioned indicator data calculation formula can be a pre-written calculation formula generated according to actual business needs. For example, in practical applications, there will be indicators that are accumulated annually or monthly, i.e., the cumulative indicator of year T-1 = the cumulative indicator of year T-2 + the real-time indicator of the day; when adding them, it is necessary to consider whether the statistical period needs to handle factors such as spanning years or months.

[0138] The real-time aggregated data and the first aggregated data are calculated and processed based on the indicator data calculation formula to obtain the corresponding calculated data.

[0139] In this embodiment, the calculated data can be obtained by summing the real-time summarized data with the first summarized data, and then used as the second summarized data.

[0140] The calculated data is used as the second summary data.

[0141] This application uses a preset indicator data calculation formula to calculate and process the real-time summarized data and the first summarized data based on the formula, obtaining corresponding calculated data, which is then used as the second summarized data. By applying the indicator data calculation formula, this application can quickly and accurately generate the second summarized data corresponding to the first specified date, which is beneficial for subsequently generating target summarized data corresponding to the first specified date based on the obtained first summarized data.

[0142] It should be emphasized that, to further ensure the privacy and security of the above results data, the results data can also be stored in a node of a blockchain.

[0143] The blockchain referred to in this application is a novel application model of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanisms, and encryption algorithms. Essentially, a blockchain is a decentralized database, a chain of data blocks linked together using cryptographic methods. Each data block contains information about a batch of network transactions, used to verify the validity of the information (anti-counterfeiting) and generate the next block. A blockchain can include an underlying blockchain platform, a platform product service layer, and an application service layer.

[0144] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) is the theory, method, technology, and application system that uses digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0145] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0146] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0147] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0148] Further reference Figure 3 As a response to the above Figure 2 To implement the method shown, this application provides an embodiment of a task processing device, which is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0149] like Figure 3 As shown, the task processing device 300 described in this embodiment includes: a judgment module 301, a switching module 302, a first acquisition module 303, a first generation module 304, and a second generation module 305. Wherein:

[0150] The judgment module 301 is used to determine whether the target application currently has a fault event corresponding to the batch running task on the first specified day; wherein, the first specified day refers to the day before the current date.

[0151] The switching module 302 is used to, if so, switch the data source currently used by the target application to a preset target data source;

[0152] The first acquisition module 303 is used to acquire target summary data corresponding to the first specified day from the target data source;

[0153] The first generation module 304 is used to generate a target batch task corresponding to the target application based on the target summary data.

[0154] The second generation module 305 is used to run the target batch task and generate result data corresponding to the target batch task.

[0155] In this embodiment, the operations performed by the above modules or units correspond one-to-one with the steps of the task processing method in the aforementioned embodiments, and will not be repeated here.

[0156] In some optional implementations of this embodiment, the task processing device further includes:

[0157] The second acquisition module is used to acquire real-time business data for the first specified day;

[0158] The processing module is used to process the indicators of the first specified day based on the real-time business data within a preset time period to obtain the real-time summary data of the first specified day.

[0159] The third acquisition module is used to acquire the first summary data corresponding to the second specified day; wherein, the second specified day refers to the day before the first specified day;

[0160] The third generation module is used to generate second summary data corresponding to the first specified day based on the real-time summary data and the first summary data;

[0161] A storage module is used to use the second summary data as the target summary data and store the target summary data in the target data source.

[0162] In this embodiment, the operations performed by the above modules or units correspond one-to-one with the steps of the task processing method in the aforementioned embodiments, and will not be repeated here.

[0163] In some optional implementations of this embodiment, the second acquisition module includes:

[0164] The first acquisition submodule is used to acquire archived logs corresponding to the target application from a preset source database;

[0165] The query submodule is used to query the target log data corresponding to the first specified day from the archived logs;

[0166] The extraction submodule is used to extract real-time business data for the first specified day from the target log data.

[0167] In this embodiment, the operations performed by the above modules or units correspond one-to-one with the steps of the task processing method in the aforementioned embodiments, and will not be repeated here.

[0168] In some optional implementations of this embodiment, the second acquisition module further includes:

[0169] The first calling submodule is used to call the preset anomaly prediction model;

[0170] The analysis submodule is used to input the target log data into the anomaly prediction model, perform anomaly analysis on the target log data through the anomaly prediction model, and generate anomaly analysis results corresponding to the target log data.

[0171] The second acquisition submodule is used to acquire the target user's communication information;

[0172] The sending submodule is used to send the anomaly analysis results to the target user's communication device based on the communication information.

[0173] In this embodiment, the operations performed by the above modules or units correspond one-to-one with the steps of the task processing method in the aforementioned embodiments, and will not be repeated here.

[0174] In some optional implementations of this embodiment, the second acquisition module further includes:

[0175] The third acquisition submodule is used to acquire preset sample data;

[0176] The partitioning submodule is used to divide the sample data into training sample data and test sample data according to a preset ratio; wherein, the training sample data includes several log sample data and category labels corresponding to the log sample data;

[0177] The training submodule is used to take the log sample data in the training sample data as the model input, take the category label in the training sample data as the model output, train the preset machine learning model, and obtain the trained machine learning model.

[0178] The testing submodule is used to test the trained machine learning model based on the test sample data;

[0179] The first determining submodule is used to use the trained machine learning model as the anomaly prediction model if the trained machine learning model passes the test.

[0180] In this embodiment, the operations performed by the above modules or units correspond one-to-one with the steps of the task processing method in the aforementioned embodiments, and will not be repeated here.

[0181] In some optional implementations of this embodiment, the processing module includes:

[0182] The fourth submodule is used to obtain preset indicator statistical rules;

[0183] The first submodule is used to invoke the preset rule engine;

[0184] The first calculation submodule is used to use the rule engine to perform indicator statistical processing on the real-time business data according to the indicator statistical rules, and obtain the corresponding indicator statistical results.

[0185] The second determining submodule is used to use the statistical results of the indicators as the real-time summary data.

[0186] In this embodiment, the operations performed by the above modules or units correspond one-to-one with the steps of the task processing method in the aforementioned embodiments, and will not be repeated here.

[0187] In some optional implementations of this embodiment, the third generation module includes:

[0188] The second calling submodule is used to call the preset indicator data calculation formula;

[0189] The second calculation submodule is used to perform calculations on the real-time summary data and the first summary data based on the indicator data calculation formula to obtain the corresponding calculated data.

[0190] The third determining submodule is used to use the calculated data as the second summary data.

[0191] In this embodiment, the operations performed by the above modules or units correspond one-to-one with the steps of the task processing method in the aforementioned embodiments, and will not be repeated here.

[0192] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0193] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected via a system bus. It should be noted that only the computer device 4 with components 41-43 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions. Its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), programmable gate arrays (FPGAs), digital digital processors (DSPs), embedded devices, etc.

[0194] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0195] The memory 41 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 41 may be an internal storage unit of the computer device 4, such as the hard disk or memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, smart memory card (SMC), secure digital card (SD) card, flash memory card, etc. of the computer device 4. Of course, the memory 41 may also include both the internal storage unit and the external storage device of the computer device 4. In this embodiment, the memory 41 is typically used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions for task processing methods. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or will be output.

[0196] In some embodiments, the processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chip. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to execute computer-readable instructions stored in the memory 41 or to process data, for example, to execute computer-readable instructions of the task processing method.

[0197] The network interface 43 may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 4 and other electronic devices.

[0198] Compared with the prior art, the embodiments of this application have the following main advantages:

[0199] In this embodiment, when a fault event corresponding to a batch processing task for a first specified day is detected in the target application, the data source currently used by the target application is switched to a preset target data source. Then, target summary data corresponding to the first specified day is obtained from the target data source. Subsequently, a target batch processing task corresponding to the target application is generated based on the target summary data. Finally, the target batch processing task is run to generate result data corresponding to the target batch processing task. In this embodiment, when the target application fails to execute a batch processing task, the data source currently used by the target application is switched to a pre-calculated target data source containing the target summary data for the batch processing task of that day. This allows for rapid recovery of the target application's batch processing task, improving processing efficiency and intelligence, and ensuring successful execution of the batch processing task.

[0200] This application also provides another embodiment, namely, providing a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor to cause the at least one processor to perform the steps of the task processing method described above.

[0201] Compared with the prior art, the embodiments of this application have the following main advantages:

[0202] In this embodiment, when a fault event corresponding to a batch processing task for a first specified day is detected in the target application, the data source currently used by the target application is switched to a preset target data source. Then, target summary data corresponding to the first specified day is obtained from the target data source. Subsequently, a target batch processing task corresponding to the target application is generated based on the target summary data. Finally, the target batch processing task is run to generate result data corresponding to the target batch processing task. In this embodiment, when the target application fails to execute a batch processing task, the data source currently used by the target application is switched to a pre-calculated target data source containing the target summary data for the batch processing task of that day. This allows for rapid recovery of the target application's batch processing task, improving processing efficiency and intelligence, and ensuring successful execution of the batch processing task.

[0203] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0204] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, the purpose of providing these embodiments is to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. A task processing method, characterized in that, Includes the following steps: Determine whether the target application currently has a fault event corresponding to the batch run task on the first specified day; where the first specified day refers to the day before the current date. If so, switch the data source currently used by the target application to the preset target data source; Obtain the target summary data corresponding to the first specified day from the target data source; Based on the target aggregated data, generate target batch processing tasks corresponding to the target application; Run the target batch processing task and generate the result data corresponding to the target batch processing task; Prior to the step of switching the data source currently used by the target application to a preset target data source, the method further includes: Obtain real-time business data for the first specified day; Within a preset time period, the indicators for the first specified day are processed based on the real-time business data to obtain the real-time summary data for the first specified day. Obtain the first summary data corresponding to the second specified day; wherein, the second specified day refers to the day preceding the first specified day; Based on the real-time aggregated data and the first aggregated data, generate second aggregated data corresponding to the first specified day; The second summary data is used as the target summary data, and the target summary data is stored in the target data source.

2. The task processing method according to claim 1, characterized in that, The step of obtaining real-time business data for the first specified day specifically includes: Retrieve archived logs corresponding to the target application from the preset source database; Retrieve the target log data corresponding to the first specified day from the archived logs; Extract the real-time business data for the first specified day from the target log data.

3. The task processing method according to claim 2, characterized in that, After the step of retrieving the target log data corresponding to the first specified day from the archived logs, the method further includes: Invoke the preset anomaly prediction model; The target log data is input into the anomaly prediction model, and the anomaly prediction model is used to perform anomaly analysis on the target log data to generate anomaly analysis results corresponding to the target log data. Obtain the target user's communication information; Based on the communication information, the anomaly analysis results are sent to the target user's communication device.

4. The task processing method according to claim 3, characterized in that, Before the step of invoking the preset anomaly prediction model, the following is also included: Obtain the preset sample data; The sample data is divided into training sample data and test sample data according to a preset ratio; wherein, the training sample data includes several log sample data and category labels corresponding to the log sample data; The log sample data in the training sample data is used as the model input, and the category label in the training sample data is used as the model output to train the preset machine learning model and obtain the trained machine learning model. The trained machine learning model is tested based on the test sample data. If the trained machine learning model passes the test, then the trained machine learning model will be used as the anomaly prediction model.

5. The task processing method according to claim 1, characterized in that, The step of processing the indicators for the first specified day based on the real-time business data to obtain the real-time summary data for the first specified day specifically includes: Obtain the preset statistical rules for indicators; Call the preset rule engine; Using the rule engine, the real-time business data is processed according to the indicator statistics rules to obtain the corresponding indicator statistics results. The statistical results of the aforementioned indicators are used as the real-time aggregated data.

6. The task processing method according to claim 1, characterized in that, The step of generating second summary data corresponding to the first specified day based on the real-time summary data and the first summary data specifically includes: Call the preset indicator data calculation formula; Based on the indicator data calculation formula, the real-time summary data and the first summary data are calculated and processed to obtain the corresponding calculated data; The calculated data is used as the second summary data.

7. A task processing device, characterized in that, include: The judgment module is used to determine whether the target application currently has a fault event corresponding to the batch task on the first specified day; wherein, the first specified day refers to the day before the current date. The switching module is used to switch the data source currently used by the target application to a preset target data source if the condition is met. The first acquisition module is used to acquire target summary data corresponding to the first specified day from the target data source; The first generation module is used to generate target batch processing tasks corresponding to the target application based on the target aggregated data. The second generation module is used to run the target batch processing task and generate result data corresponding to the target batch processing task. The task processing unit also includes: The second acquisition module is used to acquire real-time business data for the first specified day; The processing module is used to process the indicators of the first specified day based on the real-time business data within a preset time period to obtain the real-time summary data of the first specified day. The third acquisition module is used to acquire the first summary data corresponding to the second specified day; wherein, the second specified day refers to the day before the first specified day; The third generation module is used to generate second summary data corresponding to the first specified day based on the real-time summary data and the first summary data; A storage module is used to use the second summary data as the target summary data and store the target summary data in the target data source.

8. A computer device comprising a memory and a processor, the memory storing computer-readable instructions, wherein the processor, when executing the computer-readable instructions, implements the steps of the task processing method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-readable instructions, which, when executed by a processor, implement the steps of the task processing method as described in any one of claims 1 to 6.

Images