Assembly line task processing method and device, electronic equipment and storage medium

Abstract

The invention provides an assembly line task processing method and device, electronic equipment and a storage medium. The method comprises the steps of generating a task state snapshot of an assembly line task based on task resource state information of the assembly line task at a specified processing node of the assembly line task in an assembly line task processing process; when it is monitored that the assembly line task breaks down, a first task with the fault in the assembly line task is determined; executing state rollback on the task resource state of the first task based on the task state snapshot; and processing the first task based on the task resource state of the first task after rollback. The purposes of reducing the labor cost and improving the processing efficiency of the assembly line task can be achieved.

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to a pipeline task processing method, device, electronic device and storage medium. Background Art

[0002] In organizing assembly line operations, tasks are often executed in a specific order and at a specific time. However, due to human error, equipment failure, or external factors, the entire assembly line may go wrong or temporarily pause, requiring task rollback to restore normal operation of the assembly line.

[0003] The existing rollback method usually involves manually analyzing the error and then manually rolling back the erroneous resource individually, or selecting previously successful pipeline records and re-executing them as a whole. The former is time-consuming and labor-intensive, and may introduce more human errors. The latter will spend a lot of time on other tasks in the pipeline that do not affect resources, reducing the processing efficiency of pipeline tasks. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a pipeline task processing method, device, electronic device and storage medium to reduce labor costs and improve the processing efficiency of pipeline tasks. The specific technical solution is as follows:

[0005] In a first aspect of the implementation of the present application, a pipeline task processing method is first provided, comprising:

[0006] During the pipeline task processing, at a designated processing node of the pipeline task, a task status snapshot of the pipeline task is generated based on task resource status information of the pipeline task;

[0007] In the case where a failure occurs in the pipeline task, determining a first task having a failure in the pipeline task;

[0008] Based on the task status snapshot, performing a state rollback on the task resource state of the first task;

[0009] The first task is processed based on the task resource state of the first task after rolling back.

[0010] Optionally, generating a task status snapshot of the pipeline task at a designated processing node of the pipeline task based on task resource status information of the pipeline task includes:

[0011] At the task switching node of the pipeline task, recording the task resource status information of the pipeline task, and generating a task status snapshot of the pipeline task; or

[0012] After the setting phase processing of the pipeline task is completed, recording the task resource status information of the pipeline task and generating a task status snapshot of the pipeline task; or

[0013] Based on a preset snapshot generation time interval, the task resource status information of the pipeline task is recorded, and a task status snapshot of the pipeline task is generated.

[0014] Optionally, before the designated processing node of the pipeline task generates a task status snapshot of the pipeline task based on the task resource status information of the pipeline task, the method further includes:

[0015] The resource type corresponding to each task in the pipeline task is determined according to the dependency relationship and influence relationship of each task in the pipeline task on the resource.

[0016] Optionally, performing a state rollback on the task resource state of the first task based on the task state snapshot includes:

[0017] Determine, according to the resource type of the first task, a second task in the pipeline task having the same resource type as the first task;

[0018] Based on the task status snapshot, a state rollback is performed on the task resource state of the first task and the task resource state of the second task.

[0019] Optionally, performing a state rollback on the task resource state of the first task based on the task state snapshot includes:

[0020] Obtain a target task status snapshot generated most recently before the failure time of the first task;

[0021] Based on the target task state snapshot, a state rollback is performed on the task resource state of the first task.

[0022] Optionally, after processing the first task based on the task resource state of the first task after the rollback, the method further includes:

[0023] Obtaining a faulty task that occurs during the processing of the pipeline task and a fault type corresponding to the faulty task;

[0024] Collect the pipeline tasks that have not failed in the historical pipeline tasks;

[0025] Constructing a model data set according to the faulty task and the fault type, and the pipeline tasks without the fault;

[0026] Dividing the model data set into a training data set, a test data set, and a validation data set according to a preset division ratio;

[0027] Training a task fault prediction model to be trained based on the training data set to obtain a task fault prediction model;

[0028] Performing a model performance evaluation on the task fault prediction model based on the verification data set to obtain a model performance evaluation result;

[0029] When the model performance evaluation result meets the expected conditions, performing a model performance test on the task fault prediction model based on the test data set to obtain a model performance test result;

[0030] When the model performance test results meet the expected results, a final target task fault prediction model for task fault prediction is obtained.

[0031] Optionally, after obtaining the final target task fault prediction model for task fault prediction, the method further includes:

[0032] Performing fault prediction on the pipeline task to be processed based on the target task fault prediction model to obtain a predicted fault task of the pipeline task to be processed;

[0033] Obtaining a predicted fault node of the predicted fault task in the pipeline task to be processed;

[0034] During the processing of the pending pipeline task, a target task status snapshot of the pending pipeline task is generated based on the task resource status information of the pending pipeline task at the designated processing node of the pending pipeline task and the predicted fault node, so that in the event of a fault in the pending pipeline task, a state rollback can be performed based on the target task status snapshot.

[0035] In a second aspect of the implementation of the present application, a pipeline task processing device is provided, comprising:

[0036] A task status snapshot generation module is used to generate a task status snapshot of the pipeline task based on the task resource status information of the pipeline task at a designated processing node of the pipeline task during the pipeline task processing;

[0037] A first task determination module, configured to determine a first task having a fault in the pipeline tasks when a fault is detected in the pipeline tasks;

[0038] A task status rollback module, configured to perform a status rollback on the task resource status of the first task based on the task status snapshot;

[0039] The first task processing module is used to process the first task based on the task resource state of the first task after rolling back.

[0040] Optionally, the task status snapshot generation module includes:

[0041] A first snapshot generating unit, configured to record task resource status information of the pipeline task at a task switching node of the pipeline task, and generate a task status snapshot of the pipeline task;

[0042] A second snapshot generating unit, configured to record the task resource status information of the pipeline task and generate a task status snapshot of the pipeline task after the setting phase processing of the pipeline task is completed;

[0043] The third snapshot generation unit is used to record the task resource status information of the pipeline task based on a preset snapshot generation time interval, and generate a task status snapshot of the pipeline task.

[0044] Optionally, the device further comprises:

[0045] The resource type determination module is used to determine the resource type corresponding to each task in the pipeline task according to the dependency and influence relationship of each task in the pipeline task on the resource.

[0046] Optionally, the task status rollback module includes:

[0047] A second task determining unit, configured to determine, according to the resource type of the first task, a second task in the pipeline task having the same resource type as the first task;

[0048] The first state rollback unit is used to perform a state rollback on the task resource state of the first task and the task resource state of the second task based on the task state snapshot.

[0049] Optionally, the task status rollback module includes:

[0050] A target status snapshot acquiring unit, used to acquire a target task status snapshot generated most recently from the failure time of the first task;

[0051] The second state rollback unit is used to perform a state rollback on the task resource state of the first task based on the target task state snapshot.

[0052] Optionally, the device further comprises:

[0053] A fault task acquisition module is used to acquire the fault tasks that occur during the processing of the pipeline tasks, and the fault types corresponding to the fault tasks;

[0054] The pipeline task collection module is used to collect pipeline tasks that have not experienced any failure in historical pipeline tasks;

[0055] A model data set construction module, used to construct a model data set according to the fault task and the fault type, and the pipeline task without fault;

[0056] A model data set division module, used to divide the model data set into a training data set, a test data set and a validation data set according to a preset division ratio;

[0057] A fault prediction model acquisition module is used to train a fault prediction model for a task to be trained based on the training data set to obtain a task fault prediction model;

[0058] A performance evaluation result acquisition module is used to perform a model performance evaluation on the task fault prediction model based on the verification data set to obtain a model performance evaluation result;

[0059] A performance test result acquisition module is used to perform a model performance test on the task fault prediction model based on the test data set to obtain a model performance test result when the model performance evaluation result meets the expected conditions;

[0060] The target prediction model acquisition module is used to obtain the final target task fault prediction model for task fault prediction when the model performance test result reaches the expected result.

[0061] Optionally, the device further comprises:

[0062] A predicted fault task acquisition module is used to perform fault prediction on the pipeline task to be processed based on the target task fault prediction model to obtain the predicted fault task of the pipeline task to be processed;

[0063] A predicted fault node acquisition module, used to acquire the predicted fault node of the predicted fault task in the pipeline task to be processed;

[0064] A target state snapshot generation module is used to generate a target task state snapshot of the pipeline task to be processed based on the task resource state information of the pipeline task to be processed at the designated processing node of the pipeline task to be processed and the predicted fault node during the processing of the pipeline task to be processed, so as to perform a state rollback based on the target task state snapshot in the event that a fault occurs in the pipeline task to be processed.

[0065] In another aspect of the present application, there is also provided an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0066] Memory, used to store computer programs;

[0067] The processor is used to implement any of the above-mentioned pipeline task processing methods when executing the program stored in the memory.

[0068] In another aspect of the implementation of the present application, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer, the computer executes any of the above-mentioned pipeline task processing methods.

[0069] In another aspect of the implementation of the present application, a computer program product including instructions is provided, which, when executed on a computer, enables the computer to execute any of the above-mentioned pipeline task processing methods.

[0070] The solution provided by the embodiment of the present application generates a task status snapshot of the pipeline task based on the task resource status information of the pipeline task at the designated processing node of the pipeline task during the pipeline task processing. When a failure of the pipeline task is detected, the first task in the pipeline task that has a failure is determined. Based on the task status snapshot, a state rollback is performed on the task resource state of the first task. The first task is processed based on the task resource state of the first task after the rollback. By saving resource snapshots, the embodiment of the present application can restore the affected resources to the state before the transaction starts for reprocessing when the pipeline job needs to be rolled back, thereby reducing labor costs while improving the processing efficiency of pipeline tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0072] Figure 1 A flowchart of a pipeline task processing method provided in an embodiment of the present application;

[0073] Figure 2 A schematic diagram of an assembly line operation process provided in an embodiment of the present application;

[0074] Figure 3 A schematic diagram of the structure of a pipeline task processing device provided in an embodiment of the present application;

[0075] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0076] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0077] Figure 1 A flowchart of a pipeline task processing method provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the pipeline task processing method may include: step 101, step 102, step 103 and step 104.

[0078] Step 101: During the processing of a pipeline task, at a designated processing node of the pipeline task, a task status snapshot of the pipeline task is generated based on task resource status information of the pipeline task.

[0079] In this embodiment, pipeline is a method to improve system efficiency and performance by breaking down tasks into multiple stages (also called "stages" or "segments") so that each stage can work in parallel. In the field of software development, pipeline tasks include building, code checking, deployment, testing, and other links. By managing the pipeline through automated tools (such as Jenkins, etc.), automatic building, testing, and deployment of code can be achieved, improving the efficiency and quality of software development.

[0080] In this example, the task state snapshot can be used to indicate a complete backup of the task resource state of the pipeline task at a certain point in time.

[0081] During the process of processing the pipeline task, a task status snapshot of the pipeline task can be generated at the designated processing node of the pipeline task based on the task resource status information of the pipeline task. Specifically, the current resource status of all pipeline tasks can be recorded at the designated processing node of the pipeline task to create a task status snapshot. In this example, the designated processing node can be a node for processing tasks such as task switching, completion of an important stage, and a predetermined time interval. Specifically, it can be described in detail as follows.

[0082] 1. At the task switching node of the pipeline task, record the task resource status information of the pipeline task and generate a task status snapshot of the pipeline task.

[0083] In this embodiment, the task switching node refers to the processing node where the pipeline automatically or manually switches to the next task when a task is completed or a certain condition is met. For example, when a task is completed according to the preset steps and rules, the pipeline will automatically jump to the next task to continue execution. This switching is usually based on the detection of the task completion status, such as successful code compilation, test passing, etc.

[0084] In the system, a monitoring thread of the pipeline task can be set to monitor the task node where the pipeline task is located. When the task switching node of the pipeline task is monitored, the task resource status information of the pipeline task can be recorded to generate a task status snapshot of the pipeline task.

[0085] 2. After the setup phase of the pipeline task is completed, the task resource status information of the pipeline task is recorded and a task status snapshot of the pipeline task is generated.

[0086] In this example, the completion of the setting stage processing can be the completion of the important stage processing of the pipeline task. Taking the pipeline task of software development as an example, the setting stage can be the code checking stage, the deployment stage, the testing stage, etc. It can be understood that the setting stage of the pipeline task can be a stage customized by the business personnel, or a stage customized by the system, etc. This embodiment does not limit the setting stage of the pipeline task.

[0087] In the system, a monitoring thread of the pipeline task can be set to monitor the task node where the pipeline task is located. After monitoring that the set stage processing of the pipeline task is completed, the task resource status information of the pipeline task can be recorded to generate a task status snapshot of the pipeline task.

[0088] 3. Based on the preset snapshot generation time interval, record the task resource status information of the pipeline task and generate the task status snapshot of the pipeline task.

[0089] The snapshot generation time interval refers to a preset time interval for generating snapshots of pipeline tasks. In this example, the snapshot generation time interval can be 1 day, 2 days, etc., and this embodiment does not limit the specific duration of the snapshot generation time interval.

[0090] During the execution of the pipeline task, the task resource status information of the pipeline task can be recorded according to the preset snapshot generation time interval to generate a task status snapshot of the pipeline task.

[0091] After the designated processing node of the pipeline task generates a task status snapshot of the pipeline task based on the task resource status information of the pipeline task, step 102 is performed.

[0092] Step 102: When it is detected that the pipeline task fails, a first task having a failure in the pipeline task is determined.

[0093] In the process of processing pipeline tasks, artificial intelligence technology can be used to analyze the multi-dimensional data of the pipeline in real time, quickly identify the occurrence of faults and accurately locate the fault points. Specifically, a corresponding fault detection network model can be pre-established for the pipeline task. In the process of processing the pipeline task, the multi-dimensional data of the pipeline can be analyzed in real time by the fault detection network model to quickly identify the occurrence of faults and accurately locate the fault points. Furthermore, the first task with a fault in the pipeline task can be determined.

[0094] Of course, in a specific implementation, other methods can also be used to monitor whether a pipeline task fails, such as calling a fault sharing database, that is, for some simple faults or anomalies, the automated detection system can automatically call a fault sharing database, by comparing historical fault data with current fault phenomena, etc. This embodiment does not limit the fault detection method for pipeline tasks.

[0095] After determining that the first task in the pipeline has a fault, step 103 is executed.

[0096] Step 103: Based on the task status snapshot, perform a state rollback on the task resource state of the first task.

[0097] After determining that a faulty first task exists in the pipeline tasks, a state rollback can be performed on the task resource state of the first task based on the task state snapshot to roll back the current task resource state of the first task to a normal task resource state.

[0098] In a specific implementation of the present application, the resource type corresponding to each task in the pipeline task can be determined based on the dependency and influence relationship of each task in the pipeline task on the resource. When performing a task resource status rollback, the second task in the pipeline task with the same resource type as the first task can be determined based on the resource type of the first task. And based on the task status snapshot, a status rollback is performed on the task resource status of the first task and the task resource status of the second task. For example, for deployment tasks with dependencies, A and B need to be deployed, A is deployed first and then B is deployed. After B is deployed, there are problems and a rollback is required. The resources for deploying A are rolled back to the previous operations, etc. This ensures that during the rollback process, the status of the same resource is synchronized between tasks to avoid problems such as data inconsistency.

[0099] In this embodiment, in pipeline task management, determining the resource type corresponding to each task generally involves an in-depth understanding of the nature of the task, execution requirements, and dependencies between them. Pipeline operations generally involve multiple stages or tasks, and each task may require different types of resources to complete. These resources may include computing resources (such as CPU, GPU, etc.), storage resources (such as hard disk space), network resources (such as bandwidth), software resources (such as specific software licenses), human resources (such as operators with specific skills), etc. Among them, inter-task dependency: the dependencies between tasks in the pipeline can be analyzed, such as sequential dependency, parallel dependency, etc. Resource dependency: resource competition or sharing requirements between tasks can be identified, such as some resources may be required by multiple tasks at the same time.

[0100] In another specific implementation of the present application, after determining that the first task has a fault, the target task state snapshot generated most recently at the time of the fault of the first task can be obtained. Based on the target task state snapshot, a state rollback is performed on the task resource state of the first task. That is, according to the determined fault point and resource snapshot, the resource state involved is rolled back to the most recent normal snapshot state.

[0101] After the task resource state of the first task is rolled back based on the task state snapshot, step 104 is executed.

[0102] Step 104: Process the first task based on the task resource state of the first task after rolling back.

[0103] After the task resource state of the first task is rolled back based on the task state snapshot, the first task may be processed based on the task resource state of the first task after the rollback.

[0104] The embodiment of the present application classifies tasks according to their impact on resources, and by saving resource snapshots, restores the affected resources to the time before the transaction started for reprocessing when the pipeline job needs to be rolled back. This can avoid the time-consuming and labor-intensive problems caused by manual processing, reduce labor costs, and improve the processing efficiency of pipeline tasks.

[0105] Next, combine Figure 2 The snapshot rollback process of the pipeline job is described in detail as follows: Figure 2 As shown, the process (taking software development as an example) can include:

[0106] 1. Build pipeline jobs. You can create a new project on the software development platform or code hosting platform and configure the basic project information. Define pipeline stages: Divide the software development process into multiple stages, such as code review, build, unit test, integration test, deployment, etc. Then, you can write pipeline tasks, which can specifically include: code review, build tasks.

[0107] 2. After building the pipeline job, you can deploy the pipeline job to the test environment and create a snapshot of the test environment resources. Specifically, you can define the deployment process of the test environment in the pipeline tool, including code pulling, building, testing, deployment and other stages, and write automated test scripts to verify the functions and performance of the application after deployment. At the same time, you can modify the configuration files of the test environment (such as database connection information, middleware configuration, etc.) according to the test requirements to complete the deployment.

[0108] 3. Testing, which can specifically include: API (Application Programming Interface) testing and unit testing. API testing refers to the testing process of verifying and evaluating the functionality, performance, and security of the application interface. Unit testing is a test performed on the smallest testable unit in the software (such as a function, class, or class method, etc.). It aims to verify that these units work as expected and are relatively isolated from other units.

[0109] 4. Online application, that is, online application in the software development process, refers to a series of application and approval processes before the software application or system that has been developed, tested and ready for use is officially deployed to the production environment (that is, the environment that users can access). This process is designed to ensure that the software application or system has been fully verified and prepared before it is officially launched, so as to reduce risks and problems after the launch.

[0110] 5. Deploy to the production environment and create a snapshot of the production environment resources. First, evaluate whether the current hardware resources (such as servers, storage devices, etc.) and software resources (such as operating systems, databases, middleware, etc.) of the production environment meet the requirements of the new software or updated versions. If the requirements are met, the network, security, monitoring and other infrastructure of the production environment can be configured according to the requirements of the new software or updated versions. Ensure that the configuration of the production environment is as consistent as possible with the test environment to reduce unexpected problems after deployment. Package the tested software into deployable units (such as Docker images, WAR packages, etc.), and use automated deployment tools (such as Ansible, Puppet, etc.) to deploy the software packages to the production environment. Adjust the hardware resources (such as increasing CPU, memory, etc.) or software resources (such as upgrading the operating system, database, etc.) of the production environment according to the requirements of the software operation. Ensure that resource adjustments do not interrupt running services or cause data loss.

[0111] 6. Online testing. Online testing after deployment to the production environment is a key step in ensuring that the software runs stably in the production environment, meets user needs and meets established quality standards.

[0112] In the entire pipeline operation process described above, real-time fault detection can be performed through artificial intelligence and other methods, and resource snapshots can be created at specific stages of the pipeline. The creation of snapshots can be based on time (for example, at regular intervals) or events (such as task completion, the beginning or end of a key stage) to facilitate rapid rollback. After detecting an abnormality in the pipeline, efficient and consistent transactional rollback can be performed based on the resource snapshot point, ensuring that the pipeline operation can be quickly restored to the most recent correct state while ensuring data consistency.

[0113] In this embodiment, the task fault prediction model can be trained to analyze historical faults, predict possible faults and maintenance requirements, and thus implement predictive maintenance. The implementation process can be described in detail in conjunction with the following specific implementation methods.

[0114] In a specific implementation of the present application, after the above step 104, the following may also be included:

[0115] Step S1: Obtain the faulty tasks that occur during the processing of the pipeline tasks, and the fault types corresponding to the faulty tasks.

[0116] In this embodiment, the faulty tasks that occur during the processing of the pipeline tasks and the fault types of the faulty tasks can be obtained. Among them, the faulty tasks refer to the tasks that fail to be completed as expected or have abnormalities during the execution process due to some reasons during the pipeline operation. The fault type refers to the result of classifying and summarizing the faults that occur in the pipeline tasks. Different fault types reflect different causes and manifestations of the faults, which helps to quickly locate the problem and take corresponding solutions.

[0117] Step S2: Collect the pipeline tasks that have not experienced any failure in the historical pipeline tasks.

[0118] At the same time, the pipeline tasks that have not failed in the historical pipeline tasks can be collected. In the specific implementation, most pipeline management systems will record the details of task execution, including task status (success, failure, interruption, etc.), execution time, operator, resources used, and other information. According to the task status or error code and other information in the data source, filter out those tasks marked as "successful" or "error-free". Thus, the pipeline tasks that have not failed in the historical pipeline tasks can be obtained.

[0119] Step S3: construct a model data set according to the faulty task and the fault type, and the pipeline tasks that have not failed.

[0120] After obtaining the faulty tasks and the corresponding faulty tasks that occur during the processing of the pipeline tasks, as well as the pipeline tasks that do not fail, the collected faulty task and normal task data can be cleaned to handle missing values, outliers, and duplicate data. Furthermore, features useful for fault prediction can be extracted, such as task type, execution time, resource usage, dependencies, etc. And the fault type is encoded so that the model can process it (for example, using one-hot encoding or label encoding). Finally, the processed faulty task data and normal task data can be merged to form the initial model data set. Each sample should contain multiple features (such as task attributes, execution environment, etc.) and a label (fault type or "normal" as the non-fault category). Of course, if there are multiple fault types, ensure that each fault type has a sufficient sample size to avoid class imbalance problems.

[0121] Step S4: Divide the model data set into a training data set, a test data set and a validation data set according to a preset division ratio.

[0122] After obtaining the model data set, the model data set can be divided into a training data set, a test data set, and a validation data set according to a preset division ratio. For example, the division ratio is 70% training set, 15% validation set, and 15% test set, and the model data set is divided to obtain a training data set, a test data set, and a validation data set.

[0123] Step S5: training the task fault prediction model to be trained based on the training data set to obtain a task fault prediction model.

[0124] When training a fault prediction model, you can choose an appropriate machine learning or deep learning model based on the complexity of the problem and the characteristics of the data. For multi-classification problems (i.e. predicting specific fault types), common choices include decision trees, random forests, gradient boosting trees (such as XGBoost, etc.) or neural networks.

[0125] After obtaining the training data set, the training data set can be used to train the task fault prediction model to be trained to obtain the task fault prediction model. During the training process, the model performance can be optimized by adjusting the model parameters (such as learning rate, tree depth, regularization strength, etc.). Using the features and labels in the training data set to train the task fault prediction model to be trained involves selecting a suitable machine learning algorithm, setting the parameters of the algorithm, and minimizing the loss function through an iterative optimization process (such as gradient descent) to obtain a preliminarily trained task fault prediction model.

[0126] Step S6: Performing a model performance evaluation on the task fault prediction model based on the verification data set to obtain a model performance evaluation result.

[0127] After the task fault prediction model is trained, the model performance evaluation of the task fault prediction model can be performed based on the validation data set to obtain the model performance evaluation result. Specifically, the validation data set (independent of the training data set) can be used to evaluate the performance of the initially trained model. Calculate and compare the performance indicators of the model on the validation set, such as accuracy, recall, F1 score, area under the ROC curve (AUC), etc. Check whether overfitting or underfitting occurs, and adjust the model parameters or structure accordingly.

[0128] After obtaining the model performance evaluation results, specifically, the model performance evaluation results can be compared with the expected conditions. If the evaluation results do not meet the expected conditions, continue to adjust the model or collect more data for training. If the evaluation results meet the expected conditions, proceed to the next step.

[0129] Step S7: When the model performance evaluation result meets the expected conditions, a model performance test is performed on the task fault prediction model based on the test data set to obtain a model performance test result.

[0130] If the model performance evaluation results meet the expected conditions, the task fault prediction model can be tested for model performance based on the test data set to obtain the model performance test results. That is, the model is tested for performance using a test data set (independent of both the training data set and the validation data set), and the performance indicators of the model on the test set are calculated to evaluate the actual prediction ability of the model and obtain the model performance test results.

[0131] Then, you can compare the model performance test results with the expected test results. If the test results meet the expected results, it means that the model has high prediction accuracy and generalization ability. If the test results do not meet the expected results, the model has high prediction accuracy and generalization ability.

[0132] Step S8: When the model performance test result reaches the expected result, a final target task fault prediction model for task fault prediction is obtained.

[0133] When the model performance test results meet the expected results, the final target task fault prediction model for task fault prediction is obtained. After confirming that the model's performance on the test set meets expectations and has high stability and reliability, the model can be determined as the final target task fault prediction model and prepared to be deployed in the actual production environment.

[0134] It is understandable that the above model training process can also be obtained through offline environment training and directly deployed on the machine where the state snapshot is created and the state is rolled back for fault prediction. This embodiment does not limit the environment for model training.

[0135] The embodiments of the present application combine artificial intelligence and machine learning algorithms to analyze historical fault data to achieve predictive maintenance and intelligent fault detection and location.

[0136] After the target task fault prediction model is trained, the model can be used to predict the faults of the pipeline tasks, and a state snapshot can be created based on the fault prediction results. The implementation process can be described in detail in conjunction with the following specific implementation methods.

[0137] In another specific implementation of the present application, the method may further include:

[0138] Step M1: perform fault prediction on the pipeline task to be processed based on the target task fault prediction model to obtain the predicted fault task of the pipeline task to be processed.

[0139] In this embodiment, before processing the pipeline task to be processed, the pipeline task to be processed can be predicted for faults based on the target task fault prediction model to obtain the predicted fault task of the pipeline task to be processed. Specifically, the task characteristics (such as task type, input data, dependency, etc.) and resource status information (such as CPU usage, memory usage, network status, etc.) of the pipeline task to be processed can be input into the target task fault prediction model. The target task fault prediction model predicts the fault tasks that may appear in the pipeline task to be processed based on the input information to obtain the predicted fault task of the pipeline task to be processed.

[0140] After performing fault prediction on the pipeline task to be processed based on the target task fault prediction model to obtain the predicted fault task of the pipeline task to be processed, step M2 is executed.

[0141] Step M2: Obtain the predicted fault node of the predicted fault task in the pipeline task to be processed.

[0142] After the predicted fault task of the pipeline task to be processed is obtained by performing fault prediction on the pipeline task to be processed based on the target task fault prediction model, the predicted fault node of the predicted fault task in the pipeline task to be processed can be obtained. Specifically, according to the process definition of the pipeline task to be processed, the specific position of each predicted fault task in the process, that is, the predicted fault node, can be found, and the information of the predicted fault node, including node ID, node type, front and back dependencies, etc., can be recorded.

[0143] After obtaining the predicted fault node of the predicted fault task in the pipeline tasks to be processed, step M3 is executed.

[0144] Step M3: During the processing of the pending pipeline task, at the designated processing node of the pending pipeline task and the predicted fault node, a target task status snapshot of the pending pipeline task is generated based on the task resource status information of the pending pipeline task, so as to perform a state rollback based on the target task status snapshot in the event of a failure of the pending pipeline task.

[0145] In this example, the designated processing node of the pipeline task to be processed may be any one of: a task switching node of the pipeline task to be processed, a node where the set stage processing of the pipeline task to be processed is completed, and a node for setting a snapshot generation time interval.

[0146] The designated processing nodes may include predicted fault nodes or may not include predicted fault nodes. Specifically, the relationship between the designated processing nodes and the predicted fault nodes may be determined according to actual conditions, and this embodiment does not impose any limitation on this.

[0147] After obtaining the predicted fault node of the predicted fault task in the pipeline task to be processed, the target task state snapshot of the pipeline task to be processed can be generated based on the task resource state information of the pipeline task to be processed at the designated processing node of the pipeline task to be processed and the predicted fault node, so that in the case of a failure of the pipeline task to be processed, the state can be rolled back based on the target task state snapshot. Specifically, when the pipeline task to be processed starts to execute, or when the designated processing node is reached, the state information of the current task is captured and recorded, including the state of all related resources (such as file status, database status, memory status, etc.). For each predicted fault node, before or after executing to the node (select according to actual needs), the task state information is captured and recorded again. These state information are saved as target task state snapshots, and each snapshot is associated with a specific time point and task execution status. During the execution of the pipeline task, a fault detection mechanism is implemented to monitor the task execution and resource status in real time. Once the actual fault is detected to match the predicted fault, or other serious errors occur, the state rollback process is triggered. The specific rollback process can be: according to the fault information, find the target task state snapshot closest to the time when the fault occurred, and use the state information in the snapshot to roll back the pipeline task and related resources to the state recorded in the snapshot. After the state rollback is successful, decide whether to re-execute the pipeline task or skip some completed nodes based on the task process definition and the current task status. Restart the pipeline task or continue to execute the remaining nodes until the task is completed.

[0148] The embodiments of the present application combine artificial intelligence and machine learning algorithms to analyze historical fault data, achieve predictive maintenance and intelligent fault detection and location, reduce the occurrence of faults and improve rollback efficiency.

[0149] The pipeline task processing method provided in the embodiment of the present application generates a task status snapshot of the pipeline task based on the task resource status information of the pipeline task at the designated processing node of the pipeline task during the pipeline task processing. When a failure of the pipeline task is detected, the first task in the pipeline task that has a failure is determined. Based on the task status snapshot, a state rollback is performed on the task resource status of the first task. The first task is processed based on the task resource status of the first task after the rollback. By saving resource snapshots, the embodiment of the present application can restore the affected resources to the time before the transaction started for reprocessing when the pipeline job needs to be rolled back, thereby reducing labor costs while improving the processing efficiency of pipeline tasks.

[0150] Reference Figure 3 , shows a schematic diagram of the structure of a pipeline task processing device provided by an embodiment of the present application, such as Figure 3 As shown, the pipeline task processing device 300 may include the following modules:

[0151] The task status snapshot generating module 310 is used to generate a task status snapshot of the pipeline task based on the task resource status information of the pipeline task at a designated processing node of the pipeline task during the pipeline task processing;

[0152] A first task determining module 320 is used to determine a first task having a fault in the pipeline tasks when a fault is detected in the pipeline tasks;

[0153] A task status rollback module 330, configured to perform a status rollback on the task resource status of the first task based on the task status snapshot;

[0154] The first task processing module 340 is configured to process the first task based on the task resource state of the first task after rolling back.

[0155] Optionally, the task status snapshot generation module includes:

[0156] A first snapshot generating unit, configured to record task resource status information of the pipeline task at a task switching node of the pipeline task, and generate a task status snapshot of the pipeline task;

[0157] A second snapshot generating unit, configured to record the task resource status information of the pipeline task and generate a task status snapshot of the pipeline task after the setting phase processing of the pipeline task is completed;

[0158] The third snapshot generation unit is used to record the task resource status information of the pipeline task based on a preset snapshot generation time interval, and generate a task status snapshot of the pipeline task.

[0159] Optionally, the device further comprises:

[0160] The resource type determination module is used to determine the resource type corresponding to each task in the pipeline task according to the dependency and influence relationship of each task in the pipeline task on the resource.

[0161] Optionally, the task status rollback module includes:

[0162] A second task determining unit, configured to determine, according to the resource type of the first task, a second task in the pipeline task having the same resource type as the first task;

[0163] The first state rollback unit is used to perform a state rollback on the task resource state of the first task and the task resource state of the second task based on the task state snapshot.

[0164] Optionally, the task status rollback module includes:

[0165] A target status snapshot acquiring unit, used to acquire a target task status snapshot generated most recently from the failure time of the first task;

[0166] The second state rollback unit is used to perform a state rollback on the task resource state of the first task based on the target task state snapshot.

[0167] Optionally, the device further comprises:

[0168] A fault task acquisition module is used to acquire the fault tasks that occur during the processing of the pipeline tasks, and the fault types corresponding to the fault tasks;

[0169] The pipeline task collection module is used to collect pipeline tasks that have not experienced any failure in historical pipeline tasks;

[0170] A model data set construction module, used to construct a model data set according to the fault task and the fault type, and the pipeline task without fault;

[0171] A model data set division module, used to divide the model data set into a training data set, a test data set and a validation data set according to a preset division ratio;

[0172] A fault prediction model acquisition module is used to train a fault prediction model for a task to be trained based on the training data set to obtain a task fault prediction model;

[0173] A performance evaluation result acquisition module is used to perform a model performance evaluation on the task fault prediction model based on the verification data set to obtain a model performance evaluation result;

[0174] A performance test result acquisition module is used to perform a model performance test on the task fault prediction model based on the test data set to obtain a model performance test result when the model performance evaluation result meets the expected conditions;

[0175] The target prediction model acquisition module is used to obtain the final target task fault prediction model for task fault prediction when the model performance test result reaches the expected result.

[0176] Optionally, the device further comprises:

[0177] A predicted fault task acquisition module is used to perform fault prediction on the pipeline task to be processed based on the target task fault prediction model to obtain the predicted fault task of the pipeline task to be processed;

[0178] A predicted fault node acquisition module, used to acquire the predicted fault node of the predicted fault task in the pipeline task to be processed;

[0179] A target state snapshot generation module is used to generate a target task state snapshot of the pipeline task to be processed based on the task resource state information of the pipeline task to be processed at the designated processing node of the pipeline task to be processed and the predicted fault node during the processing of the pipeline task to be processed, so as to perform a state rollback based on the target task state snapshot in the event that a fault occurs in the pipeline task to be processed.

[0180] The pipeline task processing device provided in the embodiment of the present application generates a task status snapshot of the pipeline task based on the task resource status information of the pipeline task at the designated processing node of the pipeline task during the pipeline task processing. In the case where a failure of the pipeline task is detected, the first task in the pipeline task that has a failure is determined. Based on the task status snapshot, a state rollback is performed on the task resource status of the first task. The first task is processed based on the task resource status of the first task after the rollback. By saving resource snapshots, the embodiment of the present application can restore the affected resources to the state before the transaction starts when the pipeline job needs to be rolled back, thereby reducing labor costs while improving the processing efficiency of pipeline tasks.

[0181] The present application also provides an electronic device, such as Figure 4As shown, it includes a processor 401, a communication interface 402, a memory 403 and a communication bus 404, wherein the processor 401, the communication interface 402, and the memory 403 communicate with each other through the communication bus 404.

[0182] Memory 403, used for storing computer programs;

[0183] The processor 401 is used to execute the program stored in the memory 403 to implement the following steps:

[0184] During the pipeline task processing, at a designated processing node of the pipeline task, a task status snapshot of the pipeline task is generated based on task resource status information of the pipeline task;

[0185] In the case where a failure occurs in the pipeline task, determining a first task having a failure in the pipeline task;

[0186] Based on the task status snapshot, performing a state rollback on the task resource state of the first task;

[0187] The first task is processed based on the task resource state of the first task after rolling back.

[0188] Optionally, generating a task status snapshot of the pipeline task at a designated processing node of the pipeline task based on task resource status information of the pipeline task includes:

[0189] At the task switching node of the pipeline task, recording the task resource status information of the pipeline task, and generating a task status snapshot of the pipeline task; or

[0190] After the setting phase processing of the pipeline task is completed, recording the task resource status information of the pipeline task and generating a task status snapshot of the pipeline task; or

[0191] Based on a preset snapshot generation time interval, the task resource status information of the pipeline task is recorded, and a task status snapshot of the pipeline task is generated.

[0192] Optionally, before the designated processing node of the pipeline task generates a task status snapshot of the pipeline task based on the task resource status information of the pipeline task, the method further includes:

[0193] The resource type corresponding to each task in the pipeline task is determined according to the dependency relationship and influence relationship of each task in the pipeline task on the resource.

[0194] Optionally, performing a state rollback on the task resource state of the first task based on the task state snapshot includes:

[0195] Determine, according to the resource type of the first task, a second task in the pipeline task having the same resource type as the first task;

[0196] Based on the task status snapshot, a state rollback is performed on the task resource state of the first task and the task resource state of the second task.

[0197] Optionally, performing a state rollback on the task resource state of the first task based on the task state snapshot includes:

[0198] Obtain a target task status snapshot generated most recently before the failure time of the first task;

[0199] Based on the target task state snapshot, a state rollback is performed on the task resource state of the first task.

[0200] Optionally, after processing the first task based on the task resource state of the first task after the rollback, the method further includes:

[0201] Obtaining a faulty task that occurs during the processing of the pipeline task and a fault type corresponding to the faulty task;

[0202] Collect the pipeline tasks that have not failed in the historical pipeline tasks;

[0203] Constructing a model data set according to the faulty task and the fault type, and the pipeline tasks without the fault;

[0204] Dividing the model data set into a training data set, a test data set, and a validation data set according to a preset division ratio;

[0205] Training a task fault prediction model to be trained based on the training data set to obtain a task fault prediction model;

[0206] Performing a model performance evaluation on the task fault prediction model based on the verification data set to obtain a model performance evaluation result;

[0207] When the model performance evaluation result meets the expected conditions, performing a model performance test on the task fault prediction model based on the test data set to obtain a model performance test result;

[0208] When the model performance test results meet the expected results, a final target task fault prediction model for task fault prediction is obtained.

[0209] Optionally, after obtaining the final target task fault prediction model for task fault prediction, the method further includes:

[0210] Performing fault prediction on the pipeline task to be processed based on the target task fault prediction model to obtain a predicted fault task of the pipeline task to be processed;

[0211] Obtaining a predicted fault node of the predicted fault task in the pipeline task to be processed;

[0212] During the processing of the pending pipeline task, a target task status snapshot of the pending pipeline task is generated based on the task resource status information of the pending pipeline task at the designated processing node of the pending pipeline task and the predicted fault node, so that in the event of a fault in the pending pipeline task, a state rollback can be performed based on the target task status snapshot.

[0213] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0214] The communication interface is used for communication between the above terminal and other devices.

[0215] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0216] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0217] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which instructions are stored. When the computer-readable storage medium is executed on a computer, the computer executes the pipeline task processing method described in any one of the above embodiments.

[0218] In another embodiment provided by the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute the pipeline task processing method described in any one of the above embodiments.

[0219] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0220] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0221] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0222] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A pipeline task processing method, characterized in that: include: During the pipeline task processing, at a designated processing node of the pipeline task, a task status snapshot of the pipeline task is generated based on task resource status information of the pipeline task; In the case where a failure occurs in the pipeline task, determining a first task having a failure in the pipeline task; Based on the task status snapshot, performing a state rollback on the task resource state of the first task; The first task is processed based on the task resource state of the first task after rolling back.

2. The method according to claim 1, characterized in that The generating, at the designated processing node of the pipeline task, a task status snapshot of the pipeline task based on the task resource status information of the pipeline task, includes: At the task switching node of the pipeline task, recording the task resource status information of the pipeline task, and generating a task status snapshot of the pipeline task; or After the setting phase processing of the pipeline task is completed, recording the task resource status information of the pipeline task and generating a task status snapshot of the pipeline task; or Based on a preset snapshot generation time interval, the task resource status information of the pipeline task is recorded, and a task status snapshot of the pipeline task is generated.

3. The method according to claim 1, characterized in that Before generating, at a designated processing node of the pipeline task, a task status snapshot of the pipeline task based on the task resource status information of the pipeline task, the method further includes: The resource type corresponding to each task in the pipeline task is determined according to the dependency relationship and influence relationship of each task in the pipeline task on the resource.

4. The method according to claim 3, characterized in that The performing a state rollback on the task resource state of the first task based on the task state snapshot includes: Determine, according to the resource type of the first task, a second task in the pipeline task having the same resource type as the first task; Based on the task status snapshot, a state rollback is performed on the task resource state of the first task and the task resource state of the second task.

5. The method according to claim 1, characterized in that The performing a state rollback on the task resource state of the first task based on the task state snapshot includes: Obtain a target task status snapshot generated most recently before the failure time of the first task; Based on the target task state snapshot, a state rollback is performed on the task resource state of the first task.

6. The method according to claim 1, characterized in that After processing the first task based on the task resource state of the first task after the rollback, the method further includes: Obtaining a faulty task that occurs during the processing of the pipeline task and a fault type corresponding to the faulty task; Collect the pipeline tasks that have not failed in the historical pipeline tasks; Constructing a model data set according to the faulty task and the fault type, and the pipeline tasks without the fault; Dividing the model data set into a training data set, a test data set, and a validation data set according to a preset division ratio; Training a task fault prediction model to be trained based on the training data set to obtain a task fault prediction model; Performing a model performance evaluation on the task fault prediction model based on the verification data set to obtain a model performance evaluation result; When the model performance evaluation result meets the expected conditions, performing a model performance test on the task fault prediction model based on the test data set to obtain a model performance test result; When the model performance test results meet the expected results, a final target task fault prediction model for task fault prediction is obtained.

7. The method according to claim 6, characterized in that After obtaining the final target task fault prediction model for task fault prediction, the method further includes: Performing fault prediction on the pipeline task to be processed based on the target task fault prediction model to obtain a predicted fault task of the pipeline task to be processed; Obtaining a predicted fault node of the predicted fault task in the pipeline task to be processed; During the processing of the pending pipeline task, a target task status snapshot of the pending pipeline task is generated based on the task resource status information of the pending pipeline task at the designated processing node of the pending pipeline task and the predicted fault node, so that in the event of a fault in the pending pipeline task, a state rollback can be performed based on the target task status snapshot.

8. A pipeline task processing device, characterized in that: include: A task status snapshot generation module is used to generate a task status snapshot of the pipeline task based on the task resource status information of the pipeline task at a designated processing node of the pipeline task during the pipeline task processing; A first task determination module, configured to determine a first task having a fault in the pipeline tasks when a fault is detected in the pipeline tasks; A task status rollback module, configured to perform a status rollback on the task resource status of the first task based on the task status snapshot; The first task processing module is used to process the first task based on the task resource state of the first task after rolling back.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-7 when executing a program stored in a memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images