Task processing method, device and equipment and storage medium thereof

Abstract

The embodiment of the invention belongs to the technical field of operation and maintenance management, is applied to a data migration scene, and relates to a task processing method, device and equipment and a storage medium thereof.The task processing method comprises the steps that when the execution state of a data migration task is migration completion, a migration result comparison instruction is sent; comparing whether the simulation processing result is consistent with the actual processing result or not; if the comparison results of the second service cluster and the first service cluster on all the service items are consistent, the data migration task is successfully migrated; otherwise, performing exception investigation on the data migration result based on a preset task investigation strategy. When the task processing method is applied to a financial service data migration scene, due to the fact that data and service items involved in financial services are complex, operation and maintenance management personnel can be assisted to quickly recognize whether data migration from an original cluster to a new cluster is successful or not, it is guaranteed that the new cluster is put into use quickly, and the efficiency of data migration is improved. And a large amount of processing time consumed by data migration between clusters is avoided.

Description

Technical Field

[0001] The present application relates to the field of operation and maintenance management technology, and is applied to data middle-end migration scenarios, and in particular to a task processing method, device, equipment and storage medium thereof. Background Art

[0002] With the rapid development of financial services, the amount of data in the business has continued to increase, and the corresponding data management costs have also exploded. The cost of data management not only involves the workload, but also involves the problem of unreasonable use of computer resources. In order to reasonably use existing data resources and quickly deploy business, it is often necessary to use the existing data center as the source data center to migrate the entire data center to the new business cluster to provide data support for the new business.

[0003] Currently, querying the scheduling status of tasks through the page provided by the scheduling platform, i.e., failed, not started, waiting, stopped, offline, etc., involves checking the results of different batches, which makes the query time-consuming. Moreover, when the table to be analyzed involves batches that are not established by yourself, you need to know the batch name to batch query and export data. Therefore, a lot of time consumption and workload are added, and it is impossible to quickly screen migration anomalies, which affects the early commissioning of the new cluster. Summary of the invention

[0004] The purpose of the embodiments of the present application is to propose a task processing method, device, equipment and storage medium thereof to solve the problem that when performing inter-cluster migration on the existing data center, a lot of time and workload are added to task scheduling and query, and migration anomaly screening cannot be performed quickly, which affects the early commissioning of the new cluster.

[0005] In order to solve the above technical problems, the present application embodiment provides a task processing method, which adopts the following technical solutions:

[0006] A task processing method comprises the following steps:

[0007] According to the data migration instructions, start the preset data migration task;

[0008] When the execution status of the data migration task is migration completed, sending a migration result comparison instruction;

[0009] Based on the migration result comparison instruction, start the second service cluster to simulate and execute the same service item as the first service cluster, wherein the first service cluster includes the original service cluster in the data migration task, and the second service cluster includes the new service cluster pointed to by the data migration task;

[0010] Obtaining an actual processing result of the first service cluster for the service item;

[0011] Obtaining a simulation processing result of the second service cluster for the service item;

[0012] Comparing whether the simulation processing result and the actual processing result are consistent, and obtaining a comparison result;

[0013] If the comparison results of the second service cluster and the first service cluster in all service items are consistent, the data migration task is successfully migrated;

[0014] Otherwise, perform an exception check on the data migration results based on the preset task troubleshooting strategy.

[0015] Furthermore, before executing the step of starting the second service cluster to simulate and execute the same service item as the first service cluster based on the migration result comparison instruction, the method further includes:

[0016] According to the data service dependency, identify all data forms involved in executing the service item;

[0017] According to the dependency relationship between services, identify all upstream services involved in executing the service item, and all data forms involved in all upstream services respectively;

[0018] Based on all data forms involved in the service item, all upstream services involved in the service item, and all data forms respectively involved in all upstream services, construct an execution monitoring task based on the service item;

[0019] The execution monitoring task based on the service item is sent to a preset task scheduling management platform.

[0020] Furthermore, after executing the step of sending the execution monitoring task based on the service item to a preset task scheduling management platform, the method further includes:

[0021] Mapping all data forms involved in the service item, all upstream services involved in the service item, and all data forms involved in all upstream services to the target visualization interface according to a layer-by-layer correspondence relationship;

[0022] When receiving the migration result comparison instruction, pre-starting the execution monitoring task based on the service item to monitor the simulated execution of the second service cluster;

[0023] Based on the monitoring result, it is identified whether the simulation execution of the second service cluster is completed normally, and the monitoring result is mapped to the target visualization interface.

[0024] Furthermore, the step of obtaining the actual processing result of the first service cluster for the service item specifically includes:

[0025] identifying all data forms involved when the first service cluster actually processes the service item, all upstream services involved in the service item, and all data forms involved in all upstream services, to obtain a first identification result;

[0026] Based on the first recognition result, record the corresponding form name and upstream service name to obtain a first record result;

[0027] Identifying actual processing result feedback information obtained by the first service cluster performing actual processing on the service item;

[0028] The first recording result and the feedback information are collated as the actual processing result.

[0029] Furthermore, the step of obtaining the simulation processing result of the second service cluster for the service item specifically includes:

[0030] identifying all data forms involved when the second service cluster performs simulation processing on the service item, all upstream services involved in the service item, and all data forms involved in all upstream services, to obtain a second identification result;

[0031] Based on the second recognition result, record the corresponding form name and upstream service name to obtain a second recording result;

[0032] identifying simulation processing result feedback information obtained by the second service cluster performing simulation processing on the service item;

[0033] The second recording result and the feedback information are sorted as the simulation processing result.

[0034] Furthermore, after executing the step of obtaining the simulation processing result of the second service cluster for the service item, the method further includes:

[0035] According to the second recording result, marking processing is performed on the display information mapped in the target visualization interface;

[0036] If all data forms involved in the service item in the target visual interface, all upstream services involved in the service item, and all data forms involved in all upstream services are marked, the simulation execution of the second service cluster is completed normally;

[0037] If all data sheets involved in the service item in the target visualization interface, all upstream services involved in the service item, and all data sheets involved in all upstream services are not marked, the simulation execution of the second service cluster is not completed normally.

[0038] Furthermore, the step of performing abnormality troubleshooting on the data migration result based on the preset task troubleshooting strategy specifically includes:

[0039] If the simulation execution of the second service cluster is completed normally, identifying all data forms involved in the service item according to the second recording result, and performing abnormal data screening on all data forms involved in the service item;

[0040] If the simulation execution of the second service cluster is not completed normally, all upstream services involved in the service item are identified according to the second record result, and all upstream services are compared with all upstream services involved in the service item in the execution monitoring task; if, after comparison, it is identified that there are services upstream that are not executed normally, all data forms involved in the upstream services that are not executed normally are screened for abnormal data;

[0041] If, after comparison, it is identified that all upstream services are being executed normally, all data forms involved in the service items are screened for abnormal data.

[0042] In order to solve the above technical problems, the embodiment of the present application also provides a task processing device, which adopts the following technical solution:

[0043] A task processing device, comprising:

[0044] A data migration task starting module is used to start a preset data migration task according to a data migration instruction;

[0045] A migration result comparison instruction sending module, used for sending a migration result comparison instruction when the execution status of the data migration task is migration completed;

[0046] A second service cluster simulation execution module, configured to start the second service cluster to simulate and execute the same service item as the first service cluster based on the migration result comparison instruction, wherein the first service cluster includes the original service cluster in the data migration task, and the second service cluster includes the new service cluster pointed to by the data migration task;

[0047] An actual processing result acquisition module, used to acquire an actual processing result of the first service cluster for the service item;

[0048] A simulation processing result acquisition module, used to obtain the simulation processing result of the second service cluster for the service item;

[0049] A processing result consistency comparison module is used to compare whether the simulated processing result is consistent with the actual processing result to obtain a comparison result;

[0050] A migration success identification module, configured to identify that the data migration task is successfully migrated if the comparison results of the second service cluster and the first service cluster on all service items are consistent;

[0051] The exception troubleshooting module is used to perform exception troubleshooting on the data migration results based on the preset task troubleshooting strategy.

[0052] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0053] A computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the task processing method described above when executing the computer-readable instructions.

[0054] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0055] A computer-readable storage medium stores computer-readable instructions, which implement the steps of the task processing method described above when executed by a processor.

[0056] Compared with the prior art, the embodiments of the present application have the following beneficial effects:

[0057] The task processing method described in the embodiment of the present application starts the preset data migration task through the data migration instruction; when the execution status of the data migration task is migration completed, sends a migration result comparison instruction; based on the migration result comparison instruction, starts the second service cluster to simulate the execution of the same service item as the first service cluster; obtains the actual processing result of the first service cluster for the service item; obtains the simulation processing result of the second service cluster for the service item; compares whether the simulation processing result is consistent with the actual processing result to obtain a comparison result; if the comparison results of the second service cluster and the first service cluster on all service items are consistent, the data migration task is successfully migrated; otherwise, the data migration result is checked for abnormalities based on the preset task troubleshooting strategy. The task processing method described in the present application is applied to the financial business data migration scenario. Since the data and service items involved in the financial business are relatively complex, it can assist the operation and maintenance management personnel to quickly identify whether the data migration from the original cluster to the new cluster is successful, thereby ensuring the rapid commissioning of the new cluster and avoiding a large amount of processing time spent on data migration between clusters. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0059] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0060] Figure 2 is a flow chart of an embodiment of a task processing method according to the present application;

[0061] Figure 3 It is a flowchart of a specific embodiment of constructing an execution monitoring task in the task processing method described in this application;

[0062] Figure 4 It is a flowchart of a specific embodiment of visualizing monitoring results in the task processing method described in this application;

[0063] Figure 5 yes Figure 2 A flowchart of a specific embodiment of step 204 is shown;

[0064] Figure 6 yes Figure 2 A flowchart of a specific embodiment of step 205 is shown;

[0065] Figure 7 It is a flowchart of a specific embodiment of confirming simulation execution results in combination with visualization processing results in the task processing method described in the present application;

[0066] Figure 8 yes Figure 2 A flowchart of a specific embodiment of step 208 is shown;

[0067] Fig. 9 is a structural schematic diagram of an embodiment of a task processing device according to the present application;

[0068] Fig.10 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0069] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0070] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0071] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0072] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables.

[0073] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0074] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to a laptop computer 1011, a tablet computer 1012 or a mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0075] The server 103 may be a server that provides various services, such as a background server that provides support for web pages displayed on the terminal device 101 .

[0076] It should be noted that the task processing method provided in the embodiment of the present application is generally executed by a server, and accordingly, the task processing device is generally set in the server.

[0077] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0078] Continue to refer Figure 2 , shows a flow chart of an embodiment of a task processing method according to the present application. The task processing method comprises the following steps:

[0079] Step 201: start a preset data migration task according to a data migration instruction.

[0080] In this embodiment, the data migration instructions include migration instructions for the data middle platform, which is composed of a data warehouse and a data service middleware. The data warehouse is used to store data content, and the data service middleware is a data business support component. The data business includes decision-making business, prediction business, classification business, etc. based on the data content in the data warehouse. It should be understood that the data migration instructions are generally for financial business big data scenarios. For example, through the data migration instructions, the data middle platform in the original cluster A of the financial business is migrated as a whole to the new cluster B. At this time, it not only involves the migration of the data warehouse, but also involves the migration of the data service middleware.

[0081] Step 202: When the execution status of the data migration task is migration completed, a migration result comparison instruction is sent.

[0082] When the execution status of the data migration task is migration completion, a migration result comparison instruction is sent to verify the migration result of the data migration task, thereby determining whether the migration is successful.

[0083] Step 203: Based on the migration result comparison instruction, start the second service cluster to simulate and execute the same service item as the first service cluster, wherein the first service cluster includes the original service cluster in the data migration task, and the second service cluster includes the new service cluster pointed to by the data migration task.

[0084] Step 204: Obtain actual processing results of the first service cluster for the service item.

[0085] Step 205: Obtain a simulation processing result of the second service cluster for the service item.

[0086] Through the migration result comparison instruction, the second service cluster is started to simulate and execute the same service item as the first service cluster, and the processing results of the two service clusters before and after migration for the same service item are obtained, so as to determine whether the data migration task is successfully migrated.

[0087] Step 206: compare the simulation processing result and the actual processing result to see if they are consistent, and obtain a comparison result.

[0088] Step 207: If the comparison results of all service items of the second service cluster and the first service cluster are consistent, the data migration task is successfully migrated.

[0089] Step 208: Otherwise, perform an abnormality check on the data migration result based on the preset task troubleshooting strategy.

[0090] Specifically, when the comparison results of the second service cluster and the first service cluster on at least one service item are inconsistent, the data migration results are checked for abnormalities based on the preset task troubleshooting strategy. It should be understood that since it is a migration task, all service processing results of the first service cluster here are always assumed to be correct by default. By checking the data migration results for abnormalities through the preset task troubleshooting strategy, abnormal items in the second service cluster can be quickly identified for processing, so that the second service cluster can be put into use as soon as possible.

[0091] The task processing method described in this application is applied to the financial business data migration scenario. Since the data and service items involved in financial business are relatively complex, it can assist operation and maintenance management personnel to quickly identify whether the data migration from the original cluster to the new cluster is successful, thereby ensuring the rapid commissioning of the new cluster and avoiding a large amount of processing time for data migration between clusters.

[0092] In this embodiment, a preset data migration task is started through a data migration instruction; when the execution status of the data migration task is migration completion, a migration result comparison instruction is sent; based on the migration result comparison instruction, the second service cluster is started to simulate the execution of the same service item as the first service cluster; the actual processing result of the first service cluster for the service item is obtained; the simulation processing result of the second service cluster for the service item is obtained; the simulation processing result is compared with the actual processing result to obtain a comparison result; if the comparison results of the second service cluster and the first service cluster on all service items are consistent, the data migration task is successfully migrated; otherwise, the data migration result is checked for abnormalities based on the preset task troubleshooting strategy. The task processing method described in this application is applied to the data migration scenario of financial or medical and health services. Since the data and service items involved in financial or medical and health services are relatively complex, it can assist operation and maintenance management personnel to quickly identify whether the data migration from the original cluster to the new cluster is successful, thereby ensuring the rapid commissioning of the new cluster and avoiding a large amount of processing time for data migration between clusters.

[0093] Continue to refer Figure 3 In some optional implementations, before step 203, a step of constructing an execution monitoring task is also included. Figure 3 This is a flowchart of a specific embodiment of constructing an execution monitoring task in the task processing method described in this application, including the following steps:

[0094] Step 301, identifying all data forms involved in executing the service item according to the data service dependency;

[0095] Step 302, identifying all upstream services involved in executing the service item and all data forms involved in each of the upstream services according to the dependency relationship between the services;

[0096] Step 303: constructing an execution monitoring task based on the service item based on all data forms involved in the service item, all upstream services involved in the service item, and all data forms involved in all upstream services respectively;

[0097] Specifically, according to the data forms involved in each service item and the fields in the data forms that need to be called, the execution monitoring tasks corresponding to each service item are constructed. When the execution monitoring tasks are initially constructed, the call statistics parameters are reserved, the corresponding data forms are initialized to be called 0 times and the number of times each field in each data form is called is initialized to be 0 times. Each time the data form is called, the number of times the data form is called is accumulated by 1, and the number of times the called field is accumulated by 1. This facilitates the subsequent monitoring of the execution of the service items according to the above call statistics parameters.

[0098] Step 304: Send the execution monitoring task based on the service item to a preset task scheduling management platform.

[0099] By combining all data forms involved in the service item, all upstream services involved in the service item, and all data forms involved in all upstream services respectively, an execution monitoring task based on the service item is constructed, which realizes the execution monitoring of the target service item from the perspective of the data form and all upstream service items. When an abnormality occurs in the target service item, the data service dependency and the dependency between services are used to perform anomaly screening and timely identify migration anomalies.

[0100] Continue to refer Figure 4 In some optional implementations, after step 304, a step of visualizing the monitoring results is also included. Figure 4 It is a flowchart of a specific embodiment of visualizing monitoring results in the task processing method described in this application, comprising the following steps:

[0101] Step 401, mapping all data forms involved in the service item, all upstream services involved in the service item, and all data forms involved in all upstream services to a target visualization interface according to a layer-by-layer correspondence relationship;

[0102] Specifically, assume that when service item D is executed, three data forms are involved, namely, table 1, table 2 and table 3. All its upstream service items include service item A, service item B and service item C. Service item A involves two data forms, namely, table 1 and table 4. Service item B involves one data form, namely, table 5. Service item C involves four data forms, namely, table 4, table 5, table 6 and table 7. Service item A is the upstream service of service item B and service item C. At this time, the service name of service item A and the names of data forms table 1 and table 4 are displayed at the top layer of the target visualization interface. The service name of service item B and the name of data form table 5 are displayed at the middle layer of the target visualization interface. The service name of service item C and the names of data forms table 4, table 5, table 6 and table 7 are displayed. Finally, the service name of service item D and the names of data forms table 1, table 2 and table 3 are displayed at the corresponding lower layer of the target visualization interface. The purpose is to facilitate operation and maintenance personnel to intuitively monitor and analyze service item calls in combination with the visualization interface.

[0103] Step 402, when receiving the migration result comparison instruction, pre-starting the execution monitoring task based on the service item to monitor the simulated execution of the second service cluster;

[0104] Specifically, when the migration result comparison instruction is received, it indicates that the data middle platform migration from the first service cluster to the second service cluster has been completed. At this time, by starting the execution monitoring task based on the service item to monitor the simulated execution of the second service cluster, it is convenient to detect whether the second service cluster can replace the first service cluster and put into actual use.

[0105] Step 403: Based on the monitoring result, it is identified whether the simulation execution of the second service cluster is completed normally, and the monitoring result is mapped to the target visualization interface.

[0106] Specifically, the step of mapping the monitoring results to the target visualization interface includes: marking all data forms and all upstream services involved in the simulation execution of the second service cluster, all data forms involved in all upstream services, and data in specific forms, so as to enable subsequent rapid identification of the simulation execution process.

[0107] Continue to refer Figure 5 , Figure 5 yes Figure 2 The flowchart of a specific embodiment of step 204 shown includes the following steps:

[0108] Step 501, identifying all data forms involved in actual processing of the service item by the first service cluster, all upstream services involved in the service item, and all data forms involved in each of the upstream services, to obtain a first identification result;

[0109] Specifically, the first service cluster is an existing cluster in use. Combined with its actual execution results, a first identification result is obtained. Assume that when service item D is executed, it is identified that three data forms are involved, namely, table 1, table 2 and table 3. All its upstream service items include service item A, service item B and service item C. Service item A involves two data forms, namely, table 1 and table 4. Service item B involves one data form, namely, table 5. Service item C involves four data forms, namely, table 4, table 5, table 6 and table 7. Service item A is the upstream service of service item B and service item C.

[0110] Step 502: based on the first recognition result, record the corresponding form name and upstream service name to obtain a first recording result;

[0111] Specifically, according to the first recognition result, the first record result is generated as a whole, that is, the upstream service names include service item A, service item B, service item C and service item D, and the table names include table1, table2, table3, table4, table5, table6 and table7. This is to facilitate the subsequent comparative analysis with the simulation processing results.

[0112] Step 503, identifying actual processing result feedback information obtained by the first service cluster performing actual processing on the service item;

[0113] Step 504: Arrange the first recording result and the feedback information as the actual processing result.

[0114] By obtaining the actual processing results of the original cluster, it is easy to identify migration anomalies of the new cluster.

[0115] Continue to refer Figure 6 , Figure 6 yes Figure 2 The flowchart of a specific embodiment of step 205 shown includes the following steps:

[0116] Step 601, identifying all data forms involved when the second service cluster performs simulation processing on the service item, all upstream services involved in the service item, and all data forms involved in all upstream services, to obtain a second identification result;

[0117] Step 602: based on the second recognition result, record the corresponding form name and upstream service name to obtain a second recording result;

[0118] Step 603, identifying simulation processing result feedback information obtained by the second service cluster performing simulation processing on the service item;

[0119] Step 604: Arrange the second recording result and the feedback information as the simulation processing result.

[0120] By obtaining the simulation processing results of the new cluster, it is easy to discover the migration anomalies of the new cluster.

[0121] Continue to refer Figure 7 In some optional implementations, after step 205, a step of confirming the simulation execution result in combination with the visualization processing result is also included. Figure 7 It is a flowchart of a specific embodiment of confirming the simulation execution result in combination with the visual processing result in the task processing method described in the present application, comprising the following steps:

[0122] Step 701: marking the display information mapped in the target visualization interface according to the second recording result;

[0123] Specifically, according to the second recording result, the display information mapped in the target visualization interface is marked, that is, according to the simulation execution result of the second service cluster, the data form and service name actually involved in the first service cluster mapped in the target visualization interface are marked.

[0124] Step 702: If all data forms involved in the service item in the target visual interface, all upstream services involved in the service item, and all data forms involved in all upstream services are marked, the simulation execution of the second service cluster is completed normally;

[0125] Specifically, if the relevant service item name and data form name in the target visual interface are both marked, that is, consistent with the actual processing result of the first service cluster, it means that the simulation execution of the second service cluster is completed normally.

[0126] Step 703, if all data forms involved in the service item in the target visualization interface, all upstream services involved in the service item, and all data forms involved in all upstream services are not marked, the simulation execution of the second service cluster is not completed normally.

[0127] This embodiment combines a visual marking method to confirm whether the simulation execution of the second service cluster is completed normally, which is more intuitive and convenient for operation and maintenance personnel to view, avoiding the previous repeated search for execution logs, improving the efficiency of operation and maintenance screening, and assisting operation and maintenance personnel to promptly discover data migration anomalies.

[0128] In this embodiment, if the comparison results of the second service cluster and the first service cluster on all service items are inconsistent, refer to Figure 8 , to screen for migration abnormalities.

[0129] Continue to refer Figure 8 , Figure 8 yes Figure 2 The flowchart of a specific embodiment of step 208 shown includes the following steps:

[0130] Step 801: if the simulation execution of the second service cluster is completed normally, all data forms involved in the service item are identified according to the second recording result, and abnormal data screening is performed on all data forms involved in the service item;

[0131] It should be understood that if the simulation execution of the second service cluster is completed normally, it means that there is no abnormality in the execution logic of the new cluster after the data migration. The most likely possibility is that there is an abnormality in the specific data content in the data form migration of the service item. Therefore, all data forms involved in the service item are directly identified, and all data forms involved in the service item are screened for abnormal data.

[0132] Step 802: If the simulation execution of the second service cluster is not completed normally, all upstream services involved in the service item are identified according to the second recording result, and all upstream services are compared with all upstream services involved in the service item in the execution monitoring task;

[0133] Similarly, it should be understood that if the simulated execution of the second service cluster is not completed normally, it means that there may be anomalies in the execution logic of the new cluster after the data migration, and the most likely possibility is that there is an execution anomaly in the upstream service item of the service item. Therefore, the scope of anomaly screening is expanded, and all upstream services involved in the service item are identified according to the second record results. Then, migration anomaly screening is performed in combination with the execution status of all upstream services.

[0134] Step 803: if it is identified through comparison that an upstream service is not executed normally, then all data forms involved in the upstream service that is not executed normally are screened for abnormal data;

[0135] Step 804: If, after comparison, it is identified that all upstream services are executed normally, all data forms involved in the service items are screened for abnormal data.

[0136] This application starts a preset data migration task through a data migration instruction; when the execution status of the data migration task is migration completed, a migration result comparison instruction is sent; based on the migration result comparison instruction, the second service cluster is started to simulate the execution of the same service item as the first service cluster; the actual processing result of the first service cluster for the service item is obtained; the simulation processing result of the second service cluster for the service item is obtained; the simulation processing result is compared with the actual processing result to obtain a comparison result; if the comparison results of the second service cluster and the first service cluster on all service items are consistent, the data migration task is successfully migrated; otherwise, the data migration result is checked for abnormalities based on the preset task troubleshooting strategy. The task processing method described in this application is applied to the financial business data migration scenario. Since the data and service items involved in the financial business are relatively complex, it can assist the operation and maintenance management personnel to quickly identify whether the data migration from the original cluster to the new cluster is successful, thereby ensuring the rapid commissioning of the new cluster and avoiding a large amount of processing time for data migration between clusters.

[0137] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0138] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0139] In an embodiment of the present application, a preset data migration task is started through a data migration instruction; when the execution status of the data migration task is migration completed, a migration result comparison instruction is sent; based on the migration result comparison instruction, a second service cluster is started to simulate the execution of the same service item as the first service cluster; the actual processing result of the first service cluster for the service item is obtained; the simulation processing result of the second service cluster for the service item is obtained; the simulation processing result is compared with the actual processing result to obtain a comparison result; if the comparison results of the second service cluster and the first service cluster on all service items are consistent, the data migration task is successfully migrated; otherwise, the data migration result is checked for abnormalities based on the preset task troubleshooting strategy. The task processing method described in the present application is applied to the financial business data migration scenario. Since the data and service items involved in the financial business are relatively complex, it can assist the operation and maintenance management personnel to quickly identify whether the data migration from the original cluster to the new cluster is successful, thereby ensuring the rapid commissioning of the new cluster and avoiding a large amount of processing time for data migration between clusters.

[0140] Further references Fig. 9 , as a response to the above Figure 2 In order to realize the method shown in the figure, the present application provides an embodiment of a task processing device, and the device embodiment is Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0141] like Fig. 9 As shown, the task processing device 900 described in this embodiment includes: a data migration task startup module 901, a migration result comparison instruction sending module 902, a second service cluster simulation execution module 903, an actual processing result acquisition module 904, a simulation processing result acquisition module 905, a processing result consistency comparison module 906, a migration success identification module 907 and an abnormality detection module 908.

[0142] in:

[0143] The data migration task starting module 901 is used to start the preset data migration task according to the data migration instruction;

[0144] A migration result comparison instruction sending module 902 is used to send a migration result comparison instruction when the execution status of the data migration task is migration completed;

[0145] A second service cluster simulation execution module 903, configured to start the second service cluster to simulate and execute the same service item as the first service cluster based on the migration result comparison instruction, wherein the first service cluster includes the original service cluster in the data migration task, and the second service cluster includes the new service cluster pointed to by the data migration task;

[0146] An actual processing result acquisition module 904 is used to acquire an actual processing result of the first service cluster for the service item;

[0147] A simulation processing result acquisition module 905 is used to acquire a simulation processing result of the second service cluster for the service item;

[0148] A processing result consistency comparison module 906 is used to compare whether the simulation processing result and the actual processing result are consistent, and obtain a comparison result;

[0149] A migration success identification module 907, configured to determine that the data migration task is successfully migrated if the comparison results of the second service cluster and the first service cluster on all service items are consistent;

[0150] The abnormality troubleshooting module 908 is used to perform abnormality troubleshooting on the data migration result based on a preset task troubleshooting strategy.

[0151] This application starts a preset data migration task through a data migration instruction; when the execution status of the data migration task is migration completed, a migration result comparison instruction is sent; based on the migration result comparison instruction, the second service cluster is started to simulate the execution of the same service item as the first service cluster; the actual processing result of the first service cluster for the service item is obtained; the simulation processing result of the second service cluster for the service item is obtained; the simulation processing result is compared with the actual processing result to obtain a comparison result; if the comparison results of the second service cluster and the first service cluster on all service items are consistent, the data migration task is successfully migrated; otherwise, the data migration result is checked for abnormalities based on the preset task troubleshooting strategy. The task processing method described in this application is applied to the financial business data migration scenario. Since the data and service items involved in the financial business are relatively complex, it can assist the operation and maintenance management personnel to quickly identify whether the data migration from the original cluster to the new cluster is successful, thereby ensuring the rapid commissioning of the new cluster and avoiding a large amount of processing time for data migration between clusters.

[0152] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0153] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0154] To solve the above technical problems, the present application also provides a computer device. Fig.10 , Fig.10 This is a basic structural block diagram of the computer device in this embodiment.

[0155] The computer device 10 includes a memory 10a, a processor 10b, and a network interface 10c which are interconnected and communicated via a system bus. It should be noted that: Fig.10 Only a computer device 10 having a component memory 10a, a processor 10b, and a network interface 10c is shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to a microprocessor, an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital processor (DSP), an embedded device, etc.

[0156] The computer device may be a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with the user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0157] The memory 10a includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 10a can be an internal storage unit of the computer device 10, such as a hard disk or memory of the computer device 10. In other embodiments, the memory 10a can also be an external storage device of the computer device 10, such as a plug-in hard disk equipped on the computer device 10, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 10a can also include both the internal storage unit of the computer device 10 and its external storage device. In this embodiment, the memory 10a is generally used to store the operating system and various application software installed on the computer device 10, such as a computer-readable instruction of a task processing method, etc. In addition, the memory 10a can also be used to temporarily store various types of data that have been output or are to be output.

[0158] The processor 10b may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 10b is generally used to control the overall operation of the computer device 10. In this embodiment, the processor 10b is used to run the computer-readable instructions stored in the memory 10a or process data, such as computer-readable instructions for running the task processing method.

[0159] The network interface 10c may include a wireless network interface or a wired network interface. The network interface 10c is generally used to establish a communication connection between the computer device 10 and other electronic devices.

[0160] The computer device proposed in this embodiment belongs to the field of operation and maintenance management technology and is applied to the data middle station migration scenario. This application starts the preset data migration task through the data migration instruction; when the execution status of the data migration task is completed, the migration result comparison instruction is sent; based on the migration result comparison instruction, the second service cluster is started to simulate the execution of the same service item as the first service cluster; the actual processing result of the first service cluster for the service item is obtained; the simulation processing result of the second service cluster for the service item is obtained; the simulation processing result is compared with the actual processing result to obtain a comparison result; if the comparison results of the second service cluster and the first service cluster on all service items are consistent, the data migration task is successfully migrated; otherwise, the data migration result is checked for abnormalities based on the preset task troubleshooting strategy. The task processing method described in this application is applied to the financial business data migration scenario. Since the data and service items involved in the financial business are relatively complex, it can assist the operation and maintenance management personnel to quickly identify whether the data migration from the original cluster to the new cluster is successful, thereby ensuring the rapid commissioning of the new cluster and avoiding a lot of processing time for data migration between clusters.

[0161] The present application also provides another implementation, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by a processor to enable the processor to perform the steps of the task processing method as described above.

[0162] The computer-readable storage medium proposed in this embodiment belongs to the field of operation and maintenance management technology and is applied to the data middle station migration scenario. This application starts the preset data migration task through the data migration instruction; when the execution status of the data migration task is completed, the migration result comparison instruction is sent; based on the migration result comparison instruction, the second service cluster is started to simulate the execution of the same service item as the first service cluster; the actual processing result of the first service cluster for the service item is obtained; the simulation processing result of the second service cluster for the service item is obtained; the simulation processing result is compared with the actual processing result to obtain a comparison result; if the comparison results of the second service cluster and the first service cluster on all service items are consistent, the data migration task is successfully migrated; otherwise, the data migration result is checked for abnormalities based on the preset task troubleshooting strategy. The task processing method described in this application is applied to the financial business data migration scenario. Since the data and service items involved in the financial business are relatively complex, it can assist the operation and maintenance management personnel to quickly identify whether the data migration from the original cluster to the new cluster is successful, thereby ensuring the rapid commissioning of the new cluster and avoiding a large amount of processing time for data migration between clusters.

[0163] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0164] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A task processing method, characterized in that: The steps include: According to the data migration instructions, start the preset data migration task; When the execution status of the data migration task is migration completed, sending a migration result comparison instruction; Based on the migration result comparison instruction, start the second service cluster to simulate and execute the same service item as the first service cluster, wherein the first service cluster includes the original service cluster in the data migration task, and the second service cluster includes the new service cluster pointed to by the data migration task; Obtaining an actual processing result of the first service cluster for the service item; Obtaining a simulation processing result of the second service cluster for the service item; Comparing whether the simulation processing result and the actual processing result are consistent, and obtaining a comparison result; If the comparison results of the second service cluster and the first service cluster in all service items are consistent, the data migration task is successfully migrated; Otherwise, perform an exception check on the data migration results based on the preset task troubleshooting strategy.

2. The task processing method according to claim 1, characterized in that: Before executing the step of starting the second service cluster to simulate and execute the same service item as the first service cluster based on the migration result comparison instruction, the method further includes: According to the data service dependency, identify all data forms involved in executing the service item; According to the dependency relationship between services, identify all upstream services involved in executing the service item, and all data forms involved in all upstream services respectively; Based on all data forms involved in the service item, all upstream services involved in the service item, and all data forms respectively involved in all upstream services, construct an execution monitoring task based on the service item; The execution monitoring task based on the service item is sent to a preset task scheduling management platform.

3. The task processing method according to claim 2, characterized in that: After executing the step of sending the execution monitoring task based on the service item to a preset task scheduling management platform, the method further includes: Mapping all data forms involved in the service item, all upstream services involved in the service item, and all data forms involved in all upstream services to the target visualization interface according to a layer-by-layer correspondence relationship; When receiving the migration result comparison instruction, pre-starting the execution monitoring task based on the service item to monitor the simulated execution of the second service cluster; Based on the monitoring result, it is identified whether the simulation execution of the second service cluster is completed normally, and the monitoring result is mapped to the target visualization interface.

4. The task processing method according to claim 1, characterized in that: The step of obtaining the actual processing result of the first service cluster for the service item specifically includes: identifying all data forms involved when the first service cluster actually processes the service item, all upstream services involved in the service item, and all data forms involved in all upstream services, to obtain a first identification result; Based on the first recognition result, record the corresponding form name and upstream service name to obtain a first record result; Identifying actual processing result feedback information obtained by the first service cluster performing actual processing on the service item; The first recording result and the feedback information are collated as the actual processing result.

5. The task processing method according to claim 3, characterized in that: The step of obtaining the simulation processing result of the second service cluster for the service item specifically includes: identifying all data forms involved when the second service cluster performs simulation processing on the service item, all upstream services involved in the service item, and all data forms involved in all upstream services, to obtain a second identification result; Based on the second recognition result, record the corresponding form name and upstream service name to obtain a second recording result; identifying simulation processing result feedback information obtained by the second service cluster performing simulation processing on the service item; The second recording result and the feedback information are sorted as the simulation processing result.

6. The task processing method according to claim 5, characterized in that: After executing the step of obtaining the simulation processing result of the second service cluster for the service item, the method further includes: According to the second recording result, marking processing is performed on the display information mapped in the target visualization interface; If all data forms involved in the service item in the target visual interface, all upstream services involved in the service item, and all data forms involved in all upstream services are marked, the simulation execution of the second service cluster is completed normally; If all data sheets involved in the service item in the target visualization interface, all upstream services involved in the service item, and all data sheets involved in all upstream services are not marked, the simulation execution of the second service cluster is not completed normally.

7. The task processing method according to claim 6, characterized in that: The step of performing abnormality troubleshooting on the data migration result based on the preset task troubleshooting strategy specifically includes: If the simulation execution of the second service cluster is completed normally, identifying all data forms involved in the service item according to the second recording result, and performing abnormal data screening on all data forms involved in the service item; If the simulation execution of the second service cluster is not completed normally, identifying all upstream services involved in the service item according to the second recording result, and comparing all upstream services with all upstream services involved in the service item in the execution monitoring task; If, after comparison, it is identified that an upstream service is not executed normally, all data forms involved in the upstream service that is not executed normally will be screened for abnormal data; If, after comparison, it is identified that all upstream services are being executed normally, all data forms involved in the service items are screened for abnormal data.

8. A task processing device, characterized in that: include: A data migration task starting module is used to start a preset data migration task according to a data migration instruction; A migration result comparison instruction sending module, used for sending a migration result comparison instruction when the execution status of the data migration task is migration completed; A second service cluster simulation execution module, configured to start a second service cluster to simulate and execute the same service item as the first service cluster based on the migration result comparison instruction, wherein the first service cluster includes the original service cluster in the data migration task, and the second service cluster includes the new service cluster pointed to by the data migration task; An actual processing result acquisition module, used to acquire an actual processing result of the first service cluster for the service item; A simulation processing result acquisition module, used to obtain the simulation processing result of the second service cluster for the service item; A processing result consistency comparison module is used to compare whether the simulated processing result is consistent with the actual processing result to obtain a comparison result; A migration success identification module, configured to identify that the data migration task is successfully migrated if the comparison results of the second service cluster and the first service cluster on all service items are consistent; The exception troubleshooting module is used to perform exception troubleshooting on the data migration results based on the preset task troubleshooting strategy.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the task processing method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the task processing method according to any one of claims 1 to 7 are implemented.

Images