A method and system for managing engine tasks

Abstract

The present invention relates to the technical field of task management, specifically to a method and system for managing engine tasks, the method comprising the following steps: obtaining source task data information of a source system and multiple engines, establishing a communication channel between multiple engines and the source system; splitting the source task data information to obtain multiple subtasks, each subtask corresponding to a unique identifier, taking the subtask to be transmitted as a target transmission subtask, and matching a communication channel for each target transmission subtask; the communication channel comprises a task transmission channel and a synchronous transmission channel, the task transmission channel and the synchronous transmission channel correspond one to one; setting auxiliary information for each task transmission channel, the auxiliary information comprises a verification window, a verification point, and a verification chain, and the verification window comprises two verification windows. It can effectively improve the accuracy of the engine executing tasks.

Description

Technical Field

[0001] The present invention relates to the technical field of task management, and in particular to a method and system for managing engine tasks. Background Art

[0002] With the development of big data processing technology, the application of engine systems is becoming more and more common. The current engine system needs to rely on different third-party engines to process tasks according to the different functions of the engine, and the engine system needs to manage tasks.

[0003] The existing engine task management is to connect multiple engines to the source system, use the source system to transfer tasks to the corresponding engines, and execute the corresponding tasks through the computing platform. However, the tasks may be changed before being sent to the engine, so that the engine may accept different versions of tasks or the tasks may be executed multiple times, which may lead to inaccurate tasks executed by the engine.

[0004] To this end, we propose an engine task management method and system to solve the above problems. Summary of the invention

[0005] The purpose of the present invention is to provide a method and system for managing engine tasks to solve the problems raised in the above background technology.

[0006] To achieve the above object, the present invention provides the following technical solution: a method and system for managing engine tasks, the method comprising the following steps:

[0007] S1: Obtain source task data information and multiple engines from the source system, and establish communication channels between multiple engines and the source system;

[0008] S2: Split the source task data information to obtain multiple subtasks, each subtask corresponds to a unique identifier, take the subtask to be transmitted as the target transmission subtask, and match a communication channel to each target transmission subtask; the communication channel includes a task transmission channel and a synchronous transmission channel, and the task transmission channel corresponds to the synchronous transmission channel one by one;

[0009] S3: setting auxiliary information for each task transmission channel, wherein the auxiliary information includes a verification window pair, a verification point, and a verification chain, wherein the verification window pair includes two verification windows, and a verification chain is formed between the two verification windows based on the verification point;

[0010] S4: transmitting the auxiliary information to the corresponding engine based on the synchronous transmission channel; transmitting the target transmission subtask to the corresponding engine based on the task transmission channel to obtain the target execution subtask; verifying the target execution subtask based on the auxiliary information;

[0011] S5: In response to determining that the verification result is unqualified, determining a modified target execution subtask based on the verification window, transmitting and verifying the modified target execution subtask until the verification result is determined to be qualified; in response to determining that the verification result is qualified, executing the corresponding target execution subtask through the engine.

[0012] After the task is successfully transmitted, the task to be executed is verified by verifying the unique identifier corresponding to the window and the task, to prevent the task to be executed by the engine from changing during the task transmission process, so that the engine accepts different versions of tasks or the task is executed multiple times, which can effectively improve the accuracy of the engine's task execution.

[0013] Preferably, the step of matching a communication channel to each target transmission subtask includes:

[0014] Acquire multiple engines, multiple target transmission subtasks, and multiple communication channels; functionally classify the multiple engines to obtain multiple functional types;

[0015] A ranking plate is constructed based on multiple function types, and multiple engine points are set on the ranking plate, each engine point corresponding to a function type;

[0016] Obtain the target transmission subtask's requirement type for the engine, and obtain the target requirement function type;

[0017] Extracting engine points corresponding to target required functional types based on multiple functional types;

[0018] Obtain the execution requirements of the target transmission subtask, and determine the engine located at the engine point as a standby engine based on the execution requirements;

[0019] The prepared engine is matched with the engine point, and the communication channel between the engine point and the source system is matched with the target transmission subtask.

[0020] Preferably, the step of obtaining the execution requirements of the target transmission subtask and determining the engine located at the engine point based on the execution requirements includes:

[0021] Get multiple engines of the same function type to get a set of engines of the same type;

[0022] A disk queue is built based on a set of similar engines. Multiple sub-points are set on the disk queue, and each sub-point corresponds to an engine.

[0023] Obtaining the required execution index of the target transmission subtask and the actual execution index of the engine, calculating the similarity between the required execution index and the actual execution index of each engine, and obtaining multiple similarity values;

[0024] Extracting a similarity value with the largest value based on multiple similarity values ​​to obtain a maximum similarity value, and determining an actual execution indicator corresponding to the required execution indicator based on the maximum similarity value as a target execution indicator;

[0025] The sub-point of the engine corresponding to the target execution index is overlapped with the engine point to obtain the engine located at the engine point.

[0026] Preferably, the step of calculating the similarity between the required execution index and the actual execution index of each engine includes:

[0027] Obtain the required execution indicators of the target transmission subtask, evaluate the actual execution indicators of each engine, the execution indicators at least include the degree of privacy protection and the degree of execution accuracy, set a corresponding weight value for each execution indicator, judge the similarity between the required execution indicator and the actual execution indicator according to the corresponding weight value, and select the corresponding engine for the engine point according to the similarity value;

[0028] The calculation formula corresponding to the similarity is: Among them, Cys is the privacy protection degree of the engine, Cjs is the execution accuracy of the engine, Cyx is the required privacy protection degree of the target transmission subtask, Cjx is the required execution accuracy of the target transmission subtask, and a1, a2, a3, and a4 are proportional factors.

[0029] Preferably, the step of verifying the target execution subtask based on the auxiliary information includes:

[0030] Acquire multiple target transmission subtasks, match the multiple target transmission subtasks with the task transmission channels, and obtain a first matching result;

[0031] Based on the first matching result, multiple target transmission subtasks are transmitted through corresponding task transmission channels, and the task transmission channels corresponding to the target transmission subtasks are recorded as target task transmission channels;

[0032] After the multiple target transmission subtasks are successfully transmitted, the matching results of the multiple target transmission subtasks and the task transmission channels are obtained again to obtain a second matching result;

[0033] Based on the auxiliary information, determine whether the first matching result is consistent with the second matching result; if it is determined that the first matching result is consistent with the second matching result, then determine that the verification result of the target execution subtask is qualified; if it is determined that the first matching result is inconsistent with the second matching result, then determine that the verification result of the target execution subtask is unqualified, and update the unqualified target execution subtask.

[0034] Preferably, the step of determining whether the first matching result is consistent with the second matching result based on the supplementary information includes:

[0035] Acquire a target transmission subtask and a target task transmission channel based on the first matching result;

[0036] The unique identifier corresponding to the target transmission subtask in the first matching result is recorded as the first identifier, and the unique identifier corresponding to the target transmission subtask in the second matching result is recorded as the second identifier;

[0037] Obtain the first matching result, the second matching result, and the verification window corresponding to the task transmission channel, and determine two verification windows corresponding to the target task transmission channel in the first matching result and the second matching result based on the verification window, wherein one verification window corresponds to the first identifier and the other verification window corresponds to the second identifier;

[0038] Determining whether the first identifier is the same as the second identifier;

[0039] If it is determined that the first identifier is the same as the second identifier, then it is determined that the first matching result is consistent with the second matching result;

[0040] If it is determined that the first identifier is not the same as the second identifier, it is determined that the first matching result is inconsistent with the second matching result.

[0041] Preferably, the step of updating the unqualified target execution subtask includes:

[0042] Obtain a first identifier and a second identifier, and divide all verification windows into two parts based on the first identifier and the second identifier, which are respectively recorded as a first verification window set and a second verification window set;

[0043] Determine verification windows corresponding to each other between the first verification window set and the second verification window set based on the verification chain to obtain a plurality of verification window pairs;

[0044] Screening out verification windows whose first identifier and second identifier are inconsistent based on multiple verification windows, and using the verification window corresponding to the second identifier as a reference window;

[0045] Based on the verification chain, another verification window corresponding to the reference window is obtained as the target replacement verification window, and the first identifier corresponding to the target replacement verification window is replaced with a second identifier, wherein the second identifier refers to the second identifier corresponding to the reference window.

[0046] An engine task management system, applied to any one of the engine task management methods described above, comprising:

[0047] An information acquisition module is used to obtain source task data information of a source system and multiple engines, and establish communication channels between multiple engines and the source system;

[0048] The task splitting module is used to split the source task data information to obtain multiple subtasks, each subtask corresponds to a unique identifier, the subtask to be transmitted is used as the target transmission subtask, and a communication channel is matched to each target transmission subtask; the communication channel includes a task transmission channel and a synchronous transmission channel, and the task transmission channel corresponds to the synchronous transmission channel one by one;

[0049] A rule setting module, used to set auxiliary information for each task transmission channel, wherein the auxiliary information includes a verification window pair, a verification point, and a verification chain, wherein the verification window pair includes two verification windows, and a verification chain is formed between the two verification windows based on the verification point;

[0050] An information transmission module is used to transmit the auxiliary information to the corresponding engine based on the synchronous transmission channel; transmit the target transmission subtask to the corresponding engine based on the task transmission channel to obtain the target execution subtask;

[0051] A task verification module, configured to verify the target execution subtask based on the auxiliary information, and in response to determining that the verification result is unqualified, determine a modified target execution subtask based on the verification window, and transmit and verify the modified target execution subtask until the verification result is determined to be qualified;

[0052] The task execution module is used to execute the corresponding target execution subtask through the engine in response to determining that the verification result is qualified.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] 1. After the task is successfully transmitted, the task to be executed is verified by verifying the unique identifier corresponding to the window and the task, so as to prevent the task to be executed by the engine from changing during the task transmission process, so that the engine accepts different versions of tasks or the task is executed multiple times, which can effectively improve the accuracy of the engine's task execution;

[0055] 2. According to the arrangement disk and disk queue, the corresponding communication channel can be matched to the task to be transmitted, so that it can select a more suitable engine. According to the matching of the task execution requirements and the engine, the corresponding engine is selected, and the corresponding communication channel is pulled out according to the determined engine, thereby improving the accuracy of communication channel allocation, thereby effectively improving the management effect of the engine. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for describing the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

[0057] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0058] Figure 2 It is a system structure block diagram of the present invention. DETAILED DESCRIPTION

[0059] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0060] Example

[0061] See also Figure 1 to Figure 2 The present invention provides a method and system technical solution for managing engine tasks: A method for managing engine tasks, comprising the following steps:

[0062] S1: Obtain source task data information and multiple engines from the source system, and establish communication channels between multiple engines and the source system;

[0063] The steps to establish communication channels between multiple engines and source systems include:

[0064] Get the data bus protocol type to be used;

[0065] Communication rules between the engine and the source system based on the bus protocol type;

[0066] Deploy the source system and the engine based on the communication rule to obtain a communication channel, wherein the communication channel includes a task transmission channel and a synchronization transmission channel, and the task transmission channel and the synchronization transmission channel correspond one to one;

[0067] Specifically, determine the type of data bus protocol to be used (such as OMAF, DDS, etc.) and write the corresponding protocol specification file. The specification file describes the basic rules for how to communicate on the data bus.

[0068] According to the protocol specification document, write the code of the source system and engine to support the data bus protocol. Usually some middleware technologies such as publish / subscribe mode, message routing, etc. are required to transmit data between different systems. Deploy the written source system and engine to the server in the running environment and ensure that they can interact with the data bus. This may involve installing applicable drivers at the operating system level, configuring network settings, etc. Test to ensure that all systems can exchange data correctly;

[0069] S2: Split the source task data information to obtain multiple subtasks, each subtask corresponds to a unique identifier, take the subtask to be transmitted as the target transmission subtask, and match a communication channel to each target transmission subtask; the communication channel includes a task transmission channel and a synchronous transmission channel, and the task transmission channel corresponds to the synchronous transmission channel one by one;

[0070] The steps of matching a communication channel to each target transmission subtask include:

[0071] Acquire multiple engines, multiple target transmission subtasks, and multiple communication channels; functionally classify the multiple engines to obtain multiple functional types;

[0072] A ranking plate is constructed based on multiple function types, and multiple engine points are set on the ranking plate, each engine point corresponding to a function type;

[0073] Obtain the target transmission subtask's requirement type for the engine, and obtain the target requirement function type;

[0074] Extracting engine points corresponding to target required functional types based on multiple functional types;

[0075] Obtain the execution requirements of the target transmission subtask, and determine the engine located at the engine point as a standby engine based on the execution requirements;

[0076] Match the prepared engine with the engine point, and match the communication channel between the engine point and the source system with the target transmission subtask;

[0077] It should be noted that the function type may include a search function, a calculation function, etc.; the requirement type refers to whether the target transmission subtask needs to be searched or calculated, etc., and an engine with the corresponding function type is selected according to the requirement type, so as to execute the processing through the engine target transmission subtask; multiple target transmission subtasks correspond to multiple requirement types, and the requirement type corresponding to a target transmission subtask is selected from the multiple requirement types to obtain the target requirement function type. For example, there are two target transmission subtasks, and the corresponding requirement types are the search requirement function and the calculation requirement function, respectively. At this time, the requirement type corresponding to one of the target transmission subtasks is the search requirement function, then the search requirement function is extracted from the search requirement function and the calculation requirement function, and the search requirement function is used as the target requirement function type corresponding to this target transmission subtask; according to the consistency of the function type, the target transmission subtask is connected to the corresponding engine point;

[0078] Specifically, a permutation disk is set with multiple engine points on it, the type of each target transmission subtask and the type of the permutation disk are obtained, the target transmission subtasks of the same type are matched with the permutation disk, and the communication channel between the engine point on the permutation disk and the source system is the target communication channel, which is used to transmit the target transmission subtasks of the same type as the matching result; the corresponding engine point is matched according to the type requirement of the subtask, and the corresponding engine is matched according to the requirement in the execution of the subtask, so as to determine who occupies the engine point; assuming that there are 4 types of engine points, the four engine points form a permutation disk, each engine point corresponds to an engine of a functional type, and each engine point is matched with the source A communication channel is set up between the systems, and there are 4 target transmission subtasks, namely subtask 1, subtask 2, subtask 3 and subtask 4. The demand engine of each target transmission subtask is different, and there may be many engines of each type. For example, search engines include Baidu, Bing, Google, etc. Assume that the type of engine point 1 is a search engine, and the type of target transmission subtask 4 is a search engine. The corresponding demand engine is a search engine. At this time, subtask 4 is matched with the communication channel corresponding to search engine 1; set a task type match between the demand engine of the target transmission subtask and the engine corresponding to the communication channel, and start the corresponding transmission channel for the task type match;

[0079] The execution requirements of the target transmission subtask are obtained, and the engine located at the engine point is determined based on the execution requirements, and the steps of preparing the engine include: obtaining multiple engines of the same functional type to obtain a set of similar engines; constructing a disk queue based on the set of similar engines, and setting multiple sub-points on the disk queue, each sub-point corresponding to an engine; obtaining the required execution index of the target transmission subtask and the actual execution index of the engine, calculating the similarity between the required execution index and the actual execution index of each engine, and obtaining multiple similarity values; extracting the similarity value with the largest value based on the multiple similarity values ​​to obtain the maximum similarity value, and determining the actual execution index corresponding to the required execution index based on the maximum similarity value as the target execution index, overlapping the sub-point of the engine corresponding to the target execution index with the engine point, and obtaining the engine located at the engine point;

[0080] Specifically, multiple engines of the same type are grouped into a disk queue. Multiple sub-points are set on the disk queue, each sub-point corresponds to an engine, and the sub-points on the disk queue correspond to the engine points. An engine that meets the execution requirements is found from the disk queue, and its point is matched with the engine point. The corresponding engine is started according to the requirements of the target transmission subtask transmitted through the communication channel; the corresponding engine is matched according to the requirements of the execution of the subtask.

[0081] The engines are classified and placed on a disk queue of the same type. Each point corresponds to an engine. The corresponding engine is selected according to the task requirements of the subtask and placed on the corresponding pit. Different engines of the same type are selected for different task situations. For example, search engines include Bing and Google. If the task situation of the subtask requires accurate search, it is best to select the Google engine. If the task situation of the subtask requires privacy protection during search, it is best to select Bing. The corresponding engine is selected according to different focuses, so that the engine disk is placed on the corresponding pit as the target engine.

[0082] It should be noted that the step of calculating the similarity between the required execution index and the actual execution index of each engine includes: obtaining the required execution index of the target transmission subtask, evaluating the actual execution index of each engine, the execution index at least includes the degree of privacy protection and the degree of execution accuracy, setting a corresponding weight value for each execution index, judging the similarity between the required execution index and the actual execution index according to the corresponding weight value, and selecting the corresponding engine for the engine point according to the value of the similarity;

[0083] The calculation formula corresponding to the similarity is: Among them, Cys is the privacy protection degree of the engine, Cjs is the execution accuracy of the engine, Cyx is the required privacy protection degree of the target transmission subtask, Cjx is the required execution accuracy of the target transmission subtask, and a1, a2, a3, and a4 are proportional factors;

[0084] S3: setting auxiliary information for each task transmission channel, wherein the auxiliary information includes a verification window pair, a verification point, and a verification chain, wherein the verification window pair includes two verification windows, and a verification chain is formed between the two verification windows based on the verification point;

[0085] S4: transmitting the auxiliary information to the corresponding engine based on the synchronous transmission channel; transmitting the target transmission subtask to the corresponding engine based on the task transmission channel to obtain the target execution subtask; verifying the target execution subtask based on the auxiliary information;

[0086] The steps of verifying the target execution subtask based on the auxiliary information include:

[0087] Acquire multiple target transmission subtasks, match the multiple target transmission subtasks with the task transmission channels, and obtain a first matching result;

[0088] Based on the first matching result, multiple target transmission subtasks are transmitted through corresponding task transmission channels, and the task transmission channels corresponding to the target transmission subtasks are recorded as target task transmission channels;

[0089] After the multiple target transmission subtasks are successfully transmitted, the matching results of the multiple target transmission subtasks and the task transmission channels are obtained again to obtain a second matching result;

[0090] Based on the auxiliary information, determine whether the first matching result is consistent with the second matching result; if it is determined that the first matching result is consistent with the second matching result, determine that the verification result of the target execution subtask is qualified; if it is determined that the first matching result is inconsistent with the second matching result, determine that the verification result of the target execution subtask is unqualified, and update the unqualified target execution subtask;

[0091] Specifically, assuming that the source task data is split into task 1, task 2, task 3, and task 4, and the corresponding task transmission channels are channel 1, channel 2, channel 3, and channel 4 respectively, each subtask corresponds to a task transmission channel, task 1 corresponds to transmission channel 1, task 2 corresponds to channel 2, task 3 corresponds to channel 3, and task 4 corresponds to channel 4, and this matching result is used as the first matching result, and the task is transmitted according to the first matching result. Assuming that after the transmission is completed, the matching result of the target transmission subtask and the transmission channel is obtained again, which is task 1 corresponds to transmission channel 1, task 2 corresponds to channel 4, task 3 corresponds to channel 3, and task 4 corresponds to channel 2. By comparing the two matching results corresponding to the task transmission channels before and after the transmission, it can be found that the task transmission channels corresponding to task 2 and task 4 have changed. Each task transmission channel corresponds to a task engine. When the matching results are inconsistent, it means that the verification result is unqualified. Assuming that after the transmission, the matching result obtained again is exactly the same as the first matching result, it means that the verification result is qualified.

[0092] The step of determining whether the first matching result and the second matching result are consistent based on the auxiliary information includes:

[0093] Acquire a target transmission subtask and a target task transmission channel based on the first matching result;

[0094] The unique identifier corresponding to the target transmission subtask in the first matching result is recorded as the first identifier, and the unique identifier corresponding to the target transmission subtask in the second matching result is recorded as the second identifier;

[0095] Obtain the first matching result, the second matching result, and the verification window corresponding to the task transmission channel, and determine two verification windows corresponding to the target task transmission channel in the first matching result and the second matching result based on the verification window, wherein one verification window corresponds to the first identifier and the other verification window corresponds to the second identifier;

[0096] Determining whether the first identifier is the same as the second identifier;

[0097] If it is determined that the first identifier is the same as the second identifier, then it is determined that the first matching result is consistent with the second matching result;

[0098] If it is determined that the first identifier is not the same as the second identifier, it is determined that the first matching result is inconsistent with the second matching result;

[0099] Specifically, assuming that the first matching result is that task 1 corresponds to transmission channel 1, task 2 corresponds to channel 2, task 3 corresponds to channel 3, and task 4 corresponds to channel 4, and the second matching result is that task 1 corresponds to transmission channel 1, task 2 corresponds to channel 4, task 3 corresponds to channel 3, and task 4 corresponds to channel 2, obtain two verification windows corresponding to the task transmission channels, one verification window corresponds to the unique identifier of the target transmission subtask before transmission, and the other verification window corresponds to the unique identifier of the target transmission subtask after transmission, and determine whether the two unique identifiers are consistent. If they are consistent, it means that the matching results are consistent. If they are inconsistent, it means that the matching results are inconsistent. Obtain the unique identifier of the target transmission subtask corresponding to each verification window, find the verification window corresponding to the task transmission channel with the problem, and when the contents of the verification window are inconsistent, find the swap position of the task according to the window, retransmit, and then verify again;

[0100] The steps for updating the subtasks for unqualified target execution include:

[0101] Obtain a first identifier and a second identifier, and divide all verification windows into two parts based on the first identifier and the second identifier, which are respectively recorded as a first verification window set and a second verification window set;

[0102] Determine verification windows corresponding to each other between the first verification window set and the second verification window set based on the verification chain to obtain a plurality of verification window pairs;

[0103] Screening out verification windows whose first identifier and second identifier are inconsistent based on multiple verification windows, and using the verification window corresponding to the second identifier as a reference window;

[0104] Based on the verification chain, another verification window corresponding to the reference window is obtained as the target replacement verification window, and a first identifier corresponding to the target replacement verification window is replaced with a second identifier, wherein the second identifier refers to the second identifier corresponding to the reference window;

[0105] Specifically, obtain the unique identifier of the target transmission subtask corresponding to each verification window, find the verification window corresponding to the transmission channel of the task with the problem, and when the content of the verification window is inconsistent, find the swap position of the task according to the window, retransmit it, and then verify it again. For example, suppose the first matching result is that task 1 corresponds to transmission channel 1, task 2 corresponds to channel 2, task 3 corresponds to channel 3, and task 4 corresponds to channel 4. The second matching result is that task 1 corresponds to transmission channel 1, task 2 corresponds to channel 4, task 3 corresponds to channel 3, and task 4 corresponds to channel 2. At this time, there are four corresponding verification windows in the first matching result, and there are also four corresponding verification windows in the second matching result. The first four verification windows correspond to the last four verification windows one by one through the verification chain. Find the inconsistent task identifiers in the verification window, and use the identifiers corresponding to the four verification windows in the second matching result as the basis. Replace the first matching result with the second matching result, and retransmit and verify the inconsistent task identifiers until the verification result is determined to be qualified. For example, the task identifier corresponding to channel 2 in the first matching result is 2, and the task identifier corresponding to the second matching result is 4. Channel 2 is used as a verification chain to obtain the task identifiers corresponding to the channels before and after. It can be seen that the identifiers 2 and 4 have changed before and after, and the tasks before and after the identifiers are inconsistent. At this time, the second matching result is used as the benchmark, that is, the task identifier 4 corresponding to channel 2 is used as the benchmark. Therefore, it is necessary to adjust the task identifier in the first matching result, adjust task 2 to task 4, and retransmit task 4 through channel 2. After transmission, task 4 at this time is used as the target execution subtask, and the target execution subtask needs to be verified again until the identifiers corresponding to the verification windows in the two matching results are completely consistent. By obtaining multiple verification windows and the first and second identifiers corresponding to the verification windows in the verification windows, the verification windows with different first and second identifiers are selected, and the first identifier is replaced with the second identifier as the benchmark; the subtask after transmission can be verified. When it is verified that the task to be executed is inconsistent with the transmitted subtask, it is retransmitted and changed, which effectively prevents the corresponding engine from executing the subtask incorrectly, and improves the management effect of the engine;

[0106] S5: In response to determining that the verification result is unqualified, determining a modified target execution subtask based on the verification window, transmitting and verifying the modified target execution subtask until the verification result is determined to be qualified; in response to determining that the verification result is qualified, executing the corresponding target execution subtask through the engine.

[0107] An engine task management system, applied to any one of the engine task management methods described above, comprising:

[0108] An information acquisition module is used to obtain source task data information of a source system and multiple engines, and establish communication channels between multiple engines and the source system;

[0109] The task splitting module is used to split the source task data information to obtain multiple subtasks, each subtask corresponds to a unique identifier, the subtask to be transmitted is used as the target transmission subtask, and a communication channel is matched to each target transmission subtask; the communication channel includes a task transmission channel and a synchronous transmission channel, and the task transmission channel corresponds to the synchronous transmission channel one by one;

[0110] A rule setting module, used to set auxiliary information for each task transmission channel, wherein the auxiliary information includes a verification window pair, a verification point, and a verification chain, wherein the verification window pair includes two verification windows, and a verification chain is formed between the two verification windows based on the verification point;

[0111] An information transmission module is used to transmit the auxiliary information to the corresponding engine based on the synchronous transmission channel; transmit the target transmission subtask to the corresponding engine based on the task transmission channel to obtain the target execution subtask;

[0112] A task verification module, configured to verify the target execution subtask based on the auxiliary information, and in response to determining that the verification result is unqualified, determine a modified target execution subtask based on the verification window, and transmit and verify the modified target execution subtask until the verification result is determined to be qualified;

[0113] The task execution module is used to execute the corresponding target execution subtask through the engine in response to determining that the verification result is qualified.

[0114] First, obtain the corresponding task data information from the source system, split the task data into multiple subtasks, each subtask matches a communication channel, the task transmission channel is used to transmit the corresponding subtask, and the synchronous transmission channel is used to transmit the ancillary information corresponding to the subtask, the ancillary information includes the verification window, verification point and verification chain; obtain the execution order of the subtasks before adjustment and the execution order of the subtasks after adjustment; set a verification window in the source system and the engine respectively, each verification window corresponds to a verification point, and determine whether the subtasks transmitted by the transmission channel are qualified by verifying whether the subtasks in the corresponding window are qualified; in the synchronous transmission channel, set a synchronization chain in the source system and the engine to determine whether the subtasks in the task transmission channel correspond to the modified unique identifier. If they correspond, it means that the transmitted subtasks are qualified and can be executed; if not, return to the source system and select a suitable execution engine for the task. After the task is transmitted, the task to be executed is verified by verifying the unique identifier corresponding to the window and the task to prevent the task to be executed by the engine from changing during the task transmission process, so that the engine accepts different versions of tasks or the task is executed multiple times, which can effectively improve the accuracy of the engine's task execution;

[0115] Verification windows and verification points are set to prevent task confusion during subtask allocation and transmission, ensure the eligibility of subtask transmission, prevent the subtasks executed by the engine from being inconsistent with the target subtasks, and improve the accuracy of task execution on the target engine system.

[0116] In the description of this specification, the description with reference to the terms "one embodiment", "example", "specific example", etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0117] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for managing engine tasks, characterized in that: The following steps are involved: S1: Obtain source task data information and multiple engines from the source system, and establish communication channels between multiple engines and the source system; S2: Split the source task data information to obtain multiple subtasks, each subtask corresponds to a unique identifier, take the subtask to be transmitted as the target transmission subtask, and match a communication channel to each target transmission subtask; the communication channel includes a task transmission channel and a synchronous transmission channel, and the task transmission channel corresponds to the synchronous transmission channel one by one; S3: setting auxiliary information for each task transmission channel, wherein the auxiliary information includes a verification window pair, a verification point, and a verification chain, wherein the verification window pair includes two verification windows, and a verification chain is formed between the two verification windows based on the verification point; S4: transmitting the auxiliary information to the corresponding engine based on the synchronous transmission channel; Based on the task transmission channel, the target transmission subtask is transmitted to the corresponding engine to obtain the target execution subtask; Verify the target execution subtask based on the attached information; The steps of verifying the target execution subtask based on the auxiliary information include: Acquire multiple target transmission subtasks, match the multiple target transmission subtasks with the task transmission channels, and obtain a first matching result; Based on the first matching result, multiple target transmission subtasks are transmitted through corresponding task transmission channels, and the task transmission channels corresponding to the target transmission subtasks are recorded as target task transmission channels; After the multiple target transmission subtasks are successfully transmitted, the matching results of the multiple target transmission subtasks and the task transmission channels are obtained again to obtain a second matching result; Based on the auxiliary information, determine whether the first matching result is consistent with the second matching result; if it is determined that the first matching result is consistent with the second matching result, determine that the verification result of the target execution subtask is qualified; if it is determined that the first matching result is inconsistent with the second matching result, determine that the verification result of the target execution subtask is unqualified, and update the unqualified target execution subtask; The step of determining whether the first matching result is consistent with the second matching result based on the auxiliary information includes: Acquire a target transmission subtask and a target task transmission channel based on the first matching result; The unique identifier corresponding to the target transmission subtask in the first matching result is recorded as the first identifier, and the unique identifier corresponding to the target transmission subtask in the second matching result is recorded as the second identifier; Obtain the first matching result, the second matching result, and the verification window corresponding to the task transmission channel, and determine two verification windows corresponding to the target task transmission channel in the first matching result and the second matching result based on the verification window, wherein one verification window corresponds to the first identifier and the other verification window corresponds to the second identifier; Determining whether the first identifier is the same as the second identifier; If it is determined that the first identifier is the same as the second identifier, then it is determined that the first matching result is consistent with the second matching result; If it is determined that the first identifier is not the same as the second identifier, it is determined that the first matching result is inconsistent with the second matching result; The step of updating the unqualified target execution subtask comprises: Obtain a first identifier and a second identifier, and divide all verification windows into two parts based on the first identifier and the second identifier, which are respectively recorded as a first verification window set and a second verification window set; Determine verification windows corresponding to each other between the first verification window set and the second verification window set based on the verification chain to obtain a plurality of verification window pairs; Screening out verification windows whose first identifier and second identifier are inconsistent based on multiple verification windows, and using the verification window corresponding to the second identifier as a reference window; Based on the verification chain, another verification window corresponding to the reference window is obtained as the target replacement verification window, and a first identifier corresponding to the target replacement verification window is replaced with a second identifier, wherein the second identifier refers to the second identifier corresponding to the reference window; S5: In response to determining that the verification result is unqualified, determining a modified target execution subtask based on the verification window, transmitting and verifying the modified target execution subtask until the verification result is determined to be qualified; in response to determining that the verification result is qualified, executing the corresponding target execution subtask through the engine.

2. The engine task management method according to claim 1, characterized in that: The steps of matching a communication channel to each target transmission subtask include: Acquire multiple engines, multiple target transmission subtasks, and multiple communication channels; functionally classify the multiple engines to obtain multiple functional types; A ranking plate is constructed based on multiple function types, and multiple engine points are set on the ranking plate, each engine point corresponding to a function type; Obtain the target transmission subtask's requirement type for the engine, and obtain the target requirement function type; Extracting engine points corresponding to target required functional types based on multiple functional types; Obtain the execution requirements of the target transmission subtask, and determine the engine located at the engine point as a standby engine based on the execution requirements; The prepared engine is matched with the engine point, and the communication channel between the engine point and the source system is matched with the target transmission subtask.

3. The engine task management method according to claim 2, characterized in that: The steps of obtaining the execution requirements of the target transmission subtask and determining the engine located at the engine point based on the execution requirements include: Get multiple engines of the same function type to get a set of engines of the same type; A disk queue is built based on a set of similar engines. Multiple sub-points are set on the disk queue, and each sub-point corresponds to an engine. Obtaining the required execution index of the target transmission subtask and the actual execution index of the engine, calculating the similarity between the required execution index and the actual execution index of each engine, and obtaining multiple similarity values; Extracting a similarity value with the largest value based on multiple similarity values ​​to obtain a maximum similarity value, and determining an actual execution indicator corresponding to the required execution indicator based on the maximum similarity value as a target execution indicator; The sub-point of the engine corresponding to the target execution index is overlapped with the engine point to obtain the engine located at the engine point.

4. The engine task management method according to claim 3, characterized in that: The step of calculating the similarity between the required execution index and the actual execution index of each engine includes: Obtain the required execution indicators of the target transmission subtask, evaluate the actual execution indicators of each engine, the execution indicators at least include the degree of privacy protection and the degree of execution accuracy, set a corresponding weight value for each execution indicator, judge the similarity between the required execution indicator and the actual execution indicator according to the corresponding weight value, and select the corresponding engine for the engine point according to the similarity value; The calculation formula corresponding to the similarity is: Among them, Cys is the privacy protection degree of the engine, Cjs is the execution accuracy of the engine, Cyx is the required privacy protection degree of the target transmission subtask, Cjx is the required execution accuracy of the target transmission subtask, and a1, a2, a3, and a4 are proportional factors.

5. An engine task management system, applied to the engine task management method as claimed in any one of claims 1 to 4, characterized in that: include: An information acquisition module is used to obtain source task data information of a source system and multiple engines, and establish communication channels between multiple engines and the source system; The task splitting module is used to split the source task data information into multiple subtasks, each subtask corresponds to a unique identifier, the subtask to be transmitted is used as the target transmission subtask, and a communication channel is matched for each target transmission subtask; The communication channel includes a task transmission channel and a synchronization transmission channel, and the task transmission channel and the synchronization transmission channel correspond one to one; A rule setting module, used to set auxiliary information for each task transmission channel, wherein the auxiliary information includes a verification window pair, a verification point, and a verification chain, wherein the verification window pair includes two verification windows, and a verification chain is formed between the two verification windows based on the verification point; An information transmission module, used to transmit the auxiliary information to the corresponding engine based on the synchronous transmission channel; Based on the task transmission channel, the target transmission subtask is transmitted to the corresponding engine to obtain the target execution subtask; A task verification module, configured to verify the target execution subtask based on the auxiliary information, and in response to determining that the verification result is unqualified, determine a modified target execution subtask based on the verification window, and transmit and verify the modified target execution subtask until the verification result is determined to be qualified; The task execution module is used to execute the corresponding target execution subtask through the engine in response to determining that the verification result is qualified.

Images