Method, apparatus, electronic device, and storage medium for constructing a machine learning workflow

By generating and verifying the hash calculation diagram of the machine learning workflow, determining the steps that need to be run and have been successfully run, and building the target workflow, solving the resource waste problem caused by repeated execution of workflows in the existing technology, improving the efficiency and user experience of the workflow.

CN114492844BActive Publication Date: 2025-06-20JINGDONG CITY BEIJING DIGITS TECH CO LTD
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202210146721.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-02-17
Publication Date
2025-06-20
Estimated Expiration
2042-02-17

AI Technical Summary

Technical Problem

In the current technology, when machine learning workflows are run, there are problems of wasting time and resource caused by repeated execution of the exact same workflow, which affects the operational performance and user experience of the workflow.

Method used

By obtaining the target workflow template, generating the target hash calculation graph, calculating the hash value of the target node, validating the hash value to determine the workflow steps that need to be run and have been successfully run, building the target workflow to avoid duplicate execution.

Benefits of technology

It effectively solves the problem of repeated execution of the exact same workflow, improves the operation efficiency and performance of the workflow, and improves the user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114492844B_ABST
    Figure CN114492844B_ABST
Patent Text Reader

Abstract

The present disclosure provides a method, apparatus, electronic device, and storage medium for constructing a machine learning workflow. The method includes: obtaining a target workflow template; generating a target hash computation graph corresponding to the target workflow template according to the organizational structure information of the target workflow template; calculating the hash values of target nodes included in the target hash computation graph according to the parameter information corresponding to the target workflow template by using the defined calculation method of the hash computation graph; verifying the hash values of the target nodes to determine the workflow steps that need to be run and the workflow steps that have been successfully run, and then constructing a target workflow by using the workflow steps that need to be run and the workflow steps that have been successfully run. This method can solve the problems of time and resource waste caused by repeated execution of exactly the same workflow, improve the running efficiency and performance of the workflow, and thus enhance the user experience.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of logistics technologies, and in particular, to a method, apparatus, electronic device, and storage medium for constructing a machine learning workflow. Background Art

[0002] Machine learning workflow technology can solve the problems of low application development efficiency and uneven quality. Machine learning workflow technology mainly evolves from process management workflows. The current operation of workflows can be divided into data-dependency-driven and task-status-dependency-driven. In real application development scenarios, machine learning workflows may be repeatedly modified and run, and the directed acyclic graph structure of the workflow is complex. Developers need to locate the reasons for the poor overall effect based on the running results of the steps. However, in the prior art, when the workflow runs, only the calculation correctness, data exchange correctness, and running time from top to bottom are concerned, resulting in waste of time and resources caused by repeated execution of exactly the same workflow, affecting the running performance of the workflow and having a poor user experience.

[0003] It should be noted that the information disclosed in the above background art section is only used to enhance the understanding of the background of the present disclosure, and thus may include information that does not constitute the prior art known to those of ordinary skill in the art. Summary of the Invention

[0004] The purpose of the present disclosure is to provide a method, apparatus, electronic device, and storage medium for constructing a machine learning workflow. This method can solve the problem of waste of time and resources caused by repeated execution of exactly the same workflow, improve the running efficiency and performance of the workflow, and thus enhance the user experience.

[0005] Other features and advantages of the present disclosure will become apparent through the following detailed description, or will be partially learned through the practice of the present disclosure.

[0006] According to an aspect of the present disclosure, a method for constructing a machine learning workflow is provided, including: obtaining a target workflow template; generating a target hash calculation graph corresponding to the target workflow template according to the organizational structure information of the target workflow template; calculating the hash value of a target node included in the target hash calculation graph according to the parameter information corresponding to the target workflow template by using the defined calculation method of the hash calculation graph; verifying the hash value of the target node to determine the workflow steps that need to be run and the workflow steps that have been successfully run, and then constructing a target workflow by using the workflow steps that need to be run and the workflow steps that have been successfully run.

[0007] In some exemplary embodiments of the present disclosure, the calculation method of the hash calculation graph includes: the hash value calculation formula of a node is: adding the hash value of the input parameters corresponding to the node and the hash value of the configuration parameters corresponding to the node, and then calculating the sum result by using a hash algorithm; the hash value calculation formula of the output parameters corresponding to the node is: adding the hash value of the node and the hash value of the output parameter name corresponding to the node.

[0008] In some exemplary embodiments of the present disclosure, generating the target hash calculation graph corresponding to the target workflow template according to the organizational structure information of the target workflow template includes: converting the components included in the target workflow template into target nodes included in the target hash calculation graph, and converting the connection relationship between the components into the connection relationship between the target nodes to generate the target hash calculation graph.

[0009] In some exemplary embodiments of the present disclosure, calculating the hash value of the target nodes included in the target hash calculation graph by using the calculation method of the defined hash calculation graph according to the parameter information corresponding to the target workflow template includes: determining the input parameters corresponding to the target node and the configuration parameters corresponding to the target node according to the connection relationship between the target nodes and the parameters of the components; calculating the hash value of the input parameters corresponding to the target node according to the parameter information of the input parameters corresponding to the target node; querying the parameter information of the configuration parameters corresponding to the target node, and calculating the hash value of the configuration parameters corresponding to the target node according to the queried parameter information; substituting the hash value of the input parameters corresponding to the target node and the hash value of the configuration parameters corresponding to the target node into the hash value calculation formula of the node to calculate the hash value of the target node.

[0010] In some exemplary embodiments of the present disclosure, the input parameters corresponding to the target node include: the input parameters of the component corresponding to the target node, the output parameters of the upstream node of the target node; and, the configuration parameters corresponding to the target node include: the configuration parameters of the component corresponding to the target node.

[0011] In some exemplary embodiments of the present disclosure, calculating the hash value of the input parameter corresponding to the target node according to the parameter information of the input parameter corresponding to the target node includes: obtaining the parameter information of the input parameter of the component corresponding to the target node, and calculating the obtained parameter information by using a hash algorithm to obtain the hash value of the input parameter of the component corresponding to the target node; substituting the hash value of the upstream node of the target node and the parameter name of the output parameter corresponding to the upstream node of the target node into the hash value calculation formula of the output parameter corresponding to the node to calculate the hash value of the output parameter corresponding to the upstream node of the target node; and summing the hash value of the input parameter of the component corresponding to the target node and the hash value of the output parameter corresponding to the upstream node of the target node to obtain the hash value of the input parameter corresponding to the target node.

[0012] In some exemplary embodiments of the present disclosure, calculating the hash value of the configuration parameter corresponding to the target node includes: determining whether the parameter type of the configuration parameter corresponding to the target node is an index; if so, calculating the hash value of the file under the queried parameter information in a manner of interface request, and determining the hash value of the file as the hash value of the configuration parameter corresponding to the target node.

[0013] In some exemplary embodiments of the present disclosure, verifying the hash value of the target node, determining the workflow steps that need to be run and the workflow steps that have been successfully run, and then constructing a target workflow by using the workflow steps that need to be run and the workflow steps that have been successfully run includes: querying whether the hash value of the target node exists in the step information database; if so, marking the workflow step corresponding to the target node as a workflow step that has been successfully run, and reading the running information of the workflow step that has been successfully run from the step information database; if not, marking the workflow step corresponding to the target node as a workflow step that needs to be run; and generating a target workflow configuration file according to the running information of the workflow step that has been successfully run and the workflow steps that need to be run.

[0014] In some exemplary embodiments of the present disclosure, the method further includes: calling an interface to run the target workflow configuration file; and listening to the running status of each step in the target workflow configuration file, and updating the running status of the step to running successfully when the step runs successfully.

[0015] According to one aspect of the present disclosure, there is provided an apparatus for constructing a machine learning workflow, including: an acquisition module configured to acquire a target workflow template; a generation module configured to generate a target hash computation graph corresponding to the target workflow template according to the organizational structure information of the target workflow template; a calculation module configured to calculate the hash value of a target node included in the target hash computation graph according to the parameter information corresponding to the target workflow template by using the defined calculation method of the hash computation graph; and a construction module configured to verify the hash value of the target node, determine the workflow steps to be run and the workflow steps that have been successfully run, and then construct a target workflow by using the workflow steps to be run and the workflow steps that have been successfully run.

[0016] According to one aspect of the present disclosure, there is provided an electronic device, including: at least one processor; a storage device configured to store at least one program, and when the at least one program is executed by the at least one processor, the at least one processor is caused to implement the method for constructing a machine learning workflow as described in any one of the above.

[0017] According to one aspect of the present disclosure, there is provided a computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the method for constructing a machine learning workflow as described in any one of the above is implemented.

[0018] The method for constructing a machine learning workflow provided by the embodiments of the present disclosure first generates a target hash computation graph corresponding to a target workflow template, then can calculate the hash value of a target node included in the target hash computation graph by using the defined calculation method of the hash computation graph, and then by verifying the calculated hash value, obtains the workflow steps to be run and the workflow steps that have been successfully run, and further can prune the workflow steps that have been successfully run to construct a target workflow, which can solve the time and resource problems caused by the repeated execution of the same workflow in the prior art, improve the running efficiency and performance of the workflow, and enhance the user experience.

[0019] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure. Obviously, the drawings in the following description are only some embodiments of the present disclosure, and those of ordinary skill in the art can also obtain other drawings without creative efforts based on these drawings.

[0021] Figure 1A schematic diagram of an exemplary system architecture to which the method for constructing a machine learning workflow according to an exemplary embodiment of the present disclosure can be applied is shown;

[0022] Figure 2 A flowchart of the method for constructing a machine learning workflow according to an exemplary embodiment of the present disclosure is shown;

[0023] Figure 3 A schematic diagram of the structure of a workflow template according to an exemplary embodiment of the present disclosure is shown;

[0024] Figure 4 A schematic diagram of the structure of a hash calculation graph according to an exemplary embodiment of the present disclosure is shown;

[0025] Figure 5 A flowchart of calculating the hash value of a target node included in a target hash calculation graph according to an exemplary embodiment of the present disclosure is shown;

[0026] Figure 6 A general flowchart of the construction of a machine learning workflow according to an exemplary embodiment of the present disclosure is shown;

[0027] Figure 7 A flowchart of calculating the hash value of each target node included in a target hash calculation graph according to an exemplary embodiment of the present disclosure is shown;

[0028] Figure 8 A schematic diagram of the structure of an apparatus for constructing a machine learning workflow according to an exemplary embodiment of the present disclosure is shown;

[0029] Figure 9 A block diagram of an electronic device according to an exemplary embodiment of the present disclosure is shown. Detailed implementation manners

[0030] Example embodiments will now be described more fully with reference to the accompanying drawings. However, the example embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be more complete and thorough, and will fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments.

[0031] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus repeated descriptions thereof will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0032] Figure 1 A schematic diagram of an exemplary system architecture showing a method for constructing a machine learning workflow to which exemplary embodiments of the present disclosure can be applied.

[0033] As Figure 1 shown, the system architecture may include a server 101, a network 102, and a client 103. The network 102 can provide a medium for a communication link between the client 103 and the server 101. The network 102 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0034] The server 101 can be a server that provides various services, such as a back-end management server that supports the devices operated by users using the client. The back-end management server can analyze and process data such as received requests, and feedback the processing results to the client.

[0035] The client 103 can be a mobile terminal such as a mobile phone, a game console, a tablet computer, an e-book reader, smart glasses, a smart home device, an AR (Augmented Reality) device, a VR (Virtual Reality) device, etc., or the client 103 can also be a personal computer, such as a laptop computer and a desktop computer, etc.

[0036] In an exemplary embodiment of the present disclosure, a user can send a workflow processing instruction to the server through the client. The server can, for example, obtain the workflow processing instruction and obtain a target workflow template according to the workflow processing instruction; the server can, for example, generate a target hash calculation graph corresponding to the target workflow template according to the organizational structure information of the target workflow template; the server can, for example, use the defined calculation method of the hash calculation graph to calculate the hash value of the target node included in the target hash calculation graph according to the parameter information corresponding to the target workflow template; the server can, for example, verify the hash value of the target node, determine the workflow steps that need to be run and the workflow steps that have been successfully run, and then use the workflow steps that need to be run and the workflow steps that have been successfully run to construct a target workflow.

[0037] It should be understood that Figure 1 the numbers of the client, the network, and the server in are merely illustrative. The server 101 can be a single physical server, can also be a server cluster composed of multiple servers, and can also be a cloud server. According to actual needs, there can be any number of terminal devices, networks, and servers.

[0038] Next, each step of the method for constructing a machine learning pipeline in the exemplary embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings and embodiments.

[0039] Figure 2 FIG. shows a flowchart of a method for constructing a machine learning pipeline according to an exemplary embodiment of the present disclosure. The method provided by the exemplary embodiment of the present disclosure may be run on a server as shown, but the present disclosure is not limited thereto. Figure 1 As shown.

[0040] As Figure 2 shown, the method for constructing a machine learning pipeline provided by the exemplary embodiment of the present disclosure may include the following steps.

[0041] Step S201: Obtain a target workflow template.

[0042] The target workflow template is a workflow template that needs to be run, and the workflow template can be obtained according to the unique identifier of the target workflow template provided by the user. For example, the user inputs the unique identifier of the workflow template that needs to be run in the interface of the workflow system, and then, in the case of determining the unique identifier of the target workflow template, the target workflow template can be queried in the workflow template database according to the unique identifier. In addition, the unique identifier of the workflow template can be set according to the actual situation, such as the workflow name, the business scenario name of the workflow, etc. Among them, the workflow system can be regarded as a system for processing workflows, and it can provide a system interface for users to input information.

[0043] It should be noted that in the method for constructing a machine learning pipeline in the exemplary embodiment of the present disclosure, workflow templates can be created in advance according to requirements, and then the created workflow templates can be stored in the workflow template database, so as to facilitate querying the required workflow template using the unique identifier of the template. Of course, the organizational structure information of the workflow template can also be stored in the workflow template database. Among them, the organizational structure information of the workflow template may include: components included in the template, connection relationships between components, and input parameters, output parameters, general configuration parameters, and resource configuration parameters of each component. Figure 3 FIG. shows a schematic structural diagram of a workflow template according to an exemplary embodiment of the present disclosure. Figure 3The workflow template shown includes components such as reading a local file, data partitioning, file merging (by column), file merging (by row), and a linear regression component. For the data partitioning component, it is used to divide a single data set into a training set and a test set. Additionally, the general configuration parameters of the data partitioning component are {parameter name: test_size, parameter type: Float} and {parameter name: target_column, parameter type: String}, and the resource configuration parameters are {CPU: 0.5 core, memory: 1024M, GPU: 0}. Figure 3 As can also be seen from the workflow template shown, the connection relationships between the components are specifically the connection relationships between the output parameters of the upstream components and the input parameters of the downstream components.

[0044] Therefore, after determining the unique identifier of the target workflow template, the target workflow template can be obtained according to this unique identifier, and the components included in the target workflow template, the connection relationships between the components, and the parameters of the components can be obtained. Among them, the parameters of the components can include input parameters, output parameters, general configuration parameters, and resource configuration parameters.

[0045] Step S202: Generate a target hash calculation graph corresponding to the target workflow template according to the organizational structure information of the target workflow template.

[0046] According to step S201 above, after obtaining the target workflow template, the organizational structure information of the target workflow template can be obtained, that is, the components included in the target workflow template, the connection relationships between the components, and the input parameters, output parameters, general configuration parameters, and resource configuration parameters of each component. In this way, the target hash calculation graph corresponding to the target workflow template can be generated using the organizational structure information of the target workflow template. Among them, the hash calculation graph refers to a calculation graph that adds hash functions to each node.

[0047] In an exemplary embodiment of the present disclosure, generating a target hash calculation graph corresponding to the target workflow template according to the organizational structure information of the target workflow template may include: converting the components included in the target workflow template into target nodes included in the target hash calculation graph, converting the connection relationships between the components into the connection relationships between the target nodes, and generating the target hash calculation graph.

[0048] In the process of generating the target hash computation graph, the components included in the target workflow template can be transformed into target nodes included in the target hash computation graph, and the connection relationships between the components can be transformed into the connection relationships between the target nodes. Specifically, each component in the target workflow template can be regarded as a target node of the target hash computation graph, and the connection relationships between the components correspond to the connection relationships between the target nodes. In addition, the number of components included in the target workflow template is one or more. Since each component can be regarded as a node, the number of target nodes included in the target hash computation graph is also one or more. Moreover, using target nodes to represent the nodes included in the target hash computation graph is to distinguish them from the nodes in the node hash value calculation formula.

[0049] Figure 4 FIG. shows a schematic structural diagram of a hash computation graph according to an exemplary embodiment of the present disclosure, and Figure 4 the shown hash computation graph is generated according to Figure 3 the shown workflow template. As can be seen from Figure 4 it, the read local file, data partitioning, file merging (by column), file merging (by row), and linear regression components in Figure 3 are respectively transformed into nodes A, B, C, D, and E, and according to the connection relationships between the components, the relationships between the nodes are set such that node A is the upstream node of node B, node B is the upstream node of nodes C and D, and the upstream nodes of node E are nodes C and D.

[0050] Step S203: Using the defined calculation method of the hash computation graph, calculate the hash values of the target nodes included in the target hash computation graph according to the parameter information corresponding to the target workflow template.

[0051] Different from the hash chain, the hash computation graph can concatenate each node in a graph manner. After splicing the values of all directly associated upstream nodes, the value obtained by calculation through the hash algorithm is the value of the current node, that is, using the defined calculation method of the hash computation graph to calculate the hash value of the node. Among them, the calculation method of the hash computation graph can be regarded as a workflow pruning algorithm based on the hash computation graph. Pruning is to avoid some unnecessary traversal processes through a certain judgment, that is, cutting off some "branches" in the search tree. The core issue of applying pruning optimization is to design a pruning judgment method, that is, a method to determine which branches should be discarded and which branches should be retained. The workflow pruning algorithm based on the hash computation graph means using the hash computation graph for pruning. Specifically, analyze each node included in the hash computation graph to judge whether the workflow step corresponding to the node should be discarded or retained, so as to prune the repeatedly running workflow steps. That is, when running the workflow, the pruned workflow steps can be skipped to avoid the repeated execution of exactly the same workflow, improve the running efficiency of the workflow, and enhance the user experience.

[0052] In the exemplary embodiments of the present disclosure, the calculation method of the hash calculation graph may include: (1) the hash value of a node = Hash (the hash value of the input parameters corresponding to the node + the hash value of the configuration parameters corresponding to the node), where Hash() refers to calculating the hash value of the content in ( ) using a hash algorithm; (2) the hash value of the output parameters corresponding to the node = the hash value of the node + the hash value of the output parameter name. The hash algorithm ensures that the same input can obtain the same output, and the hash algorithm can prevent different inputs from obtaining the same output. Currently, there are multiple hash algorithms that can be used, such as md5, sha1, sha256, and sha512. Considering calculation efficiency, storage requirements, and collision probability, etc., the md5 algorithm is preferably used.

[0053] In the hash value calculation formula of the node, the hash value of the input parameters corresponding to the node = the hash value of all the input parameters of the node + the hash value of the parameter names of all the input parameters of the node, and the hash value of the configuration parameters corresponding to the node = the hash value of all the configuration parameters of the node + the hash value of the parameter names of all the configuration parameters of the node. Since for different components included in the same workflow template, their corresponding resource configuration parameters are the same, therefore, the configuration parameters here can be general configuration parameters. In addition, considering that a node can have multiple output parameters, in order to make a distinction, the hash value of the parameter name is introduced in the calculation formula.

[0054] Figure 5 The flowchart shows calculating the hash value of the target node included in the target hash calculation graph according to the exemplary embodiments of the present disclosure. As Figure 5 shown, using the defined calculation method of the hash calculation graph, according to the parameter information corresponding to the target workflow template, calculating the hash value of the target node included in the target hash calculation graph may include:

[0055] Step S501, determine the input parameters corresponding to the target node and the configuration parameters corresponding to the target node according to the connection relationship between the target nodes and the parameters of the components;

[0056] Step S502, calculate the hash value of the input parameters corresponding to the target node according to the parameter information of the input parameters corresponding to the target node;

[0057] Step S503, query the parameter information of the configuration parameters corresponding to the target node, and calculate the hash value of the configuration parameters corresponding to the target node according to the queried parameter information;

[0058] Step S504, substitute the hash value of the input parameters corresponding to the target node and the hash value of the configuration parameters corresponding to the target node into the hash value calculation formula of the node, and calculate the hash value of the target node.

[0059] Among them, step S501 is to determine the input parameters and configuration parameters corresponding to the target node; step S502 is to calculate the hash value of the input parameters corresponding to the target node according to the information of the input parameters; step S503 is to calculate the hash value of the configuration parameters corresponding to the target node according to the information of the configuration parameters; step S504 is to substitute the hash values of the input parameters and configuration parameters calculated in the above steps into the defined hash value calculation formula of the node, so as to obtain the hash value of the target node. It should be noted that as described above, the number of target nodes included in the target hash calculation graph is also one or more. Therefore, it is necessary to use the method described in steps S501 to S504 to calculate the hash value of each target node. Next, steps S501 to S504 will be described in detail.

[0060] First of all, in step S501, the input parameters corresponding to the target node may include: the input parameters of the component corresponding to the target node, the output parameters of the upstream node corresponding to the target node; the configuration parameters corresponding to the target node may include: the configuration parameters of the component corresponding to the target node. Of course, if a certain node does not have an upstream node, then the input parameters corresponding to this node are the input parameters of the component corresponding to this node.

[0061] For the sake of easy understanding, take Figure 4 nodes A, B, C, D, and E in it as an example for illustration. Node A corresponds to the component of reading the local file, and node A has no upstream node. Then the input parameters corresponding to node A are the input parameter a of the component of reading the local file, and the configuration parameters corresponding to node A are the configuration parameters of the component of reading the local file. Node B corresponds to the data partitioning component, and the upstream node of node B is node A. Then the input parameters corresponding to node B are the input parameter b of the data partitioning component and the output parameter a1 corresponding to node A, and the configuration parameters corresponding to node B are the configuration parameters of the data partitioning component. Node C corresponds to the file merging (by column) component, and the upstream node of node C is node B. Then the input parameters corresponding to node C are the input parameter c of the file merging (by column) component and the output parameter b1 corresponding to node B, and the configuration parameters corresponding to node C are the configuration parameters of the file merging (by column) component. Node D corresponds to the file merging (by row) component, and the upstream node of node D is node B. Then the input parameters corresponding to node D are the input parameter d of the file merging (by row) component and the output parameter b2 corresponding to node B, and the configuration parameters corresponding to node D are the configuration parameters of the file merging (by row) component. Node E corresponds to the linear regression component, and the upstream nodes of node E are node C and node D. Then the input parameters corresponding to node E are the input parameter e of the linear regression component, the output parameter c1 corresponding to node C, and the output parameter d1 of node D, and the configuration parameters corresponding to node E are the configuration parameters of the linear regression component.

[0062] In step S502, according to the parameter information of the input parameter corresponding to the target node, calculate the hash value of the input parameter corresponding to the target node. The specific implementation can be as follows:

[0063] (1) Obtain the parameter information of the input parameter of the component corresponding to the target node, and use the hash algorithm to calculate the obtained parameter information to obtain the hash value of the input parameter of the component corresponding to the target node.

[0064] The user provides the parameter information of the input parameter of the component corresponding to the target node. For example, the user inputs the actual operation parameter information of the input parameters of each component of the target workflow template in the interface of the workflow system. In this way, the provided parameter information can be calculated to obtain the corresponding hash value. For example, obtain the specific parameter value m1 of the input parameter b of the data partitioning component corresponding to node B, then use the hash algorithm to perform hash calculation on m1, and use the hash algorithm to perform hash calculation on the parameter name of b, and sum the two calculation results to obtain the hash value of the input parameter b of the data partitioning component corresponding to node B.

[0065] It should be noted that if there are multiple input parameters of the component corresponding to the target node, then the hash value of each input parameter needs to be calculated and then summed to obtain the hash value of the input parameter of the component corresponding to the target node.

[0066] (2) Substitute the hash value of the upstream node of the target node and the parameter name of the output parameter corresponding to the upstream node of the target node into the hash value calculation formula of the output parameter corresponding to the node to calculate the hash value of the output parameter corresponding to the upstream node of the target node.

[0067] As described above, the input parameter corresponding to the target node includes the output parameter corresponding to the upstream node of the target node. Therefore, the hash value calculation formula of the output parameter corresponding to the node can be used to calculate the hash value of the output function corresponding to the upstream node. For example, the input parameter corresponding to node B includes the output parameter a1 corresponding to node A. Then, sum the hash value of node A and the hash value of the parameter name of a1 to obtain the hash value of the output parameter a1 corresponding to the upstream node A of node B. Of course, if there are multiple output parameters corresponding to the upstream node of the target node, then the hash value of each output parameter needs to be calculated and then summed to obtain the hash value of the output parameter corresponding to the upstream node of the target node. It should also be noted that the output parameter corresponding to the upstream node of the target node refers to the parameter that is output from the upstream node and input to the target node. For example, Figure 4 the output parameter b1 in is the output parameter corresponding to the upstream node B of node C, which is the input parameter of node C, while the parameter b2 is the output parameter of node B, but it cannot be regarded as the input parameter of node C.

[0068] (3) Sum the hash value of the input parameters of the component corresponding to the target node and the hash value of the output parameters of the upstream node corresponding to the target node to obtain the hash value of the input parameters corresponding to the target node. For example, the hash value of the input parameters corresponding to node B = the hash value of input parameter b + the hash value of node A + the hash value of the parameter name of a1.

[0069] Step S502 is to calculate the hash value of the input parameters corresponding to the target node, and step S503 is to query the parameter information of the configuration parameters corresponding to the target node, and then calculate the hash value of the configuration parameters corresponding to the target node according to the queried parameter information.

[0070] In step S503, by querying the parameter configuration database, the actual operation parameter information of the configuration parameters of each component of the target workflow template is obtained, mainly including the actual values of the general configuration parameters of the component and the operation resource configuration parameters. Among them, the types of the general configuration parameters of the component include values and indexes, such as file paths; the operation resource configuration parameters include the number of CPUs, the number of GPU cores, and the memory size. In addition, if the type of the general configuration parameter of the component is an index, the hash value of the corresponding file needs to be calculated first by means of an interface request, and the obtained hash value is used as the value of the parameter configuration. Therefore, in the exemplary embodiment of the present disclosure, calculating the hash value of the configuration parameters corresponding to the target node may include: determining whether the parameter type of the configuration parameters corresponding to the target node is an index; if so, calculating the hash value of the file under the queried parameter information by means of an interface request, and determining the hash value of the file as the hash value of the configuration parameters corresponding to the target node.

[0071] Through step S502, the hash value of the input parameters corresponding to the target node is calculated. Through step S503, the hash value of the configuration parameters corresponding to the target node is calculated. Then, in step S504, the hash value of the input parameters corresponding to the target node calculated through step S502 and the hash value of the configuration parameters corresponding to the target node calculated through step S503 can be substituted into the hash value calculation formula of the node, and finally the hash value of the target node can be obtained. It should be noted that in the exemplary embodiment of the present disclosure, for the target nodes included in the target hash calculation graph, the hash values of each target node need to be calculated sequentially from top to bottom starting from the target node without upstream nodes.

[0072] Step S204: Verify the hash value of the target node, determine the workflow steps that need to be run and the workflow steps that have been successfully run, and then construct the target workflow by using the workflow steps that need to be run and the workflow steps that have been successfully run.

[0073] The hash value of each target node is calculated through step S203. In step S204, the calculated hash value needs to be verified. The specific verification process can be as follows: query whether the hash value of the target node exists in the step information database; if so, mark the workflow step corresponding to the target node as a successfully run workflow step, and read the running information of the successfully run workflow step from the step information database; if not, mark the workflow step corresponding to the target node as a workflow step to be run; generate a target workflow configuration file based on the running information of the successfully run workflow step and the workflow step to be run.

[0074] After calculating and obtaining the hash value of a certain target node, it is queried whether the hash value already exists in the step information database. Specifically, if the calculated hash value already exists in the step information database, it means that steps with the same parameter configuration and the same definition (i.e., exactly the same) have been successfully run at some time before. Therefore, only the previous running status and information need to be directly queried in the step information database and assigned to the workflow step corresponding to the target node, and the workflow step corresponding to the target node is marked as a successfully run workflow step. If the hash value of the target node is not in the step information database, it means that no steps identical to the workflow step corresponding to the target node have been run. Therefore, the workflow step corresponding to the target node needs to be marked as a workflow step to be run.

[0075] After analyzing the hash values of the target nodes included in the target hash calculation graph, the successfully run workflow steps and the workflow steps to be run included in the target workflow template can be determined, and the running information of the successfully run workflow steps can be obtained. Furthermore, a target workflow configuration file can be generated. That is to say, after calculating the hash values of all the target nodes included in the target hash calculation graph, the successfully run workflow steps (i.e., the workflow steps that do not need to be run again) will be skipped, that is, pruning is performed on the successfully run workflow steps, and the workflow steps to be run can also be obtained. At this time, the running information of the workflow steps to be run and the successfully run workflow steps can be used to reconstruct the target workflow configuration file.

[0076] In addition, in the exemplary embodiment of the present disclosure, the workflow processing method may further include: calling an interface to run the target workflow configuration file; and listening to the running status of each step in the target workflow configuration file, and updating the running status of the step to successfully run when the step runs successfully.

[0077] An application call interface is established between the workflow system and the underlying workflow executor, and the workflow system calls the interface to run the workflow configuration file. Moreover, the workflow system continuously monitors the running status of each step of the workflow. After a certain step of the workflow runs successfully, the running information of this step is updated in the step information database. Generally speaking, the step information database records all steps, the workflows to which the steps belong, and the running information of the steps, so that the step and the workflow to which the step belongs can be queried based on the running information of a certain step.

[0078] The workflow processing method provided by the embodiments of the present disclosure first generates a target hash calculation graph corresponding to the target workflow template, and then can use the defined calculation method of the hash calculation graph to calculate the hash values of the target nodes included in the target hash calculation graph. Then, by verifying the calculated hash values, the workflow steps that need to be run and the workflow steps that have been successfully run can be obtained. Furthermore, the workflow steps that have been successfully run can be pruned to construct the target workflow, which can solve the time and resource problems caused by the repeated execution of the same workflow in the prior art, improve the running efficiency and performance of the workflow, and enhance the user experience. Moreover, the step information database records all steps, the workflows to which the steps belong, and the running information of the steps, and it is possible to find the workflow to which a certain step belongs based on the step, and it is also possible to query the running result of this step and the connection with other steps, realizing the traceability of the workflow running result.

[0079] Figure 6 Shows the overall flowchart of the construction of a machine learning workflow according to an exemplary embodiment of the present disclosure, as Figure 6As shown in the figure, the overall process of workflow construction is as follows: According to the unique identifier of the workflow template provided by the user, read the target workflow template from the workflow template database; convert the components included in the target workflow template into target nodes included in the target hash calculation graph, and convert the connection relationships between the components into connection relationships between the target nodes to generate the target hash calculation graph; starting from the target nodes without upstream nodes, calculate the hash values of each target node included in the target hash calculation graph from top to bottom in sequence, and then query whether there is such a hash value of the target node in the step information database; if so, mark the workflow step corresponding to the target node as a successfully run workflow step, and read the running information of the successfully run workflow step from the step information database; if not, mark the workflow step corresponding to the target node as a workflow step to be run; generate a target workflow configuration file based on the running information of the successfully run workflow steps and the workflow steps to be run; call the interface to run the target workflow configuration file; monitor the running status of each step in the target workflow configuration file, and update the running status of the step to successfully run in the case of successful step operation, and store the running result of the step in the step information database.

[0080] Figure 7 FIG. shows a flowchart of calculating the hash value of each target node included in the target hash calculation graph according to an exemplary embodiment of the present disclosure. As Figure 7 shown, the specific process of calculating the hash value of each target node may include:

[0081] Step S701: Determine the input parameters corresponding to the target node and the configuration parameters corresponding to the target node according to the connection relationship between the target nodes and the parameters of the component.

[0082] Step S702: Obtain the parameter information of the input parameters of the component corresponding to the target node, and calculate the obtained parameter information using a hash algorithm to obtain the hash value of the input parameters of the component corresponding to the target node.

[0083] Step S703: Substitute the hash value of the upstream node of the target node and the parameter name of the output parameter corresponding to the upstream node of the target node into the hash value calculation formula of the output parameter corresponding to the node, and calculate the hash value of the output parameter corresponding to the upstream node of the target node.

[0084] Step S704: Sum the hash value of the input parameters of the component corresponding to the target node and the hash value of the output parameter corresponding to the upstream node of the target node to obtain the hash value of the input parameters corresponding to the target node.

[0085] Step S705: Query the parameter information of the configuration parameters corresponding to the target node. According to the queried parameter information, determine whether the parameter type of the configuration parameters corresponding to the target node is an index. If so, execute Step S706; if not, execute Step S707;

[0086] Step S706: Calculate the hash value of the file under the queried parameter information by means of an interface request, and determine the hash value of the file as the hash value of the configuration parameters corresponding to the target node;

[0087] Step S707: Calculate the hash value of the queried parameter information using a hash algorithm;

[0088] Step S708: Substitute the hash value of the input parameters corresponding to the target node and the hash value of the configuration parameters corresponding to the target node into the hash value calculation formula of the node to calculate the hash value of the target node.

[0089] The following is an embodiment of the apparatus of the present disclosure, which can be used to execute the method embodiment of the present disclosure. For details not disclosed in the embodiment of the apparatus of the present disclosure, please refer to the method embodiment of the present disclosure.

[0090] Figure 8 FIG. shows a schematic structural diagram of a construction apparatus for a machine learning workflow according to an exemplary embodiment of the present disclosure.

[0091] As Figure 8 shown, the construction apparatus 800 for a machine learning workflow may include: an acquisition module 801, a generation module 802, a calculation module 803, and a construction module 804.

[0092] Among them, the acquisition module 801 can be used to: acquire a target workflow template; the generation module 802 can be used to: generate a target hash calculation graph corresponding to the target workflow template according to the organizational structure information of the target workflow template; the calculation module 803 can be used to: calculate the hash value of the target node included in the target hash calculation graph according to the parameter information corresponding to the target workflow template by using the defined calculation method of the hash calculation graph; the construction module 804 can be used to: verify the hash value of the target node, determine the workflow steps that need to be run and the workflow steps that have been successfully run, and then use the workflow steps that need to be run and the workflow steps that have been successfully run to construct the target workflow.

[0093] Among them, the calculation method of the hash calculation graph may include: (1) The hash value calculation formula of the node is: sum the hash value of the input parameters corresponding to the node and the hash value of the configuration parameters corresponding to the node, and then calculate the sum result using a hash algorithm; (2) The hash value calculation formula of the output parameters corresponding to the node is: sum the hash value of the node and the hash value of the output parameter name corresponding to the node.

[0094] In an exemplary embodiment of the present disclosure, the generation module 802 may further be configured to: convert the components included in the target workflow template into target nodes included in the target hash calculation graph, convert the connection relationships between the components into connection relationships between the target nodes, and generate the target hash calculation graph.

[0095] In an exemplary embodiment of the present disclosure, the calculation module 803 may further be configured to: determine the input parameters corresponding to the target nodes and the configuration parameters corresponding to the target nodes according to the connection relationships between the target nodes and the parameters of the components; calculate the hash values of the input parameters corresponding to the target nodes according to the parameter information of the input parameters corresponding to the target nodes; query the parameter information of the configuration parameters corresponding to the target nodes, and calculate the hash values of the configuration parameters corresponding to the target nodes according to the queried parameter information; substitute the hash values of the input parameters corresponding to the target nodes and the hash values of the configuration parameters corresponding to the target nodes into the hash value calculation formula of the nodes, and calculate the hash values of the target nodes.

[0096] Among them, the input parameters corresponding to the target nodes may include: the input parameters of the components corresponding to the target nodes, the output parameters of the upstream nodes corresponding to the target nodes; and, the configuration parameters corresponding to the target nodes may include: the configuration parameters of the components corresponding to the target nodes.

[0097] In an exemplary embodiment of the present disclosure, the calculation module 803 may further be configured to: obtain the parameter information of the input parameters of the components corresponding to the target nodes, calculate the obtained parameter information by using a hash algorithm, and obtain the hash values of the input parameters of the components corresponding to the target nodes; substitute the hash values of the upstream nodes corresponding to the target nodes and the parameter names of the output parameters corresponding to the upstream nodes of the target nodes into the hash value calculation formula of the output parameters corresponding to the nodes, and calculate the hash values of the output parameters of the upstream nodes corresponding to the target nodes; sum the hash values of the input parameters of the components corresponding to the target nodes and the hash values of the output parameters of the upstream nodes corresponding to the target nodes, and obtain the hash values of the input parameters corresponding to the target nodes.

[0098] In an exemplary embodiment of the present disclosure, the calculation module 803 may further be configured to: determine whether the parameter type of the configuration parameters corresponding to the target nodes is an index; if so, calculate the hash value of the file under the queried parameter information by using an interface request method, and determine the hash value of the file as the hash value of the configuration parameters corresponding to the target nodes.

[0099] In an exemplary embodiment of the present disclosure, the building module 804 can also be used to: query whether the hash value of the target node exists in the step information database; if so, mark the workflow step corresponding to the target node as a successfully run workflow step, and read the running information of the successfully run workflow step from the step information database; if not, mark the workflow step corresponding to the target node as a workflow step to be run; generate a target workflow configuration file according to the running information of the successfully run workflow step and the workflow step to be run.

[0100] In an exemplary embodiment of the present disclosure, the above device may further include: a running module. The running module is used to: call an interface to run the target workflow configuration file; and, monitor the running status of each step in the target workflow configuration file, and update the running status of the step to successfully run in the case of successful step running, and store the running result of the step in the step information database.

[0101] It should be noted that the block diagrams shown in the above figures are functional entities, and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0102] Figure 9 A block diagram of an electronic device according to an exemplary embodiment of the present disclosure is shown. It should be noted that Figure 9 The electronic device shown is only an example, and should not bring any limitation to the functions and usage scope of the embodiments of the present invention.

[0103] As Figure 9 shown, the electronic device 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 902 or the program loaded from the storage section 908 into the random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the system 900 are also stored. The CPU 901, ROM 902, and RAM 903 are connected to each other through a bus 904. The input / output (I / O) interface 905 is also connected to the bus 904.

[0104] The following components are connected to the I / O interface 905: an input section 906 including a keyboard, a mouse, etc.; an output section 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc. as well as a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, a modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is mounted on the drive 910 as needed so that a computer program read therefrom is installed into the storage section 908 as needed.

[0105] Specifically, according to an embodiment of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, an embodiment of the present invention includes a computer program product that includes a computer program carried on a computer-readable medium, and the computer program includes program codes for performing the methods shown in the flowcharts. In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 909, and / or installed from the removable medium 911. When the computer program is executed by a central processing unit (CPU) 901, the above-described functions defined in the system of the present invention are performed.

[0106] It should be noted that the computer-readable medium shown in the present invention can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of a computer-readable storage medium can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present invention, a computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, and this computer-readable medium can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on a computer-readable medium can be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0107] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code, and the above module, program segment, or part of code contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, as well as the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0108] The units involved in the embodiments of the present invention can be implemented in software or in hardware. The described units can also be provided in a processor. For example, it can be described as: a processor includes a sending unit, an obtaining unit, a determining unit, and a first processing unit. Among them, the names of these units do not constitute a limitation on the unit itself in some cases. For example, the sending unit can also be described as "the unit that sends a picture acquisition request to the connected server".

[0109] As another aspect, the present disclosure also provides a computer-readable storage medium, which may be included in the electronic device described in the above embodiments; or may exist alone without being assembled into the electronic device. The above computer-readable storage medium carries one or more programs. When the above one or more programs are executed by an electronic device, the electronic device implements the methods described in the following embodiments. For example, the electronic device can implement the respective steps as Figure 2 shown.

[0110] According to one aspect of the present disclosure, there is provided a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the various alternative implementations of the above embodiments.

[0111] It should be understood that any number of elements in the drawings of the present disclosure are for illustration rather than limitation, and any naming is only for distinction without any limiting meaning.

[0112] Those skilled in the art will readily conceive of other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure, which follow the general principles of the present disclosure and include known common general knowledge or conventional technical means in the technical field not disclosed in the present disclosure. The specification and embodiments are only regarded as exemplary, and the true scope and spirit of the present disclosure are pointed out by the following claims.

[0113] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.

Claims

1. A method for constructing a machine learning workflow, characterized in that, Including: Obtain a target workflow template; Generate a target hash calculation graph corresponding to the target workflow template according to the organizational structure information of the target workflow template; Using the defined calculation method of the hash calculation graph, calculate the hash value of the target node included in the target hash calculation graph according to the parameter information corresponding to the target workflow template; Verify the hash value of the target node, determine the workflow steps that need to be run and the workflow steps that have been successfully run, and then use the workflow steps that need to be run and the workflow steps that have been successfully run to construct a target workflow; The step of using the defined calculation method of the hash calculation graph to calculate the hash value of the target node included in the target hash calculation graph according to the parameter information corresponding to the target workflow template includes: determining the input parameters corresponding to the target node and the configuration parameters corresponding to the target node according to the connection relationship between the target nodes and the parameters of the components; calculating the hash value of the input parameters corresponding to the target node according to the parameter information of the input parameters corresponding to the target node; querying the parameter information of the configuration parameters corresponding to the target node, and calculating the hash value of the configuration parameters corresponding to the target node according to the queried parameter information; substituting the hash value of the input parameters corresponding to the target node and the hash value of the configuration parameters corresponding to the target node into the node hash value calculation formula in the calculation method of the hash calculation graph to calculate the hash value of the target node; Wherein, the component is a component in the target workflow template, and the target node is obtained by converting the component.

2. The method according to claim 1, characterized in that, The calculation method of the hash calculation graph includes: The hash value calculation formula of the node is: adding the hash value of the input parameters corresponding to the node and the hash value of the configuration parameters corresponding to the node, and then calculating the sum result using a hash algorithm; The hash value calculation formula of the output parameters corresponding to the node is: adding the hash value of the node and the hash value of the output parameter name corresponding to the node.

3. The method according to claim 2, characterized in that, The step of generating a target hash calculation graph corresponding to the target workflow template according to the organizational structure information of the target workflow template includes: converting the components included in the target workflow template into target nodes included in the target hash calculation graph, and converting the connection relationship between the components into the connection relationship between the target nodes to generate the target hash calculation graph.

4. The method according to claim 3, characterized in that, The input parameters corresponding to the target node include: the input parameters of the component corresponding to the target node, the output parameters of the upstream node of the target node; and, the configuration parameters corresponding to the target node include: the configuration parameters of the component corresponding to the target node.

5. The method according to claim 4, characterized in that, The step of calculating the hash value of the input parameters corresponding to the target node according to the parameter information of the input parameters corresponding to the target node includes: Obtain the parameter information of the input parameters of the component corresponding to the target node, and calculate the obtained parameter information using a hash algorithm to obtain the hash value of the input parameters of the component corresponding to the target node; Substitute the hash value of the upstream node of the target node and the parameter name of the output parameter corresponding to the upstream node of the target node into the hash value calculation formula of the output parameter corresponding to the node, and calculate the hash value of the output parameter corresponding to the upstream node of the target node; Sum the hash value of the input parameter of the component corresponding to the target node and the hash value of the output parameter corresponding to the upstream node of the target node to obtain the hash value of the input parameter corresponding to the target node.

6. The method according to claim 3, characterized in that, The calculating the hash value of the configuration parameter corresponding to the target node includes: Determine whether the parameter type of the configuration parameter corresponding to the target node is an index; If so, calculate the hash value of the file under the queried parameter information by means of an interface request, and determine the hash value of the file as the hash value of the configuration parameter corresponding to the target node.

7. The method according to claim 1, characterized in that, The verifying the hash value of the target node, determining the workflow steps to be run and the workflow steps that have been successfully run, and then constructing a target workflow by using the workflow steps to be run and the workflow steps that have been successfully run includes: Query whether the hash value of the target node exists in the step information database; If so, mark the workflow step corresponding to the target node as a workflow step that has been successfully run, and read the running information of the workflow step that has been successfully run from the step information database; If not, mark the workflow step corresponding to the target node as a workflow step to be run; Generate a target workflow configuration file according to the running information of the workflow steps that have been successfully run and the workflow steps to be run.

8. The method according to claim 7, characterized in that, The method further includes: Calling an interface to run the target workflow configuration file; and Listening to the running status of each step in the target workflow configuration file, and updating the running status of the step to successful when the step runs successfully.

9. A device for constructing a machine learning workflow, characterized in that, including: An acquisition module for acquiring a target workflow template; A generation module for generating a target hash calculation graph corresponding to the target workflow template according to the organizational structure information of the target workflow template; A calculation module for calculating the hash value of the target node included in the target hash calculation graph according to the calculation method of the defined hash calculation graph and the parameter information corresponding to the target workflow template; A construction module for verifying the hash value of the target node, determining the workflow steps to be run and the workflow steps that have been successfully run, and then constructing a target workflow by using the workflow steps to be run and the workflow steps that have been successfully run; The calculation module is further configured to determine the input parameter corresponding to the target node and the configuration parameter corresponding to the target node according to the connection relationship between the target nodes and the parameters of the component; calculate the hash value of the input parameter corresponding to the target node according to the parameter information of the input parameter corresponding to the target node; Query the parameter information of the configuration parameter corresponding to the target node, and calculate the hash value of the configuration parameter corresponding to the target node according to the queried parameter information; Substitute the hash value of the input parameter corresponding to the target node and the hash value of the configuration parameter corresponding to the target node into the hash value calculation formula of the node in the calculation method of the hash calculation graph to calculate the hash value of the target node; Wherein, the component is a component in the target workflow template, and the target node is obtained by transforming the component.

10. An electronic device, characterized in that, Including: At least one processor; A storage device for storing at least one program, which when executed by the at least one processor causes the at least one processor to implement the method according to any one of claims 1 to 8.

11. A computer-readable storage medium storing computer-executable instructions thereon, characterized in that, When the executable instruction is executed by the processor, it implements the method according to any one of claims 1 to 8.

Citation Information

Patent Citations

  • Workflow management

    WO2021086362A1