Machine learning task management method and apparatus, electronic device, and storage medium

Abstract

The application provides a kind of management method, device, electronic equipment and storage medium of machine learning task, belong to machine learning technical field, method includes: according to the task range and task content of each machine learning task, construct node, based on the node relationship of each two nodes, node edge is constructed, to obtain graph model;Based on graph model and node vector prediction model, obtain node vector;All node vectors are stored in the form of K-dimensional tree, to obtain knowledge base, based on knowledge base management machine learning task.The application constructs the graph model of machine learning task by node and node edge, realizes the unified representation of machine learning task, converts machine learning task into node vector by graph model, to obtain knowledge base, effectively represents the relationship between different machine learning tasks, facilitates the sharing and reuse of machine learning task, improves the management efficiency of machine learning task, is conducive to improving the application efficiency of machine learning task.

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, and in particular to a method, apparatus, electronic device, and storage medium for managing machine learning tasks. Background Technology

[0002] With the rapid development of information and communication technologies, communication networks are becoming increasingly complex, and traditional network management and optimization methods are no longer sufficient to meet current needs. Machine learning, as an effective data analysis tool, is widely used in various aspects of communication networks. However, training machine learning models typically requires a large amount of data and computing resources, and the knowledge transfer efficiency between different tasks is low. This limits the effectiveness of machine learning in communication networks. Most existing knowledge representation methods for machine learning tasks are designed for specific domains and lack knowledge sharing and invocation between tasks.

[0003] In summary, the application efficiency of existing machine learning tasks is low. Summary of the Invention

[0004] This invention provides a method, apparatus, electronic device, and storage medium for managing machine learning tasks, in order to address the shortcomings of low application efficiency of machine learning tasks in the prior art and improve the application efficiency of machine learning tasks.

[0005] In a first aspect, the present invention provides a method for managing machine learning tasks, comprising: constructing nodes for each machine learning task according to the task scope and task content of each machine learning task; constructing node edges between each pair of nodes based on the node relationship between each pair of nodes, to obtain a graph model of multiple machine learning tasks; obtaining node vectors for each node based on the graph model and a node vector prediction model; storing all node vectors in the form of a K-dimensional tree to obtain a knowledge base for the machine learning tasks; and managing the machine learning tasks based on the knowledge base.

[0006] According to a machine learning task management method provided by the present invention, the step of obtaining the node vector of each node based on the graph model and the node vector prediction model includes: in the graph model, taking each node as the starting point, generating multiple node sequences for each node through a pre-set random walk step size; inputting the multiple node sequences of each node into the node vector prediction model to obtain the node vector of each node output by the node vector prediction model; wherein, the node vector prediction model is trained based on sample node sequences on the basis of a preset language model.

[0007] According to a machine learning task management method provided by the present invention, the node vector prediction model is obtained based on the following steps: obtaining training samples based on multiple sample node sequences of each sample node; training the parameters of a preset language model based on the gradient descent algorithm and the training samples to obtain multiple sample node vectors output by the preset language model, wherein the parameters include the window size of the preset language model; clustering the multiple sample node vectors based on the K-means clustering algorithm and setting a label for each clustering result; determining at least the precision and recall of the preset language model based on the sample node vectors and the labels of the clustering results; and determining that the training of the preset language model is complete based at least on the precision and the recall, thereby obtaining the node vector prediction model.

[0008] According to a method for managing machine learning tasks provided by the present invention, the step of constructing nodes for each machine learning task based on the task scope and task content of each machine learning task includes: determining a node scope based on the task scope, wherein the task scope includes at least the task type, task objective, and task domain of the machine learning task; determining node content based on the task content, wherein the task content includes at least the task name, task description, task input data, task required data, and task output data of the machine learning task; and obtaining each node based on the node scope and node content of each machine learning task.

[0009] According to a method for managing machine learning tasks provided by the present invention, the step of constructing node edges between every two nodes based on the node relationship between every two nodes to obtain a graph model of multiple machine learning tasks includes: determining the type of node edge according to the relationship values ​​and weight values ​​of multiple relationship types of each node relationship; determining a cell graph based on the type of node edge, wherein the cell graph includes the node edge and the two nodes of the node edge; and merging all the cell graphs into the graph model.

[0010] According to a machine learning task management method provided by the present invention, the step of determining the type of node edge based on the relation values ​​and weight values ​​of multiple relation types of each node relationship, and determining the cell graph based on the type of node edge, includes: performing a weighted summation of the relation values ​​and weight values ​​of each relation type of each node relationship to obtain a target score for each node relationship, wherein the target score is used to characterize the direction of the node relationship; when the target score is greater than a set score, determining the type of the node edge as a first edge with direction, and determining the cell graph as a directed graph based on the first edge; when the target score is less than or equal to the set score, determining the type of the node edge as a second edge without direction, and determining the cell graph as an undirected graph based on the second edge.

[0011] According to a method for managing machine learning tasks provided by the present invention, the step of managing the machine learning tasks based on the knowledge base includes: constructing retrieval information of the knowledge base according to a hash table or a nearest neighbor search method; obtaining, based on the retrieval information, a vector of associated nodes with a new machine learning task from the knowledge base; determining a transfer learning strategy based on the feature information and requirement information of the new machine learning task; and, based on the transfer learning strategy, inputting the associated node vector into a preset model of the new machine learning task for training to obtain a task model of the new machine learning task.

[0012] Secondly, the present invention provides a management device for machine learning tasks, comprising: a graph model determination module, which constructs nodes for each machine learning task according to the task scope and task content of each machine learning task, and constructs node edges for each pair of nodes based on the node relationship between each pair of nodes, so as to obtain a graph model of multiple machine learning tasks; a node vector determination module, which obtains a node vector for each node based on the graph model and a node vector prediction model; and a knowledge base determination module, which stores all the node vectors in the form of a K-dimensional tree to obtain a knowledge base for the machine learning tasks, and manages the machine learning tasks based on the knowledge base.

[0013] Thirdly, the present invention provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement a management method for any of the machine learning tasks described above.

[0014] Fourthly, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements a management method for machine learning tasks as described above.

[0015] The present invention provides a method, apparatus, electronic device, and storage medium for managing machine learning tasks. Based on the task scope and content of each machine learning task, nodes for each machine learning task are constructed. Based on the node relationships between any two nodes, node edges are constructed between any two nodes to obtain a graph model of multiple machine learning tasks. Based on the graph model and a node vector prediction model, node vectors for each node are obtained. All node vectors are stored in the form of a K-dimensional tree to obtain a knowledge base for the machine learning tasks. The machine learning tasks are then managed based on this knowledge base. This invention constructs a graph model of machine learning tasks through nodes and node edges, achieving a unified representation of machine learning tasks. By converting machine learning tasks into node vectors through the graph model to obtain a knowledge base, it effectively represents the relationships between different machine learning tasks, facilitating the sharing and reuse of machine learning tasks, improving the management efficiency of machine learning tasks, and ultimately enhancing the application efficiency of machine learning tasks. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0017] Figure 1 This is one of the flowcharts illustrating the machine learning task management method provided by the present invention;

[0018] Figure 2 This is the second flowchart of the machine learning task management method provided by the present invention;

[0019] Figure 3 This is a schematic diagram of the structure of the machine learning task management device provided by the present invention;

[0020] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0021] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0022] The following is combined with Figures 1-4This invention describes the machine learning task management method, apparatus, and electronic device provided in embodiments of the present invention.

[0023] Figure 1 This is one of the flowcharts illustrating the machine learning task management method provided by this invention, such as... Figure 1 As shown, a method for managing machine learning tasks includes, but is not limited to, steps S100 and S300, each of which is detailed below:

[0024] S100: Based on the task scope and content of each machine learning task, construct the nodes for each machine learning task. Based on the node relationship between every two nodes, construct the node edges between every two nodes to obtain a graph model for multiple machine learning tasks.

[0025] It should be noted that the execution subject of the embodiments of the present invention can be a server, computer equipment, such as a mobile phone, tablet computer, laptop computer, handheld computer, vehicle electronic equipment, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc.

[0026] Machine learning tasks are specific problems or objectives that model training needs to solve, such as network fault prediction, traffic prediction, and resource allocation.

[0027] Machine learning tasks are represented using a graph model, which abstracts each task as a node in the graph model. The relationship between any two machine learning tasks is abstracted as an edge between nodes in the graph model. The configuration information of each machine learning task, such as model parameters and feature identifiers, is used as the configuration information of each node, and this configuration information is represented in a unified format. For example, the configuration information of each node can be represented in tabular form.

[0028] Based on the task scope and task content of each machine learning task, construct nodes for each machine learning task. Specifically, based on the task scope, determine the node scope, which should at least include the task type, task objective, and task domain of the machine learning task; based on the task content, determine the node content, which should at least include the task name, task description, task input data, task required data, and task output data of the machine learning task; based on the node scope and node content of each machine learning task, obtain each node.

[0029] Obtain the task scope for each machine learning task, including task type, task objective, and task domain. Use the task scope of each machine learning task as the node scope of each node. Obtain the task content for each machine learning task, including task name, task description, task input data, required data, and task output data. Use the task content of each machine learning task as the node content of each node. Based on the node scope and node content, abstract each machine learning task into a node. Based on the node scope and node content of any two nodes, determine the node relationships between them.

[0030] S200: Based on graph models and node vector prediction models, obtain the node vector of each node.

[0031] A node vector prediction model is pre-trained. Nodes in the graph model are represented as vectors using graph embedding techniques. A node sequence for each node is obtained based on the graph model. This node sequence is then input into the node vector prediction model to obtain the node vector for each node.

[0032] S300: Store all node vectors in the form of a K-dimensional tree to obtain the knowledge base for machine learning tasks, and manage machine learning tasks based on the knowledge base.

[0033] K-Dimensional (KD) trees are multidimensional data structures often used to solve problems such as nearest neighbor search. In a KD tree, each node represents a point in a K-dimensional space. By continuously partitioning the space to build the tree structure, efficient organization and querying of data can be achieved.

[0034] Choose a node vector as the root node of the KD-tree. Recursively partition the space based on the values ​​of this node vector across all dimensions to construct the KD-tree. In each partitioning step, select one dimension to divide the space, enabling faster finding of nearest neighbors during queries. Insert all node vectors into the tree one by one according to the above KD-tree construction rules to obtain the KD-tree. This KD-tree constitutes a knowledge base for machine learning tasks. Multiple machine learning tasks are managed through this knowledge base; for example, by extracting the node vector and associated node vectors corresponding to each machine learning task.

[0035] The machine learning task management method provided in this invention constructs nodes for each machine learning task based on its scope and content, and constructs node edges between every two nodes based on their relationships, thus obtaining a graph model of multiple machine learning tasks. Based on the graph model and a node vector prediction model, the node vector of each node is obtained. All node vectors are stored in the form of a K-dimensional tree to obtain a knowledge base for the machine learning tasks, and the machine learning tasks are managed based on this knowledge base. This invention constructs a graph model of machine learning tasks through nodes and node edges, achieving a unified representation of machine learning tasks. By converting machine learning tasks into node vectors through the graph model to obtain the knowledge base, it effectively represents the relationships between different machine learning tasks, facilitating the sharing and reuse of machine learning tasks, improving the management efficiency of machine learning tasks, and ultimately enhancing the application efficiency of machine learning tasks.

[0036] Based on the above embodiments, the node vector of each node is obtained based on the graph model and the node vector prediction model, including steps S210 to S220, each step of which is as follows:

[0037] S210: In the graph model, starting from each node, multiple node sequences are generated for each node by using a pre-set random walk step size.

[0038] S220: Input multiple node sequences of each node into the node vector prediction model to obtain the node vector of each node output by the node vector prediction model; wherein, the node vector prediction model is trained based on the sample node sequence on the basis of the preset language model.

[0039] The preset language model includes a skip-gram language model. The step size for random walks is preset. A node is selected in the graph model, and starting from that node, multiple node sequences are randomly generated based on the step size.

[0040] A Skip-Gram language model is pre-trained based on the sample node sequences of multiple sample nodes to obtain a node vector prediction model. This model can convert nodes into node vectors based on their node sequences.

[0041] Multiple node sequences of a given node are input into a trained node vector prediction model. The node vector prediction model captures the semantic and contextual relationships of each node sequence and updates the vector representation of that node based on these relationships, ultimately obtaining the node vector.

[0042] This invention provides a low-dimensional vector representation of each node by inputting the node sequence into the node vector prediction model, which is beneficial for improving the management efficiency of machine learning tasks.

[0043] Based on the above embodiments, the node vector prediction model is obtained based on steps S221 to S225, and the specific steps are as follows:

[0044] S221: Obtain training samples based on the sequence of multiple sample nodes for each sample node.

[0045] S222: Based on the gradient descent algorithm and training samples, train the parameters of the preset language model to obtain multiple sample node vectors output by the preset language model. The parameters include the window size of the preset language model.

[0046] S223: Based on the K-means clustering algorithm, cluster multiple sample node vectors and set a label for each clustering result.

[0047] S224: Based on the sample node vectors and the labels from the clustering results, at least determine the accuracy and recall of the preset language model.

[0048] S225: Based at least on accuracy and recall, determine that the training of the preset language model is complete in order to obtain the node vector prediction model.

[0049] Obtain multiple sample node sequences for each sample node. Based on these sequences, obtain training samples. Set parameters for the preset language model (Skip-Gram language model), such as window size. Train the Skip-Gram language model using gradient descent and the training samples. During training, obtain multiple sample node vectors output by the Skip-Gram language model. Use K-means clustering to cluster these sample node vectors and assign a label to each cluster result. Calculate the difference between the sample node vector and the label of its corresponding cluster result. Based on this difference, calculate evaluation metrics such as accuracy and recall of the Skip-Gram language model. When the accuracy is greater than a set accuracy and the recall is less than a set recall, the Skip-Gram language model training is complete, and a node vector prediction model is obtained.

[0050] This invention employs a gradient descent algorithm to train a pre-defined language model, clusters the training results to obtain labels for sample node sequences, and then obtains information such as the accuracy and recall of the pre-defined language model, thereby improving the accuracy of determining evaluation metrics such as accuracy and recall. By using accuracy and recall to determine the completion of the pre-defined language model training, it is beneficial to improve the accuracy of the node vector prediction model.

[0051] Based on the above embodiments, node edges are constructed between every two nodes based on the node relationships between them to obtain graph models for multiple machine learning tasks, including steps S110 to S120, each step of which is as follows:

[0052] S110: Determine the type of node edge based on the relation values ​​and weight values ​​of multiple relation types for each node relationship. Based on the type of node edge, determine the cell graph, which includes the node edge and the two nodes of the node edge.

[0053] S120: Merge all unit diagrams into a graphical model.

[0054] The relationship between any two nodes is determined based on their scope and content. Relationship types include dependency, similarity, and mutual exclusion. A weight value for each relationship type is determined based on the scope and content of each pair of nodes.

[0055] Based on the relation values ​​and weight values ​​of multiple relation types for each node relationship, the type of node edge is determined. Based on the type of node edge, the unit graph is determined. Specifically, the relation values ​​and weight values ​​of each relation type for each node relationship are weighted and summed to obtain the target score for each node relationship. The target score is used to represent the direction of the node relationship. When the target score is greater than a set score, the type of node edge is determined to be a first edge with direction. Based on the first edge, the unit graph is determined to be a directed graph. When the target score is less than or equal to the set score, the type of node edge is determined to be a second edge without direction. Based on the second edge, the unit graph is determined to be an undirected graph.

[0056] For example, regarding the relationship between nodes A and B, the weight of dependency is 0.7, similarity is 0.3, and mutual exclusion is 0. A pre-set value for each relationship type is used; for example, dependency is scored at 100 points, similarity at 60 points, and mutual exclusion at 0 points. The weighted sum of the relationship values ​​and weights for each relationship type between nodes A and B is calculated to obtain the target score for their relationship. For example, the target score is 100 × 0.7 + 60 × 0.3 + 0 × 0 = 88. The target score is set to 50. Since the target score is greater than 50, the edge type between nodes A and B is designated as a first-sided directed edge, and the cell graph of nodes A and B is a directed graph.

[0057] If the target score for the node relationship between nodes C and D is 10, which is less than the set score of 50, then the node edges of nodes C and D are of type undirected second edges, and the cell graph of nodes C and D is an undirected graph.

[0058] Determine the unit graph between every two nodes, merge all the unit graphs, and obtain the graph model.

[0059] The embodiments of the present invention construct a graph model based on the weight value of the relation type and the relation value, which improves the accuracy of determining the unit graph and is beneficial to improving the accuracy of the graph model.

[0060] Based on the above embodiments, the management of machine learning tasks based on a knowledge base includes steps S400 to S700, each of which is detailed below:

[0061] S400: Construct retrieval information for the knowledge base based on hash tables or nearest neighbor search methods.

[0062] S500: Based on the retrieved information, obtain the associated node vectors with the new machine learning task from the knowledge base.

[0063] S600: Determine the transfer learning strategy based on the feature and requirement information of the new machine learning task.

[0064] S700: Based on the transfer learning strategy, the associated node vectors are input into the preset model of the new machine learning task for training, so as to obtain the task model of the new machine learning task.

[0065] The knowledge base supports efficient data storage, retrieval, and update operations. Retrieval information for the knowledge base is constructed using hash tables or nearest neighbor search methods. When a new machine learning task needs to be completed, the knowledge base is retrieved using the retrieval information to find related node vectors relevant to the new machine learning task. The characteristics and requirements of the new machine learning task are analyzed to determine appropriate transfer learning strategies, such as fine-tuning and feature extraction. Based on the selected transfer learning strategy, the related node vectors are applied to the model training for the new machine learning task, resulting in the task model for the new machine learning task.

[0066] The embodiments of the present invention obtain related node vectors through knowledge base and retrieval information, which helps to improve the efficiency of training preset models for new machine learning tasks and improves the application efficiency of machine learning tasks.

[0067] This invention provides a method for managing machine learning tasks, such as... Figure 2 As shown.

[0068] (1) Representation of machine learning tasks: Machine learning tasks are represented by graph models, which are represented as unit graphs composed of nodes and edges. Nodes represent machine learning tasks, and edges represent the relationships between machine learning tasks. Nodes in the graph model are represented as node vectors using graph embedding techniques.

[0069] (2) Storage and indexing of machine learning tasks: A knowledge base is built based on node vectors. Relevant machine learning tasks are quickly retrieved and invoked based on the retrieved information.

[0070] (3) Invocation and transfer of machine learning tasks: When training the task model for a new machine learning task, the associated node vectors of the new machine learning task are retrieved by searching information. According to the selected transfer learning strategy, the associated node vectors are used in the training of the model for the child machine learning task to improve training efficiency and task model performance.

[0071] The machine learning task management method provided in this invention realizes a unified identifier for machine learning tasks, provides a unified knowledge representation framework for different machine learning tasks, facilitates the sharing and reuse of machine learning tasks, improves the ability to express reasoning, and the graph model can represent complex relationships and attribute information in communication networks, providing rich reasoning operations and improving the practicality of machine learning tasks.

[0072] This invention also provides a management device for machine learning tasks, such as... Figure 3 As shown, Figure 3 This is a schematic diagram of the machine learning task management device provided by the present invention. It should be noted that the machine learning task management device provided in this embodiment can execute the machine learning task management method described in any of the above embodiments during actual operation; however, this embodiment will not elaborate further on this.

[0073] Reference Figure 3 Embodiments of the present invention provide a management device for machine learning tasks, comprising:

[0074] The graph model determination module 301 constructs nodes for each machine learning task based on the task scope and content of each machine learning task, and constructs node edges for each pair of nodes based on the node relationship between each pair of nodes, so as to obtain a graph model for multiple machine learning tasks.

[0075] The node vector determination module 302 obtains the node vector of each node based on the graph model and the node vector prediction model.

[0076] The knowledge base determination module 303 stores all node vectors in the form of a K-dimensional tree to obtain the knowledge base for the machine learning task, and manages the machine learning task based on the knowledge base.

[0077] The machine learning task management device provided in this invention constructs nodes for each machine learning task based on its scope and content, and constructs node edges between every two nodes based on their relationships, thereby obtaining a graph model of multiple machine learning tasks. Based on the graph model and a node vector prediction model, it obtains the node vector for each node. All node vectors are stored in the form of a K-dimensional tree to obtain a knowledge base for the machine learning tasks, and the machine learning tasks are managed based on this knowledge base. This invention constructs a graph model of machine learning tasks through nodes and node edges, achieving a unified representation of machine learning tasks. By converting machine learning tasks into node vectors through the graph model to obtain the knowledge base, it effectively represents the relationships between different machine learning tasks, facilitating the sharing and reuse of machine learning tasks, improving the management efficiency of machine learning tasks, and ultimately enhancing the application efficiency of machine learning tasks.

[0078] In one embodiment, the node vector determination module 302 is used to: in the graph model, starting from each node, generate multiple node sequences for each node by using a pre-set random walk step size; input the multiple node sequences of each node into the node vector prediction model to obtain the node vector of each node output by the node vector prediction model; wherein, the node vector prediction model is trained based on the sample node sequences on the basis of a preset language model.

[0079] In one embodiment, the node vector determination module 302 is used to determine a node vector prediction model based on the following steps: obtaining training samples based on multiple sample node sequences for each sample node; training the parameters of a preset language model based on the gradient descent algorithm and the training samples to obtain multiple sample node vectors output by the preset language model, the parameters including the window size of the preset language model; clustering the multiple sample node vectors based on the K-means clustering algorithm and setting labels for each clustering result; determining at least the accuracy and recall of the preset language model based on the sample node vectors and the labels of the clustering results; and determining that the preset language model training is complete based at least on the accuracy and recall, thereby obtaining the node vector prediction model.

[0080] In one embodiment, the graph model determination module 301 is used to: determine the node range based on the task range, wherein the task range includes at least the task type, task objective, and task domain of the machine learning task; determine the node content based on the task content, wherein the task content includes at least the task name, task description, task input data, task required data, and task output data of the machine learning task; and obtain each node based on the node range and node content of each machine learning task.

[0081] In one embodiment, the graph model determination module 301 is used to: determine the type of node edge based on the relation values ​​and weight values ​​of multiple relation types of each node relationship; determine the cell graph based on the type of node edge, the cell graph including the node edge and the two nodes of the node edge; and merge all cell graphs into a graph model.

[0082] In one embodiment, the graph model determination module 301 is used to: perform a weighted summation of the relation values ​​and weight values ​​for each relation type of each node relation to obtain a target score for each node relation, the target score being used to characterize the direction of the node relation; when the target score is greater than a set score, determine the type of the node edge as a first edge with direction, and based on the first edge, determine the cell graph as a directed graph; when the target score is less than or equal to the set score, determine the type of the node edge as a second edge without direction, and based on the second edge, determine the cell graph as an undirected graph.

[0083] In one embodiment, the knowledge base determination module 303 is used to: construct retrieval information of the knowledge base according to a hash table or a nearest neighbor search method; obtain the associated node vectors with the new machine learning task from the knowledge base based on the retrieval information; determine the transfer learning strategy based on the feature information and requirement information of the new machine learning task; and input the associated node vectors into the preset model of the new machine learning task for training based on the transfer learning strategy to obtain the task model of the new machine learning task.

[0084] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention, such as... Figure 4 As shown, the electronic device may include a processor 410, a communication interface 420, a memory 430, and a communication bus 440. The processor 410, communication interface 420, and memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute a machine learning task management method. This method includes: constructing nodes for each machine learning task based on its scope and content; constructing node edges between every two nodes based on their relationships to obtain a graph model of multiple machine learning tasks; obtaining the node vector for each node based on the graph model and a node vector prediction model; storing all node vectors in the form of a K-dimensional tree to obtain a knowledge base for the machine learning tasks; and managing the machine learning tasks based on the knowledge base.

[0085] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0086] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements a method for managing machine learning tasks provided in the above embodiments. The method includes: constructing nodes for each machine learning task based on the task scope and task content of each machine learning task; constructing node edges between each pair of nodes based on the node relationship between each pair of nodes to obtain a graph model of multiple machine learning tasks; obtaining node vectors for each node based on the graph model and a node vector prediction model; storing all node vectors in the form of a K-dimensional tree to obtain a knowledge base for machine learning tasks; and managing machine learning tasks based on the knowledge base.

[0087] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0088] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0089] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for managing machine learning tasks, characterized in that, include: Based on the task scope and content of each machine learning task, a node for each machine learning task is constructed, and a node edge is constructed between each pair of nodes based on the node relationship between each pair of nodes, so as to obtain a graph model of multiple machine learning tasks. Based on the graph model and node vector prediction model, obtain the node vector of each node; All the node vectors are stored in the form of a K-dimensional tree to obtain the knowledge base for the machine learning task. The retrieval information of the knowledge base is constructed based on a hash table or a nearest neighbor search method; Based on the retrieval information, obtain the associated node vectors with the newly added machine learning task from the knowledge base; Based on the feature information and requirement information of the newly added machine learning task, a transfer learning strategy is determined. Based on the transfer learning strategy, the associated node vectors are input into the preset model of the new machine learning task for training to obtain the task model of the new machine learning task; the K-dimensional tree is used for efficient organization and querying of data.

2. The method for managing machine learning tasks according to claim 1, characterized in that, The step of obtaining the node vector for each node based on the graph model and the node vector prediction model includes: In the graph model, starting from each node, multiple node sequences are generated for each node by using a pre-set random walk step size; Multiple node sequences of each node are input into a node vector prediction model to obtain the node vector of each node output by the node vector prediction model; wherein, the node vector prediction model is trained based on sample node sequences on the basis of a preset language model.

3. The method for managing machine learning tasks according to claim 2, characterized in that, The node vector prediction model is obtained based on the following steps: Training samples are obtained based on the sequence of multiple sample nodes for each sample node; Based on the gradient descent algorithm and the training samples, the parameters of the preset language model are trained to obtain multiple sample node vectors output by the preset language model, wherein the parameters include the window size of the preset language model; Based on the K-means clustering algorithm, multiple sample node vectors are clustered, and a label is set for each clustering result; Based on the sample node vectors and the labels of the clustering results, at least the accuracy and recall of the preset language model are determined; Based at least on the accuracy and recall, it is determined that the training of the preset language model is complete, so as to obtain the node vector prediction model.

4. The method for managing machine learning tasks according to claim 1, characterized in that, The step of constructing nodes for each machine learning task based on its scope and content includes: Based on the task scope, the node scope is determined, wherein the task scope includes at least the task type, task objective, and task domain of the machine learning task; Based on the task content, determine the node content, which includes at least the task name, task description, task input data, task required data, and task output data of the machine learning task. Each node is obtained based on the node range and node content for each machine learning task.

5. The method for managing machine learning tasks according to claim 1, characterized in that, The process of constructing node edges between every two nodes based on the node relationships between them to obtain graph models for multiple machine learning tasks includes: Based on the relationship values ​​and weight values ​​of multiple relationship types for each node relationship, the type of node edge is determined, and based on the type of node edge, a cell graph is determined, wherein the cell graph includes the node edge and the two nodes of the node edge; All the unit diagrams are merged into the graph model.

6. The method for managing machine learning tasks according to claim 5, characterized in that, The step of determining the type of node edge based on the relationship values ​​and weight values ​​of multiple relationship types for each node relationship, and determining the cell graph based on the type of node edge, includes: The relationship value and the weight value for each relationship type of each node relationship are weighted and summed to obtain the target score for each node relationship. The target score is used to characterize the direction of the node relationship. When the target score is greater than the set score, the type of the node edge is determined to be a first edge with a direction, and based on the first edge, the cell graph is determined to be a directed graph; When the target score is less than or equal to the set score, the node edge is determined to be a second edge without direction, and based on the second edge, the cell graph is determined to be an undirected graph.

7. A management device for machine learning tasks, characterized in that, include: The graph model determination module constructs nodes for each machine learning task based on the task scope and content of each machine learning task, and constructs node edges between each pair of nodes based on the node relationship between each pair of nodes, so as to obtain graph models for multiple machine learning tasks. The node vector determination module obtains the node vector of each node based on the graph model and the node vector prediction model. The knowledge base determination module stores all the node vectors in the form of a K-dimensional tree to obtain the knowledge base for the machine learning task; and constructs the retrieval information of the knowledge base according to a hash table or the nearest neighbor search method. Based on the retrieval information, obtain the associated node vectors with the new machine learning task from the knowledge base; based on the feature information and requirement information of the new machine learning task, determine the transfer learning strategy. Based on the transfer learning strategy, the associated node vectors are input into the preset model of the new machine learning task for training to obtain the task model of the new machine learning task; the K-dimensional tree is used for efficient organization and querying of data.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the management method for the machine learning task as described in any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the management method for the machine learning task as described in any one of claims 1 to 6.

Images