Model building method, device and electronic equipment based on machine learning platform
By creating multiple nodes in the machine learning platform and running independently, only the algorithm of the node to be adjusted is required for parameter adjustment, which solves the problem of low parameter adjustment efficiency in the existing technology and realizes efficient model training.
Patent Information
- Application Number
- CN202311628007.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-11-30
- Publication Date
- 2024-12-27
- Estimated Expiration
- 2043-11-30
AI Technical Summary
In machine learning technology, algorithm parameter adjustment efficiency is low, and the entire container code needs to be repeatedly run, resulting in a longer time consumption.
By creating multiple nodes in the target container, each node has an initial algorithm with preset parameters, obtaining input data for training, adjusting preset parameters, and generating a target model, you only need to run the algorithm of the node to be adjusted to improve the efficiency of parameter adjustment.
Improve the efficiency of parameter adjustment and model training efficiency, and reduce resource usage and time consumption.
Smart Images
Figure CN117591244B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of machine learning technology, and specifically to a model building method, device and electronic device based on a machine learning platform. Background Art
[0002] In machine learning technology, in order to adjust the parameters of the algorithm corresponding to the model, the model code needs to be input into the container and the entire algorithm of the model needs to be run through the container. In related technologies, in order to adjust the parameters in the algorithm corresponding to the model, the code in the entire container needs to be run to obtain the training results, and then the parameters are adjusted based on the training results. Since the entire code needs to be run repeatedly when adjusting the parameters, it takes a long time to adjust the parameters, resulting in low efficiency in adjusting the algorithm parameters.
[0003] It can be seen that the related art has the problem of low efficiency in adjusting algorithm parameters. Summary of the invention
[0004] The embodiments of the present application provide a model building method, device, electronic device and readable storage medium based on a machine learning platform to solve the problem of low algorithm parameter adjustment efficiency in the related art.
[0005] To solve the above problems, this application is implemented as follows:
[0006] In a first aspect, an embodiment of the present application provides a model building method based on a machine learning platform, comprising:
[0007] Starting a target container, and creating a plurality of nodes in a page of the target container, wherein each node includes an initial algorithm with preset parameters;
[0008] Acquire input data of a target node, and train an initial algorithm of the target node based on the input data to obtain a training result, wherein the target node is any node among the multiple nodes;
[0009] Adjust the preset parameters based on the training results to obtain target parameters corresponding to the target node;
[0010] A target model is generated based on the target parameters respectively corresponding to the multiple nodes.
[0011] In a second aspect, the embodiment of the present application further provides a model building device based on a machine learning platform, comprising:
[0012] A creation module, used to start a target container and create multiple nodes in a page of the target container, wherein each node includes an initial algorithm with preset parameters;
[0013] A first training module is used to obtain input data of a target node, and train an initial algorithm of the target node based on the input data to obtain a training result, wherein the target node is any node among the multiple nodes;
[0014] An adjustment module, used to adjust the preset parameters based on the training results to obtain target parameters corresponding to the target node;
[0015] A generation module is used to generate a target model based on the target parameters corresponding to the multiple nodes.
[0016] In a third aspect, an embodiment of the present application further provides an electronic device comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed by the processor, the steps in the model building method based on the machine learning platform as described in the first aspect above are implemented.
[0017] In a fourth aspect, an embodiment of the present application further provides a readable storage medium for storing a program, which, when executed by a processor, implements the steps in the model building method based on the machine learning platform as described in the first aspect above.
[0018] In an embodiment of the present application, by starting a target container, a plurality of nodes are created in a page of the target container, wherein each node includes an initial algorithm with preset parameters; the input data of the target node is obtained, and the initial algorithm of the target node is trained based on the input data to obtain a training result, wherein the target node is any one of the plurality of nodes; the preset parameters are adjusted based on the training result to obtain the target parameters corresponding to the target node; and a target model is generated based on the target parameters corresponding to the plurality of nodes. In this way, in the process of adjusting the preset parameters of the initial algorithm, it is not necessary to run the initial algorithm of all nodes in the entire target container. It is only necessary to run the algorithm of the node corresponding to the preset parameters to be adjusted, so as to adjust the preset parameters to be adjusted and obtain the adjusted target parameters to improve the adjustment efficiency of the parameters. At the same time, after adjusting the target parameters corresponding to the plurality of nodes, the target model is generated based on the target parameters corresponding to the plurality of nodes, and the training of the model is completed, thereby improving the efficiency of the model training. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.
[0020] Figure 1is a schematic diagram of an interface of a machine learning platform provided in an embodiment of the present application;
[0021] Figure 2 It is a flow chart of a model building method based on a machine learning platform provided in an embodiment of the present application;
[0022] Figure 3 is a schematic diagram of multiple nodes in a target container provided by an embodiment of the present application;
[0023] Figure 4 is a model training flow chart provided in an embodiment of the present application;
[0024] Figure 5 is a training diagram of node X and node Y provided in an embodiment of the present application;
[0025] Figure 6 is a connection diagram of multiple nodes provided in an embodiment of the present application;
[0026] Figure 7 It is a structural diagram of a model building device based on a machine learning platform provided in an embodiment of the present application;
[0027] Figure 8 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0028] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.
[0029] Modeling is one of the key technologies widely used in machine learning technology. In related technologies, modeling requires writing algorithm code according to the corresponding scenario of modeling, and running the algorithm code in the container. After repeated parameter adjustment and running, the model can be optimized. In related technologies, each time the algorithm code is run, it is necessary to start the container, run all the algorithm codes in the container, and then save the running results to the disk. When it is necessary to analyze the running results, it is necessary to read the running results in the disk into the memory, and then process the running results in the memory. Therefore, in related technologies, in order to adjust the parameters of the algorithm, it is necessary to start the container multiple times to run the algorithm, and then read the running results in the disk into the memory, resulting in more resources being occupied. At the same time, running the entire algorithm will also consume a lot of time, resulting in low efficiency in adjusting the algorithm parameters.
[0030] In order to improve the efficiency of parameter adjustment of the algorithm, the embodiment of the present application provides a machine learning platform, such as Figure 1 The interface of the machine learning platform is shown, through which multiple containers can be started, and the target container is any one of the multiple containers. Multiple nodes can be newly created in the target container, each of the multiple nodes is used to add an initial algorithm, the initial algorithm includes preset parameters, and the multiple nodes include the target node;
[0031] After the target container is started, the target node is used to obtain input data, and to test the initial algorithm of the target node based on the input data to obtain a training result.
[0032] The above-mentioned target container is a container including multiple nodes, each of which can run independently. By adding an initial algorithm to each node, the initial algorithm corresponding to the node can be run by running a certain node, and there is no need to run all the algorithms of the container, so as to reduce the time required for adjusting parameters.
[0033] The above nodes are virtual nodes configured in the target container. Each node corresponds to at least one operator in the target container, and each operator can run independently. When model training is required, an initial algorithm is added to each node, and the initial algorithm is trained through the operator corresponding to the node, thereby realizing the training of different algorithms in sequence.
[0034] See also Figure 2 , Figure 2 is a flow chart of a model building method based on a machine learning platform provided in an embodiment of the present application, such as Figure 2 As shown, the following steps are included:
[0035] Step S1: Start a target container and create multiple nodes in a page of the target container, wherein each node includes an initial algorithm with preset parameters.
[0036] The target container is Figure 1 The container shown in the figure needs to start the target container before model training, and perform model training through the nodes in the target container. The above multiple nodes are multiple nodes created on the page of the target container. The number of nodes is related to the model to be trained. The model corresponds to one or more initial algorithms to be trained, and each initial algorithm corresponds to an independent node.
[0037] Each initial algorithm includes preset parameters, and the preset parameters are parameters in each initial algorithm. The preset parameters in the initial algorithm are adjusted through training data, thereby realizing the training of the initial algorithm.
[0038] Step S2: obtaining input data of a target node, and training an initial algorithm of the target node based on the input data to obtain a training result, wherein the target node is any node among the multiple nodes;
[0039] The above input data is the input data of the target node, and the input data is used to train the initial algorithm of the target node and adjust the preset parameters of the initial algorithm until the preset parameters meet the set requirements. The above training result is the result obtained by testing the input data by the initial algorithm in the target node. After the initial algorithm in the target node obtains the training result through the test, the training result is output so that the preset parameters are adjusted based on the training result.
[0040] Step S3: adjusting the preset parameters based on the training results to obtain target parameters corresponding to the target node.
[0041] The above target parameters are adjusted parameters. The above preset parameters are adjusted based on the training results, and the preset parameters can be adjusted until the target parameters that meet the training end conditions are obtained. For example, the preset parameters are trained based on the training results until the training results output by the target node are within the set value range, and the value of the preset parameters in the target container is the target parameter; another example is setting the number of adjustments, and when the number of times the preset parameters are adjusted based on the training results reaches the number of adjustments, the value of the preset parameters in the target container is the target parameter.
[0042] Step S4: Generate a target model based on the target parameters corresponding to the multiple nodes respectively.
[0043] The above-mentioned multiple nodes are respectively set with preset parameters. By adjusting the preset parameters of the initial algorithms of the multiple nodes, the target parameters corresponding to the multiple nodes are obtained to realize the training of the algorithm of each node; then the adjusted algorithms are generated based on the target parameters corresponding to the multiple nodes, and the adjusted algorithms of the multiple nodes are assembled to generate the trained target model.
[0044] In an embodiment of the present application, by starting a target container, a plurality of nodes are created in a page of the target container, wherein each node includes an initial algorithm with preset parameters; the input data of the target node is obtained, and the initial algorithm of the target node is trained based on the input data to obtain a training result, wherein the target node is any one of the plurality of nodes; the preset parameters are adjusted based on the training result to obtain the target parameters corresponding to the target node; and a target model is generated based on the target parameters corresponding to the plurality of nodes. In this way, in the process of adjusting the preset parameters of the initial algorithm, it is not necessary to run the initial algorithm of all nodes in the entire target container. It is only necessary to run the algorithm of the node corresponding to the preset parameters to be adjusted, so as to adjust the preset parameters to be adjusted and obtain the adjusted target parameters to improve the adjustment efficiency of the parameters. At the same time, after adjusting the target parameters corresponding to the plurality of nodes, the target model is generated based on the target parameters corresponding to the plurality of nodes, and the training of the model is completed, thereby improving the efficiency of the model training.
[0045] In one embodiment, before obtaining the input data of the target node, the method further includes:
[0046] Configuring at least one operator for each of the multiple nodes, and determining the initial algorithm based on the at least one operator;
[0047] Establish a connection relationship between the multiple nodes, the connection relationship is used to characterize the upper node and / or lower node corresponding to each node in the multiple nodes, the input data of each node is the output data of the corresponding upper node, and the output data of each node is the input data of the corresponding lower node.
[0048] The above-mentioned operator is configured for each node and is used to calculate the input data of the node. By configuring at least one operator for each node, establishing the operation order between operators, the input-output relationship between operators and / or configuring parameters in the operator, the initial algorithm of each node is determined. Among them, the preset parameters include the parameters in the operator, and the preset parameters are adjusted according to the training results, specifically, the parameters in at least one operator configured for the node are adjusted.
[0049] The above connection relationship is the connection relationship between every two nodes. When there is a connection relationship between two nodes, the output data of one node is the input data of another node. By determining the connection relationship between multiple nodes, a connection network is formed in which two nodes are mutually associated. The connection network is a virtual connection network composed of multiple nodes. Through the connection network, the upper node and the lower node of each node can be determined based on the connection network.
[0050] For example, Figure 3As shown, the target container includes four nodes, node 1, node 2, node 3 and node 4 are connected in sequence. When the algorithm of node 2 needs to be trained, the parent node of node 2 is determined to be node 1. In this case, the output data of node 1 is obtained as the input data of node 2, and the initial algorithm of node 2 is trained based on the output data of node 1.
[0051] Furthermore, when the target node has no upper node, that is, when the target node is the first node in the connection network, the input data of the target node is the input data of the target container. In this case, the initial algorithm of the target node is trained based on the input data of the target container.
[0052] In an embodiment of the present application, at least one operator is configured for each of the multiple nodes, and the initial algorithm is determined based on the at least one operator; a connection relationship is established between the multiple nodes to form a connection network, and the upper and lower nodes of each node can be determined based on the connection network, thereby realizing the training of the initial algorithm of the target node through the output data of the upper node corresponding to the target node.
[0053] In one embodiment, the step of acquiring input data of the target node and training the initial algorithm of the target node based on the input data to obtain a training result includes:
[0054] In the case where the target node has an upper-level node, obtaining output data of the upper-level node of the target node as input data of the target node;
[0055] Processing the input data based on the at least one operator of the target node to obtain the training result; wherein the initial algorithm includes the at least one operator, each operator is configured with an operator initial parameter, and the preset parameters include the operator initial parameter of the at least one operator;
[0056] The adjusting the preset parameters based on the training results to obtain the target parameters corresponding to the target node includes:
[0057] Adjusting the initial operator parameters of each operator of the at least one operator of the target node based on the training result until a training end condition is met, thereby obtaining the operator result parameters corresponding to the at least one operator;
[0058] Determine a target parameter of the target node according to an operator result parameter corresponding to the at least one operator.
[0059] It should be noted that, when the target node has an upper-level node, the output data of the upper-level nodes of the target node are used as the input data of the target node, and when the target node does not have an upper-level node, that is, when the target node is the first node in the target container, the input data of the target node is the input data of the target container.
[0060] At least one operator is configured in the above initial algorithm, and the operator includes initial parameters. The adjustment of the preset parameters of the initial algorithm is specifically to adjust the operator initial parameters of each operator in at least one operator of the target node based on the training results until the training cutoff condition is met, so as to obtain the adjusted initial parameters corresponding to each operator, that is, the operator result parameters corresponding to each operator. Then, the target parameters are determined by the operator result parameters corresponding to each operator in the target node, and the target parameters are the operator training results combined according to the operation order between each operator and / or the input-output relationship between the operators.
[0061] The training cutoff condition may be one of the following:
[0062] (1) Calculate the loss value of the initial algorithm including the operator training results of each operator, and the loss value is less than or equal to the set loss threshold;
[0063] (2) The number of times the initial parameters of each operator are adjusted based on the training results is equal to the set number;
[0064] (3) The adjusted initial algorithm including the operator training results of each operator is used to calculate the input data, and the obtained training results are within the set range.
[0065] When the above training cutoff conditions are met, the preset parameters finally adjusted are the target parameters.
[0066] The above loss value is used to characterize whether the adjusted initial parameters in the operator in the current initial algorithm meet the training expectations. When the loss value is less than or equal to the set loss threshold, the adjusted initial parameters in the operator in the current initial algorithm meet the training expectations. In this case, the adjusted initial parameters in the operator in the current initial algorithm are the operator result parameters. When the loss value is greater than the set loss threshold, the adjusted initial parameters in the operator need to be further adjusted.
[0067] In addition, in addition to determining whether the preset parameters meet the training expectations through the loss value, it is also possible to determine whether the training expectations are met by adjusting the number of times the initial parameters are adjusted in the operator. When the number of times the initial parameters are adjusted in the operator based on the training results is equal to the set number of times, the training is considered to meet expectations. In this case, the last adjusted parameter is set as the operator result parameter.
[0068] Furthermore, whether the training expectations are met can also be determined by whether the training results are within a set range, wherein, when the training results output by the initial algorithm including the adjusted initial parameters in the operator are within the set range, the adjusted initial parameters in the operator are considered to be the operator result parameters.
[0069] In an embodiment of the present application, when the target node has an upper-level node, the output data of the upper-level node of the target node is obtained as the input data of the target node; the input data is processed based on at least one operator of the target node to obtain a training result; wherein the initial algorithm includes at least one operator, each operator is configured with an operator initial parameter, and the preset parameters include the operator initial parameter of at least one operator; based on the training result, the operator initial parameter of each operator of at least one operator of the target node is adjusted until the training end condition is met, and the operator result parameter corresponding to at least one operator is obtained; according to the operator result parameter corresponding to at least one operator, the target parameter of the target node is determined, so as to determine the target parameter of the target node through the training result, thereby realizing independent training of a node in the target container.
[0070] In one embodiment, after adjusting the preset parameters based on the training results to obtain the target parameters corresponding to the target node, the method further includes:
[0071] Performing an operation on the input data based on the at least one operator in the target node to obtain output data of the target node, wherein the at least one operator has a corresponding operator result parameter;
[0072] The output data of the target node and / or the target parameter of the target node are stored in the memory corresponding to the target container.
[0073] In the related art, the output data of the target container needs to be output to the disk first, and then read from the disk to the memory to process the output data. In this way, during the training process of the target container, it is necessary to frequently input and output data to the disk, resulting in more hardware resources being occupied, and the output data cannot be obtained quickly, which is inefficient.
[0074] In an embodiment of the present application, the training results are stored in the memory corresponding to the target container, specifically, the output data of the target node and / or the target parameters of the target node are stored in the memory corresponding to the target container. When the output data of the target node and / or the target parameters of the target node are needed, there is no need to read the training results from the disk into the memory, and the training results can be directly obtained from the memory, thereby avoiding input and output of data to the disk, reducing the occupancy of hardware resources, and improving the efficiency of data acquisition.
[0075] The above-mentioned storing the output data of the target node and / or the target parameter of the target node in the memory corresponding to the target container can be specifically achieved by setting the target container to a memory mode, in which the output data and / or the target parameter of each node in the target container are directly stored in the memory.
[0076] In one embodiment, the method further comprises:
[0077] According to the connection relationship between the multiple nodes, the initial algorithms of the multiple nodes are trained in sequence to obtain the target parameters corresponding to the multiple nodes respectively; wherein, when the current node is not the first node among the multiple nodes, the output data of the upper node is obtained from the memory corresponding to the target container as the input data of the current node;
[0078] The target container is closed.
[0079] It should be noted that after the initial parameter training of at least one operator configured in the target node is completed and the operator result parameters are obtained, there may be other nodes in the target container that need to be trained. Before training, it is necessary to obtain the input data of each node through the connection relationship between multiple nodes. When the output data of the target node and / or the target parameter of the target node is stored in the memory corresponding to the target container, the input data of the node to be trained can be directly obtained from the memory to achieve rapid training of the initial parameters of at least one operator configured in the node.
[0080] Among them, the target container includes multiple nodes. During the training process, the initial parameters of at least one operator configured for each node can be trained in turn according to the connection relationship until all nodes are trained, and then the target container is closed to avoid frequent startup and shutdown of the target container, thereby reducing the resource occupation of the machine learning platform caused by frequent startup and shutdown of the target container.
[0081] Furthermore, after all nodes are trained and before the target container is closed, the data generated during the training process stored in the memory needs to be returned to the machine learning platform to facilitate the subsequent establishment of the target model. The generated data includes the output data after each node is trained and the target parameters after each node is trained.
[0082] In one embodiment, after storing the output data of the target node and / or the target parameter of the target node in the memory corresponding to the target container, the method further includes:
[0083] In response to an operation instruction to view the output data of the target node and / or the target parameter of the target node, acquiring the output data of the target node and / or the target parameter of the target node from a memory corresponding to the target container;
[0084] The output data of the target node and / or the target parameter of the target node are displayed.
[0085] The device for displaying the output data of the target node and / or the target parameters of the target node may be a device for running the target container, or may be a device that is communicatively connected to the device for running the target container. By displaying the output data of the target node and / or the target parameters of the target node, the preset parameters may be adjusted.
[0086] The above-mentioned viewing operation instruction is an instruction issued by a device that displays the output data of the target node and / or the target parameters of the target node. In an embodiment of the present application, when receiving the viewing operation instruction, the output data of the target node and / or the target parameters of the target node are obtained from the memory corresponding to the target container, and then the output data of the target node and / or the target parameters of the target node are displayed, thereby realizing the rapid acquisition of the output data of the target node and / or the target parameters of the target node, and then realizing the adjustment of the preset parameters in the initial algorithm based on the target data.
[0087] Among them, when the operation instruction of viewing is received and the target node has not yet completed training, the displayed data is the data in the training process (that is, the output data of the target node during the training process and / or the initial parameters adjusted in the operator of the target node); when the operation instruction of viewing is received and the target node has completed training, the displayed data is the output data of the target node after training and / or the target parameters of the target node.
[0088] In one embodiment, the multiple nodes include a data reading node, a data processing node, a model training node, and a model evaluation node, and the at least one operator configured in each node is used to execute the function corresponding to each node. For example, each operator configured in the data reading node is an algorithm or function for executing the data reading function, and each operator configured in the model training node is an algorithm or function for executing the model training function. In some embodiments, at least one operator in each node has a connection relationship, the input data of the first operator in the current node is the output data of the previous node, and the input data of other operators in the current node is the output data of the previous level operator corresponding to the operator. At least one operator in each node processes the input data of the node according to the connection relationship, realizes the function corresponding to the node, and finally obtains the output data of the node.
[0089] It should be noted that the specific contents executed by different nodes in the target container are different. For example, the data reading node is only used to read the input data of the node in the memory; the data processing node is used to process the read data, such as normalization or filtering; the model training node is used to train and adjust the operator including the initial parameters of the operator to obtain the operator result parameters; the model evaluation node is used to evaluate the operator after the model training node is processed once (that is, the initial algorithm in the operator is adjusted once) to determine whether the current training result meets the training cutoff condition. The training of the preset parameters of the initial algorithm in the node is completed through different nodes.
[0090] Among them, at least one operator configured in different nodes is used to execute the function of the node. For example, the operator in the data processing node is used to normalize or filter the input data; the operator in the model training node is used to calculate the data; the operator in the model evaluation node is used to calculate the loss value or the number of training times, etc.
[0091] In one embodiment, generating a target model based on target parameters corresponding to the plurality of nodes respectively includes:
[0092] Connecting the multiple nodes based on the connection relationship of the multiple nodes to generate a modeling workflow, wherein the multiple nodes correspond to the target parameters respectively;
[0093] The modeling workflow is run to generate the target model.
[0094] It should be noted that after the preset parameters of the initial algorithm of each node are trained in the target container, the output data of each node and / or the target parameters of each node are obtained from the memory corresponding to the target container. At this time, each initial algorithm has completed the training and the trained algorithm is obtained. The machine learning platform connects multiple nodes based on the connection relationship of multiple nodes to generate a modeling workflow. The nodes in the modeling workflow include the trained algorithms. By running the modeling workflow, the target model for completing the model training is obtained. The target model is obtained by training the target container in this application, which can effectively improve the training efficiency.
[0095] Furthermore, the present application also provides the following embodiments: Figure 4 The model training diagram shown in Figure 4 As shown in the figure, the model training process includes:
[0096] 1) Start the target container. Before starting the target container, set the central processing unit (CPU) and memory that the target container needs to occupy;
[0097] 2) After starting the target container, create multiple nodes in the target container, including node X and node Y, and add operators to each node in turn. The operators in each node correspond to an initial algorithm of the node. The operators can be assembled by dragging and dropping on the page of the target container;
[0098] 3) Establish a connection relationship between the nodes, and splice the three nodes in the target container based on the connection relationship to form Figure 5 The workflow of multiple nodes shown in the figure, in which node X is the parent node of node Y, and the output data of node X is the input data of node Y;
[0099] 4) Train the initial algorithm in node X (i.e., train the initial algorithms corresponding to operator 1, operator 2, and operator 3 in node X), and adjust the preset parameters in the initial algorithm based on the training results until they meet the training expectations;
[0100] 5) Storing the output data of node X in memory so that the output data can be obtained through memory for model training or preview;
[0101] 6) Obtain the output data of node X from the memory, and train the initial algorithm in node Y based on the output data of node X (i.e., train the initial algorithms corresponding to operators 4, 5, and 6 in node X);
[0102] 7) After the preset parameters in the initial algorithm of all nodes in the target container are adjusted, all nodes are assembled to form Figure 6 The structure shown in the figure runs all nodes to generate a training model.
[0103] In this way, by training the initial algorithm in each node in turn, there is no need to run all the operators in the entire container every time when adjusting the parameters. It is only necessary to run the operators of the target nodes corresponding to the parameters to be adjusted to adjust the operator parameters corresponding to the model, thereby improving the efficiency of parameter adjustment.
[0104] See also Figure 7 , Figure 7 is a structural diagram of a model building device based on a machine learning platform provided in an embodiment of the present application, such as Figure 7 As shown, the model building device 700 based on the machine learning platform includes:
[0105] A creation module 701 is used to start a target container and create multiple nodes in a page of the target container, wherein each node includes an initial algorithm with preset parameters;
[0106] A first training module 702 is used to obtain input data of a target node, and train an initial algorithm of the target node based on the input data to obtain a training result, wherein the target node is any node among the multiple nodes;
[0107] An adjustment module 703 is used to adjust the preset parameters based on the training results to obtain target parameters corresponding to the target node;
[0108] The generating module 704 is used to generate a target model based on the target parameters respectively corresponding to the multiple nodes.
[0109] In one embodiment, before the first training module 702, the model building device 700 based on the machine learning platform includes:
[0110] A configuration module, configured to configure at least one operator for each of the plurality of nodes, and determine the initial algorithm based on the at least one operator;
[0111] An establishment module is used to establish a connection relationship between the multiple nodes, wherein the connection relationship is used to characterize the upper node and / or lower node corresponding to each node in the multiple nodes, the input data of each node is the output data of the corresponding upper node, and the output data of each node is the input data of the corresponding lower node.
[0112] In one embodiment, the first training module 702 includes:
[0113] an acquisition unit, configured to acquire output data of the upper-level node of the target node as input data of the target node when the target node has an upper-level node;
[0114] A training unit, configured to process the input data based on the at least one operator of the target node to obtain the training result; wherein the initial algorithm includes the at least one operator, each operator is configured with an operator initial parameter, and the preset parameters include the operator initial parameter of the at least one operator;
[0115] The adjustment module 703 includes:
[0116] A first adjustment unit, configured to adjust an initial operator parameter of each operator of at least one operator of the target node based on the training result until a training end condition is met, thereby obtaining an operator result parameter corresponding to the at least one operator;
[0117] The second adjustment unit is used to determine the target parameter of the target node according to the operator result parameter corresponding to the at least one operator.
[0118] In one embodiment, after the adjustment module 703, the model building device 700 based on the machine learning platform further includes:
[0119] A computing module, configured to compute the input data based on the at least one operator in the target node to obtain output data of the target node, wherein the at least one operator has a corresponding operator result parameter;
[0120] A storage module is used to store the output data of the target node and / or the target parameter of the target node in a memory corresponding to the target container.
[0121] In one embodiment, the model building device 700 based on the machine learning platform further includes:
[0122] A second training module is used to train the initial algorithms of the multiple nodes in sequence according to the connection relationship between the multiple nodes to obtain target parameters corresponding to the multiple nodes respectively; wherein, when the current node is not the first node among the multiple nodes, the output data of the upper node is obtained from the memory corresponding to the target container as the input data of the current node;
[0123] The closing module is used to close the target container.
[0124] In one embodiment, after the storage module, the model building device 700 based on the machine learning platform further includes:
[0125] a response module, configured to obtain the output data of the target node and / or the target parameter of the target node from a memory corresponding to the target container in response to an operation instruction to view the output data of the target node and / or the target parameter of the target node;
[0126] The display module is used to display the output data of the target node and / or the target parameter of the target node.
[0127] In one embodiment, one of the nodes is a data reading node, a data processing node, a model training node or a model evaluation node, and the at least one operator configured for each node is an operator for performing data reading, data processing, model training or model evaluation.
[0128] In one embodiment, the generating module 704 includes:
[0129] A first generating unit, configured to connect the plurality of nodes based on a connection relationship between the plurality of nodes to generate a modeling workflow, wherein the plurality of nodes correspond to the target parameters respectively;
[0130] The second generating unit is used to run the modeling workflow to generate the target model.
[0131] The model building device based on the machine learning platform provided in the embodiment of the present application is capable of implementing each process of each embodiment of the above-mentioned model building method based on the machine learning platform. The technical features correspond one to one and can achieve the same technical effect. To avoid repetition, they will not be described here.
[0132] It should be noted that the model building device based on the machine learning platform in the embodiment of the present application can be a device, or it can be a component, integrated circuit, or chip in an electronic device.
[0133] The present application also provides an electronic device, see Figure 8 , Figure 8 801, a processor 802, and a program or instruction stored in the memory 801 and executed by the processor 802. Figure 2 Any steps in the corresponding method embodiments and achieving the same beneficial effects will not be repeated here.
[0134] The processor 802 may be a CPU, an ASIC, an FPGA or a GPU.
[0135] Those skilled in the art will appreciate that all or part of the steps of implementing the above-mentioned embodiment method can be completed by hardware associated with program instructions, and the program can be stored in a readable medium.
[0136] The present application also provides a readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the above-mentioned Figure 2 Any steps in the corresponding method embodiments can achieve the same technical effect, and will not be repeated here to avoid repetition. The storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0137] The terms "first", "second" etc. in the embodiments of the present application are used to distinguish similar objects, and need not be used to describe a specific order or sequential order. In addition, the terms "include" and "have" and any variation thereof are intended to cover non-exclusive inclusions, for example, the process, method, system, product or equipment comprising a series of steps or units need not be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products or equipment. In addition, "and / or" is used in the present application to represent at least one of connected objects, such as A and / or B and / or C, representing comprising independent A, independent B, independent C, and A and B all exist, B and C all exist, A and C all exist, and 7 situations that A, B and C all exist.
[0138] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.
[0139] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the relevant technology, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a second terminal device, etc.) to execute the methods of each embodiment of the present application.
[0140] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.
Claims
1. A model building method based on a machine learning platform, applied to the machine learning platform, characterized in that: include: Starting a target container, and creating a plurality of nodes in a page of the target container, wherein each node includes an initial algorithm with preset parameters; Acquire input data of a target node, and train an initial algorithm of the target node based on the input data to obtain a training result, wherein the target node is any node among the multiple nodes; Adjust the preset parameters based on the training results to obtain target parameters corresponding to the target node; Generate a target model based on the target parameters respectively corresponding to the multiple nodes; Before obtaining the input data of the target node, the method further includes: At least one operator is configured for each of the multiple nodes, and the initial algorithm is determined based on the at least one operator, where the operators in each node correspond to an initial algorithm of the node, and the operators are assembled in a drag-and-drop manner on the page of the target container; Establishing a connection relationship between the multiple nodes, wherein the connection relationship is used to characterize the upper node and / or lower node corresponding to each node in the multiple nodes, the input data of each node is the output data of the corresponding upper node, and the output data of each node is the input data of the corresponding lower node; The generating a target model based on the target parameters respectively corresponding to the plurality of nodes comprises: Connecting the multiple nodes based on the connection relationship of the multiple nodes to generate a modeling workflow; Running the modeling workflow to generate the target model; The step of acquiring input data of the target node and training the initial algorithm of the target node based on the input data to obtain a training result includes: In the case where the target node has an upper-level node, obtaining output data of the upper-level node of the target node as input data of the target node; Processing the input data based on the at least one operator of the target node to obtain the training result; wherein each operator is configured with an operator initial parameter, and the preset parameters include the operator initial parameter of the at least one operator; The adjusting the preset parameters based on the training results to obtain the target parameters corresponding to the target node includes: Adjusting the initial operator parameters of each operator of the at least one operator of the target node based on the training result until a training cutoff condition is met, thereby obtaining an operator result parameter corresponding to the at least one operator; Determine a target parameter of the target node according to an operator result parameter corresponding to the at least one operator, wherein the target parameter is an operator training result combined according to an operation sequence between each operator and / or an input-output relationship between operators; The training cutoff condition is one of the following: Calculate a loss value of an initial algorithm including operator training results of each operator, wherein the loss value is less than or equal to a set loss threshold; The number of times the initial parameters of each operator are adjusted based on the training result is equal to the set number of times; The adjusted initial algorithm including the operator training results of each operator calculates the input data, and the obtained training results are within the set range; After adjusting the preset parameters based on the training results to obtain the target parameters corresponding to the target node, the method further includes: Performing operation on the input data based on the at least one operator in the target node to obtain output data of the target node; Storing the output data of the target node and / or the target parameter of the target node in the memory corresponding to the target container; After storing the output data of the target node and / or the target parameter of the target node in the memory corresponding to the target container, the method further includes: In response to an operation instruction to view the output data of the target node and / or the target parameter of the target node, acquiring the output data of the target node and / or the target parameter of the target node from a memory corresponding to the target container; Display output data of the target node and / or target parameters of the target node; Before starting the target container, the method further includes: Set the CPU and memory required by the target container; Before training the initial algorithms of the multiple nodes in sequence according to the connection relationship between the multiple nodes, obtaining the target parameters respectively corresponding to the multiple nodes, completing the training of all nodes, and closing the target container, the method further includes: The output data of the target node and the target parameters of the target node stored in the memory are returned to the machine learning platform, and the output data of the target node and the target parameters of the target node are used for subsequent establishment of a target model.
2. The method according to claim 1, characterized in that: The multiple nodes include data reading nodes, data processing nodes, model training nodes and model evaluation nodes.
3. A model building device based on a machine learning platform, characterized in that: include: A creation module, used to start a target container and create multiple nodes in a page of the target container, wherein each node includes an initial algorithm with preset parameters; A first training module is used to obtain input data of a target node, and train an initial algorithm of the target node based on the input data to obtain a training result, wherein the target node is any node among the multiple nodes; An adjustment module, used to adjust the preset parameters based on the training results to obtain target parameters corresponding to the target node; A generating module, used for generating a target model based on target parameters respectively corresponding to the plurality of nodes; The model building device based on the machine learning platform also includes a configuration module and a building module. Before the input data of the target node is obtained and the initial algorithm of the target node is trained based on the input data to obtain the training result, the target node is any node among the multiple nodes. The configuration module is used to configure at least one operator for each of the multiple nodes, and determine the initial algorithm based on the at least one operator, the operator in each node corresponds to an initial algorithm of the node, and the operator is assembled in a drag-and-drop manner on the page of the target container; The establishing module is used to establish a connection relationship between the multiple nodes, the connection relationship is used to characterize the upper node and / or lower node corresponding to each node in the multiple nodes, the input data of each node is the output data of the corresponding upper node, and the output data of each node is the input data of the corresponding lower node; The generation module comprises: A first generating unit, configured to connect the plurality of nodes based on the connection relationship between the plurality of nodes to generate a modeling workflow; A second generating unit, used for running the modeling workflow to generate the target model; The first training module includes: an acquisition unit, configured to acquire output data of the upper-level node of the target node as input data of the target node when the target node has an upper-level node; A training unit, configured to process the input data based on the at least one operator of the target node to obtain the training result; wherein each operator is configured with an operator initial parameter, and the preset parameters include the operator initial parameter of the at least one operator; The adjustment module comprises: A first adjustment unit, configured to adjust an operator initial parameter of each operator of at least one operator of the target node based on the training result until a training cutoff condition is met, thereby obtaining an operator result parameter corresponding to the at least one operator; A second adjustment unit, configured to determine a target parameter of the target node according to an operator result parameter corresponding to the at least one operator, wherein the target parameter is an operator training result combined according to an operation sequence between each operator and / or an input-output relationship between operators; The training cutoff condition is one of the following: Calculate a loss value of an initial algorithm including operator training results of each operator, wherein the loss value is less than or equal to a set loss threshold; The number of times the initial parameters of each operator are adjusted based on the training result is equal to the set number of times; The adjusted initial algorithm including the operator training results of each operator calculates the input data, and the obtained training results are within the set range; The model building device based on the machine learning platform also includes a computing module and a storage module. After adjusting the preset parameters based on the training results to obtain the target parameters corresponding to the target node, The computing module is used to perform a calculation on the input data based on the at least one operator in the target node to obtain output data of the target node; The storage module is used to store the output data of the target node and / or the target parameter of the target node in the memory corresponding to the target container; The model building device based on the machine learning platform also includes a response module and a display module. After storing the output data of the target node and / or the target parameter of the target node in the memory corresponding to the target container, The response module is used to obtain the output data of the target node and / or the target parameter of the target node from the memory corresponding to the target container in response to the operation instruction to view the output data of the target node and / or the target parameter of the target node; The display module is used to display the output data of the target node and / or the target parameter of the target node; The model building device based on the machine learning platform also sets the central processing unit CPU and memory required to be occupied by the target container before starting the target container; The model building device based on the machine learning platform also trains the initial algorithms of the multiple nodes in sequence according to the connection relationship between the multiple nodes, obtains the target parameters corresponding to the multiple nodes respectively, completes the training of all nodes, and before closing the target container, returns the output data of the target node and the target parameters of the target node stored in the memory to the machine learning platform, and the output data of the target node and the target parameters of the target node are used for subsequent establishment of the target model.
4. An electronic device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and executable on the processor, wherein the computer program, when executed by the processor, implements the steps in the model building method based on the machine learning platform as described in any one of claims 1 to 2.