Neural network automatic construction and training method
By splitting the training of neural network model into multiple subtasks and using containerization technology to automatically build the training environment, the problems of inefficiency and high threshold for non-professional personnel when training neural network models are solved, and efficient and reproducible Training process.
Patent Information
- Application Number
- CN202411077827.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-07
- Publication Date
- 2024-12-06
AI Technical Summary
Non-professionals face inefficient training and unreasonable resource allocation when training complex models and large-scale dataset processing, which increases training barriers, especially in the absence of efficient hardware support.
Treat neural network model training as a series of composable components. Through the processes of environment construction, model construction and log monitoring, containerization technology and visual processing are adopted to split the training tasks into multiple subtasks, and the training environment and recommendation models are automatically built. Architecture, reduce manual adjustment workload and improve training efficiency.
The automated training environment construction and training of neural network models is realized, which reduces the technical threshold, improves training efficiency and reproducibility, and simplifies the troubleshooting and performance tuning process.
Smart Images

Figure CN119089984A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of neural network model training, and in particular to a method for automatically constructing and training a neural network. Background Art
[0002] With the rapid development of deep learning technology, neural network models, especially deep neural networks (DNNs), have shown excellent performance in many fields such as image recognition, natural language processing, and speech recognition. However, high-performance models are often accompanied by huge network structures and large computing requirements, which leads to a sharp increase in training time and becomes a major bottleneck restricting model development and application.
[0003] To address this challenge, researchers have proposed the concept of pre-trained models. A pre-trained model refers to a neural network model that has been pre-trained on a large-scale general dataset. The learned data features provide a good starting point for subsequent fine-tuning for specific tasks. This strategy greatly accelerates the model training process for specific tasks and improves the convergence speed and final performance of the model.
[0004] Although pre-trained models significantly reduce the time cost of model training, there are still the following challenges in practical applications: In order to adapt to different task scenarios, pre-trained models usually require a certain degree of structural adjustment or parameter fine-tuning. This process not only requires professional knowledge, but also it is difficult for non-professionals to determine the best model architecture and hyperparameter settings without sufficient guidance. Even when using pre-trained models, the fine-tuning process may still consume a lot of computing resources and time for large models, especially in the absence of efficient hardware support.
[0005] At present, in order to solve the above problems, some platforms provide automated tools for model training, such as automatic hyperparameter optimization (AutoML); however, these tools are often limited to specific model types or algorithms, and may still face problems such as low training efficiency and unreasonable resource allocation when dealing with complex models and large-scale data sets. It can be seen that the current problem of neural network training efficiency always exists, which increases the training threshold for non-professionals. Summary of the invention
[0006] Based on the current status of the background technology, the purpose of the present invention is to solve the problem that it is difficult for non-professionals to train neural network models, and therefore proposes a method for automatic construction and training of neural networks; the method involves processes such as environment construction, model construction and log monitoring, which can effectively realize the automatic training environment construction and training instance application during neural network training, thereby smoothly improving the training efficiency.
[0007] The present invention adopts the following technical solutions to achieve the purpose:
[0008] A method for automatically constructing and training a neural network comprises the following steps:
[0009] S1. The complete neural network model training is used as the parent task, and the parent task includes multiple subtasks that are trained separately;
[0010] S2. Select the corresponding model to be trained based on the type of the parent task, and configure the corresponding model components to complete the model building process of neural network model training;
[0011] S3. After selecting the model to be trained, the corresponding container is built in an image-based manner to complete the environment construction process for neural network model training;
[0012] S4. After completing the model construction and environment construction, generate a training task execution command according to the determined model parameters and environment parameters to carry out the training process of the model to be trained;
[0013] S5. Use visual processing to monitor the training process, and perform the training process of multiple subtasks in sequence. After the multiple subtasks are trained in sequence, the last trained subtask is used as the training result of the parent task to obtain a neural network model that has completed the training.
[0014] Specifically, in step S1, the type of the parent task is determined by the training task, which includes visual tasks, auditory tasks and language tasks; each subtask includes data preparation, model definition, training configuration, model training, model evaluation, model optimization and optimal storage.
[0015] Specifically, in step S2, the corresponding model to be trained is selected in advance from the model library according to the type of the parent task, and the optimizer, evaluation index and data set required for model optimization are configured for the selected model to be trained as the corresponding model component.
[0016] Furthermore, a preset script is used to capture the hyperparameter list in the model to be trained, a Python command is generated through interactive settings, and the Python command is written into the model to be trained and called during the training process of each subtask.
[0017] Specifically, in step S3, based on the selected model to be trained, a shell script is called to pull the basic image corresponding to the model to be trained from the container image library and build the corresponding container.
[0018] Preferably, for the constructed container, after setting the internal and external directory mapping, building the log directory, the running log directory and the pre-training directory in the container, the environment construction process is completed; the parent task is executed in the constructed environment.
[0019] Specifically, in step S4, the model parameters include the specific architecture of the model to be trained and the corresponding hyperparameters; the hyperparameters are specified by the training task, including the number of layers of the specific architecture, the number of nodes in each layer, the activation function, the loss function and the optimizer.
[0020] Specifically, in step S4, based on the built and set container, the environmental parameters include the computing resources, operating system, programming language version, deep learning framework and dependent libraries used in the training process; when generating the training task execution command, the data set location, preprocessing method, batch size and training rounds for each subtask are specified at the same time.
[0021] Furthermore, in step S5, for the training process of each subtask, the corresponding operation log is parsed and visualized, the operation log is monitored using a timer, and the training data in the operation log is captured for visualization; the training process of each subtask is divided into a training phase and a verification phase. When the training of a subtask is completed, the obtained weight file is preferentially stored in the pre-training directory set in the container; the training process of each subtask is performed in turn, and the hyperparameters are adjusted accordingly when the training of each subtask is completed. When the training of the last subtask is completed, the optimal weight file is obtained as the training result of the parent task.
[0022] The present invention also provides a computer device system, including a memory, a processor and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the aforementioned neural network automatic construction and training method.
[0023] In summary, due to the adoption of this technical solution, the beneficial effects of the present invention are as follows:
[0024] The present invention regards the construction and training of neural network models as a series of combinable components, which users can flexibly select and configure according to specific needs without in-depth understanding of the underlying code. The present invention also provides a standardized training environment construction template to ensure the stability and reproducibility of the training process. After splitting the training task into multiple links, each link can be optimized independently, which is convenient for troubleshooting and performance tuning.
[0025] Based on the task description input by the user, the application of the method of the present invention can automatically recommend the appropriate model architecture and pre-trained model, reducing the workload of manual adjustment. Based on the component-based design concept, the present invention can develop an intuitive graphical user interface, allowing non-professionals to complete model training through simple operations such as dragging, thereby greatly reducing the threshold for using neural network model technology. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1 The overall process diagram of the method of the present invention is briefly described;
[0027] Figure 2 It is a schematic diagram of the overall training process from the perspective of the parent task in the method of the present invention;
[0028] Figure 3 A schematic diagram of the complete process of constructing and training a neural network model according to the method of the present invention;
[0029] Figure 4 Schematic diagram of the overall architecture of the system of the present invention. DETAILED DESCRIPTION
[0030] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings here can be arranged and designed in various different configurations.
[0031] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention claimed for protection, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0032] Example 1
[0033] A method for automatically building and training a neural network. The overall process of this method can be briefly described in Figure 1 , each step in this embodiment is summarized as follows:
[0034] S1. The complete neural network model training is used as the parent task, and the parent task includes multiple subtasks that are trained separately;
[0035] S2. Select the corresponding model to be trained based on the type of the parent task, and configure the corresponding model components to complete the model building process of neural network model training;
[0036] S3. After selecting the model to be trained, the corresponding container is built in an image-based manner to complete the environment construction process for neural network model training;
[0037] S4. After completing the model construction and environment construction, generate a training task execution command according to the determined model parameters and environment parameters to carry out the training process of the model to be trained;
[0038] S5. Use visual processing to monitor the training process, and perform the training process of multiple subtasks in sequence. After the multiple subtasks are trained in sequence, the last trained subtask is used as the training result of the parent task to obtain a neural network model that has completed the training.
[0039] In this embodiment, each subtask can represent an iteration of model training or a training instance under different configurations. These subtasks are repeated in a serial manner, which helps to gradually train, verify and adjust the model. The application of containerization technology ensures that each subtask runs in the same software environment, provides environmental consistency, and makes the training process portable and repeatable on different hardware platforms.
[0040] The generation of training task execution commands can be scripted automatically or manually, which is convenient for simplifying the startup and management of the training process. Monitoring during the training process can help researchers make more informed decisions, such as terminating poorly performing subtasks early or increasing resources for well-performing subtasks. After applying the above method steps of this embodiment, the optimal or most satisfactory trained neural network model can be finally obtained, and the method improves the efficiency and reliability of model training.
[0041] The following is a detailed introduction to each step of the method in this embodiment. From the perspective of the complete neural network model training in this embodiment, that is, the method training diagram from the perspective of the parent task can be found in Figure 2 ; The detailed construction and training process during the method execution can be found in Figure 3 .
[0042] In step S1 of this embodiment, the type of the parent task is determined by the training task, which includes visual tasks, auditory tasks and language tasks; each subtask includes data preparation, model definition, training configuration, model training, model evaluation, model optimization and preferential storage.
[0043] Among them, data preparation involves data collection; data cleaning, removing noise, outliers and duplicate data, etc.; data annotation; data preprocessing, such as scaling, normalization, encoding categorical variables, etc.; data enhancement, transforming existing data to increase diversity and ultimately improve the generalization ability of the model, etc. This part can form the data set in the model component.
[0044] The model definition part may involve architecture selection, parameter initialization, loss function selection and optimizer configuration, which also corresponds to some model components; training configuration involves operations such as hyperparameter adjustment, early stopping and learning rate adjustment; model training involves iteration and monitoring process; model evaluation can be carried out through test sets, performance indicator calculation and error analysis; model optimization includes hyperparameter optimization and architecture modification; preferential storage means the process of storing the corresponding optimal weight file after the subtask training is completed.
[0045] In step S2 of this embodiment, Figure 3As shown in the figure, the corresponding model to be trained is selected from the model library in advance according to the type of the parent task, and the optimizer, evaluation index and data set required for model optimization are configured for the selected model to be trained as the corresponding model component. Figure 2 and Figure 3 , use the preset script to grab the hyperparameter list in the model to be trained, generate Python commands through interactive settings, write the Python commands into the model to be trained, and call them during the training process of each subtask.
[0046] In this embodiment, an existing script is written or used to identify and extract hyperparameters in the model code, which are variables set when the model is defined or when the training cycle starts. The preset script should have the ability to parse the model code to identify these hyperparameters and list them one by one. Once the hyperparameters are identified, the specific values of these hyperparameters are set through an interactive interface; the interactive interface can be a command line interface, a graphical user interface GUI, or manual input through a configuration file.
[0047] Based on the hyperparameter values provided, the preset script generates corresponding Python command lines, which can directly modify the hyperparameters of the model to be trained, and are integrated into the training process of the model. The present embodiment writes Python commands into a batch file, a shell script or is directly embedded into the training process so that it can be executed before each subtask starts. At the beginning of each subtask, the Python command will be called to ensure that the latest hyperparameter settings are used. In this way, each subtask can be operated under different hyperparameter configurations, thereby exploring the hyperparameter space and finding the optimal configuration.
[0048] The above is a detailed introduction to the model building process for neural network model training, followed by an introduction to the environment building process for neural network model training.
[0049] In step S3 of this embodiment, based on the selected model to be trained, the shell script is called to pull the basic image corresponding to the model to be trained from the container image library to build the corresponding container. For the built container, after setting the internal and external directory mapping, building the log directory, running the log directory and the pre-training directory in the container, the environment construction process is completed; the parent task is executed in the built environment. For container construction and settings, please refer to Figure 3 's hint.
[0050] Based on step S3, the model parameters in step S4 include the specific architecture of the model to be trained and the corresponding hyperparameters; the hyperparameters are specified by the training task, and also involve the aforementioned hyperparameter-related content. In this embodiment, they may include the number of layers of the specific architecture, the number of nodes in each layer, the activation function, the loss function, and the optimizer. At the same time, based on the container that is built and set, the environmental parameters include the computing resources, operating system, programming language version, deep learning framework, and dependent libraries used in the training process; when generating the training task execution command, the data set location, preprocessing method, batch size, and training rounds for each subtask are specified at the same time.
[0051] Finally, in step S5, Figure 2 and Figure 3 As shown in the figure, for the training process of each subtask, the corresponding operation log is parsed and visualized, the operation log is monitored using a timer, and the training data in the operation log is captured for visualization. The monitored and captured operation log can be recorded as a monitoring log, and the recorded content includes the training stage, verification stage, test results, expected time and actual time, etc., which can be used as a reference for adjusting the training of the next subtask.
[0052] In this embodiment, the training process of each subtask can be divided into a training phase and a verification phase. When the training of a subtask is completed, the obtained weight file is preferentially stored in the pre-training directory set in the container; the training process of each subtask is performed in turn, and the hyperparameters are adjusted accordingly when the training of each subtask is completed. When the training of the last subtask is completed, the optimal weight file is obtained as the training result of the parent task.
[0053] Example 2
[0054] Based on Example 1, this example introduces a computer device system, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the steps of the method for automatic construction and training of a neural network in Example 1.
[0055] In this embodiment, these computer programs / instructions may be stored in a computer-readable memory that can direct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more steps in the method.
[0056] like Figure 4As shown, after the entire computer device system is equipped with the method of Example 1, according to the functional architecture relationship, the container engine is connected to the Java backend and the file system in sequence; the Java backend is connected to the web front end and the database respectively. The container engine is the relevant content of the image and container in the implementation method, and the file system is divided into data set files, pre-trained files, model files, picture files (when training image neural network models), build log files, and operation log files by category; and the content of the file system can be stored in the database on the hardware storage.
Claims
1. A method for automatically constructing and training a neural network, characterized in that: The steps include: S1. The complete neural network model training is used as the parent task, and the parent task includes multiple subtasks that are trained separately; S2. Select the corresponding model to be trained based on the type of the parent task, and configure the corresponding model components to complete the model building process of neural network model training; S3. After selecting the model to be trained, the corresponding container is built in an image-based manner to complete the environment construction process for neural network model training; S4. After completing the model construction and environment construction, generate a training task execution command according to the determined model parameters and environment parameters to carry out the training process of the model to be trained; S5. Use visual processing to monitor the training process, and perform the training process of multiple subtasks in sequence. After the multiple subtasks are trained in sequence, the last trained subtask is used as the training result of the parent task to obtain a neural network model that has completed the training.
2. The method for automatically constructing and training a neural network according to claim 1, characterized in that: In step S1, the type of the parent task is determined by the training task, which includes visual tasks, auditory tasks, and language tasks; each subtask includes data preparation, model definition, training configuration, model training, model evaluation, model optimization, and optimal storage.
3. The method for automatically constructing and training a neural network according to claim 1, characterized in that: In step S2, the corresponding model to be trained is selected in advance from the model library according to the type of the parent task, and the optimizer, evaluation index and data set required for model optimization are configured for the selected model to be trained as the corresponding model component.
4. The method for automatically constructing and training a neural network according to claim 3, characterized in that: Use the preset script to grab the hyperparameter list in the model to be trained, generate Python commands through interactive settings, write the Python commands into the model to be trained, and call them during the training process of each subtask.
5. The method for automatically constructing and training a neural network according to claim 1, characterized in that: In step S3, based on the selected model to be trained, a shell script is called to pull the basic image corresponding to the model to be trained from the container image library and build the corresponding container.
6. The method for automatically constructing and training a neural network according to claim 5, characterized in that: For the built container, after setting the internal and external directory mapping, build log directory, run log directory, and pre-training directory in the container, the environment building process is completed; the parent task is executed in the built environment.
7. The method for automatically constructing and training a neural network according to claim 6, characterized in that: In step S4, the model parameters include the specific architecture of the model to be trained and the corresponding hyperparameters; the hyperparameters are specified by the training task, including the number of layers of the specific architecture, the number of nodes in each layer, the activation function, the loss function and the optimizer.
8. The method for automatically constructing and training a neural network according to claim 6, characterized in that: In step S4, based on the built and set container, the environment parameters include computing resources, operating system, programming language version, deep learning framework and dependent libraries used in the training process; When generating the training task execution command, specify the dataset location, preprocessing method, batch size, and training rounds for each subtask.
9. The method for automatically constructing and training a neural network according to claim 6, characterized in that: In step S5, for the training process of each subtask, the corresponding operation log is parsed and visualized, the operation log is monitored using a timer, and the training data in the operation log is captured for visualization; the training process of each subtask is divided into a training phase and a verification phase. When the training of a subtask is completed, the obtained weight file is preferentially stored in the pre-training directory set in the container; The training process of each subtask is carried out in turn, and the hyperparameters are adjusted accordingly when the training of each subtask is completed. When the training of the last subtask is completed, the optimal weight file is obtained as the training result of the parent task.
10. A computer device system, comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method for automatic construction and training of a neural network according to any one of claims 1 to 9.
Citation Information
Cited By
Model-based visual code-free neural network construction method
CN120046703A
