Method, System, Device and Medium for Model Training Based on a Training Platform

Abstract

The present invention provides a method, system, device and medium for model training based on a training platform. The method includes: constructing a model file management module for maintaining model file data; starting the model file management module when creating a training task; according to the target model file data corresponding to the training task, the model file processing module mounts the target model file data through the model file management module; and the training task performs deep learning model training based on the mounted target model file data. The aim is to improve the convenience of deep learning model training, thereby enhancing the training efficiency of deep learning model training.

Description

Technical Field

[0001] The present invention relates to the technical field of operating system installation, and particularly to a method, system, device and medium for model training based on a training platform. Background Art

[0002] At present, several ways of training a deep learning model through model file data include: one is to copy the model file data to a public directory before the deep learning model training starts, and mount the public directory when running the training task, so that the training script can read the model file data in the public directory mounted in the container; the other is to write code for downloading the model file data in the training script, and download the model file data in real time when running the script, and then perform deep learning model training after the download is completed. Both of these methods have problems with the ease of use of deep learning model training, making it necessary to perform real-time download or copy of the model file data every time deep learning model training is carried out, resulting in insufficient convenience of deep learning model training. Summary of the Invention

[0003] In view of this, the present invention provides a method, system, device and medium for model training based on a training platform. The aim is to improve the convenience of deep learning model training, thereby improving the training efficiency of deep learning model training.

[0004] In the first aspect of the embodiments of the present invention, a method for model training based on a training platform is provided. The method includes:

[0005] Construct a model file management module for maintaining model file data;

[0006] When creating a training task, start the model file management module;

[0007] According to the target model file data corresponding to the training task, the model file processing module mounts the target model file data through the model file management module;

[0008] The training task performs deep learning model training based on the mounted target model file data.

[0009] Optionally, the model file data maintained by the model file management module includes: externally imported model file data and model file data generated during the execution of the training task.

[0010] Optionally, the model file management module is used to maintain model file data, including:

[0011] Adding the model file data to the database corresponding to the model file management module;

[0012] The model file management module adds the location information and parameter information of the model file data to the corresponding model table.

[0013] Optionally, for the target model file data corresponding to the training task, the model file processing module mounts the target model file data through the model file management module, including:

[0014] Create a training task and display the maintained model table through the model file management module;

[0015] Select the corresponding target model file data from the model table according to the training task;

[0016] According to the selected target model file data, the model file processing module obtains the model file data from the corresponding database for mounting.

[0017] Optionally, the method further includes:

[0018] Customize the mounting path of the corresponding target model file data according to the training task;

[0019] According to the mounting path, the model file processing module mounts the target model file data from the mounting path.

[0020] Optionally, the method further includes:

[0021] Verify the determined target model file data;

[0022] When the verification passes, according to the determined target model file data, the model file processing module obtains the target model file data for mounting;

[0023] When the verification fails, according to the verification result, prompt the user with corresponding warning information, and the warning information at least includes: the determined target model file data does not exist, the determined target model file data is repeated.

[0024] Optionally, the method further includes:

[0025] When executing the training task through a script, customize the mounting path of the model file data in the container during the execution of the script;

[0026] According to the mounting path, the model file processing module mounts the model file data from the mounting path.

[0027] In the second aspect of the embodiments of the present invention, a system for a method of model training based on a training platform is provided. The system includes:

[0028] The system includes:

[0029] A model file management module for maintaining model file data;

[0030] A startup module for starting the model file management module when creating a training task;

[0031] A mounting module for mounting the target model file data by the model file processing module through the model file management module according to the target model file data corresponding to the training task;

[0032] A training module for performing deep learning model training on the training task based on the mounted target model file data.

[0033] In a third aspect of the embodiments of the present invention, an electronic device is further provided, including a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete communication with each other through the communication bus;

[0034] The memory is used for storing a computer program;

[0035] The processor is used for implementing the steps of a method for model training based on a training platform provided in the first aspect of the present invention when executing the program stored on the memory.

[0036] In a fourth aspect of the embodiments of the present invention, a computer-readable storage medium is further provided, on which a computer program is stored, and when the program is executed by a processor, it implements a method for model training based on a training platform as described in the first aspect of the present invention.

[0037] Regarding the prior art, the present invention has the following advantages:

[0038] A method for model training based on a training platform provided by the embodiments of the present invention constructs a model file management module in a deep learning model training platform, and maintains a model file data group through the model file management module; when a user creates a training task, the model file management module is started, and the model file management module provides the maintained model file data group for the user to select. When the target model file data corresponding to the training task selected by the user is obtained, the model file management module mounts the target model file data; the model is trained based on the mounted target model file data during the execution of the training task. Thus, the present invention constructs a model file management module, maintains a large model file data group through this model file management module, and directly mounts the target model file data for the training task selected by the user through the model file management module when the user creates a training task, without the need for copying and real-time downloading of model file data, thereby improving the convenience of deep learning model training and thus improving the training efficiency of deep learning model training.

[0039] The above description is only an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention, it can be implemented according to the content of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and understandable, the specific embodiments of the present invention are given below. Brief Description of the Drawings

[0040] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art.

[0041] Figure 1 It is a flowchart of a method for model training based on a training platform provided by an embodiment of the present invention;

[0042] Figure 2 It is another flowchart of a method for model training based on a training platform provided by an embodiment of the present invention;

[0043] Figure 3 It is a schematic diagram of a system for a method for model training based on a training platform provided by an embodiment of the present invention. Detailed Description of the Embodiments

[0044] The exemplary embodiments of the present invention will be described in more detail below with reference to the drawings.

[0045] Before describing the present invention, the background proposed by the present invention will be described first. At present, there are mainly the following two ways to train a deep learning model through model file data. One is to copy the model file data to a public directory before starting the deep learning model training, and mount the public directory when running the training task, so that the training script can read the model file data in the public directory mounted in the container; the other is to write code for downloading the model file data in the training script, and download the model file data in real time when running the script, and then perform deep learning model training after the download is completed. However, both of these two ways have problems with the ease of use of deep learning model training, which makes it necessary to perform real-time downloading or copying of model file data every time deep learning model training is performed, thus resulting in insufficient convenience of deep learning model training and also reducing the training efficiency of deep learning model training.

[0046] In view of this, the present invention proposes a new method for model training based on a training platform. By constructing a model file management module, a large model file data group is maintained by this model file management module; when a user creates a training task, the model file management module is started, and this model file management module will provide the maintained model file data group for the user to select. At this time, the user can directly select one or more target model file data from the provided model file data group for this training task. After the user selects the corresponding one or more target model file data, the model file management module will immediately mount the one or more target model file data from the maintained model file data group for this training task, without the need for real-time downloading or copying of the model file data, thereby being able to improve the convenience of deep learning model training and thus improve the training efficiency of deep learning model training.

[0047] Figure 1 The flowchart of a method for model training based on a training platform provided by an embodiment of the present invention is as Figure 1 shown, and the method includes:

[0048] Step S101: Construct a model file management module for maintaining model file data;

[0049] Step S102: When creating a training task, start the model file management module;

[0050] Step S103: According to the target model file data corresponding to the training task, the model file processing module mounts the target model file data through the model file management module;

[0051] Step S104: The training task performs deep learning model training based on the mounted target model file data.

[0052] In an embodiment of the present invention, a model file management module is integrated in a deep learning model training platform for maintaining and managing a model file data group. The model file data group includes a large amount of model file data, and the model file data group is stored in a database corresponding to the model file management model. When a user creates a training task through the deep learning model training platform, the model file management module is synchronously started. After the model file management module is synchronously started, all the model file data in the model file data group maintained by the model file management module is displayed to the user through the deep learning model training platform. The user can select the target model file data for this training task from all the model file data in the model file data group maintained by the displayed model file management module. Among them, the target model file data may include one or more types of model file data. After the user selects the target model file data for this training task from all the model file data in the model file data group maintained by the displayed model file management module, the model file data mounting processor in the model file processing module mounts the selected target model file data from the corresponding database to a container so that the training task can be executed based on the target model file data selected by the user. Among them, the container represents the unit in which the training task runs (docker container). After the model file data mounting processor in the model file processing module completes the mounting of the target model file data, the training is carried out on the mounted target model file data in the corresponding container for model training.

[0053] A method for model training based on a training platform provided by an embodiment of the present invention constructs a model file management module in a deep learning model training platform, and maintains a model file data group through the model file management module; when a user creates a training task, the model file management module is started. The model file management module provides the maintained model file data group for the user to select. When the target model file data corresponding to the training task selected by the user is obtained, the model file data mounting processor in the model file processing module mounts the target model file data; during the execution of the training task, model training is carried out based on the mounted target model file data. Thus, the present invention constructs a model file management module, which maintains a large model file data group through this model file management module. When a user creates a training task, the model file data mounting processor in the model file processing module directly mounts the target model file data for this training task selected by the user, without the need to copy and download the model file data in real time, thereby improving the convenience of deep learning model training and thus improving the training efficiency of deep learning model training.

[0054] In the present invention, the model file data maintained by the model file management module includes: externally imported model file data, and model file data generated during the execution of the training task.

[0055] In an embodiment of the present invention, to enhance the diversity of the model file data managed and maintained by the model file management module and the convenience of adding the model file data to the database corresponding to the model file management module, the present invention can import external model file data into the database corresponding to the model file management module, and can directly store the corresponding generated model file data in the database corresponding to the model file management module when the deep learning model is successfully trained during the execution of the training task.

[0056] In the present invention, the model file management module is used to maintain model file data, including: adding the model file data to the database corresponding to the model file management module; adding the location information and parameter information of the model file data to the corresponding model table through the model file management module.

[0057] In an embodiment of the present invention, one implementation manner of the model file management module in the present invention to maintain model file data is: adding the model file data that needs to be managed and maintained to the database corresponding to the model file management module. At the same time, the model file data group maintained by the model file management module has a corresponding model table, which records the location information and parameter information of each model file data in the model file data group stored in the database. After adding the model file data that needs to be managed and maintained to the database corresponding to the model file management module, adding the currently added model file data to the model file management module to the model table, and recording the location information and parameter information of the currently added model file data to the model file management module in the model table. The parameter information of the model file data includes at least the scenario, name, and version number.

[0058] In the present invention, for the target model file data corresponding to the training task, the model file processing module mounts the target model file data through the model file management module, including: creating a training task, and displaying and maintaining the model table through the model file management module; selecting the corresponding target model file data from the model table according to the training task; according to the selected target model file data, the model file processing module obtains the model file data from the corresponding database for mounting.

[0059] In an embodiment of the present invention, an implementation manner in which the model file data mounting processor in the model file processing module mounts the target model file data according to the target model file data corresponding to the training task is as follows: When a user creates a training task through a deep learning model training platform, the model file management module displays the managed and maintained model table to the user through the deep learning model training platform. The user selects the target model file data for the training task from the model table according to the training task. Among them, the target model file data may include one or more. According to the target model file data selected by the user from the displayed model table, the model file data mounting processor in the model file processing module obtains the target model file data from the corresponding database and mounts it into the corresponding container so that the training task can be executed based on the target model file data selected by the user.

[0060] In the present invention, the method further includes: customizing the mounting path of the corresponding target model file data according to the training task; according to the mounting path, the model file processing module mounts the target model file data from the mounting path.

[0061] In an embodiment of the present invention, the present invention aims to improve the diversity of deep learning model training. Another implementation manner in which the model file data mounting processor in the model file processing module of the present invention mounts the target model file data is as follows: When a user creates a training task through a deep learning model training platform, the model file management module displays the managed and maintained model table to the user through the deep learning model training platform. When the model table does not have the model file data required by the user or the user wants to use the model file data from other places according to their own needs, the model file management module of the present invention can also provide the user with a function of customizing the mounting path of the model file data. The user can customize the mounting path of the model file data according to their own needs. According to the mounting path, the model file data mounting processor in the model file processing module obtains the corresponding target model file data from the mounting path and mounts it into the corresponding container so that the training task can be executed based on the target model file data. For example, the user customizes one or more local mounting paths. At this time, the model file data mounting processor in the model file processing module will obtain the corresponding one or more target model file data from the positions pointed to by the one or more mounting paths based on the one or more mounting paths and mount them into the corresponding container so that the training task can be executed based on the one or more target model file data.

[0062] In an embodiment of the present invention, since the present invention can simultaneously select multiple target model file data for deep learning model training for a training task, when a user determines multiple target model file data for deep learning model training, some of the multiple target model file data can come from the target model file data selected by the user from the model table, and some of the multiple target model file data can come from the target model file data obtained from the user-defined mounting path. Wherein, the some target model file data may include one or more, and the other part of the target model file data may also include one or more.

[0063] Exemplarily, the user creates a training task through a deep learning model training platform, and this training task requires multiple target model file data for deep learning model training. Among the multiple target model file data, the target model file data belonging to the target model file data group maintained by the model file management module are target model file data a, target model file data b, and target model file data c, and the target model file data that do not belong to the target model file data group maintained by the model file management module among the multiple target model file data are target model file data d and target model file data e. For target model file data a, target model file data b, and target model file data c, the training platform can directly display the model table maintained by the model file management module to the user, and the user directly selects target model file data a, target model file data b, and target model file data c from the model table. For the target model file data d and target model file data e that do not belong to the target model file data group maintained by the model file management module, in the case of determining the storage paths of target model file data d and target model file data e respectively, in a custom manner, based on the storage paths of target model file data d and target model file data e respectively, the mounting paths of target model file data d and target model file data e are defined. Thus, a complete training task is created based on target model file data a, target model file data b, target model file data c determined by the user through the model table maintained by the model file management module and target model file data d, target model file data e determined by the user through the custom mounting path, so that the training task can perform deep learning model training through target model file data a, target model file data b, target model file data c, target model file data d, and target model file data e.

[0064] In the present invention, the method further includes: verifying the determined target model file data; when the verification passes, according to the determined target model file data, the model file processing module obtains the target model file data for mounting; when the verification fails, according to the verification result, corresponding warning information is prompted to the user, and the warning information at least includes: the determined target model file data does not exist, and the determined target model file data is repeated.

[0065] In an embodiment of the present invention, since the present invention can simultaneously select multiple target model file data for deep learning model training for a training task, in order to prevent the training task from not being executable due to the non-existence of the target model file data to be mounted in the user-defined mounting path or the deletion of the target model file data selected by the user from the model table, and in order to prevent the target model file data in the user-defined mounting path from being repeated with the target model file data selected by the user from the model table, resulting in multiple identical target model file data performing deep learning model training simultaneously, thus causing ineffective training and wasting computing resources. Therefore, the model file data validator in the model file processing module verifies the target model file data determined by the user through the model table and / or through the user-defined mounting path to determine whether these determined target model file data exist and whether there are duplicate identical target model file data. When the verification passes, that is, these determined target models all exist and there are no duplicate identical target model file data, at this time, the model file data mounting processor in the model file processing module mounts each target model file data. When the verification fails, when the verification result is that a certain target model file data does not exist, the deep learning model training platform alarms the user to inform the user that the certain target model file data does not exist, and when the verification result is that some target model file data is repeated, the deep learning model training platform alarms the user to inform the user that the some target model file data is repeated, and prompts the user to select one target model file data from the some target model file data for subsequent deep learning model training.

[0066] Exemplarily, the user creates a training task through a deep learning model training platform, and determines multiple target model file data through the model table and / or by customizing the mounting path, including: target model file data a1, target model file data a2, target model file data a3, target model file data a4, target model file data a5. Among them, target model file data a1, target model file data a2, and target model file data a3 are all determined through the model table maintained by the model file management module, while target model file data a4 and target model file data a5 are both determined by the user by customizing the mounting path of the model file data. The model file data validator in the model file processing module validates whether the target model file data a1, target model file data a2, target model file data a3, target model file data a4, and target model file data a5 exist and whether they are duplicates, and obtains the corresponding validation results. The validation result is finally that the target model file data a1 does not exist, and at the same time, it is shown that the target model file data a2 and the target model file data a4 belong to duplicate model file data. According to this validation result, an alarm is sent to the user through the page of the deep learning model training platform to inform the user that the target model file data a1 determined through the model table maintained by the model file management module does not exist, and at the same time, an alarm is sent to the user through the page of the deep learning model training platform to inform the user that the target model file data a2 determined through the model table maintained by the model file management module and the target model file data a4 determined by the user by customizing the mounting path of the model file data belong to duplicate model file data. After the user deletes the selected target model file data a1 based on the alarm information and retains one of the target model file data a2 and the target model file data a4, namely the target model file data a2, and then validates each target model file data determined by the user again, that is, validates the target model file data a2, target model file data a3, and target model file data a5, there will no longer be non-existent target model file data and duplicate target model file data at this time, and the validation passes. At this time, the model file data mounting processor in the model file processing module mounts the target model file data a2, target model file data a3, and target model file data a5 into the corresponding container so that the training task can perform deep learning model training normally.

[0067] In the present invention, the method further includes: when executing a training task through a script, customizing the mounting path of the model file data in the container during the execution of the script; according to the mounting path, the model file processing module mounts the model file data from the mounting path.

[0068] In an embodiment of the present invention, the method further includes: when executing a training task through a script, customizing the mounting path of the model file data in the container during the execution of the script; according to the mounting path, the model file data mounting processor in the model file processing module mounts the model file data from the mounting path.

[0069] In an embodiment of the present invention, when a user wants to execute deep learning model training through a script used by another user, in order to make the present invention applicable to the scenario where the user does not want to modify the script. The present invention also provides an implementation: when the user executes a deep learning model training task through a script used by another user, the deep learning model training platform will mount the model file data under a default mounting path, and the model file data pointed to by the mounting path in the script is different from the model file data under the model mounting path mounted by the platform. At this time, the deep learning model training platform will not be able to execute the script to perform deep learning model training. This is because the mounting path executed by the script is different from the default mounting path of the deep learning model training platform. For example, the mounting path executed by the script is / model / xxx, which points to model file data 1, while the default mounting path of the platform is / a / 1 (the model name is a and the version is 1), which points to model file data 2. The script wants to execute the model file data under the mounting path / model / xxx, while the deep learning model training platform actually defaults to mount the model file data under the mounting path / a / 1, and the two are inconsistent. In order to achieve the goal of correctly executing the script without modifying the script, the model file management module constructed by the present invention also has the function of customizing the mounting path of the model file data in the container. When the user executes a deep learning model training task through a script used by another user, the mounting path of the model file data in the container during the execution of the script is customized through the model file management module in the deep learning model training platform, that is, the default mounting path of the deep learning model training platform is customized to the mounting path in the script. As in the above example, the default mounting path / a / 1 of the deep learning model training platform is customized to the mounting path / model / xxx of the model file data in the script.

[0070] In an embodiment of the present invention, as Figure 2As shown in the figure, an implementation of a method for model training based on a training platform provided by the present invention is as follows: First, the user creates a training task through a deep learning model training platform, and at the same time determines the target model file data for the training task from the model file data group managed by the model file management module and / or determines the target model file data for the training task by customizing the mounting path of the model file data. The sources of each model file data in the model file data group managed and maintained by the model file management module include model file data imported externally and model file data generated during the training process of the training task. After the user selects the corresponding target model file data from the model table provided by the model file management module through the deep learning model training platform and / or determines the corresponding target model file data by customizing the mounting path of the model file data, the model file processing module starts to work at this time. First, the model file data validator in the model file processing module starts to work, and the target model file data determined by the user through the model table and / or through the custom mounting path is verified by the model file data validator in the model file processing module to determine whether these determined target model file data exist and whether there are duplicate identical target model file data.

[0071] When the verification passes, that is, when all these determined target model files exist and there are no duplicate identical target model file data, the model file data mounting processor in the model file processing module will work at this time. The model file data mounting processor in the model file processing module mounts the target model file data determined by the user through the model table and / or through the custom mounting path into the corresponding container so that the training task can be correctly executed in the container.

[0072] When the verification fails, for example, when the verification result indicates that a certain target model file data A does not exist, an alarm is sent to the user through the deep learning model training platform to inform the user that the certain target model file data A does not exist. When the verification result shows that some target model file data (such as target model file data B, target model file data C, target model file data D) is repeated, an alarm is sent to the user through the deep learning model training platform to inform the user that the some target model file data (target model file data B, target model file data C, target model file data D) is repeated, and the user is prompted to select one target model file data from the some target model file data (i.e., only retain one of the target model file data B, target model file data C, target model file data D) for subsequent deep learning model training. After the user performs corresponding correction operations based on the alarm information (for example, when it is prompted that the target model file does not exist, a new model file data is re-determined or the non-existent target model file data is deleted; when it is prompted that there are multiple duplicate target model file data, one of the multiple duplicate target model file data is retained for the execution of subsequent training tasks), the target model file data determined by the user through the model table and / or through the custom mounting path is verified again. When the verification passes, that is, when it is determined that these target model files all exist and there are no duplicate identical target model file data, the model file data mounting processor in the model file processing module will work. The model file data mounting processor in the model file processing module mounts the target model file data determined by the user through the model table and / or through the custom mounting path to the corresponding container so that the training task can be correctly executed in the container; when the verification fails again, corresponding alarms will continue to be sent to the user until the final verification passes, and then the model file data mounting processor in the model file processing module will work to mount the target model file data determined by the user through the model table and / or through the custom mounting path to the corresponding container so that the training task can be correctly executed in the container.

[0073] The present invention provides a method for model training based on a training platform, which is mainly applicable to scenarios where a training task uses one or more model file data for training and the training script directly uses the model file data for training. It solves the usability problem when using model file data for training tasks. Through the deep learning model training method in the present invention, when creating a training task, it is possible to effectively select the model file data to be used and / or the mounting path of the custom model file data through the deep learning model training platform page. After successful creation, the deep learning model training platform will automatically mount the model file data with the selected and / or custom-mounted path to the task container, and the training script can directly use the model file data with the selected and / or custom-mounted path. For scenarios where users want to continue using historical scripts without modifying them, it is also possible to customize the mounting path of the model file data in the container to the mounting path of a model file data maintained by default in the deep learning model training platform, so as to meet the requirement of directly using model file data for training without modifying the script. Through the method for model training based on a training platform provided by the present invention, users can efficiently use model file data for deep learning model training of training tasks, without the need to modify model file parameters in the training script or copy / download model file data in the public directory, which can effectively improve the training efficiency of algorithm personnel.

[0074] The second aspect of the present invention provides a system for the method of model training based on a training platform, as Figure 3 shown. The system 300 includes:

[0075] A model file management module 301 for maintaining model file data;

[0076] A start module 302 for starting the model file management module when creating a training task;

[0077] A mounting module 303 for mounting the target model file data according to the target model file data corresponding to the training task, and the model file processing module mounts the target model file data through the model file management module;

[0078] A training module 304 for performing deep learning model training on the training task based on the mounted target model file data.

[0079] Optionally, the model file data maintained by the model file management module 301 includes: externally imported model file data, and model file data generated during the execution of the training task.

[0080] Optionally, the model file management module 301 includes:

[0081] A storage module, configured to add model file data to the database corresponding to the model file management module;

[0082] A data addition module, configured to add the location information and parameter information of the model file data to the corresponding model table through the model file management module.

[0083] Optionally, the mounting module 303 includes:

[0084] A model table display module, configured to create a training task and display the maintained model table through the model file management module;

[0085] A target model file data determination module, configured to select corresponding target model file data from the model table according to the training task;

[0086] A mounting sub-module, configured to obtain the model file data from the corresponding database for mounting according to the selected target model file data by the model file processing module.

[0087] Optionally, the system 300 further includes:

[0088] A first mounting path customization module, configured to customize the mounting path of the corresponding target model file data according to the training task;

[0089] A first mounting module, configured to mount the target model file data from the mounting path by the model file processing module according to the mounting path.

[0090] Optionally, the system 300 further includes:

[0091] A verification module, configured to verify the determined target model file data;

[0092] A first verification module, configured to, when the verification is passed, obtain the target model file data for mounting by the model file processing module according to the determined target model file data;

[0093] A second verification module, configured to, when the verification fails, prompt the user with corresponding warning information according to the verification result, where the warning information at least includes: the determined target model file data does not exist, the determined target model file data is repeated.

[0094] Optionally, the system 300 further includes:

[0095] A second mounting path customization module, configured to customize the mounting path of the model file data in the container during the execution of the script when executing the training task through the script;

[0096] The second mounting module is used to mount the model file data by the model file processing module from the mounting path according to the mounting path.

[0097] The present invention provides a system for a method of model training based on a training platform, which is mainly applicable to scenarios where a training task uses one or more model file data for training, and the training script directly uses the model file data for training. It solves the problem of usability when a training task uses model file data for training. Through the deep learning model training method in the present invention, when creating a training task, it is possible to effectively select the model file data and / or the mounting path of the custom model file data through the deep learning model training platform page. After successful creation, the deep learning model training platform will automatically mount the model file data with the selected and / or custom-mounted path into the task container, and the training script can directly use the model file data with the selected and / or custom-mounted path. For scenarios where users want to continue using historical scripts without modifying them, it is also possible to customize the mounting path of the model file data in the container by customizing the mounting path of a model file data maintained by default in the deep learning model training platform to the mounting path of the model file data defined in the historical script, realizing the requirement of directly using model file data for training without modifying the script. Through the system for a method of model training based on a training platform provided by the present invention, users can efficiently use model file data for deep learning model training of training tasks, without modifying model file parameters in the training script or copying / downloading model file data in a public directory, which can effectively improve the training efficiency of algorithm personnel.

[0098] In the third aspect of the embodiments of the present invention, an electronic device is further provided. The electronic device includes a processor, a communication interface, a memory, and a communication bus. Among them, the processor, the communication interface, and the memory complete mutual communication through the communication bus;

[0099] The memory is used to store a computer program;

[0100] The processor is used to implement the steps of the method for a model training based on a training platform provided in the first aspect of the present invention when executing the program stored on the memory.

[0101] In the fourth aspect of the embodiments of the present invention, a computer-readable storage medium is further provided, on which a computer program is stored, and when the program is executed by a processor, it implements the method for a model training based on a training platform as described in the first aspect of the present invention.

[0102] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from a website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, hard disk, magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)).

[0103] It should be noted that, in this document, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "comprising a..." does not exclude the existence of additional identical elements in the process, method, article or device comprising the element.

[0104] Each embodiment in this specification is described in a related manner. For the same or similar parts among the embodiments, reference can be made to each other. Each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and reference can be made to the corresponding part of the method embodiment for the relevant content.

[0105] The above are only the preferred embodiments of the present invention and are not intended to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention are all included in the protection scope of the present invention.

Claims

1. A method for model training based on a training platform, characterized in that, the method includes: Construct a model file management module for maintaining model file data; When creating a training task, start the model file management module; According to the target model file data corresponding to the training task, the model file processing module mounts the target model file data through the model file management module; The training task performs deep learning model training based on the mounted target model file data; When executing the training task through a script, customize the mounting path of the model file data in the container during the script execution process; According to the mounting path, the model file processing module mounts the model file data from the mounting path; Among them, the model file processing module mounts the target model file data through the model file management module according to the target model file data corresponding to the training task, including: Create a training task, and display the maintained model table through the model file management module; According to the training task, select the corresponding target model file data from the model table; According to the selected target model file data, the model file processing module obtains the model file data from the corresponding database for mounting.

2. The method according to claim 1, characterized in that, The model file data maintained by the model file management module includes: externally imported model file data, and model file data generated during the execution of the training task.

3. The method according to claim 1, characterized in that, The model file management module is used to maintain model file data, including: Add the model file data to the database corresponding to the model file management module; Add the location information and parameter information of the model file data to the corresponding model table through the model file management module.

4. The method according to claim 1, characterized in that, The method further includes: According to the training task, customize the mounting path of the corresponding target model file data; According to the mounting path, the model file processing module mounts the target model file data from the mounting path.

5. The method according to claim 1, characterized in that, The method further includes: Verify the determined target model file data; When the verification passes, according to the determined target model file data, the model file processing module obtains the target model file data for mounting; When the verification fails, according to the verification result, prompt the user with the corresponding warning information, and the warning information at least includes: the determined target model file data does not exist, and the determined target model file data is repeated.

6. A system for a method of model training based on a training platform, characterized in that, the system includes: A model file management module for maintaining model file data; A startup module for starting the model file management module when creating a training task; A mounting module for the model file processing module to mount the target model file data through the model file management module according to the target model file data corresponding to the training task; A training module for performing deep learning model training on the training task based on the mounted target model file data; A second mount path customization module for customizing the mount path of the model file data in the container during the execution of the script when performing the training task through the script; A second mounting module for the model file processing module to mount the model file data from the mount path according to the mount path; Wherein, the mounting module includes: a model table display module for creating a training task and displaying the maintained model table through the model file management module; a target model file data determination module for selecting corresponding target model file data from the model table according to the training task; a mounting sub-module for the model file processing module to obtain the model file data from the corresponding database for mounting according to the selected target model file data.

7. An electronic device, characterized in that, it includes a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory complete communication with each other through the communication bus; the memory is used for storing a computer program; the processor, when executing the program stored on the memory, implements the steps in the method for model training based on a training platform according to any one of claims 1-5.

8. A computer-readable storage medium, on which a computer program is stored, characterized in that, when the program is executed by a processor, it implements the method for model training based on a training platform according to any one of claims 1-5.

Images