Model training method, electronic device and storage medium

Abstract

The present application relates to the technical field of machine learning, and specifically discloses a model training method, an electronic device and a storage medium, the method comprising: obtaining a pre-trained super network model; determining a plurality of target sub network models from a preset number of sub network models of the super network model; obtaining a plurality of mainstream network models trained based on open source data; splicing each of the target sub network models as a first trunk network with a first branch network to obtain a plurality of first spliced ​​networks, and splicing each of the mainstream network models as a second trunk network with a second branch network to obtain a plurality of second spliced ​​networks; fine-tuning and testing the plurality of the first spliced ​​networks and the second spliced ​​networks to determine the target network model; performing transfer learning on the target network model to obtain the required model, thereby realizing the convenience and intelligence of model training and improving the user experience.

Description

Technical Field

[0001] The present application relates to the field of machine learning technology, and in particular to a model training method, electronic device and storage medium. Background Art

[0002] Neural Architecture Search (NAS) is one of the hot topics in the field of Automated Machine Learning (AutoML). By designing cost-effective search methods, neural networks with strong generalization capabilities and friendly hardware requirements can be automatically obtained, thus saving a lot of manpower and material resources. The main working principle of NAS is to first define the search space, then find candidate network structures through search strategies, evaluate them, and conduct the next round of search based on feedback until the target network structure is found, and then perform automatic machine learning based on the target network structure to obtain the required model.

[0003] However, the existing NAS search process is long. Every time after obtaining user data, it needs to search from the beginning to obtain the network structure. When using the network structure, it needs to be pre-trained, and then transfer learning is performed on the user data to obtain the required model. Due to the long search waiting time, it is not possible to provide users with a friendly experience. Summary of the invention

[0004] The embodiments of the present application provide a model training method, an electronic device, and a storage medium, which aim to solve the problem of long waiting time for searching a neural network structure, save time costs, and provide users with a more friendly experience.

[0005] In a first aspect, the present application provides a method for training a model, the method comprising:

[0006] Acquire a pre-trained hypernetwork model, wherein the hypernetwork model includes a preset number of subnetwork models;

[0007] Determining a plurality of target sub-network models from a preset number of sub-network models of the super-network model;

[0008] Obtain multiple mainstream network models trained based on open source data;

[0009] Each of the target sub-network models is used as a first trunk network and is spliced ​​with a first branch network to obtain a plurality of first spliced ​​networks, and each of the mainstream network models is used as a second trunk network and is spliced ​​with a second branch network to obtain a plurality of second spliced ​​networks, wherein the network structures of the first branch networks spliced ​​after each of the first trunk networks are the same and the parameters are shared, and the network structures of the second branch networks spliced ​​after each of the second trunk networks are the same and the parameters are not shared;

[0010] Fine-tune and test a plurality of the first splicing networks and the second splicing networks to determine a target network model;

[0011] Perform transfer learning on the target network model to obtain the required model.

[0012] In a second aspect, the embodiment of the present application further provides another model training method, the method comprising:

[0013] Acquire a pre-trained hypernetwork model, wherein the hypernetwork model includes a preset number of subnetwork models;

[0014] Determining a plurality of target sub-network models from a preset number of sub-network models of the super-network model;

[0015] Obtain multiple mainstream network models trained based on open source data;

[0016] Each of the target sub-network models is used as a first trunk network and is spliced ​​with a first branch network to obtain a plurality of first spliced ​​networks, and each of the mainstream network models is used as a second trunk network and is spliced ​​with a second branch network to obtain a plurality of second spliced ​​networks;

[0017] Fine-tune and test a plurality of the first splicing networks and the second splicing networks to determine a target network model;

[0018] Perform transfer learning on the target network model to obtain the required model.

[0019] In a third aspect, the present application embodiment further provides another model training method, the method comprising:

[0020] Acquire a pre-trained hypernetwork model, wherein the hypernetwork model includes a preset number of subnetwork models;

[0021] Determining a plurality of target sub-network models from a preset number of sub-network models of the super-network model;

[0022] Obtain multiple mainstream network models trained based on open source data;

[0023] Determining a target network model according to the plurality of target sub-network models and the plurality of mainstream network models;

[0024] Perform transfer learning on the target network model to obtain the required model.

[0025] In a fourth aspect, an embodiment of the present application provides an electronic device, including a memory and a processor;

[0026] The memory is used to store computer programs;

[0027] The processor is used to execute the computer program and, when executing the computer program, implement the model training method provided in any one of the embodiments of the present application.

[0028] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the processor enables the processor to implement any one of the model training methods provided in the embodiments of the present application.

[0029] The model training method, electronic device and storage medium provided in the embodiments of the present application can not only greatly save the search time of neural architecture search (NAS) by combining a pre-trained hypernetwork model with a mainstream network model and then splicing a trunk network with a branch network, but also improve the accuracy of the model.

[0030] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory and cannot limit the disclosure of the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0032] Figure 1 A method for training a hypernetwork model provided in an embodiment of the present application;

[0033] Figure 2 is a schematic diagram of a super network structure provided in an embodiment of the present application;

[0034] Figure 3 is a schematic flow chart of a model training method provided in an embodiment of the present application;

[0035] Figure 4 This is a schematic flow chart of selecting a sub-network model that meets the preset model constraint conditions from a super-network model provided by an embodiment of the present application.

[0036] Figure 5 It is a schematic flow chart of obtaining multiple mainstream network models trained based on open source data provided by an embodiment of the present application;

[0037] Figure 6 This is a schematic diagram of a scenario in which a model training method provided in an embodiment of the present application is applied to a server;

[0038] Figure 7 is a schematic flow chart of another model training method provided in an embodiment of the present application;

[0039] Figure 8 It is a schematic block diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0040] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0041] The flowcharts shown in the accompanying drawings are only examples and do not necessarily include all the contents and operations / steps, nor must they be executed in the order described. For example, some operations / steps may also be decomposed, combined or partially merged, so the actual execution order may change according to actual conditions.

[0042] The term "and / or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0043] At present, Neural Architecture Search (NAS) is one of the hot topics in the field of automatic machine learning (Auto-ML). By designing economical and efficient search methods, neural networks with strong generalization capabilities and friendly hardware requirements can be automatically obtained, thus saving a lot of manpower and material resources.

[0044] However, the existing NAS search process is long. After obtaining user data, it needs to search from the beginning to obtain the network structure. When using the network structure, it needs to be pre-trained, and then transfer learning is performed on the user data to obtain the required model. Due to the long search waiting time, it is not possible to provide users with a friendly experience.

[0045] To this end, the present application provides a model training method, an electronic device and a storage medium to solve the above problems.

[0046] In conjunction with the accompanying drawings, some embodiments of the present application are described in detail below. In the absence of conflict, the following embodiments and features in the embodiments can be combined with each other.

[0047] Since the model training method provided in the embodiment of the present application is implemented through a hypernetwork model, before introducing the model training method, the training method of the hypernetwork model is first introduced.

[0048] See also Figure 1 , Figure 1 A method for training a hypernetwork model provided in an embodiment of the present application is shown, and the hypernetwork model obtained through training can be used to determine multiple target sub-network models.

[0049] like Figure 1 As shown, the training method of the hypernetwork model includes: steps S101 to S104.

[0050] S101. Obtain open source datasets.

[0051] Open source data can be a public training set obtained from the Internet, such as an open source image set (imagenet). Specifically, the corresponding open source data set can be searched in the open source data according to the model requirement information provided by the user, and used to train the hypernetwork model. The model requirement information includes information corresponding to the function, type and scope of application of the model, that is, description information of what the user wants to do with the model.

[0052] Exemplarily, the open source dataset may also be a dataset related to CV (computer vision) tasks or NLP (natural language processing) tasks, etc.

[0053] In some embodiments, the model requirement information includes at least one of the following: task type, terminal type, application scenario, and computing power requirement. Among them, the task type indicates the practical scenario of the target AI model required by the user, such as the task that the target AI model needs to process, such as classification, detection, video, natural language processing, etc. The terminal type indicates the deployment environment of the target AI model, such as the model of the terminal where the target AI model is deployed, the processor type and model of the terminal, etc. For example, the processor type of the terminal may include CPU and / or GPU. The application scenario may include at least one of the following: small sample detection, small object detection, unbalanced sample detection, etc. The computing power requirement indicates the ability of the model to process tasks.

[0054] S102: Obtain a preset hypernetwork, where the hypernetwork includes a first number of channels and a second number of layers.

[0055] The super network includes a first number of channels and a second number of layers, and the first number and the second number are greater than a preset number threshold to ensure that the neural network is a super network. The preset number threshold is specifically 100, for example, the width of the first layer of the model is 128 channels, and the depth of the model is 101 layers, wherein the super network is specifically a neural network.

[0056] It is understandable that the number of channels and layers of the hypernetwork may be any other number and is not specifically limited herein.

[0057] S103: Randomly shut down channels and / or layers of the super network, and use the open source dataset to train a batch of data on the remaining network.

[0058] Randomly shut down the channels and / or layers of the hypernetwork. After disconnecting some channels and some layers, a new network structure will be formed. The new network structure is trained based on the open source dataset, that is, a batch of data is trained. The remaining network refers to the new network structure after disconnecting some channels and some layers.

[0059] For example, Figure 2 As shown in the figure, the depth of the hypernetwork structure is 4 layers, and each layer of the network width includes 6 channels. The number of channels in the 4 layers is 6-6-6-6. In actual applications, it is much larger than 4 layers and each layer includes 6 channels. After randomly disconnecting the channels of width and / or the layers of depth, the new network structure becomes 3 layers deep and 5-5-6 wide. Among them, each layer and each channel can be randomly disconnected. It should be noted that Figure 2 The ones marked with “×” are disconnected layers or channels.

[0060] S104, until the preset hypernetwork converges to obtain a pre-trained hypernetwork model.

[0061] Repeat the steps of randomly shutting down the channels and / or layers of the preset hypernetwork and training a batch of data on the remaining network using the open source data set until the preset hypernetwork converges to obtain a pre-trained hypernetwork model.

[0062] Specifically, an error can be defined by using the back propagation algorithm to determine whether the super network converges. If the super network convergence meets the agreed error condition, that is, the super network convergence is determined to be successful, and a pre-trained super network model is obtained; if the super network convergence does not meet the agreed error condition, steps S103 and S104 are repeated until the preset super network converges to obtain a pre-trained super network model. It can be seen that the super network model is a neural network model including a preset number of sub-network models. This can improve the efficiency and accuracy of subsequent model training and greatly reduce the time cost of training.

[0063] In some embodiments, after obtaining a pre-trained hypernetwork model, a subnetwork model can be directly extracted from the pre-trained hypernetwork model and sent to the user so that the user can directly use the subnetwork model, which is suitable for the case where the user does not have high requirements for model accuracy and application scenarios. In this way, when facing users with different needs, suitable models can be provided in a targeted manner, which reduces unnecessary training and improves user experience.

[0064] According to the hypernetwork model training method provided above, a pre-trained hypernetwork model can be obtained. The model training methods provided in the embodiments of the present application are all based on the hypernetwork model. Therefore, the model training methods provided in the embodiments of the present application can be introduced on this basis.

[0065] See also Figure 3 , Figure 3 This is a schematic flow chart of a model training method provided in an embodiment of the present application. The model training method can be applied to electronic devices, can realize efficient model training, improve the convenience of users to obtain the required model, reduce the time cost of training the model, and thus improve the user experience.

[0066] The electronic device is, for example, a terminal device or a server. The terminal device may be a mobile phone, a tablet computer, a laptop computer, a desktop computer, a personal digital assistant, a wearable device, or other electronic device. The server may be an independent server, a server cluster, or a cloud server (Elasti cCompute Service, ECS). In some embodiments, the electronic device includes a GPU to improve the training efficiency of the model.

[0067] like Figure 3 As shown, the model training method includes steps S201 to S206.

[0068] S201. Obtain a pre-trained super network model, where the super network model includes a preset number of sub-network models.

[0069] In an embodiment of the present application, a super network model obtained by the super network model training method can be obtained, and the super network model includes a preset number of sub-network models, and the preset number is, for example, 100 or more, which is not limited here.

[0070] S202: Determine a plurality of target sub-network models from a preset number of sub-network models of the super-network model.

[0071] The super network model is sampled according to the sampling algorithm, and then the sub-network model that meets the preset model constraint conditions is screened out according to the preset model constraint conditions as the target sub-network model. The sub-network model is continuously sampled until the preset value is met, thereby determining multiple target sub-network models, the preset value is, for example, M, and M can be set by the user according to actual conditions, and is not specifically limited here.

[0072] Exemplarily, for example, the value range of M is set to 10-50, and specifically M can be set to be equal to 20.

[0073] The preset sampling algorithm includes at least one of a random sampling algorithm, an evolutionary algorithm-based sampling algorithm and a gradient-based sampling algorithm.

[0074] The random sampling algorithm randomly selects a sub-network model from the super-network model; the sampling algorithm based on the evolutionary algorithm utilizes the parallel iteration of the evolutionary algorithm to improve the sampling compliance while ensuring the sampling accuracy; the gradient-based sampling algorithm such as gradient descent can obtain corresponding changes in the output according to the sampling results, thereby improving the probability of obtaining the optimal sub-network. The above three sampling methods can all be applied in the embodiments of the present application, but the latter two have higher sampling efficiency and accuracy.

[0075] In some embodiments, taking the random sampling algorithm as an example, Figure 4 As shown, the step of selecting a sub-network model that meets the preset model constraints from the super-network model specifically includes the following steps:

[0076] S2021. Randomly select a sub-network model from the super-network model;

[0077] S2022: Determine whether the computation amount of the sub-network model is less than a preset computation amount threshold, and whether the model parameter amount of the sub-network model is less than a preset parameter amount threshold;

[0078] S2023. If the computational complexity of the sub-network model is less than the preset computational complexity threshold or the model parameter quantity of the sub-network model is less than the preset parameter quantity threshold, the sub-network model is selected.

[0079] Specifically, a sub-network model is randomly selected from a preset number of sub-network models in the super-network model, the amount of computation and the amount of model parameters of the selected sub-network model are obtained, and it is determined whether the amount of computation of the selected sub-network model is less than the preset amount of computation (FLOPS) threshold, and whether the amount of model parameters of the selected sub-network model is less than the preset parameter amount threshold; if the amount of computation of the sub-network model is less than the preset amount of computation threshold, and the amount of model parameters of the sub-network model is less than the preset parameter amount threshold, the sub-network model is selected; if the amount of computation of the sub-network model is greater than or equal to the preset amount of computation threshold, or the amount of model parameters of the sub-network model is greater than or equal to the preset parameter amount threshold, the sub-network model is discarded.

[0080] Specifically, if the amount of computation of the sub-network model is greater than or equal to the preset computation amount threshold, and the amount of model parameters of the sub-network model is less than the preset parameter amount threshold, the sub-network model is discarded. If the amount of computation of the sub-network model is less than the preset computation amount threshold, and the amount of model parameters of the sub-network model is greater than or equal to the preset parameter amount threshold, the sub-network model is discarded.

[0081] Among them, determine whether the computing amount of the selected sub-network model is less than the preset computing amount threshold, the computing amount is the number of floating-point operations per second (FLOPS), which is used to determine whether the computing amount of the sub-network model meets the requirements; determine whether the model parameter amount of the sub-network model is less than the preset parameter amount threshold, the model parameter amount may include the number of weights, which is used to determine whether the model parameter amount of the sub-network model meets the requirements.

[0082] In some embodiments, it is determined whether the number of collected sub-network models meets the preset value. If the number of collected sub-network models meets the preset value, the loop is terminated to determine the selected sub-network model; if the number of collected sub-network models does not meet the preset value, the above steps S2021-S2023 are repeated until the number of collected sub-network models meets the preset value. This can improve the accuracy of the selected sub-network model, prevent the optimal model from being missed due to chance, and improve the fault tolerance rate.

[0083] The preset value is a pre-set quantity value, which can be 50 or any other quantity.

[0084] In some embodiments, the number of target sub-network models that need to be determined can also be determined based on the number of mainstream network models. For example, the number of target sub-network models to be determined is equal to or greater than the number of mainstream network models, or the number of mainstream network models can be the same as or in a certain proportion to the number of target sub-network models that need to be determined. Of course, other relationships can also be used, which are not specifically limited here. The accuracy and efficiency of the searched network structure can be improved by the number of mainstream network models.

[0085] In some embodiments, the number of target sub-network models to be determined may be agreed upon first, and then the number of mainstream network models to be obtained may be limited.

[0086] In some embodiments, the collected sub-network models can also be tested and evaluated to obtain the accuracy of the sub-network models and rank the test and evaluation results to determine multiple target sub-network models. This allows a clear understanding of the ranking of the sub-network models, thereby selecting the optimal model.

[0087] Specifically, the collected multiple sub-network models are tested and evaluated according to the test set to obtain test evaluation results of the multiple sub-network models, and the test evaluation results are used to represent the accuracy.

[0088] Specifically, the test set is used to test and evaluate the collected multiple sub-network models to obtain test evaluation results about the multiple sub-network models, wherein the test set can be test data used to test and evaluate the output test evaluation results of the sub-network models.

[0089] Exemplarily, taking an image set as an example, multiple sub-network models can be tested using object recognition test data. Multiple sub-network models learn the test data and output test evaluation results. The specific output test evaluation results can be probability values, thereby calculating the accuracy of multiple sub-network models based on the test evaluation results.

[0090] It is understandable that the multiple sub-network models can be subjected to multiple test evaluations and multiple test evaluation results and test accuracy rates can be obtained.

[0091] In some embodiments, the multiple sub-network models may also be sorted according to the test evaluation results of the multiple sub-network models to obtain the sorting results of the multiple sub-network models.

[0092] The multiple sub-network models are sorted according to the test evaluation results of the multiple sub-network models on the test data, so as to obtain the sorting results of the multiple sub-network models.

[0093] Exemplarily, the multiple sub-network models may be sorted according to scores, such as sorting from sub-network models with high scores to sub-network models with low scores or sorting from sub-network models with low scores to sub-network models with high scores.

[0094] In some embodiments, a plurality of target sub-network models are determined according to the sorting result and the number of the target sub-network models. According to the number of target sub-network models to be determined, a corresponding number of target sub-network models are selected through the sorting result.

[0095] Exemplarily, if the number of mainstream network models obtained is 10, the number of target sub-network models that need to be determined is also 10. According to the sorting result, for example, the sub-network models with high scores are sorted in sequence, and the top 10 sub-network models are selected as target sub-network models.

[0096] In some embodiments, if multiple test evaluations are performed, the test score proportions can be first allocated according to the importance of the tests, such as 60% for highly important tests and 40% for less important tests, and finally a comprehensive score is obtained.

[0097] For example, if a person recognition test and an animal recognition test are performed, the person recognition test score accounts for 60% and the animal recognition test score accounts for 40%. If the person recognition test score of sub-network model A is 100 points and the animal recognition test score is 90 points, then the comprehensive score of sub-model A is 96 points; if the person recognition test score of sub-network model B is 90 points and the animal recognition test score is 100 points, then the comprehensive score of sub-model B is 94 points. Therefore, the ranking order of sub-model A is higher than that of sub-model B.

[0098] S203. Obtain multiple mainstream network models trained based on open source data.

[0099] Among them, the mainstream network model is trained based on open source data. The network model that has been trained and frequently used by users can be obtained from the open source website as the mainstream network model.

[0100] In some embodiments, the model complexity of the obtained multiple mainstream network models is different. The model complexity includes at least one of the model operation amount and the model parameter amount. By selecting mainstream network models with different model complexities, the accuracy of network structure search can be improved, thereby improving the accuracy of the model required by the user.

[0101] In some embodiments, Figure 5 As shown, multiple mainstream network models trained based on open source data are obtained, which specifically includes the following steps:

[0102] S2031. Determine the type of the pre-trained hypernetwork model.

[0103] The type of the hypernetwork model, such as detection, classification, and recognition, indicates the purpose of the hypernetwork model. For example, if the pre-trained hypernetwork model is image recognition, the type of the hypernetwork model is recognition.

[0104] S2032. According to the type, select at least one open source network model that matches the type of the pre-trained super network model from a public open source network model set as a seed model.

[0105] According to the determined type of the super network model, one or more open source network models of matching type are selected from a public open source network model set and used as seed models, wherein the open source network models have different model complexities.

[0106] Exemplarily, if the type of the determined hypernetwork model is a detection type, multiple open source network models for detection are selected from a public open source network model set, such as a network model for person detection and a network model for animal detection, and used as seed models, such as using mobilenetV2 (lightweight convolutional neural network) as a seed model.

[0107] S2033. Obtain user requirement information for the model, where the requirement information includes the accuracy of the model and / or the magnitude of the model.

[0108] The user's requirement information for the model is obtained, such as the accuracy of the model and / or the magnitude of the model. For example, the user's requirement for the accuracy of the model is 95% and the magnitude of the model is 2M (mega) parameters.

[0109] S2034. Determine a conversion processing strategy for the model according to the user's demand information for the model, where the conversion processing strategy includes at least one of an enlargement processing strategy and a compression processing strategy.

[0110] Specifically, if the demand information has high requirements for the model, such as the accuracy requirement is 90% and the model has a parameter level of 2M (megabytes), the demand information with high requirements corresponds to the increase processing strategy. On the contrary, if the demand information has low requirements for the model, such as the accuracy requirement is 80% and the model has a parameter level of 1M (megabytes), the demand information with low requirements corresponds to the compression processing strategy.

[0111] Exemplarily, the enlargement processing strategy can be performed by adding RBF (Radial Basis Neural Network) or SE (Squeeze-and-Excitation) to expand the convolution layer, thereby fusing features of the local area, or designing more channel features to increase.

[0112] For example, by adding an attention mechanism to the model, a model that can be used for text summarization, reading comprehension, language modeling, syntactic analysis, and the like can be trained.

[0113] Exemplarily, the compression processing strategy can compress the model parameters to obtain the mainstream network model.

[0114] S2035. According to the conversion processing strategy, the seed model is converted to obtain multiple mainstream network models.

[0115] For example, taking MobilenetV2 as a seed model, through an enlargement processing strategy or a compression processing strategy, a variety of MobilenetV2 variant models with different FLOPS can be obtained.

[0116] For example, Figure 6 As shown, Figure 6 This is a schematic diagram of a scenario when a model training method provided by an embodiment of the present application is applied to a server. Before executing the model training method, the server can obtain model requirement information from the terminal device, that is, a description of what the user wants to do with the model. The model training method is executed to generate a target model, and the generated model can also be sent to the terminal device so that the terminal device can perform operations such as model testing or deployment.

[0117] Specifically, the client can be an application program (APP). When the user opens the APP, the APP displays a demand information interface so that the user can fill in the demand information on the demand information interface, and obtains the demand information filled in by the user on the demand information interface, and sends the demand information to the server. The server determines the type of the hypernetwork model, the type of the open source network model set or the accuracy and magnitude of the model based on the demand information, and executes the model training method provided in the embodiment of the present application to perform model training, and sends the trained model to the user for user use.

[0118] In some embodiments, the client may also obtain the user's voice. For example, the client may be provided with a voice button to prompt the user to publish demand information through voice, and the user's model demand information may be obtained by recognizing the user's voice.

[0119] S204, taking each of the target sub-network models as a first trunk network and splicing it with the first branch network to obtain a plurality of first spliced ​​networks, and taking each of the mainstream network models as a second trunk network and splicing it with the second branch network to obtain a plurality of second spliced ​​networks.

[0120] The multiple target sub-network models determined by the hypernetwork model are respectively used as the first backbone network and spliced ​​with the first branch network to obtain multiple first spliced ​​networks, and the multiple mainstream network models trained based on open source data are respectively used as the second backbone network and spliced ​​with the second branch network to obtain multiple second spliced ​​networks. The backbone network refers to the sub-network structure in the hypernetwork model and the mainstream network model trained based on open source data, and the branch network is a network connected to the backbone network. The structure of this network can be connected with all the backbone networks, that is, the backbone network and the branch network are connected in series and the branch network is spliced ​​behind the backbone network.

[0121] In some embodiments, the network structure of each first branch network spliced ​​after the first backbone network is the same and the parameters are shared, and the network structure of each second branch network spliced ​​after the second backbone network is the same and the parameters are not shared. The parameters include branch network parameters, and the branch network parameters may include a backbone network (backbone), a head network (head), a neck network (neck), a learning rate, a weight decay (weight decay), etc.

[0122] Exemplarily, the branch network structure and branch network parameters of each first branch network are the same. Since the first trunk networks are extracted from the super network, the parameters of the trunk network structure can be fixed using the same branch network parameters. The branch network structure of each second branch network is the same, but the branch network parameters are different. Since the second trunk network is obtained from open source data, different branch network parameters need to be configured to fix the parameters of the trunk network structure. By sharing the parameters of the first branch network, the network structure search time can be further shortened, thereby improving the efficiency of model training.

[0123] In some embodiments, the network structure of the second branch network can be set to be the same as the network structure of the first branch network, thereby facilitating fine-tuning and testing of the first splicing network and the second splicing network, thereby improving the accuracy of model training.

[0124] S205: fine-tune and test the plurality of the first spliced ​​networks and the second spliced ​​networks to determine a target network model.

[0125] The first spliced ​​network and the second spliced ​​network are fine-tuned and tested respectively, and a target network model is determined according to the test evaluation results, and the target network model is used for transfer learning to obtain a model required by the user. In this way, the network parameters can be dynamically fine-tuned to train a target model that better meets the user's needs.

[0126] Exemplarily, the first splicing network and the second splicing network may be fine-tuned respectively, wherein branch network parameters such as a head network (head) or a neck network (neck) are fine-tuned mainly based on user data.

[0127] Exemplarily, the first concatenated network and the second concatenated network are fine-tuned for N cycles (epochs), wherein fine-tuning one epoch means fine-tuning all the concatenated networks in the training set once, and N can be any number, generally 3 or 5.

[0128] In some embodiments, the fine-tuned first concatenated network and the second concatenated network are both tested and evaluated, and the test and evaluation results are sorted to determine the target network model.

[0129] Exemplarily, the same test evaluation is performed on the first spliced ​​network and the second spliced ​​network, and test evaluation results are obtained. All test evaluation results are sorted, and the best spliced ​​network is selected to determine the target network model.

[0130] For example, if multiple test evaluations are performed, the test score proportions can be first allocated according to the test importance, such as 60% for tests with high importance and 40% for tests with low importance, and finally a comprehensive score is obtained, and the model with the high comprehensive score is selected as the target network model.

[0131] S206: Perform transfer learning on the target network model to obtain the required model.

[0132] Specifically, the target network model is transferred using the scenario data provided by the user, and the model required by the user can be efficiently obtained through transfer learning. The transfer learning is to achieve the learning effect of completing or improving the target field or task by transferring annotated data or knowledge structure from related fields.

[0133] In some embodiments, transfer learning of the target network model can be performed based on a transfer learning algorithm of feature mapping or based on the migration of shared parameters to perform effective weight distribution, so that the instance distribution of the domain of the target network model is close to the instance distribution of the target domain, thereby establishing a reliable learning model with higher classification accuracy in the target domain, thereby obtaining a more suitable model.

[0134] In some embodiments, if the target network model is selected from an open source network model set, then return to S201-S203 of executing the model training method, redetermine the target sub-network model, then splice, fine-tune and test the redetermined target sub-network model with the first branch network to obtain a redetermined network model, compare the test evaluation results of the redetermined network model with the test evaluation results of the target network model, and finally determine the target network model based on the test evaluation results until the source of the determined target network model is a super network model, which is beneficial to improving the applicability and accuracy of the target network model and avoiding missing a better network model due to the randomness of sampling.

[0135] See also Figure 7 , Figure 7 This is a schematic flow chart of another model training method provided in an embodiment of the present application. The model training method can be applied to a server, realizes efficient model training, improves the convenience of users obtaining the required model, reduces the time cost of training the model, and improves the user experience.

[0136] like Figure 7 As shown, the model training method includes steps S301 to S305.

[0137] S301, obtaining a pre-trained super network model, wherein the super network model includes a preset number of sub-network models;

[0138] S302, determining a plurality of target sub-network models from a preset number of sub-network models of the super-network model;

[0139] S303, obtaining multiple mainstream network models trained based on open source data;

[0140] S304, determining a target network model according to the plurality of target sub-network models and the plurality of mainstream network models;

[0141] S305: Perform transfer learning on the target network model to obtain the required model.

[0142] In the embodiments of the present application, multiple target sub-network models and multiple mainstream network models can be directly fine-tuned, and the target network model can be determined without splicing with the branch network.

[0143] Specifically, multiple target sub-network models and multiple mainstream network models trained based on open source data are fine-tuned and tested, and the target network model is determined according to the test evaluation results, which is used to perform transfer learning on the target network model to obtain the required model.

[0144] In some embodiments, multiple fine-tuned target sub-network models and multiple mainstream network models are tested and evaluated, and the test and evaluation results are sorted to determine the target network model.

[0145] Exemplarily, the same test evaluation is performed on the multiple target sub-network models and the multiple mainstream network models, and the test evaluation results are obtained, all the test evaluation results are sorted, and the best network model is selected to determine the target network model.

[0146] See also Figure 8 , Figure 8 4 is a schematic diagram of an electronic device 400 provided in an embodiment of the present application. The electronic device may be a server or a terminal.

[0147] like Figure 8 As shown, the electronic device 400 includes a processor 402 and a memory 401 connected via a system bus, wherein the memory may include a non-volatile storage medium and an internal memory.

[0148] The non-volatile storage medium can store an operating system and a computer program. The computer program includes program instructions, which, when executed, can cause the processor to execute any model training method.

[0149] The processor is used to provide computing and control capabilities to support the operation of the entire electronic device.

[0150] The internal memory provides an environment for the operation of the computer program in the non-volatile storage medium. When the computer program is executed by the processor, the processor can execute any model training method.

[0151] Those skilled in the art will understand that the structure of the electronic device is merely a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the electronic device to which the scheme of the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0152] It should be understood that the processor may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0153] In some embodiments, the processor is used to run a computer program stored in the memory to implement the following steps:

[0154] Obtain a pre-trained super network model, wherein the super network model includes a preset number of sub-network models; determine multiple target sub-network models from the preset number of sub-network models of the super network model; obtain multiple mainstream network models trained based on open source data; splice each of the target sub-network models as a first trunk network with a first branch network to obtain multiple first spliced ​​networks, and splice each of the mainstream network models as a second trunk network with a second branch network to obtain multiple second spliced ​​networks, wherein the network structure of each first branch network spliced ​​after the first trunk network is the same and the parameters are shared, and the network structure of each second branch network spliced ​​after the second trunk network is the same and the parameters are not shared; fine-tune and test the multiple first spliced ​​networks and the second spliced ​​networks to determine the target network model; perform transfer learning on the target network model to obtain the required model.

[0155] In some embodiments, when the processor implements obtaining a pre-trained hypernetwork model, it is specifically used to:

[0156] Obtain an open source data set; obtain a preset hypernetwork, the hypernetwork comprising a first number of channels and a second number of layers; randomly shut down the channels and / or layers of the preset hypernetwork, and use the open source data set to train a batch of data on the remaining network; repeatedly execute the steps of randomly shutting down the channels and / or layers of the preset hypernetwork, and using the open source data set to train a batch of data on the remaining network, until the preset hypernetwork converges, thereby obtaining a pre-trained hypernetwork model.

[0157] In some embodiments, the processor is used to select a sub-network model that satisfies a preset model constraint from a super-network model, specifically:

[0158] A sub-network model is randomly selected from the super-network model; and it is determined whether the computational complexity of the selected sub-network model is less than a preset computational complexity threshold, and whether the model parameter quantity of the selected sub-network model is a preset parameter quantity threshold.

[0159] In some embodiments, the processor is specifically used to:

[0160] The number of target sub-network models to be determined is determined according to the number of mainstream network models; the multiple sub-network models collected are tested according to the test set to obtain the accuracy of the multiple sub-network models; the multiple sub-network models are sorted according to the accuracy of the multiple sub-network models to obtain the sorting results of the multiple sub-network models; and multiple target sub-network models are determined according to the sorting results and the number of target sub-network models.

[0161] In some embodiments, the processor is used to obtain multiple mainstream network models trained based on open source data, specifically for:

[0162] Determine the type of the pre-trained hypernetwork model; select at least one open source network model that matches the type of the pre-trained hypernetwork model from a public open source network model set as a seed model according to the type; obtain user demand information for the model, the demand information includes the accuracy of the model and / or the magnitude of the model; determine a conversion processing strategy for the model according to the user's demand information for the model, the conversion processing strategy includes at least an enlargement processing strategy and a compression processing strategy; convert the seed model according to the conversion processing strategy to obtain multiple mainstream network models.

[0163] An embodiment of the present application also provides a computer-readable storage medium, on which a computer program is stored. The computer program includes program instructions, and when the program instructions are executed, any one of the model training methods provided in the embodiments of the present application is implemented.

[0164] The computer-readable storage medium may be an internal storage unit of the electronic device described in the foregoing embodiment, such as a hard disk or memory of the electronic device. The computer-readable storage medium may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the electronic device.

[0165] Furthermore, the computer-readable storage medium may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function, etc.; the data storage area may store data created according to the use of the blockchain node, etc.

[0166] The application refers to a new application model of computer technologies such as storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. of the blockchain language model. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.

[0167] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A model training method, applied to image processing, text processing or video processing, It is characterized in that The method comprises: Acquire a pre-trained hypernetwork model, wherein the hypernetwork model includes a preset number of subnetwork models; Determining a plurality of target sub-network models from a preset number of sub-network models of the super-network model; Obtain multiple mainstream network models trained based on open source data; Each of the target sub-network models is used as a first trunk network, and is spliced ​​with the first branch network to obtain a plurality of first spliced ​​networks; and each of the mainstream network models is used as a second trunk network, and is spliced ​​with the second branch network to obtain a plurality of second spliced ​​networks; Fine-tune and test a plurality of the first spliced ​​networks and the second spliced ​​networks to determine a target network model; Perform transfer learning on the target network model to obtain the required model.

2. The method according to claim 1, It is characterized in that The method further comprises: Access open source datasets; Obtaining a preset hypernetwork, wherein the hypernetwork includes a first number of channels and a second number of layers; Randomly shut down channels and / or layers of the hypernetwork, and train a batch of data on the remaining network using the open source dataset; Repeat the steps of randomly shutting down the channels and / or layers of the preset hypernetwork and training a batch of data on the remaining network using the open source data set until the hypernetwork converges to obtain a pre-trained hypernetwork model.

3. The method according to claim 1, It is characterized in that The step of determining a plurality of target sub-network models from a preset number of sub-network models of the super-network model comprises: Based on a preset sampling algorithm, sub-network models that meet preset model constraints are selected from the super-network model until the number of collected sub-network models meets a preset value; The collected multiple sub-network models are tested and evaluated to determine multiple target sub-network models.

4. The method according to claim 3, It is characterized in that The method of selecting a sub-network model that satisfies a preset model constraint condition from the super-network model based on a preset sampling algorithm comprises: Randomly selecting a sub-network model from the super-network model; Determine whether the computation amount of the sub-network model is less than a preset computation amount threshold, and whether the model parameter amount of the sub-network model is less than a preset parameter amount threshold; If the computation amount of the sub-network model is less than the preset computation amount threshold, and the model parameter amount of the sub-network model is less than the preset parameter amount threshold, the sub-network model is selected.

5. The method according to claim 3, It is characterized in that The testing and evaluating of the collected multiple sub-network models to determine multiple target sub-network models includes: Determine the number of target sub-network models that need to be determined based on the number of mainstream network models; Testing the collected multiple sub-network models according to the test set to obtain the accuracy of the multiple sub-network models; Sorting the multiple sub-network models according to their accuracy rates to obtain a sorting result of the multiple sub-network models; and According to the sorting result and the number of the target sub-network models, a plurality of target sub-network models are determined.

6. The method according to claim 1, It is characterized in that The method of obtaining multiple mainstream network models trained based on open source data includes: Determining the type of the pre-trained hypernetwork model; According to the type, selecting at least one open source network model that matches the type of the pre-trained super network model from a public open source network model set as a seed model; Acquiring user requirement information for the model, wherein the requirement information includes accuracy of the model and / or magnitude of the model; Determining a conversion processing strategy for the model according to the user's demand information for the model; According to the conversion processing strategy, the seed model is converted to obtain multiple mainstream network models.

7. The method according to claim 1, It is characterized in that The transfer learning of the target network model to obtain the required model includes: Obtaining the backbone network of the target network model; Determine the corresponding branch network after fine-tuning and testing according to the target network model; The backbone network and the branch network are spliced ​​together, and transfer learning training is performed on the scene data provided by the user to obtain the required model.

8. The method according to any one of claims 1 to 7, It is characterized in that The network structure of the second branch network is the same as the network structure of the first branch network.

9. The method according to any one of claims 1 to 7, It is characterized in that The model complexity of the multiple mainstream network models is different, wherein the model complexity includes at least one of the model calculation amount and the model parameter amount.

10. The method according to any one of claims 1 to 7, It is characterized in that The first branch networks spliced ​​after each of the first trunk networks have the same network structure and share parameters, and the second branch networks spliced ​​after each of the second trunk networks have the same network structure and do not share parameters.

11. An electronic device, It is characterized in that The electronic device comprises a memory and a processor; The memory is used to store computer programs; The processor is configured to execute the computer program and implement, when executing the computer program: A model training method as described in any one of claims 1 to 10.

12. A computer-readable storage medium, It is characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor enables the processor to implement the model training method as described in any one of claims 1-10.

Images