Network model training method and device

Abstract

The present application discloses a network model training method and device thereof, belonging to the field of model training technology. The network model training method comprises: obtaining a first representation network and a second representation network, wherein the second representation network is an augmented network of the first representation network; inputting image data into the first representation network and the second representation network respectively for feature extraction to obtain first feature data and second feature data, wherein, during the feature extraction process, the data stream of the second representation network includes the data stream of the first representation network; and performing supervised training on the first representation network according to the first feature data and the second feature data to obtain a target model.

Description

Technical Field

[0001] The present application belongs to the field of model training technology, and specifically relates to a network model training method and device thereof. Background Art

[0002] Face recognition technology has been widely used in daily life. Face recognition models deployed on mobile smart devices must ensure both model accuracy and model lightweight. In related technologies, a model distillation training method is used to train face recognition models. A lightweight network structure model and a large network structure model that meet performance requirements are designed. First, the large network structure model is trained to achieve the optimal performance. Then, the lightweight network structure model is trained to achieve the optimal performance by combining the large network structure model with supervision. The trained lightweight network structure model is deployed to mobile smart devices.

[0003] In the model training process of related technologies, two models need to be trained separately and then jointly supervised, resulting in low training model efficiency. How to simplify the model training process and improve the model training efficiency has become an urgent problem to be solved. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a network model training method and device thereof, which realizes the training of two network models at the same time, simplifies the model training process, and improves the efficiency of training models.

[0005] In a first aspect, an embodiment of the present application provides a network model training method, comprising: obtaining a first representation network and a second representation network, wherein the second representation network is an augmented network of the first representation network; inputting image data into the first representation network and the second representation network respectively for feature extraction to obtain first feature data and second feature data, wherein, during the feature extraction process, the data stream of the second representation network includes the data stream of the first representation network; and performing supervised training on the first representation network based on the first feature data and the second feature data to obtain a target model.

[0006] In a second aspect, an embodiment of the present application provides a network model training device, comprising: an acquisition module, used to acquire a first representation network and a second representation network, wherein the second representation network is an augmented network of the first representation network; an extraction module, used to input image data into the first representation network and the second representation network respectively for feature extraction to obtain first feature data and second feature data, wherein, during the feature extraction process, the data stream of the second representation network includes the data stream of the first representation network; a training module, used to perform supervised training on the first representation network according to the first feature data and the second feature data to obtain a target model.

[0007] In a third aspect, an embodiment of the present application provides an electronic device, including a processor and a memory, wherein the memory can be a program or instruction running on the processor, and when the program or instruction is executed by the processor, the steps of the method of the first aspect are implemented.

[0008] In a fourth aspect, an embodiment of the present application provides a readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the method of the first aspect are implemented.

[0009] In a fifth aspect, an embodiment of the present application provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the steps of the method of the first aspect.

[0010] In a sixth aspect, an embodiment of the present application provides a computer program product, which is stored in a storage medium and is executed by at least one processor to implement the method of the first aspect.

[0011] In the embodiment of the present application, a first representation network and a second representation network which is an augmented network of the first representation network are selected. During the training process, the first representation network and the second representation network are trained simultaneously, which simplifies the model training process, improves the training efficiency of the first representation network and the second representation network, and performs additional supervision through the second representation network to assist in training the first representation network, thereby avoiding the first representation network from falling into a local optimum too early, thereby improving the accuracy of the first representation network. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 A schematic diagram of the process of the network model training method provided in the embodiment of the present application is shown;

[0013] Figure 2 One of the schematic diagrams showing an embodiment of the present application provides a first characterization network and a second characterization network for feature extraction;

[0014] Figure 3 A second schematic diagram showing a first characterization network and a second characterization network for feature extraction according to an embodiment of the present application is shown;

[0015] Figure 4 A third schematic diagram showing a first characterization network and a second characterization network for feature extraction according to an embodiment of the present application is shown;

[0016] Figure 5 A structural block diagram of a network model training device provided in an embodiment of the present application is shown;

[0017] Figure 6 A structural block diagram of an electronic device provided in an embodiment of the present application is shown;

[0018] Figure 7 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0019] The following will be combined with the drawings in the embodiments of the present application to clearly describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application belong to the scope of protection of this application.

[0020] The terms "first", "second", etc. in the specification and claims of the present application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of one type, and the number of objects is not limited. For example, the first object can be one or more. In addition, "and / or" in the specification and claims represents at least one of the connected objects, and the character " / " generally indicates that the objects associated with each other are in an "or" relationship.

[0021] The following is combined with Figures 1 to 7 , the network model training method, network model training device, electronic device and storage medium provided in the embodiments of the present application are described in detail through specific embodiments and their application scenarios.

[0022] In an embodiment of the present application, a network model training method is provided. Figure 1 A schematic diagram of the process of the network model training method provided in the embodiment of the present application is shown, Figure 1 As shown, the network model training method includes:

[0023] Step 102, obtaining a first representation network and a second representation network;

[0024] Wherein, the second representation network is an augmented network of the first representation network;

[0025] It should be noted that the first representation network is a subset of the second representation network. In the subsequent training process, the first representation network is supervised and assisted by the second representation network, which can prevent the first representation network from falling into the local optimum too early, thereby improving the accuracy of the first representation network. Optionally, the first representation network is a student network and the second representation network is a teacher network.

[0026] It should be noted that there are two different ways to enhance the first characterization network. One is the "widening" network enhancement method, that is, increasing the number of channels of each layer of the network in the first characterization network. The other is the "deepening" network enhancement method, that is, increasing the number of network layers in the first characterization network. The embodiment of the present application can obtain two second characterization networks with different structures through different network enhancement methods. This embodiment does not limit the network enhancement method.

[0027] Step 104, inputting the image data into the first representation network and the second representation network respectively for feature extraction to obtain first feature data and second feature data;

[0028] In the process of training the network model, a large amount of image data is required to train the network model. The image data can be optionally input into the first representation network and the second representation network in the form of a data set, and the data set includes a plurality of image data. In the process of feature extraction, the data stream of the second representation network includes the data stream of the first representation network;

[0029] It should be noted that network enhancement includes two different ways of network enhancement: "widening" or "deepening" the first representation network, and different forms of network enhancement correspond to different feature extraction processes.

[0030] Optionally, the first characterization network is enhanced by "widening" to obtain a second characterization network. In the process of feature extraction, there are downsampling and upsampling processes. After the image data is input into the first characterization network and the second characterization network, the first characterization network and the second characterization network respectively perform feature extraction on the image data, and the first characterization network and the second characterization network extract first extraction results and second extraction results. The first characterization network continues to perform feature extraction on the first extraction result, and the second characterization network performs feature extraction on the first extraction result and the second extraction result, that is, in the subsequent feature extraction process, the second characterization network uses the first extraction result obtained by the previous feature extraction of the first characterization network and the second extraction result obtained by the previous feature extraction of the second characterization network as data for extraction, so that the data flow of the second characterization network during the feature extraction process includes the data flow of the first characterization network during the feature extraction process.

[0031] Optionally, the first representation network is enhanced by "deepening" to obtain a second representation network. In the feature extraction process, the first representation network sequentially downsamples and upsamples the image data, and the output sampling results are used as the first feature data. The above sampling results are input into the second representation network for upsampling, and the output sampling results are used as the second feature data.

[0032] It should be noted that the image data is input into the first representation network and the second representation network after preprocessing. Exemplarily, the image data in the image data is cropped for portrait detection to obtain a portrait image. Common training data enhancements such as random noise, blur, rotation, occlusion, etc. are then added to the portrait image, and then uniformly scaled to a fixed size to obtain preprocessed image data.

[0033] Optionally, in the embodiment of the present application, the size of the image data is selected to be between 64 pixels and 256 pixels. The more pixels in the image data, the more detailed features are provided, and the higher the accuracy of the first feature data and the second feature data extracted by the first characterization network and the second characterization network. Exemplarily, the image data is selected to be image data of 112×112 pixels.

[0034] In an embodiment of the present application, the first feature data extracted by the first representation network and the second feature data extracted by the second representation network may be selected as image data, the first representation network outputs a first feature map, and the second representation network outputs a second feature map.

[0035] Step 106: Perform supervised training on the first representation network based on the first feature data and the second feature data to obtain a target model.

[0036] After the feature extraction is completed, the first representation network is supervised and trained by extracting the first feature data and the second feature data, so as to obtain a target model, that is, the target model is a model obtained by training the first representation network. The target model is a lightweight network structure, so that the target model is adapted to the mobile terminal device.

[0037] In the embodiment of the present application, during the feature extraction process of the first representation network and the second representation network, the data stream of the second representation network includes the data stream of the first representation network, so that the first representation network is enhanced by two different methods of "deepening" or "widening", so that the first representation network in the training process is a subset of the second representation network, and the parameters of the first representation network and the second representation network are shared. The second representation network is used to supervise and assist the training of the first representation network. The first representation network and the second representation network are trained simultaneously.

[0038] It should be noted that in the embodiments of the present application, the image data are all selected as image data including portraits, and the target model obtained by training is a portrait recognition model.

[0039] In the embodiment of the present application, a first representation network and a second representation network which is an augmented network of the first representation network are selected. During the training process, the first representation network and the second representation network are trained simultaneously, which simplifies the model training process, improves the training efficiency of the first representation network and the second representation network, and performs additional supervision through the second representation network to assist in training the first representation network, thereby avoiding the first representation network from falling into a local optimum too early, thereby improving the accuracy of the first representation network.

[0040] In some embodiments of the present application, image data is input into a first characterization network and a second characterization network for feature extraction to obtain first feature data and second feature data, including: downsampling the image data through the first characterization network and the second characterization network to obtain a first downsampling result and a second downsampling result; inputting the first downsampling result into the first characterization network for upsampling processing to obtain a first upsampling result; inputting the first downsampling result and the second downsampling result into the second characterization network for upsampling processing to obtain a second upsampling result; determining the first feature data according to the first upsampling result, and determining the second feature data according to the first upsampling result and the second upsampling result.

[0041] In an embodiment of the present application, when the first representation network is trained using the "widening" network enhancement method, the first representation network and the second representation network simultaneously downsample and upsample the image data in sequence, and during the upsampling and downsampling processes, the data stream of the second representation network includes the data stream of the first representation network.

[0042] Specifically, the first representation network and the second representation network downsample the image data. After the downsampling is completed, the first representation network outputs the first downsampling result, and the second representation network outputs the second downsampling result. In the upsampling process, the downsampling result is upsampled, wherein the first representation network only upsamples the first downsampling result to obtain the first upsampling result. In order to perform "widening" network enhancement, the first downsampling result and the second downsampling result are both input into the second representation network for upsampling processing to obtain the second upsampling result. After the upsampling is completed, the first representation network outputs the first feature data according to the first upsampling result, and the second representation network outputs the second feature data according to the second sampling result.

[0043] Figure 2 One of the schematic diagrams of an embodiment of the present application providing a first characterization network and a second characterization network for feature extraction is shown.

[0044] like Figure 2As shown, optionally, 112×112 pixel image data is input, the first representation network downsamples the image data to obtain 14×14 pixel image data, and the second representation network downsamples the image data to obtain 14×14 pixel image data. The first representation network upsamples the image data downsampled by the first representation network to obtain 28×28 pixel image data. The second representation network combines the image data downsampled by the first representation network and the image data downsampled by the second representation network and upsamples them to obtain 28×28 pixel image data. The first representation network outputs first feature data based on the image data output after upsampling on the first representation network, and the second representation network outputs second feature data based on the image data output after upsampling on the first representation network and the image data output after upsampling on the second representation network.

[0045] In an embodiment of the present application, when the second representation network is a second representation network obtained by the first representation network through a "widening" network enhancement method, when the first representation network and the second representation network perform feature extraction, the data stream of each sampling of the second representation network includes the previous sampling results of the first representation network and the second representation network, thereby improving the accuracy of feature extraction. Since the second representation network has a wider structure and a stronger network expression capability, the first representation network is prevented from entering the local optimum too early, thereby improving the accuracy of the trained target model for portrait recognition.

[0046] In some embodiments of the present application, the number of downsampling processes is multiple times; after downsampling the image data through the first representation network and the second representation network to obtain the first downsampling result and the second downsampling result, it also includes: downsampling the first downsampling result obtained by the previous downsampling process through the first representation network, and updating the first downsampling result; downsampling the first downsampling result and the second downsampling result obtained by the previous downsampling process through the second representation network, and updating the second downsampling result.

[0047] In an embodiment of the present application, the number of downsampling processes is set to multiple times. When the first representation network performs multiple downsampling processes, only the previous downsampling result of the first representation network is downsampled. When the second representation network performs multiple downsampling processes, it is necessary to downsample the previous downsampling results of the first representation network and the second representation network.

[0048] Specifically, both the first representation network and the second representation network perform multiple downsampling processes on the image data, and the first representation network and the second representation network perform the downsampling process the same number of times, and the first representation network and the second representation network perform the downsampling process simultaneously. In the process of the first representation network performing multiple downsampling processes on the image data, the first representation network performs downsampling processes again on the first downsampling result obtained from its previous downsampling process, and updates the first downsampling result. In the process of the second representation network performing multiple downsampling processes on the image data, the second representation network performs downsampling processes again after merging the second downsampling result obtained from its previous downsampling process and the first downsampling result obtained from the previous downsampling process of the first representation network, thereby updating the first downsampling result.

[0049] It should be noted that the number of downsampling processes is related to the size of the image data input to the first representation network and the second representation, and the size of the feature maps corresponding to the first feature data and the second feature data required to be output by the first representation network and the second representation network.

[0050] In the embodiment of the present application, by setting the number of times the first representation network and the second representation network perform downsampling to multiple times, the image data is made to be larger in size. Since the image data input to the first representation network and the second representation network has a larger size, the image data provides more detailed features, thereby further improving the detection accuracy of the target model.

[0051] In some embodiments of the present application, the number of downsampling processes is multiple times; before determining the first feature data based on the first upsampling result, and determining the second feature data based on the second upsampling result, it also includes: upsampling the first upsampling result obtained by the previous upsampling process through the first characterization network, and updating the first upsampling result; upsampling the first upsampling result and the second upsampling result obtained by the previous upsampling process through the second characterization network, and updating the second upsampling result.

[0052] In an embodiment of the present application, the number of upsampling processes is set to multiple times. When the first representation network performs multiple upsampling processes, only the previous upsampling result of the first representation network is upsampled. When the second representation network performs multiple upsampling processes, it is necessary to upsample the previous upsampling results of the first representation network and the second representation network.

[0053] Specifically, both the first representation network and the second representation network perform multiple upsampling processes on the image data, and the first representation network and the second representation network perform the upsampling process the same number of times, and the first representation network and the second representation network perform upsampling processes simultaneously. In the process of the first representation network performing multiple upsampling processes on the image data, the first representation network performs upsampling processes again on the first upsampling result obtained by its previous upsampling process, thereby updating the first upsampling result. In the process of the second representation network performing multiple upsampling processes on the image data, the second representation network combines the second upsampling result obtained by its previous upsampling process and the first upsampling result obtained by the previous upsampling process of the first representation network and performs upsampling processes again, thereby updating the first upsampling result.

[0054] It should be noted that the number of upsampling processes is related to the size of the image data input to the first representation network and the second representation, and the size of the feature maps corresponding to the first feature data and the second feature data required to be output by the first representation network and the second representation network.

[0055] In the embodiment of the present application, by setting the number of times the first characterization network and the second characterization network are upsampled to multiple times, the size of the feature graph corresponding to the output first feature data and the second feature data is made larger. Since the larger the size of the output feature graph, the smaller the lower bound of the error of the target model, the higher the detection accuracy of the target model is further improved.

[0056] It should be noted that the number of upsampling and downsampling performed by the first representation network and the second representation network is related to the size of the image data input to the first representation network and the second representation network, and is related to the size of the feature map corresponding to the output first feature data and the second feature data.

[0057] Figure 3 A second schematic diagram showing an embodiment of the present application providing a first characterization network and a second characterization network for feature extraction.

[0058] like Figure 3 As shown, optionally, the size of the image data input to the first representation network and the second representation network is 112×112 pixels, and the size of the feature map corresponding to the first feature data and the second feature data output by the first representation network and the second representation network is 28×28 pixels. The number of downsampling processes is designed to be 4 times, and the sizes of the image data obtained by the 4 downsampling processes are 56×56 pixels, 28×28 pixels, 14×14 pixels, and 7×7 pixels, respectively. The number of upsampling processes is designed to be 2 times, and the sizes of the image data obtained by the 2 upsampling processes are 14×14 pixels and 28×28 pixels, respectively.

[0059] It should be noted that the second representation network can be selected not only as the representation network obtained by enhancing the first representation network by "widening" the network, but also as a redesigned second representation network, the redesigned second representation network structure is wider than the first representation network, that is, the second representation network has stronger expression ability.

[0060] In some embodiments of the present application, image data is input into a first representation network and a second representation network for feature extraction to obtain first feature data and second feature data, including: downsampling the image data through the first representation network to obtain a third downsampling result; inputting the third downsampling result into the first representation network for upsampling to obtain a third upsampling result; inputting the third upsampling result into the second representation network for upsampling to obtain a fourth upsampling result; determining the first feature data according to the third upsampling result; and determining the second feature data according to the fourth upsampling result.

[0061] In the embodiment of the present application, when the first representation network is trained by the "speeding up" network enhancement method, the image data is sequentially downsampled and upsampled by the first representation network. The data stream of the second representation network includes the data stream of the first representation network, that is, the second representation network performs upsampling again on the upsampling result output by the first representation network.

[0062] Specifically, the first representation network downsamples the image data, obtains a third downsampled result after the downsample is completed, the first representation network upsamples the third downsampled result to obtain a third upsampled result, and the first representation network outputs the first feature data according to the third upsampled result. The third upsampled result obtained by upsampling on the first representation network is transmitted to the second representation network, and the second representation network upsamples the third upsampled result again to obtain a fourth upsampled result. The second representation network outputs the second feature data according to the fourth upsampled result.

[0063] Figure 4 A third schematic diagram showing an embodiment of the present application providing a first characterization network and a second characterization network for feature extraction.

[0064] like Figure 4As shown, optionally, in an embodiment of the present application, image data of 112×112 pixels is input, and the first representation network downsamples the image data to obtain image data of 14×14 pixels. The first representation network upsamples the image data downsampled by the first representation network to obtain image data of 28×28 pixels. The first representation network outputs first feature data based on the image data of 28×28 pixels obtained by the upsampling process. The second representation network upsamples the image data of 28×28 pixels upsampled by the first representation network to obtain image data of 56×56 pixels, and the second representation network outputs second feature data based on the image data of 56×56 pixels obtained by the upsampling process.

[0065] In an embodiment of the present application, when the second representation network is a second representation network obtained by the first representation network through a "deepening" network enhancement method, when the first representation network and the second representation network perform feature extraction, the second representation network upsamples the image data obtained after downsampling and upsampling processing by the first representation network, so that the size of the feature map corresponding to the second feature data output by the second representation network is larger than the size of the feature map corresponding to the first feature data output by the first representation network. Therefore, the network expression ability of the second representation network is stronger, which avoids the first representation network from entering the local optimum too early, and improves the accuracy of the target model obtained by training for portrait recognition.

[0066] It should be noted that the first characterization network and the second characterization network need to output a plurality of first feature data and a plurality of second feature data. Exemplarily, the first characterization network and the second characterization network output 103 first feature data and 103 second feature data, respectively.

[0067] In some embodiments of the present application, supervised training is performed on the first representation network based on the first feature data and the second feature data to obtain a target model, including: determining a first heat map and a second heat map based on the first feature data and the second feature data; determining a first coordinate value of the first heat map, and a second coordinate value of the second heat map; training a preset loss function through the first coordinate value, the second coordinate value and the preset coordinate value to obtain a target model, and the preset loss function is associated with the first representation network.

[0068] In an embodiment of the present application, the first feature data and the second feature data include corresponding feature maps, that is, the first feature data include the first feature map output by the first characterization network, and the second feature data include the second feature map output by the second characterization network. According to the first feature map and the second feature map, the corresponding first thermodynamic map and the second thermodynamic map can be calculated, and the coordinate values ​​of the first thermodynamic map and the second thermodynamic map are calculated. According to the first coordinate value and the second coordinate value, and the preset coordinate value, the preset loss function related to the first characterization network is trained, thereby realizing supervised training of the first characterization network.

[0069] Optionally, the preset loss function is as follows:

[0070]

[0071] Wherein, N is the number of the first feature data and the second feature data, y_teacher is the second coordinate value, y_student is the first coordinate value, y_gt is the preset coordinate value, and α is a constant.

[0072] It should be noted that N can be selected as 103, and α is used to balance the teacher network supervision loss and the student network supervision loss. In the first half of the supervised training process, α=1 is set, and in the second half, α=2 is set.

[0073] In the embodiment of the present application, the second representation network is obtained by network augmentation of the first representation network, so the first representation network is part of the second representation network, and the weights are shared. During the model training process, the gradient flow of the second representation network parameters passes through the first representation network. Since the second representation network has a wider structure and stronger network expression ability, it can prevent the first representation network from entering the local optimum earlier. The second representation network and the first representation network are trained at the same time to improve the training efficiency, and the expression ability of the second representation network is also used to supervise the training of the first representation network.

[0074] The network model training method provided in the embodiment of the present application can be executed by a network model training device or a control module in the network model training device for executing the network model training method. In the embodiment of the present application, the method of executing the network model training by the network model training device is taken as an example to illustrate the network model training device provided in the embodiment of the present application.

[0075] In some embodiments of the present application, a network model training device is provided. Figure 5 The structural block diagram of the network model training device provided in the embodiment of the present application is shown as follows: Figure 5 As shown, the network model training device 500 includes:

[0076] An acquisition module 502 is used to acquire a first representation network and a second representation network, wherein the second representation network is an augmented network of the first representation network;

[0077] An extraction module 504 is used to input the image data into the first representation network and the second representation network respectively for feature extraction to obtain first feature data and second feature data, wherein during the feature extraction process, the data stream of the second representation network includes the data stream of the first representation network;

[0078] The training module 506 is used to perform supervised training on the first representation network according to the first feature data and the second feature data to obtain a target model.

[0079] In the embodiment of the present application, a first representation network and a second representation network which is an augmented network of the first representation network are selected. During the training process, the first representation network and the second representation network are trained simultaneously, which simplifies the model training process, improves the training efficiency of the first representation network and the second representation network, and performs additional supervision through the second representation network to assist in training the first representation network, thereby avoiding the first representation network from falling into a local optimum too early, thereby improving the accuracy of the first representation network.

[0080] In some embodiments of the present application, the network model training device 500 further includes:

[0081] A downsampling module, used to perform downsampling processing on the image data through the first representation network and the second representation network to obtain a first downsampling result and a second downsampling result;

[0082] An upsampling module, used for inputting the first downsampling result into the first representation network for upsampling processing to obtain a first upsampling result;

[0083] An upsampling module, used for inputting the first downsampling result and the second downsampling result into the second representation network for upsampling processing to obtain a second upsampling result;

[0084] The first determining module is used to determine the first characteristic data according to the first up-sampling result, and to determine the second characteristic data according to the first up-sampling result and the second up-sampling result.

[0085] In an embodiment of the present application, when the second representation network is a second representation network obtained by the first representation network through a "widening" network enhancement method, when the first representation network and the second representation network perform feature extraction, the data stream of each sampling of the second representation network includes the previous sampling results of the first representation network and the second representation network, thereby improving the accuracy of feature extraction. Since the second representation network has a wider structure and a stronger network expression capability, the first representation network is prevented from entering the local optimum too early, thereby improving the accuracy of the trained target model for portrait recognition.

[0086] In some embodiments of the present application, the number of downsampling processes is multiple times;

[0087] A downsampling module, configured to perform downsampling processing on a first downsampling result obtained by a previous downsampling processing through a first representation network, and update the first downsampling result;

[0088] The downsampling module is used to downsample the first downsampling result and the second downsampling result obtained by the previous downsampling process through the second representation network, and update the second downsampling result.

[0089] In the embodiment of the present application, by setting the number of times the first representation network and the second representation network perform downsampling to multiple times, the image data is made to be larger in size. Since the image data input to the first representation network and the second representation network has a larger size, the image data provides more detailed features, thereby further improving the detection accuracy of the target model.

[0090] In some embodiments of the present application, the number of downsampling processes is multiple times;

[0091] An upsampling module, configured to perform upsampling processing on a first upsampling result obtained by performing upsampling processing on the previous time through a first representation network, and update the first upsampling result;

[0092] The upsampling module is used to perform upsampling processing on the first upsampling result and the second upsampling result obtained by the previous upsampling processing through the second representation network, and update the second upsampling result.

[0093] In the embodiment of the present application, by setting the number of times the first characterization network and the second characterization network are upsampled to multiple times, the size of the feature graph corresponding to the output first feature data and the second feature data is made larger. Since the larger the size of the output feature graph, the smaller the lower bound of the error of the target model, the higher the detection accuracy of the target model is further improved.

[0094] In some embodiments of the present application, the downsampling module is used to perform downsampling processing on the image data through the first representation network to obtain a third downsampling result;

[0095] An upsampling module, used for inputting the third downsampling result into the first representation network for upsampling processing to obtain a third upsampling result;

[0096] An upsampling module, used for inputting the third upsampling result into the second representation network for upsampling processing to obtain a fourth upsampling result;

[0097] The network model training device 500 further includes:

[0098] A second determining module, used to determine the first feature data according to the third upsampling result;

[0099] The first determining module is used to determine the second feature data according to the fourth up-sampling result.

[0100] In an embodiment of the present application, when the second representation network is a second representation network obtained by the first representation network through a "deepening" network enhancement method, when the first representation network and the second representation network perform feature extraction, the second representation network upsamples the image data obtained after downsampling and upsampling processing by the first representation network, so that the size of the feature map corresponding to the second feature data output by the second representation network is larger than the size of the feature map corresponding to the first feature data output by the first representation network. Therefore, the network expression ability of the second representation network is stronger, which avoids the first representation network from entering the local optimum too early, and improves the accuracy of the target model obtained by training for portrait recognition.

[0101] In some embodiments of the present application, the network model training device 500 further includes:

[0102] A third determination module, configured to determine a first thermal map and a second thermal map according to the first characteristic data and the second characteristic data;

[0103] A fourth determination module, used to determine a first coordinate value of the first thermal map and a second coordinate value of the second thermal map;

[0104] The training module 506 is further used to train the preset loss function through the first coordinate value, the second coordinate value and the preset coordinate value to obtain the target model, and the preset loss function is associated with the first representation network.

[0105] In the embodiment of the present application, the second representation network is obtained by network augmentation of the first representation network, so the first representation network is part of the second representation network, and the weights are shared. During the model training process, the gradient flow of the second representation network parameters passes through the first representation network. Since the second representation network has a wider structure and stronger network expression ability, it can prevent the first representation network from entering the local optimum earlier. The second representation network and the first representation network are trained at the same time to improve the training efficiency, and the expression ability of the second representation network is also used to supervise the training of the first representation network.

[0106] The network model training device of the electronic device in the embodiment of the present application can be an electronic device, or a component in the electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal, or it can be other devices other than a terminal. Exemplarily, the electronic device can be a mobile phone, a tablet computer, a laptop computer, a PDA, a car electronic device, a mobile Internet device (Mobile Internet Device, MID), an augmented reality (augmented reality, AR) / virtual reality (virtual reality, VR) device, a robot, a wearable device, an ultra-mobile personal computer (ultra-mobile personal computer, UMPC), a netbook or a personal digital assistant (personal digital assistant, PDA), etc., and can also be a server, a network attached storage (Network Attached Storage, NAS), a personal computer (personal computer, PC), a television (television, TV), a teller machine or a self-service machine, etc., which is not specifically limited in the embodiment of the present application.

[0107] The network model training device of the electronic device in the embodiment of the present application may be a device having an operating system. The operating system may be an Android operating system, an iOS operating system, or other possible operating systems, which are not specifically limited in the embodiment of the present application.

[0108] The network model training device for an electronic device provided in the embodiment of the present application can implement each process implemented in the above method embodiment, and will not be described again here to avoid repetition.

[0109] Optionally, an embodiment of the present application also provides an electronic device, which includes a network model training device as in any of the above embodiments, and thus has all the beneficial effects of the network model training device in any of the embodiments, which will not be elaborated herein.

[0110] Optionally, an embodiment of the present application further provides an electronic device, Figure 6 A structural block diagram of an electronic device according to an embodiment of the present application is shown. Figure 6 As shown, the electronic device 600 includes a processor 602, a memory 604, and a program or instruction stored in the memory 604 and executable on the processor 602. When the program or instruction is executed by the processor 602, each process of the above-mentioned network model training method embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.

[0111] It should be noted that the electronic devices in the embodiments of the present application include the above-mentioned mobile electronic devices and non-mobile electronic devices.

[0112] Figure 7 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of the present application.

[0113] The electronic device 700 includes but is not limited to: a radio frequency unit 701, a network module 702, an audio output unit 703, an input unit 704, a sensor 705, a display unit 706, a user input unit 707, an interface unit 708, a memory 709, a processor 710 and other components.

[0114] Those skilled in the art will appreciate that the electronic device 700 may also include a power source (such as a battery) for supplying power to each component, and the power source may be logically connected to the processor 710 through a power management system, thereby implementing functions such as managing charging, discharging, and power consumption management through the power management system. Figure 7 The electronic device structure shown in the figure does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently, which will not be described in detail here.

[0115] The processor 710 is configured to obtain a first representation network and a second representation network, wherein the second representation network is an augmented network of the first representation network;

[0116] The processor 710 is used to input the image data into the first representation network and the second representation network respectively for feature extraction to obtain first feature data and second feature data, wherein in the feature extraction process, the data stream of the second representation network includes the data stream of the first representation network;

[0117] Processor 710 is used to perform supervised training on the first representation network according to the first feature data and the second feature data to obtain a target model.

[0118] In the embodiment of the present application, a first representation network and a second representation network which is an augmented network of the first representation network are selected. During the training process, the first representation network and the second representation network are trained simultaneously, which simplifies the model training process, improves the training efficiency of the first representation network and the second representation network, and performs additional supervision through the second representation network to assist in training the first representation network, thereby avoiding the first representation network from falling into a local optimum too early, thereby improving the accuracy of the first representation network.

[0119] Further, the processor 710 is used to perform downsampling processing on the image data through the first representation network and the second representation network to obtain a first downsampling result and a second downsampling result;

[0120] The processor 710 is configured to input the first down-sampling result into the first representation network for up-sampling processing to obtain a first up-sampling result;

[0121] The processor 710 is configured to input the first down-sampling result and the second down-sampling result into a second representation network for up-sampling processing to obtain a second up-sampling result;

[0122] The processor 710 is configured to determine first feature data according to the first up-sampling result, and determine second feature data according to the first up-sampling result and the second up-sampling result.

[0123] In an embodiment of the present application, when the second representation network is a second representation network obtained by the first representation network through a "widening" network enhancement method, when the first representation network and the second representation network perform feature extraction, the data stream of each sampling of the second representation network includes the previous sampling results of the first representation network and the second representation network, thereby improving the accuracy of feature extraction. Since the second representation network has a wider structure and a stronger network expression capability, the first representation network is prevented from entering the local optimum too early, thereby improving the accuracy of the trained target model for portrait recognition.

[0124] Furthermore, the number of downsampling processes is multiple times;

[0125] The processor 710 is configured to perform downsampling processing on a first downsampling result obtained by performing downsampling processing last time through a first representation network, and update the first downsampling result;

[0126] The processor 710 is configured to perform downsampling processing on the first downsampling result and the second downsampling result obtained by the previous downsampling processing through the second representation network, and update the second downsampling result.

[0127] In the embodiment of the present application, by setting the number of times the first representation network and the second representation network perform downsampling to multiple times, the image data is made to be larger in size. Since the image data input to the first representation network and the second representation network has a larger size, the image data provides more detailed features, thereby further improving the detection accuracy of the target model.

[0128] Furthermore, the number of downsampling processes is multiple times;

[0129] The processor 710 is configured to perform upsampling processing on a first upsampling result obtained by performing upsampling processing last time through a first representation network, and update the first upsampling result;

[0130] The processor 710 is configured to perform upsampling processing on the first upsampling result and the second upsampling result obtained by the previous upsampling processing through the second representation network, and update the second upsampling result.

[0131] In the embodiment of the present application, by setting the number of times the first characterization network and the second characterization network are upsampled to multiple times, the size of the feature graph corresponding to the output first feature data and the second feature data is made larger. Since the larger the size of the output feature graph, the smaller the lower bound of the error of the target model, the higher the detection accuracy of the target model is further improved.

[0132] Further, the processor 710 is used to perform downsampling processing on the image data through the first representation network to obtain a third downsampling result;

[0133] The processor 710 is configured to input the third down-sampling result into the first representation network for up-sampling processing to obtain a third up-sampling result;

[0134] The processor 710 is configured to input the third upsampling result into the second representation network for upsampling processing to obtain a fourth upsampling result;

[0135] The processor 710 is configured to determine first feature data according to the third upsampling result;

[0136] The processor 710 is configured to determine second feature data according to the fourth up-sampling result.

[0137] In an embodiment of the present application, when the second representation network is a second representation network obtained by the first representation network through a "deepening" network enhancement method, when the first representation network and the second representation network perform feature extraction, the second representation network upsamples the image data obtained after downsampling and upsampling processing by the first representation network, so that the size of the feature map corresponding to the second feature data output by the second representation network is larger than the size of the feature map corresponding to the first feature data output by the first representation network. Therefore, the network expression ability of the second representation network is stronger, which avoids the first representation network from entering the local optimum too early, and improves the accuracy of the target model obtained by training for portrait recognition.

[0138] Further, the processor 710 is used to determine a first thermal map and a second thermal map according to the first characteristic data and the second characteristic data;

[0139] The processor 710 is configured to determine a first coordinate value of the first thermal map and a second coordinate value of the second thermal map;

[0140] The processor 710 is used to train a preset loss function through the first coordinate value, the second coordinate value and the preset coordinate value to obtain a target model, and the preset loss function is associated with the first representation network.

[0141] In the embodiment of the present application, the second representation network is obtained by network augmentation of the first representation network, so the first representation network is part of the second representation network, and the weights are shared. During the model training process, the gradient flow of the second representation network parameters passes through the first representation network. Since the second representation network has a wider structure and stronger network expression ability, it can prevent the first representation network from entering the local optimum earlier. The second representation network and the first representation network are trained at the same time to improve the training efficiency, and the expression ability of the second representation network is also used to supervise the training of the first representation network.

[0142] It should be understood that in the embodiment of the present application, the input unit 704 may include a graphics processor (Graphics Processing Unit, GPU) 7041 and a microphone 7042, and the graphics processor 7041 processes the image data of the static picture or video obtained by the image capture device (such as a camera) in the video capture mode or the image capture mode. The display unit 706 may include a display panel 7061, and the display panel 7061 may be configured in the form of a liquid crystal display, an organic light emitting diode, etc. The user input unit 707 includes a touch panel 7071 and at least one of other input devices 7072. The touch panel 7071 is also called a touch screen. The touch panel 7071 may include at least two parts of a touch detection device and a touch controller. Other input devices 7072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, and a joystick, which will not be repeated here.

[0143] The memory 709 can be used to store software programs and various data. The memory 709 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required for at least one function (such as a sound playback function, an image playback function, etc.), etc. In addition, the memory 709 may include a volatile memory or a non-volatile memory, or the memory 709 may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), a static random access memory (SRAM), a dynamic random access memory (DRAM), a synchronous dynamic random access memory (SDRAM), a double data rate synchronous dynamic random access memory (DDRSDRAM), an enhanced synchronous dynamic random access memory (ESDRAM), a synchronous link dynamic random access memory (SLDRAM) and a direct memory bus random access memory (DRRAM). The memory 709 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

[0144] The processor 710 may include one or more processing units; optionally, the processor 710 integrates an application processor and a modem processor, wherein the application processor mainly processes operations related to an operating system, a user interface, and application programs, and the modem processor mainly processes wireless communication signals, such as a baseband processor. It is understandable that the modem processor may not be integrated into the processor 710.

[0145] An embodiment of the present application also provides a readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the above-mentioned method embodiment are implemented and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.

[0146] The processor is the processor in the electronic device in the above embodiment. The readable storage medium includes a computer readable storage medium, such as a computer read-only memory ROM, a random access memory RAM, a magnetic disk or an optical disk.

[0147] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface and the processor are coupled, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned network model training method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0148] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0149] An embodiment of the present application provides a computer program product, which is stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the above-mentioned network model training method embodiment and can achieve the same technical effect. To avoid repetition, it will not be repeated here.

[0150] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise one..." do not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted, or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0151] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a computer software product, which is stored in a storage medium (such as ROM / RAM, a disk, or an optical disk), and includes a number of instructions for a terminal (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of each embodiment of the present application.

[0152] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms without departing from the purpose of the present application and the scope of protection of the claims, all of which are within the protection of the present application.

Claims

1. A network model training method, characterized in that: include: Acquire a first representation network and a second representation network, wherein the second representation network is an augmented network of the first representation network; Inputting the image data into the first representation network and the second representation network respectively for feature extraction to obtain first feature data and second feature data, wherein during the feature extraction process, the data stream of the second representation network includes the data stream of the first representation network; Performing supervised training on the first representation network according to the first feature data and the second feature data to obtain a target model; The step of inputting the image data into the first representation network and the second representation network for feature extraction to obtain first feature data and second feature data includes: Downsampling the image data by using the first representation network and the second representation network to obtain a first downsampling result and a second downsampling result; Inputting the first down-sampling result into the first representation network for up-sampling processing to obtain a first up-sampling result; Inputting the first down-sampling result and the second down-sampling result into the second representation network for up-sampling processing to obtain a second up-sampling result; The first feature data is determined according to the first up-sampling result, and the second feature data is determined according to the first up-sampling result and the second up-sampling result.

2. The network model training method according to claim 1, characterized in that: The number of times of downsampling processing is multiple times; After downsampling the image data by the first representation network and the second representation network to obtain a first downsampling result and a second downsampling result, the method further includes: Performing downsampling processing on the first downsampling result obtained by the previous downsampling processing through the first representation network, and updating the first downsampling result; Downsampling the first downsampling result and the second downsampling result obtained by the previous downsampling process is performed through the second representation network to update the second downsampling result.

3. The network model training method according to claim 1, characterized in that: The number of times of downsampling processing is multiple times; Before determining the first feature data according to the first up-sampling result and determining the second feature data according to the second up-sampling result, the method further includes: Upsampling the first upsampling result obtained by the previous upsampling process through the first representation network to update the first upsampling result; The first upsampling result and the second upsampling result obtained by the previous upsampling process are upsampled by the second representation network to update the second upsampling result.

4. The network model training method according to any one of claims 1 to 3, characterized in that: The step of performing supervised training on the first representation network according to the first feature data and the second feature data to obtain a target model includes: Determining a first thermal map and a second thermal map according to the first characteristic data and the second characteristic data; Determining a first coordinate value of the first heat map and a second coordinate value of the second heat map; The preset loss function is trained by using the first coordinate value, the second coordinate value and the preset coordinate value to obtain the target model, and the preset loss function is associated with the first representation network.

5. A network model training device, characterized in that: include: An acquisition module, configured to acquire a first representation network and a second representation network, wherein the second representation network is an augmented network of the first representation network; An extraction module, used for inputting the image data into the first representation network and the second representation network respectively for feature extraction to obtain first feature data and second feature data, wherein during the feature extraction process, the data stream of the second representation network includes the data stream of the first representation network; A training module, used for performing supervised training on the first representation network according to the first feature data and the second feature data to obtain a target model; A downsampling module, configured to perform downsampling processing on the image data through the first representation network and the second representation network to obtain a first downsampling result and a second downsampling result; an upsampling module, configured to input the first downsampling result into the first representation network for upsampling processing to obtain a first upsampling result; The upsampling module is used to input the first downsampling result and the second downsampling result into the second representation network for upsampling processing to obtain a second upsampling result; A first determining module is configured to determine the first feature data according to the first up-sampling result, and to determine the second feature data according to the first up-sampling result and a second up-sampling result.

6. The network model training device according to claim 5, characterized in that: The number of times of downsampling processing is multiple times; The downsampling module is configured to perform downsampling processing on the first downsampling result obtained by the previous downsampling processing through the first representation network, and update the first downsampling result; The downsampling module is used to downsample the first downsampling result and the second downsampling result obtained by the previous downsampling process through the second representation network, and update the second downsampling result.

7. The network model training device according to claim 5, characterized in that: The number of times of downsampling processing is multiple times; The upsampling module is used to perform upsampling processing on a first upsampling result obtained by the previous upsampling processing through the first representation network, and update the first upsampling result; The upsampling module is used to perform upsampling processing on the first upsampling result and the second upsampling result obtained by the previous upsampling processing through the second representation network, and update the second upsampling result.

8. The network model training device according to any one of claims 5 to 7, characterized in that: include: A third determination module, configured to determine a first heat map and a second heat map according to the first feature data and the second feature data; a fourth determination module, configured to determine a first coordinate value of the first thermal map and a second coordinate value of the second thermal map; The training module is also used to train a preset loss function through the first coordinate value, the second coordinate value and the preset coordinate value to obtain the target model, and the preset loss function is associated with the first representation network.

Images