A method, device, computer equipment and storage medium for generating a neural network

Abstract

The present disclosure provides a method, apparatus, computer device and storage medium for generating a neural network, wherein the method includes: obtaining a teacher neural network trained using a first training sample set, and obtaining a second training sample set; wherein the first training sample set includes training samples corresponding to multiple preset image processing tasks, and the second training sample set includes training samples corresponding to a target task, and the multiple preset image processing tasks include the target task; inputting the second training sample set into the teacher neural network and the student neural network to be trained, and respectively obtaining first image feature information corresponding to the second training sample and second image feature information corresponding to the second training sample; based on the obtained first image feature information and second image feature information, generating a first prediction loss, and training the student neural network to be trained based on the first prediction loss until a first preset training cutoff condition is met, thereby obtaining a trained student neural network.

Description

Technical Field

[0001] The present disclosure relates to the field of deep learning technology, and in particular to a method, apparatus, computer device, and storage medium for generating a neural network. Background Art

[0002] Pre-trained models trained on large-scale datasets often have good generalization capabilities. Therefore, people try to migrate the trained pre-trained models to downstream target tasks to improve the processing performance of the target tasks.

[0003] However, in real-world applications, because the target tasks are specific to a specific domain, such as detecting pedestrians on a street, lightweight model structures with high execution speed are required to improve the execution efficiency of the target tasks. However, pre-trained models are often large-scale neural networks with specific structures. Therefore, migrating pre-trained models to the target tasks and using them to process the target tasks is inefficient. Summary of the Invention

[0004] The embodiments of the present disclosure at least provide a method, apparatus, computer device, and storage medium for generating a neural network.

[0005] In a first aspect, an embodiment of the present disclosure provides a method for generating a neural network, comprising:

[0006] Obtaining a teacher neural network trained using a first training sample set, and obtaining a second training sample set; wherein the first training sample set includes training samples corresponding to a plurality of preset image processing tasks, and the second training sample set includes training samples corresponding to a target task, and the plurality of preset image processing tasks include the target task;

[0007] Inputting the second training sample set into the teacher neural network and the student neural network to be trained, processing the second training sample in the second training sample set by the teacher neural network to obtain first image feature information corresponding to the second training sample, and processing the second training sample in the second training sample set by the student neural network to be trained to obtain second image feature information corresponding to the second training sample;

[0008] Based on the obtained first image feature information and the second image feature information, a first prediction loss is generated, and the student neural network to be trained is trained based on the first prediction loss until a first preset training cutoff condition is met, thereby obtaining a trained student neural network, wherein the first prediction loss represents the prediction error of the student network to be trained in the target task.

[0009] This method uses a first training sample set of multiple preset image processing tasks to train a teacher neural network, thereby obtaining a teacher neural network with higher performance. That is, using the teacher neural network to execute the preset image processing task can obtain a relatively accurate image processing result. Furthermore, using a second training sample set corresponding to the target task, the first image feature information and the second image feature information are respectively processed by the teacher neural network and the student neural network to be trained. The first image feature information is output by the teacher neural network that has been trained and has higher processing performance. Therefore, the first image feature information can be used as training supervision information for the student neural network to be trained. Afterwards, the student neural network to be trained is continuously trained using the first prediction loss generated by the first image feature information and the second image feature information. The processing capability of the teacher neural network with higher performance can be transferred to the student neural network, so that the trained student neural network also has higher performance. At the same time, the processing capability of the teacher neural network is transferred to the student neural network with a simpler network structure, which is conducive to improving the execution efficiency of the target task.

[0010] In an optional implementation, obtaining the second training sample set includes:

[0011] Acquire a third training sample set corresponding to the target task, wherein the third training sample set includes real sample data;

[0012] The third training sample set is processed by a derivative neural network for generating a derivative image to obtain a second training sample set corresponding to the target task; wherein the distribution of the second training samples in the second training sample set is similar to the distribution of the third training samples in the third training sample set.

[0013] In an optional embodiment, the processing of the third training sample set by a derivative neural network for generating a derivative image to obtain a second training sample set corresponding to the target task includes:

[0014] performing masking processing on at least part of the third training samples in the third training sample set;

[0015] The third training sample after masking is input into the derivative neural network to obtain a second training sample set corresponding to the target task.

[0016] In an optional embodiment, the first preset training cutoff condition includes a first training cutoff sub-condition and a second training cutoff sub-condition;

[0017] The step of training the student neural network to be trained based on the first prediction loss until a first preset training cutoff condition is satisfied to obtain a trained student neural network includes:

[0018] Training the student neural network to be trained based on the first prediction loss until a first training cutoff sub-condition is met, thereby obtaining a student neural network that has completed preliminary training;

[0019] generating a second prediction loss based on a difference between the label information of the third training sample corresponding to the target task and a processing result of the third training sample by the preliminarily trained student neural network, wherein the second prediction loss represents a prediction error of the preliminarily trained student neural network in the target task;

[0020] Based on the second prediction loss, the student neural network that has been initially trained is trained until a second training cutoff sub-condition is met, thereby obtaining a finally trained student neural network.

[0021] In an optional embodiment, the derived neural network is generated as follows:

[0022] Using the third training sample set as conditional information of a conditional generative network, and training the conditional generative network based on the first training sample set, wherein the conditional generative network includes a generator for generating image data;

[0023] The generator in the trained conditional generative network is used as the derived neural network.

[0024] In an optional implementation, using the third training sample set as conditional information of a conditional generative network, and training the conditional generative network based on the first training sample set includes:

[0025] Determining a third prediction loss of the generator based on the distribution of a plurality of virtual image data generated by the generator in the conditional generative network under the supervision of a first training sample in the first training sample set, and the distribution of a plurality of third training samples in the third training sample set;

[0026] The conditional generative network is trained according to the third prediction loss.

[0027] In an optional implementation, the training of the conditional generative network based on the first training sample set further includes:

[0028] Inputting a plurality of first training samples in the first training sample set into a generator in a conditional generative network to obtain third image feature information of the plurality of first training samples respectively;

[0029] generating a fourth prediction loss based on the third image feature information and standard feature information corresponding to the first training sample, wherein the standard feature information includes feature information obtained by extracting features from the first training sample using a trained feature extraction network;

[0030] The step of training the conditional generative network according to the third prediction loss comprises:

[0031] Based on the third prediction loss and the fourth prediction loss, the generator in the conditional generative network is trained until a second preset training cutoff condition is met, thereby obtaining a generator in the conditional generative network that has completed training.

[0032] In an optional implementation, generating a fourth prediction loss based on the third image feature information and standard feature information corresponding to the first training sample includes:

[0033] Determining codebook sequence information based on standard feature information corresponding to the first training sample; the codebook sequence information includes a feature vector corresponding to each of a plurality of first feature points; the plurality of first feature points are selected from feature points corresponding to the plurality of first training samples, and feature vectors corresponding to different first feature points have similar distributions;

[0034] The fourth prediction loss is generated based on the third image feature information and the codebook sequence information.

[0035] In an optional implementation, generating the fourth prediction loss based on the third image feature information and the codebook sequence information includes:

[0036] Based on the third image feature information, screening out a feature vector sequence matching the first training sample from the codebook sequence information;

[0037] Generating a prediction sub-loss corresponding to the first training sample based on the feature vector sequence;

[0038] The fourth prediction loss is generated based on the prediction sub-loss corresponding to each first training sample.

[0039] In an optional embodiment, the conditional generative network further includes a discriminator for adversarial training with the generator;

[0040] The third prediction loss and the fourth prediction loss are determined based on a true / false discrimination result of the discriminator on the virtual image data and the first training sample.

[0041] In a second aspect, an embodiment of the present disclosure further provides an apparatus for generating a neural network, comprising:

[0042] an information acquisition module, configured to acquire a teacher neural network trained using a first training sample set, and to acquire a second training sample set; wherein the first training sample set includes training samples corresponding to a plurality of preset image processing tasks, and the second training sample set includes training samples corresponding to a target task, wherein the plurality of preset image processing tasks include the target task;

[0043] a network processing module, configured to input the second training sample set into the teacher neural network and the student neural network to be trained, process the second training samples in the second training sample set through the teacher neural network to obtain first image feature information corresponding to the second training samples, and process the second training samples in the second training sample set through the student neural network to be trained to obtain second image feature information corresponding to the second training samples;

[0044] A network training module is used to generate a first prediction loss based on the obtained first image feature information and the second image feature information, and train the student neural network to be trained based on the first prediction loss until a first preset training cutoff condition is met, thereby obtaining a trained student neural network, wherein the first prediction loss represents the prediction error of the student network to be trained in the target task.

[0045] In an optional implementation, the information acquisition module is configured to acquire a third training sample set corresponding to the target task, wherein the third training sample set includes real sample data;

[0046] The third training sample set is processed by a derivative neural network for generating a derivative image to obtain a second training sample set corresponding to the target task; wherein the distribution of the second training samples in the second training sample set is similar to the distribution of the third training samples in the third training sample set.

[0047] In an optional implementation, the information acquisition module is configured to perform masking on at least part of the third training samples in the third training sample set;

[0048] The third training sample after masking is input into the derivative neural network to obtain a second training sample set corresponding to the target task.

[0049] In an optional embodiment, the first preset training cutoff condition includes a first training cutoff sub-condition and a second training cutoff sub-condition;

[0050] The network training module is configured to train the student neural network to be trained based on the first prediction loss until a first training cutoff sub-condition is met, thereby obtaining a student neural network that has completed preliminary training;

[0051] generating a second prediction loss based on a difference between the label information of the third training sample corresponding to the target task and a processing result of the third training sample by the preliminarily trained student neural network, wherein the second prediction loss represents a prediction error of the preliminarily trained student neural network in the target task;

[0052] Based on the second prediction loss, the student neural network that has been initially trained is trained until a second training cutoff sub-condition is met, thereby obtaining a finally trained student neural network.

[0053] In an optional embodiment, the apparatus further includes a network generation module configured to generate a derivative neural network in the following manner:

[0054] Using the third training sample set as conditional information of a conditional generative network, and training the conditional generative network based on the first training sample set, wherein the conditional generative network includes a generator for generating image data;

[0055] The generator in the trained conditional generative network is used as the derived neural network.

[0056] In an optional embodiment, the network generation module is configured to determine a third prediction loss of the generator based on the distribution of multiple virtual image data generated by the generator in the conditional generation network under the supervision of the first training sample in the first training sample set, and the distribution of multiple third training samples in the third training sample set;

[0057] The conditional generative network is trained according to the third prediction loss.

[0058] In an optional embodiment, the network generation module is further configured to input a plurality of first training samples in the first training sample set into a generator in the conditional generation network to obtain third image feature information of the plurality of first training samples respectively;

[0059] generating a fourth prediction loss based on the third image feature information and standard feature information corresponding to the first training sample, wherein the standard feature information includes feature information obtained by extracting features from the first training sample using a trained feature extraction network;

[0060] The network generation module is used to train the generator in the conditional generation network based on the third prediction loss and the fourth prediction loss until a second preset training cutoff condition is met, thereby obtaining a generator in the conditional generation network that has completed training.

[0061] In an optional embodiment, the network generation module is configured to determine codebook sequence information based on standard feature information corresponding to the first training sample; the codebook sequence information includes a feature vector corresponding to each of a plurality of first feature points; the plurality of first feature points are selected from feature points corresponding to the plurality of first training samples, and the feature vectors corresponding to the plurality of different first feature points have similar distributions;

[0062] The fourth prediction loss is generated based on the third image feature information and the codebook sequence information.

[0063] In an optional implementation, the network generation module is configured to filter out a feature vector sequence matching the first training sample from the codebook sequence information based on the third image feature information;

[0064] Generating a prediction sub-loss corresponding to the first training sample based on the feature vector sequence;

[0065] The fourth prediction loss is generated based on the prediction sub-loss corresponding to each first training sample.

[0066] In an optional embodiment, the conditional generative network further includes a discriminator for adversarial training with the generator;

[0067] The third prediction loss and the fourth prediction loss are determined based on a true / false discrimination result of the discriminator on the virtual image data and the first training sample.

[0068] In a third aspect, an embodiment of the present disclosure further provides a computer device, comprising: a processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the above-mentioned first aspect, or any possible method of generating a neural network in the first aspect are performed.

[0069] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the above-mentioned first aspect, or any possible method of generating a neural network in the first aspect are executed.

[0070] For a description of the effects of the apparatus, computer equipment, and storage medium for generating a neural network, please refer to the description of the method for generating a neural network, which will not be repeated here.

[0071] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following briefly introduces the drawings required for use in the embodiments. The drawings herein are incorporated into and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and, together with the specification, are used to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For those of ordinary skill in the art, other relevant drawings can be obtained based on these drawings without inventive effort.

[0073] Figure 1 A flowchart of a method for generating a neural network provided by an embodiment of the present disclosure is shown;

[0074] Figure 2 A schematic diagram of the process structure of knowledge distillation provided by an embodiment of the present disclosure is shown;

[0075] Figure 3 A schematic diagram of a process for determining a derived training sample provided by an embodiment of the present disclosure is shown;

[0076] Figure 4 A specific flow chart of training a student neural network to be trained provided by an embodiment of the present disclosure is shown;

[0077] Figure 5 A schematic diagram showing the process structure of further training a student neural network that has completed preliminary training, provided by an embodiment of the present disclosure;

[0078] Figure 6 A schematic diagram of a device for generating a neural network provided by an embodiment of the present disclosure is shown;

[0079] Figure 7 A schematic structural diagram of a computer device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0080] In order to make the purpose, technical solutions and advantages of the embodiments of the present disclosure clearer, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the disclosure for which protection is sought, but merely represents selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work are within the scope of protection of the present disclosure.

[0081] In addition, the terms "first," "second," and the like in the description and claims of the embodiments of the present disclosure and in the accompanying drawings are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, such that the embodiments described herein can be practiced in an order other than that shown or described herein.

[0082] In this document, "multiple or several" refers to two or more. "And / or" describes the relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can mean: A exists alone, A and B exist simultaneously, or B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0083] Research has found that pre-trained models trained on large-scale datasets often have excellent generalization capabilities. Therefore, attempts have been made to transfer pre-trained models to downstream target tasks to improve their performance. However, in practical applications, since target tasks are specific to specific domains, such as detecting pedestrians on the street, fast and lightweight model structures are required to improve their execution efficiency. However, pre-trained models are often large-scale neural networks with specific structures. Therefore, transferring pre-trained models to target tasks and using them to process them is inefficient.

[0084] Based on the above research, the present disclosure provides a method for generating a neural network, which uses a first training sample set of multiple preset image processing tasks to train a teacher neural network, and can obtain a teacher neural network with higher performance, that is, using the teacher neural network to perform the preset image processing task, it is possible to obtain a more accurate image processing result. Further, using the second training sample set corresponding to the target task, the first image feature information and the second image feature information are obtained respectively after being processed by the teacher neural network and the student neural network to be trained, wherein the first image feature information is output by the teacher neural network that has been trained and has higher processing performance. Therefore, the first image feature information can be used as training supervision information for the student neural network to be trained; thereafter, the first prediction loss generated by the first image feature information and the second image feature information is used to continuously train the student neural network to be trained, and the processing power of the teacher neural network with higher performance can be transferred to the student neural network, so that the trained student neural network also has higher performance. At the same time, the processing power of the teacher neural network is transferred to the student neural network with a simpler network structure, which is conducive to improving the execution efficiency of the target task.

[0085] The defects existing in the solutions in the prior art are the results obtained by the inventor after practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed by the present disclosure for the above problems below should be the contributions made by the inventor to the present disclosure during the disclosure process.

[0086] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0087] To facilitate understanding of this embodiment, a method for generating a neural network disclosed in this embodiment is first described in detail. The method for generating a neural network provided in this embodiment is generally executed by a computer device with certain computing capabilities. In some possible implementations, this method for generating a neural network can be implemented by a processor invoking computer-readable instructions stored in a memory.

[0088] The following describes the method for generating a neural network provided by the embodiment of the present disclosure by taking a computer device as an example of an execution subject.

[0089] See also Figure 1 FIG. 1 is a flow chart of a method for generating a neural network according to an embodiment of the present disclosure, wherein the method includes steps S101 to S103, wherein:

[0090] S101: Obtain a teacher neural network trained using a first training sample set, and obtain a second training sample set; wherein the first training sample set includes training samples corresponding to multiple preset image processing tasks, the second training sample set includes training samples corresponding to a target task, and the multiple preset image processing tasks include the target task.

[0091] The multiple preset image processing tasks may include image processing tasks for multiple scenarios. These multiple scenarios include, for example, natural scenes, surveillance scenes, and acquisition scenes. The multiple preset image processing tasks may include image classification tasks for images collected in at least one of the natural, surveillance, and acquisition scenes; and / or image detection tasks for images collected in at least one of the natural, surveillance, and acquisition scenes; and / or image segmentation tasks for images collected in at least one of the natural, surveillance, and acquisition scenes, and so on.

[0092] The first training sample set may include training samples corresponding to multiple preset image processing tasks, that is, first training samples, such as samples of multiple image processing tasks in multiple scenes in the above example. For example, the first training sample set may include images in natural scenes, images in monitoring scenes, and images in acquisition scenes. Or for example, the first training sample set may include sample images of image classification tasks in natural scenes, sample images of image detection tasks in natural scenes, sample images of semantic segmentation tasks in natural scenes, sample images of image classification tasks in monitoring scenes, sample images of image detection tasks in monitoring scenes, sample images of semantic segmentation tasks in monitoring scenes, sample images of image classification tasks in acquisition scenes, sample images of image detection tasks in acquisition scenes, and sample images of image segmentation tasks in acquisition scenes, etc.

[0093] Multiple preset image processing tasks may include a target task. In other words, a target task is one or more specific tasks among the multiple preset image processing tasks. Target tasks can include specific tasks in specific scenarios. For example, pedestrian detection on a certain street, chef behavior detection in a kitchen, vehicle classification at a certain intersection, or ore segmentation on a coal mine conveyor belt.

[0094] The second training sample set is the training sample set corresponding to the target task. Optionally, the second training sample set may include pseudo data derived from the real sample data corresponding to the target task, such as virtual image data (i.e., pseudo image data) generated based on real captured images. Optionally, in addition to pseudo data, the second training sample set may also include real sample data corresponding to the target task, i.e., sample data collected in a real scene, such as images of pedestrians captured on a certain street for a pedestrian detection task on a certain street. Optionally, the second training sample set may also only include real sample data corresponding to the target task.

[0095] The teacher neural network may be a neural network trained using a first training sample set for multiple preset image processing tasks, and is used to perform the preset image processing tasks. The first training sample set may be a training sample set used for large-scale pre-training, which may include training samples for multiple preset image processing tasks. Training the teacher neural network using this large-scale pre-trained first training sample set can produce a higher-performance teacher neural network, allowing the trained teacher neural network to perform the preset image processing tasks and obtain relatively accurate image processing results.

[0096] S102: Input the second training sample set into the teacher neural network and the student neural network to be trained, process the second training sample in the second training sample set through the teacher neural network to obtain first image feature information corresponding to the second training sample, and process the second training sample in the second training sample set through the student neural network to be trained to obtain second image feature information corresponding to the second training sample.

[0097] This step can employ knowledge distillation for transfer learning, using the output of a pre-trained teacher neural network as a supervisory signal to train the student neural network. During the knowledge distillation process, the teacher neural network and the student neural network each extract image features from the input sample data, obtaining first and second image feature information for the second training sample set, respectively.

[0098] The processing of the second training sample by the teacher neural network and the processing of the second training sample by the student neural network to be trained are both based on the image processing of the target task, that is, the processing is performed with the image processing result corresponding to the target task as the goal.

[0099] Taking the target task of pedestrian detection on a certain street as an example, the teacher neural network determines whether the pedestrian has run a red light based on the detection results of the pedestrian image; the detection results are used to supervise the processing of the pedestrian image by the student neural network to be trained, so that the image processing results obtained by the student neural network to be trained are similar to the detection results of the pedestrian image by the teacher neural network, that is, the student neural network to be trained performs image detection processing with the goal of detecting whether the pedestrian has run a red light.

[0100] See also Figure 2 As shown, it is a schematic diagram of the process structure of knowledge distillation; it includes multiple second training samples 21 (such as natural environment sample images), a teacher neural network 22, first image feature information 221 output by the teacher neural network, a student neural network 23 to be trained, and second image feature information 231 output by the student neural network 23 to be trained. Exemplarily, multiple natural environment sample images 21 can be input into the teacher neural network 22 and the student neural network 23 to be trained respectively. By processing the multiple natural environment sample images 21 through the teacher neural network 22, the first image feature information 221 corresponding to each natural environment sample image can be obtained; by processing the multiple natural environment sample images through the student neural network 23 to be trained, the second image feature information 231 corresponding to each natural environment sample image can be obtained.

[0101] The first image feature information may include information representing image features extracted by any network layer in the teacher neural network from the corresponding second training sample, such as a feature map output by an intermediate layer, or a feature vector output by the last feature extraction layer or fully connected layer.

[0102] The second image feature information may include information representing image features extracted by any network layer in the student neural network to be trained for the corresponding second training sample, such as a feature map output by an intermediate layer, or a feature vector output by the last feature extraction layer or fully connected layer.

[0103] For example, the network structure of the teacher neural network can be a relatively complex network structure, obtained by training with a first training sample set of multiple preset image processing tasks; the network structure of the student neural network to be trained can be a relatively simple network layer. The network structure complexity of the teacher network is higher than that of the student network.

[0104] S103: Based on the obtained first image feature information and second image feature information, a first prediction loss is generated, and the student neural network to be trained is trained based on the first prediction loss until a first preset training cutoff condition is met, thereby obtaining a trained student neural network, wherein the first prediction loss represents the prediction error of the student network to be trained in the target task.

[0105] The first image feature information and the second image feature information obtained based on the same second training sample can be used as a set of training data. Subsequently, knowledge distillation processing is performed. Specifically, the first image feature information in the set of training data can be used as standard supervisory data. The difference between the second image feature information and the first image feature information is calculated, and the difference is used as the prediction error of the student neural network to be trained in the target task, thereby obtaining the prediction loss of the student neural network to be trained. Subsequently, based on multiple second training samples, multiple sets of training data containing the first image feature information and the second image feature information are obtained, thereby obtaining the prediction losses of multiple student neural networks to be trained; based on the prediction losses of the multiple student neural networks to be trained, a first prediction loss is determined. The student neural network to be trained is trained using the multiple first prediction losses, and the parameters of the student neural network to be trained are adjusted. Based on the student neural network with adjusted parameters, the process returns to the step of inputting the second training sample set into the student neural network to obtain the second image feature information and recalculating the first prediction loss. Thus, through multiple rounds of iterative parameter adjustment, the parameters are adjusted until the first preset training cutoff condition is met, resulting in a trained student neural network.

[0106] The process of iteratively training the student neural network is the process of continuously aligning the second image feature information output by the student neural network with the first image feature information in the same training data. This is also the process of continuously transferring the processing power of the teacher neural network to the student neural network. This completes knowledge distillation, allowing the trained student neural network to learn the processing power of the teacher neural network, resulting in a higher-performance student neural network.

[0107] The first prediction loss can be determined by, specifically, calculating the average of the prediction errors corresponding to multiple second training samples in the second training sample set and using the average as the first prediction loss. Alternatively, the multiple prediction errors can be weighted to obtain the first prediction loss. The weight corresponding to each prediction loss can be set based on empirical values and is not specifically limited in the present embodiment.

[0108] The difference between the second image feature information and the first image feature information is calculated. For example, the difference between the feature vector indicated by the second image feature information and the feature vector indicated by the first image feature information can be calculated using an L2 distance metric. Here, the feature vector indicated by the second image feature information and the feature vector indicated by the first image feature information have the same dimension.

[0109] The first prediction loss can be used for back propagation to obtain the gradient information of the network layer in the student neural network to be trained, and the gradient information can be used to adjust the network parameters of the student neural network that has completed the preliminary training to achieve an iterative training of the student neural network to be trained.

[0110] The first preset training cutoff condition may include a first prediction loss reaching a convergence condition, such as the first prediction loss being less than a first preset loss threshold, or a condition that the number of training iterations is greater than or equal to a first preset training number threshold. The first preset loss threshold and the first preset training number threshold may be set based on empirical values and are not specifically limited in the present embodiment.

[0111] In the above-mentioned S101 to S103, a teacher neural network with higher performance is trained using a first training sample set of multiple preset image processing tasks, that is, a teacher neural network with higher performance is obtained by using the teacher neural network to perform the preset image processing tasks, which can obtain more accurate image processing results. Furthermore, a second training sample set corresponding to the target task is used, and the first image feature information and the second image feature information are obtained respectively after being processed by the teacher neural network and the student neural network to be trained, wherein the first image feature information is output by the teacher neural network that has been trained and has higher processing performance. Therefore, the first image feature information can be used as training supervision information for the student neural network to be trained; thereafter, the student neural network to be trained is continuously trained using the first prediction loss generated by the first image feature information and the second image feature information, which can transfer the processing power of the teacher neural network with better performance for multiple preset image processing tasks to the student neural network used to perform the target task, so that the trained student neural network also has better performance in the target task, thereby realizing the use of a lightweight neural network to perform specific image processing tasks, obtaining better image processing results, and improving the execution efficiency of the image processing task.

[0112] In some embodiments, the target task is a specific task in a specific scenario, and the amount of sample data for the target task in the real scenario is limited. Using this limited amount of sample data makes it difficult to train a high-performance student neural network. Therefore, the student neural network to be trained can be trained using multiple sample data corresponding to the target task, as well as pseudo data derived from the multiple sample data, thereby expanding the amount of sample data corresponding to the target task. In other words, using the second training sample set to train the student neural network to be trained can result in a high-performance student neural network.

[0113] The process of obtaining the second training sample set corresponding to the target task can refer to the following steps:

[0114] S1011: Acquire a third training sample set corresponding to the target task, where the third training sample set includes real sample data;

[0115] S1012: Processing the third training sample set through a derivative neural network for generating a derivative image to obtain a second training sample set corresponding to the target task; wherein the distribution of the second training samples in the second training sample set is similar to the distribution of the third training samples in the third training sample set.

[0116] In this step, the derived neural network may be a network that can restore samples with a similar distribution to the training samples based on the training samples.

[0117] The third training sample set may include real sample data of the target task, that is, sample data obtained in a real scenario.

[0118] In specific implementation, for each of the multiple third training samples, the third training sample can be processed by a pre-trained derivative neural network, and the sample features of the third training sample can be extracted using the derivative neural network. Based on the sample features of the third training sample, derivative sample features corresponding to multiple third training samples are generated; then, virtual image data is generated based on the multiple derivative sample features. In this way, one or more virtual image data can be generated based on one third training sample, and the virtual image data can be used as a second training sample and added to the second training sample set.

[0119] Based on this, when there are a limited number of third training samples, a training sample set whose number is much larger than the number of third training samples, ie, the second training sample set, can be obtained by using a derivative neural network.

[0120] The distribution of the second training samples in the second training sample set is similar to the distribution of the third training samples in the third training sample set. Here, the distribution of multiple samples can be represented by the distribution of features extracted from the multiple samples in the feature space, or the distribution characteristics of the multiple samples can be determined using the signal representation of the samples in the frequency domain. During the training process of the above-mentioned derivative neural network, the distribution characteristics of the third training samples in the third training sample set can be added to the supervisory information so that after the derivative neural network training is completed, a second training sample set with a distribution similar to that of the third training samples in the third training sample set can be generated.

[0121] The above-mentioned use of the derivative neural network to process multiple third training samples can obtain multiple second training samples with a distribution similar to the corresponding third training samples. Then, by using a large number of second training samples, the performance of the trained student neural network in performing the target task can be improved.

[0122] In some embodiments, a derivative neural network can be used to restore the masked third training samples from the masked portion of the third training samples to generate multiple derivative images. Specifically, at least a portion of the third training samples in the third training sample set are masked; the masked third training samples are then input into the derivative neural network to obtain a second training sample set corresponding to the target task.

[0123] The mask processing may use a selected mask plate, such as an image, graphic or object, to (globally or locally) block or extract the third training sample to control the image processing area or processing process.

[0124] For example, a partial area of the third training sample may be occluded (see Figure 3 (e.g., the display effect in the second training sample), for example, starting from the bottom of the second training sample, occluding half or one-third of the entire sample. Here, the occlusion area of the occluded portion of the sample can be set based on empirical values or the entire area of the second training sample, and is not specifically limited in the present embodiment.

[0125] See also Figure 3 As shown, it is a flow chart of determining the derived training samples; it includes the third training sample 31 after mask processing, including a mask 311; a derived neural network 32; and a plurality of second training samples 33.

[0126] The process of determining the second training sample includes: first, inputting the masked third training sample 31 into the derivative neural network 32, and using the network layers in the derivative neural network 32 to extract partial sample features of the masked third training sample 31; the derivative neural network 32 can restore the sample features filtered out by the masking process based on the retained partial sample features of the masked third training sample 31, thereby obtaining multiple second training samples 33. Each second training sample 33 in the multiple second training samples 33 has similar features to the portion of the masked third training sample 31 not processed by the mask 311. The derivative neural network 32 can also restore different situations for the portion processed by the mask 311. Therefore, a single masked third training sample 31 can be restored using the derivative neural network 32 to obtain multiple second training samples 33.

[0127] See also Figure 4 As shown, it is a specific flow chart for training the student neural network to be trained, including S401 to S403; wherein:

[0128] S401: Training the student neural network to be trained based on the first prediction loss until a first training cutoff sub-condition is met, thereby obtaining a student neural network that has completed preliminary training.

[0129] In this step, the first training cutoff sub-condition may include that the first prediction loss is less than a first preset loss threshold, or may also include that the number of training iterations is greater than or equal to a first preset training number threshold.

[0130] Here, the process of training the student neural network to be trained using the first prediction loss can be referred to the detailed description of S103 above, and the repeated parts will not be repeated here.

[0131] S402: Generate a second prediction loss based on the difference between the label information of the third training sample corresponding to the target task and the processing result of the student neural network that has been preliminarily trained on the third training sample, wherein the second prediction loss represents the prediction error of the student neural network that has been preliminarily trained in the target task.

[0132] In this step, the label information may include a vector label corresponding to the target task, wherein the vector label may be a label value specified by the user, and the vector label is used to indicate the vector corresponding to the label value, that is, the label vector of the third training sample corresponding to the target task.

[0133] Here, the label information may be obtained by filtering from a preset label set, which may be set based on the target task. For example, if the target task is a classification task, the preset label set may include the category label vector corresponding to the classification task. If the target task is a detection task, the preset label set may include the detection result label vector corresponding to the detection task.

[0134] During specific implementation, the third training sample is input into the student neural network that has completed preliminary training. The student neural network that has completed preliminary training processes the third training sample to obtain a processing result of the third training sample, such as a sample vector of the third training sample. Afterwards, the label information can be used as training supervision information of the student neural network that has completed preliminary training. Based on the sample vector of the third training sample and the label vector of the third training sample corresponding to the target task, a second prediction loss can be generated.

[0135] See also Figure 5As shown, it is a schematic diagram of the process structure for further training the student neural network that has completed preliminary training; it includes a third training sample 51 (i.e., a natural environment sample image), a student neural network 52 that has completed preliminary training, and a sample vector 521 output by the student neural network 52 that has completed preliminary training. For example, multiple natural environment sample images 51 can be input into the student neural network 52 that has completed preliminary training. The multiple natural environment sample images 51 are processed by the student neural network 52 that has completed preliminary training to obtain sample vectors 521 corresponding to each natural environment sample image. The difference between each sample vector and its corresponding label vector is calculated respectively to obtain multiple prediction losses of the student neural network that has completed preliminary training. Afterwards, a second prediction loss is generated based on the multiple prediction losses. For example, the average loss of the multiple prediction losses can be calculated and the average loss is used as the second prediction loss; alternatively, the multiple prediction losses can be weighted to obtain the second prediction loss. The weight corresponding to each prediction loss can be set according to an empirical value, and the embodiment of the present disclosure is not specifically limited.

[0136] S403: Based on the second prediction loss, the student neural network that has been initially trained is trained until the second training cutoff sub-condition is met, thereby obtaining a finally trained student neural network.

[0137] The second prediction loss is back-propagated to calculate the gradient information of the network layer in the student neural network that has been preliminarily trained, and the gradient information is used to adjust the network parameters of the student neural network that has been preliminarily trained, thereby achieving an iterative training of the student neural network that has been preliminarily trained.

[0138] The second preset training cutoff condition may include a condition that the second prediction loss is less than a second preset loss threshold, or may also include a condition that the number of training iterations is greater than or equal to a second preset training number threshold. Here, the second preset loss threshold and the second preset training number threshold can be set based on empirical values and are not specifically limited in the present embodiment.

[0139] In the above S401 to S403, the first prediction loss is used to train the student neural network to be trained. When the first training cutoff sub-condition is met, a preliminarily trained student neural network can be obtained. The preliminarily trained student neural network has transferred and learned the higher processing performance of the teacher neural network. Afterwards, the preliminarily trained student neural network is further trained using the second prediction loss constructed by the difference between the label information of the third training sample corresponding to the target task (the label information can be used as training supervision information of the preliminarily trained student neural network) and the processing result of the preliminarily trained student neural network on the third training sample. The final trained student neural network with higher processing capability for the target task can be obtained. Therefore, in the application process, the final trained student neural network is used to process the target task to obtain high-precision image processing results.

[0140] The derived neural network in S1012 can be generated as follows:

[0141] S1012-1: Using the third training sample set as conditional information of a conditional generative network, and training the conditional generative network based on the first training sample set, wherein the conditional generative network includes a generator for generating image data.

[0142] Here, the first training sample set may include training samples corresponding to multiple preset image processing tasks. Furthermore, when the preset image processing task is a target task, the training samples in the first training sample set also include training samples corresponding to the target task. The training samples used to train the conditional generation network and the training samples used to train the teacher neural network may be the same, different, or overlap.

[0143] In a specific implementation, the first training sample set can be input into a conditional generative network for processing. The conditional generative network includes a generator for generating image data. The generator can be constructed using a neural network. Optionally, the generator can be a partial network of the conditional generative network. The conditional generative network can also include a discriminator for adversarial training with the generator. The first training sample is input into the generator, and the generator can generate multiple virtual image data under the supervision of the first training sample. The better the generator's training effect, the more the generated virtual image data approximates the real data.

[0144] During generator training, the third training sample set can be used as conditional information and input into the conditional generative network along with the first training set. A supervision item corresponding to the conditional information is added to the supervision information of the conditional generative network, so that the virtual image data generated by the generator in the trained conditional generative network meets the conditions corresponding to the third training sample set. Optionally, the conditions corresponding to the third training sample set include being similar to the distribution of third training samples in the third training sample set.

[0145] Afterwards, the second training sample set is used as conditional information of the conditional generative network to further train the conditional generative network.

[0146] Optionally, when the conditional generative network also includes a discriminator, the discriminator can be used to judge the authenticity of the virtual image generated by the generator. When the discriminator cannot distinguish the authenticity of the virtual image data and the real image data, it is determined that the generator training is completed.

[0147] S1012-2: Use the generator in the trained conditional generative network as a derivative neural network.

[0148] Optionally, when the conditional generative network includes a discriminator, it is determined whether the generator has been trained based on the discrimination result of the discriminator, and the trained generator is used as the derived neural network.

[0149] For S1012-1, the third training sample set is used as the conditional information of the conditional generative network to train the conditional generative adversarial network. Specifically, based on the distribution of multiple virtual image data generated by the generator in the conditional generative network under the supervision of the first training sample in the first training sample set and the distribution of multiple third training samples in the third training sample set, the third prediction loss of the generator is determined; and the conditional generative network is trained according to the third prediction loss.

[0150] The third training sample set is used as the conditional information of the conditional generation network, that is, the distribution of multiple third training samples in the third training sample set is used as the conditional information, and the difference between the distribution of the virtual image data generated by the generator and the distribution of the third training samples is further determined. According to the difference, the third prediction loss is constructed.

[0151] The conditional generative network is trained based on the third prediction loss. For example, the third prediction loss can be used for backpropagation to calculate gradient information of network layers in the conditional neural network, and the gradient information can be used to adjust network parameters of the conditional generative network to achieve a single iteration of training for the conditional generative network.

[0152] Alternatively, when the conditional generative network includes a discriminator, the third prediction loss can also be determined based on the authenticity discrimination results of the discriminator in the conditional generative network on the virtual image data (i.e., pseudo data) and the real first training sample or the real third training sample (i.e., real data). The closer the virtual image data is to the real data, the more difficult it is to obtain an accurate authenticity discrimination result based on the discriminator. The lower the accuracy of the discriminator's authenticity discrimination result, the smaller the third prediction loss is determined.

[0153] For S1012-1, the conditional generative network is trained based on the first training sample set. Specifically, multiple first training samples in the first training sample set are input into the generator in the conditional generative network to obtain third image feature information of the multiple first training samples respectively; based on the third image feature information and the standard feature information corresponding to the first training samples, a fourth prediction loss is generated; based on the third prediction loss and the fourth prediction loss, the generator in the conditional generative network is trained until the second preset training cutoff condition is met, and a generator in the conditional generative network that has completed training is obtained.

[0154] Here, the third image feature information may include the sample features extracted by the network layer in the generator of the corresponding first training sample, such as the sample feature map of L1×W1×D1, where L1 represents the number of second feature points in the length direction of the sample feature map, W1 represents the number of second feature points in the width direction of the sample feature map, and D1 represents the dimension of each second feature point.

[0155] The standard feature information may include feature information obtained by extracting features from the first training sample using a trained feature extraction network. The feature extraction network may be a network used to extract image features in any image processing task.

[0156] For example, the fourth prediction loss can be generated by using the difference between the standard feature information corresponding to the first training sample and the third image feature information. For example, the average of multiple differences can be used as the fourth prediction loss; alternatively, the multiple differences can be weighted and the result used as the fourth prediction loss. The weight corresponding to each difference can be set based on empirical values and is not specifically limited in this embodiment.

[0157] Alternatively, when the conditional generative network includes a discriminator, the fourth prediction loss can also be determined based on the discriminator's authenticity discrimination result for the first training sample. For example, the generator extracts third image feature information of the first training sample, and the discriminator obtains an authenticity discrimination result for the third image feature information. The lower the accuracy of the authenticity discrimination result, the smaller the fourth prediction loss is determined.

[0158] The generator in the conditional generative network is trained. Exemplarily, a fused prediction loss can be determined based on the third prediction loss and the fourth prediction loss. The fused prediction loss can be used for backpropagation to obtain gradient information of the network layer in the conditional generative network. The gradient information is used to adjust the network parameters of the conditional generative network to achieve a single iterative training of the conditional generative network. Exemplarily, the conditional generative network can be trained by backpropagation based on the third prediction loss and the fourth prediction loss.

[0159] The second preset training cutoff condition may include the fusion prediction loss being less than a third preset loss threshold, or the number of training iterations being greater than or equal to a third preset training number threshold. Here, the third preset loss threshold and the third preset training number threshold may be set based on empirical values and are not specifically limited in the present embodiment.

[0160] Generate the fourth prediction loss. In some embodiments, codebook sequence information may be determined based on standard feature information corresponding to the first training sample; and the fourth prediction loss may be generated based on the third image feature information and the codebook sequence information.

[0161] The codebook sequence information includes a feature vector corresponding to each of the multiple first feature points; the multiple first feature points are selected from the feature points corresponding to the multiple first training samples, and the distribution of the feature vectors corresponding to the multiple different first feature points is similar.

[0162] Here, the standard feature information can be feature information obtained by performing feature extraction on the first training sample through a trained feature extraction network, such as a standard feature map of L2×W2×D2, where L2 represents the number of feature points in the length direction of the standard feature map, W2 represents the number of feature points in the width direction of the standard feature map, and D2 represents the dimension of each feature point.

[0163] Multiple first feature points can be screened from a standard feature map indicated by standard feature information corresponding to each first training sample in the multiple first training samples, and feature vectors corresponding to the screened multiple first feature points can be encoded to determine codebook sequence information. Exemplarily, the codebook sequence information includes a feature vector corresponding to each first feature point in the multiple first feature points and an encoded value of each feature vector. Here, the feature vector included in the codebook sequence information is defined as a codebook vector.

[0164] Here, the screened first feature points meet a preset requirement, that is, the distributions of feature vectors corresponding to a plurality of different first feature points are similar.

[0165] A fourth prediction loss is generated based on the third image feature information and the codebook sequence information. Specifically, based on the third image feature information, a feature vector sequence matching the first training sample is selected from the codebook sequence information; based on the feature vector sequence, a predictor loss corresponding to the first training sample is generated; and based on the predictor loss corresponding to each first training sample, a fourth prediction loss is generated.

[0166] A codebook vector that matches the feature vector corresponding to the second feature point in the third image feature information can be filtered out from the codebook sequence information, and the encoding value corresponding to the filtered codebook vector is determined. According to the position of the second feature point in the third image feature information, the filtered codebook vectors are sorted to determine the feature vector sequence that matches the first training sample, that is, the feature vector sequence that matches the third image feature information corresponding to the first training sample. The feature vector sequence can be represented by the encoding value.

[0167] The screening requirement may include that the distribution between the screened codebook vectors and the feature vectors corresponding to the corresponding second feature points is similar.

[0168] Based on the feature vector sequence, a prediction sub-loss corresponding to the first training sample is generated. In a specific implementation, the feature vector sequence can be divided into multiple feature vector sub-sequences; then, each feature vector sub-sequence in the multiple feature vector sub-sequences is input into a generator in a conditional generative network, and each feature vector sub-sequence is processed by the generator to obtain a predicted feature vector sub-sequence corresponding to each processed feature vector sub-sequence; based on the feature vector sequence and each predicted feature vector sub-sequence, the prediction sub-loss corresponding to the first training sample is determined; and based on the prediction sub-loss corresponding to each first training sample, a fourth prediction loss is generated.

[0169] Based on the feature vector sequence and each predicted feature vector subsequence, the prediction sub-loss corresponding to the first training sample is determined. In a specific implementation, the feature vector sequence is used as the standard vector sequence, the prediction difference information of each predicted feature vector subsequence is determined, and the prediction difference information of each predicted feature vector subsequence is fused to determine the prediction sub-loss corresponding to the first training sample.

[0170] Based on the prediction sub-loss corresponding to each first training sample, a fourth prediction loss is generated. For example, the average of the multiple prediction sub-losses can be calculated and used as the fourth prediction loss for the derivative neural network to be trained; alternatively, the multiple prediction sub-losses can be weighted and the weighted result used as the fourth prediction loss. The weight corresponding to each prediction sub-loss can be set based on empirical values and is not specifically limited in the present embodiment.

[0171] For example, the known codebook sequence information includes 1000×128-dimensional feature vectors, where 1000 indicates there are 1000 codebook vectors, numbered 1, 2, 3, ..., 999, and 1000. Each codebook vector has 128 dimensions. The L1×W1×D1 sample feature map corresponding to the third image feature information can be 20×20×128. 400 codebook vectors that match the feature vector corresponding to the second feature point of the sample feature map are selected from the 1000 codebook vectors to determine a feature vector sequence that matches the first training sample, such as [3, 5, 12, 20, ..., 480, 500]. The feature vector sequence is then divided into multiple feature vector subsequences, such as [3, 5], [3, 5, 12], [3, 5, 12, 20], ..., [3, 5, 12, 20, ..., 480, 500]. Each of the multiple feature vector subsequences is input into the generator. The generator can predict the next encoding value of the last encoding value in each feature vector subsequence and output the corresponding predicted feature vector subsequence, for example, [3,5] corresponds to [3,5,A], [3,5,12] corresponds to [3,5,12,B], [3,5,12,20] corresponds to [3,5,12,20,C], ..., [3,5,12,20,...,480] corresponds to [3,5,12,20,...,480,D]. Then, based on the feature vector sequence [3,5,12,20,...,480,500], the prediction difference information of each predicted feature vector subsequence is determined, and the prediction difference information of each predicted feature vector subsequence is fused to determine the prediction sub-loss corresponding to the first training sample.

[0172] Those skilled in the art will understand that in the above-mentioned method of the specific implementation method, the writing order of each step does not mean a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of each step should be determined by its function and possible internal logic.

[0173] Based on the same inventive concept, the embodiments of the present disclosure also provide a device for generating a neural network corresponding to the method for generating a neural network. Since the principle of solving the problem by the device in the embodiments of the present disclosure is similar to the method for generating a neural network in the embodiments of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0174] Reference Figure 6 FIG. 6 is a schematic diagram of a device for generating a neural network according to an embodiment of the present disclosure, wherein the device includes: an information acquisition module 601, a network processing module 602, and a network training module 603; wherein,

[0175] An information acquisition module 601 is configured to acquire a teacher neural network trained using a first training sample set, and to acquire a second training sample set; wherein the first training sample set includes training samples corresponding to a plurality of preset image processing tasks, and the second training sample set includes training samples corresponding to a target task, wherein the plurality of preset image processing tasks include the target task;

[0176] a network processing module 602 configured to input the second training sample set into the teacher neural network and the student neural network to be trained, process the second training sample in the second training sample set through the teacher neural network to obtain first image feature information corresponding to the second training sample, and process the second training sample in the second training sample set through the student neural network to be trained to obtain second image feature information corresponding to the second training sample;

[0177] The network training module 603 is used to generate a first prediction loss based on the obtained first image feature information and the second image feature information, and train the student neural network to be trained based on the first prediction loss until a first preset training cutoff condition is met, thereby obtaining a trained student neural network, wherein the first prediction loss represents the prediction error of the student network to be trained in the target task.

[0178] In an optional implementation, the information acquisition module 601 is configured to acquire a third training sample set corresponding to the target task, wherein the third training sample set includes real sample data;

[0179] The third training sample set is processed by a derivative neural network for generating a derivative image to obtain a second training sample set corresponding to the target task; wherein the distribution of the second training samples in the second training sample set is similar to the distribution of the third training samples in the third training sample set.

[0180] In an optional implementation, the information acquisition module 601 is configured to perform masking on at least part of the third training samples in the third training sample set;

[0181] The third training sample after masking is input into the derivative neural network to obtain a second training sample set corresponding to the target task.

[0182] In an optional embodiment, the first preset training cutoff condition includes a first training cutoff sub-condition and a second training cutoff sub-condition;

[0183] The network training module 603 is configured to train the student neural network to be trained based on the first prediction loss until a first training cutoff sub-condition is met, thereby obtaining a student neural network that has completed preliminary training;

[0184] generating a second prediction loss based on a difference between the label information of the third training sample corresponding to the target task and a processing result of the third training sample by the preliminarily trained student neural network, wherein the second prediction loss represents a prediction error of the preliminarily trained student neural network in the target task;

[0185] Based on the second prediction loss, the student neural network that has been initially trained is trained until a second training cutoff sub-condition is met, thereby obtaining a finally trained student neural network.

[0186] In an optional embodiment, the apparatus further includes a network generation module 604 configured to generate a derivative neural network in the following manner:

[0187] Using the third training sample set as conditional information of a conditional generative network, and training the conditional generative network based on the first training sample set, wherein the conditional generative network includes a generator for generating image data;

[0188] The generator in the trained conditional generative network is used as the derived neural network.

[0189] In an optional embodiment, the network generation module 604 is configured to determine a third prediction loss of the generator based on the distribution of multiple virtual image data generated by the generator in the conditional generative network under the supervision of the first training sample in the first training sample set, and the distribution of multiple third training samples in the third training sample set;

[0190] The conditional generative network is trained according to the third prediction loss.

[0191] In an optional embodiment, the network generation module 604 is further configured to input a plurality of first training samples in the first training sample set into a generator in the conditional generation network to obtain third image feature information of the plurality of first training samples respectively;

[0192] generating a fourth prediction loss based on the third image feature information and standard feature information corresponding to the first training sample, wherein the standard feature information includes feature information obtained by extracting features from the first training sample using a trained feature extraction network;

[0193] The network generation module 604 is used to train the generator in the conditional generation network based on the third prediction loss and the fourth prediction loss until a second preset training cutoff condition is met, thereby obtaining a generator in the conditional generation network that has completed training.

[0194] In an optional embodiment, the network generation module 604 is configured to determine codebook sequence information based on standard feature information corresponding to the first training sample; the codebook sequence information includes a feature vector corresponding to each of a plurality of first feature points; the plurality of first feature points are selected from the feature points corresponding to the plurality of first training samples, and the feature vectors corresponding to the plurality of different first feature points have similar distributions;

[0195] The fourth prediction loss is generated based on the third image feature information and the codebook sequence information.

[0196] In an optional implementation, the network generation module 604 is configured to filter out a feature vector sequence matching the first training sample from the codebook sequence information based on the third image feature information;

[0197] Generating a prediction sub-loss corresponding to the first training sample based on the feature vector sequence;

[0198] The fourth prediction loss is generated based on the prediction sub-loss corresponding to each first training sample.

[0199] In an optional embodiment, the conditional generative network further includes a discriminator for adversarial training with the generator;

[0200] The third prediction loss and the fourth prediction loss are determined based on a true / false discrimination result of the discriminator on the virtual image data and the first training sample.

[0201] For descriptions of the processing flow of each module in the apparatus for generating a neural network and the interaction flow between each module, reference can be made to the relevant descriptions in the above-mentioned embodiment of the method for generating a neural network, which will not be described in detail here.

[0202] Based on the same technical concept, the embodiment of the present application also provides a computer device. Figure 7 FIG. 1 is a schematic diagram of the structure of a computer device provided in an embodiment of the present application, including:

[0203] Processor 71 , memory 72 and bus 73 . The memory 72 stores machine-readable instructions executable by the processor 71, and the processor 71 is used to execute the machine-readable instructions stored in the memory 72. When the machine-readable instructions are executed by the processor 71, the processor 71 performs the following steps: S101: obtaining a teacher neural network trained using a first training sample set, and obtaining a second training sample set; wherein the first training sample set includes training samples corresponding to multiple preset image processing tasks, and the second training sample set includes training samples corresponding to a target task, and the multiple preset image processing tasks include the target task; S102: inputting the second training sample set into the teacher neural network and the student neural network to be trained, processing the second training sample in the second training sample set by the teacher neural network to obtain first image feature information corresponding to the second training sample, and processing the second training sample in the second training sample set by the student neural network to be trained to obtain second image feature information corresponding to the second training sample; S103: generating a first prediction loss based on the obtained first image feature information and second image feature information, and training the student neural network to be trained based on the first prediction loss until a first preset training cutoff condition is met, thereby obtaining a trained student neural network, wherein the first prediction loss represents the prediction error of the student network to be trained in the target task.

[0204] The above-mentioned memory 72 includes internal memory 721 and external memory 722; the memory 721 here is also called internal memory, which is used to temporarily store the calculation data in the processor 71, as well as the data exchanged with the external memory 722 such as the hard disk. The processor 71 exchanges data with the external memory 722 through the internal memory 721. When the computer device is running, the processor 71 and the memory 72 communicate through the bus 73, so that the processor 71 executes the execution instructions mentioned in the above method embodiment.

[0205] The present disclosure also provides a computer-readable storage medium having a computer program stored thereon. When executed by a processor, the computer program executes the steps of the method for generating a neural network described in the above method embodiment. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0206] The present disclosure also provides a computer program product comprising computer instructions, which, when executed by a processor, implement the steps of the method for generating a neural network. The computer program product may be any product capable of implementing the method for generating a neural network. Part or all of the contributions to the prior art in the computer program product may be embodied in the form of a software product (e.g., a software development kit (SDK)). The software product may be stored in a storage medium, and the computer instructions contained therein may cause a related device or processor to execute part or all of the steps of the method for generating a neural network.

[0207] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules is only a logical function division. There may be other division methods in actual implementation. For example, multiple modules or components can be combined, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or modules can be electrical, mechanical or other forms.

[0208] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0209] In addition, each functional module in each embodiment of the present disclosure may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0210] If the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0211] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The scope of protection of the present disclosure is not limited thereto. Although the present disclosure has been described in detail with reference to the above-described embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily conceive of changes to the technical solutions described in the above-described embodiments within the technical scope disclosed in the present disclosure, or replace some of the technical features therein with equivalents. Such modifications, changes, or replacements do not deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure shall be subject to the scope of protection of the claims.

Claims

1. A method for generating a neural network, characterized in that include: Obtaining a teacher neural network trained using a first training sample set, and obtaining a second training sample set; wherein the first training sample set includes training samples corresponding to a plurality of preset image processing tasks, and the second training sample set includes training samples corresponding to a target task, and the plurality of preset image processing tasks include the target task; Inputting the second training sample set into the teacher neural network and the student neural network to be trained, processing the second training sample in the second training sample set by the teacher neural network to obtain first image feature information corresponding to the second training sample, and processing the second training sample in the second training sample set by the student neural network to be trained to obtain second image feature information corresponding to the second training sample; Based on the obtained first image feature information and the second image feature information, a first prediction loss is generated, and the student neural network to be trained is trained based on the first prediction loss until a first preset training cutoff condition is met, thereby obtaining a trained student neural network, wherein the first prediction loss represents a prediction error of the student neural network to be trained in the target task; The second training sample set is generated by a derivative neural network for generating derivative images; the derivative neural network includes a generator in a trained conditional generative network; the conditional generative network is trained using a fourth prediction loss; the first training sample set includes multiple first training samples; The fourth predicted loss is generated in the following manner: Determining codebook sequence information based on standard feature information corresponding to the first training sample; the codebook sequence information includes a feature vector corresponding to each of a plurality of first feature points; the plurality of first feature points are selected from feature points corresponding to the plurality of first training samples, and feature vectors corresponding to different first feature points have similar distributions; generating the fourth prediction loss based on the third image feature information and the codebook sequence information; The third image feature information is generated by inputting the multiple first training samples into a generator in a conditional generation network.

2. The method according to claim 1, characterized in that The obtaining of the second training sample set includes: Acquire a third training sample set corresponding to the target task, wherein the third training sample set includes real sample data; The third training sample set is processed by a derivative neural network for generating a derivative image to obtain a second training sample set corresponding to the target task; wherein the distribution of the second training samples in the second training sample set is similar to the distribution of the third training samples in the third training sample set.

3. The method according to claim 2, characterized in that The step of processing the third training sample set by a derivative neural network for generating a derivative image to obtain a second training sample set corresponding to the target task includes: performing masking processing on at least part of the third training samples in the third training sample set; The third training sample after masking is input into the derivative neural network to obtain a second training sample set corresponding to the target task.

4. The method according to any one of claims 1 to 3, characterized in that The first preset training cutoff condition includes a first training cutoff sub-condition and a second training cutoff sub-condition; The step of training the student neural network to be trained based on the first prediction loss until a first preset training cutoff condition is satisfied to obtain a trained student neural network includes: Training the student neural network to be trained based on the first prediction loss until a first training cutoff sub-condition is met, thereby obtaining a student neural network that has completed preliminary training; generating a second prediction loss based on a difference between label information of a third training sample corresponding to the target task and a processing result of the third training sample by the preliminarily trained student neural network, wherein the second prediction loss represents a prediction error of the preliminarily trained student neural network in the target task; Based on the second prediction loss, the student neural network that has been initially trained is trained until a second training cutoff sub-condition is met, thereby obtaining a finally trained student neural network.

5. The method according to claim 2, characterized in that The derived neural network is generated as follows: Using the third training sample set as conditional information of a conditional generative network, and training the conditional generative network based on the first training sample set, wherein the conditional generative network includes a generator for generating image data; The generator in the trained conditional generative network is used as the derived neural network.

6. The method according to claim 5, characterized in that Using the third training sample set as conditional information of a conditional generative network, and training the conditional generative network based on the first training sample set, including: Determining a third prediction loss of the generator based on the distribution of a plurality of virtual image data generated by the generator in the conditional generative network under the supervision of a first training sample in the first training sample set, and the distribution of a plurality of third training samples in the third training sample set; The conditional generative network is trained according to the third prediction loss.

7. The method according to claim 6, characterized in that The step of training the conditional generative network based on the first training sample set further includes: Inputting a plurality of first training samples in the first training sample set into a generator in a conditional generative network to obtain third image feature information of the plurality of first training samples respectively; generating a fourth prediction loss based on the third image feature information and standard feature information corresponding to the first training sample, wherein the standard feature information includes feature information obtained by extracting features from the first training sample using a trained feature extraction network; The step of training the conditional generative network according to the third prediction loss comprises: Based on the third prediction loss and the fourth prediction loss, the generator in the conditional generative network is trained until a second preset training cutoff condition is met, thereby obtaining a generator in the conditional generative network that has completed training.

8. The method according to claim 7, characterized in that The generating the fourth prediction loss based on the third image feature information and the codebook sequence information includes: Based on the third image feature information, screening out a feature vector sequence matching the first training sample from the codebook sequence information; Generating a prediction sub-loss corresponding to the first training sample based on the feature vector sequence; The fourth prediction loss is generated based on the prediction sub-loss corresponding to each first training sample.

9. The method according to claim 8, characterized in that The conditional generation network also includes a discriminator for adversarial training with the generator; The third prediction loss and the fourth prediction loss are determined based on a true / false discrimination result of the discriminator on the virtual image data and the first training sample.

10. A device for generating a neural network, characterized in that include: an information acquisition module, configured to acquire a teacher neural network trained using a first training sample set, and to acquire a second training sample set; wherein the first training sample set includes training samples corresponding to a plurality of preset image processing tasks, and the second training sample set includes training samples corresponding to a target task, wherein the plurality of preset image processing tasks include the target task; a network processing module, configured to input the second training sample set into the teacher neural network and the student neural network to be trained, process the second training samples in the second training sample set through the teacher neural network to obtain first image feature information corresponding to the second training samples, and process the second training samples in the second training sample set through the student neural network to be trained to obtain second image feature information corresponding to the second training samples; a network training module, configured to generate a first prediction loss based on the obtained first image feature information and the second image feature information, and train the student neural network to be trained based on the first prediction loss until a first preset training cutoff condition is met, thereby obtaining a trained student neural network, wherein the first prediction loss represents a prediction error of the student neural network to be trained in the target task; The second training sample set is generated by a derivative neural network for generating derivative images; the derivative neural network includes a generator in a trained conditional generative network; the conditional generative network is trained using a fourth prediction loss; the first training sample set includes multiple first training samples; The fourth predicted loss is generated in the following manner: Determining codebook sequence information based on standard feature information corresponding to the first training sample; the codebook sequence information includes a feature vector corresponding to each of a plurality of first feature points; the plurality of first feature points are selected from feature points corresponding to the plurality of first training samples, and feature vectors corresponding to different first feature points have similar distributions; generating the fourth prediction loss based on the third image feature information and the codebook sequence information; The third image feature information is generated by inputting the multiple first training samples into a generator in a conditional generation network.

11. A computer device, characterized in that: include: A processor, a memory, and a bus, wherein the memory stores machine-readable instructions executable by the processor. When the computer device is running, the processor and the memory communicate via the bus. When the machine-readable instructions are executed by the processor, the steps of the method for generating a neural network according to any one of claims 1 to 9 are performed.

12. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, executes the steps of the method for generating a neural network according to any one of claims 1 to 9.

Images