Model training method, device, equipment, storage medium and program product
By generating and training the positive and negative sample feature sets of students' models, the problem of lack of in-class information constraints in the existing technology is solved, and the knowledge distillation accuracy is improved, which is suitable for image recognition tasks of mobile devices.
Patent Information
- Application Number
- CN202110650902.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-06-11
- Publication Date
- 2024-11-22
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing knowledge distillation technology lacks inter-class information constraints during model compression, resulting in insufficient distillation accuracy and difficulty in operating effectively on mobile terminals or embedded devices.
By obtaining the sample image set, input it into the pre-trained teacher model, generating a positive and negative sample feature set, and training the student model based on these feature pairs, building a loss function to approximate the output of the teacher model, realizing knowledge of the relationship between positive and negative sample feature pairs based on the positive and negative sample feature pairs Distilled.
The accuracy of knowledge distillation is improved, and the output of the student model is approached by the teacher model, achieving the purpose of small in-class distances and large inter-class distances. It is suitable for image recognition tasks on mobile or embedded devices.
Smart Images

Figure CN113326852B_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to the field of computer vision and deep learning technology, and in particular to model training methods, devices, electronic devices, storage media, and program products. Background Art
[0002] With the widespread application of deep learning in multiple fields such as image recognition, speech recognition, and natural language processing, most models are too computationally cumbersome to run on mobile or embedded devices. Therefore, the models need to be compressed. Knowledge distillation technology is one of the important technologies in model compression.
[0003] Traditional knowledge distillation technology introduces soft targets related to the teacher model as part of the total loss to induce the training of the student model and achieve knowledge transfer. Most knowledge distillation algorithms are completed by fitting the features of the teacher model and the student model. Specifically, a teacher model is first trained on a labeled sample set, and then the features learned by the teacher model are transferred to the student model. During the transfer process, the student model learns the output of the teacher model on the one hand, and the real label of the image on the other hand. After the distillation process is completed, the student model is the image recognition model in the image recognition task. Summary of the invention
[0004] The present disclosure provides a model training method, apparatus, device, storage medium and program product and a method, apparatus, device, storage medium and program product for generating information.
[0005] According to a first aspect of the present disclosure, a model training method is provided, comprising: obtaining a sample image set, wherein the sample image set includes sample images with annotated labels; inputting sample images in the sample image set into a pre-trained first model to obtain a first feature set corresponding to the sample images in the input sample image set, wherein the first model is a trained teacher model; based on the first feature set, determining a first positive sample feature set corresponding to the first feature set and a first negative sample feature set corresponding to the first feature set, and generating sample feature pairs of the first model characterizing the first positive sample feature set and the first negative sample feature set; taking the sample images in the sample image set as input and the second feature set corresponding to the sample images in the input sample image set as output, training the second model to obtain a feature extraction model, wherein the second model is a student model to be trained, and a loss function of the second model is constructed based on the sample feature pairs of the first model and the sample feature pairs of the second model, and the sample feature pairs of the second model are used to characterize the positive sample feature sets in the second feature set and the negative sample feature sets in the second feature set.
[0006] According to a second aspect of the present disclosure, a method for generating information is provided, comprising: acquiring a target image; inputting the target image into a pre-trained feature extraction model to generate a feature set corresponding to the target image, wherein the feature extraction model is trained by a method such as any one of the embodiments of the above-mentioned model training method.
[0007] According to a third aspect of the present disclosure, a model training device is provided, comprising: an acquisition unit, configured to acquire a sample image set, wherein the sample image set includes sample images with labeled labels; an input unit, configured to input sample images in the sample image set into a pre-trained first model, and obtain a first feature set corresponding to the sample images in the input sample image set, wherein the first model is a trained teacher model; a generation unit, configured to determine, based on the first feature set, a first positive sample feature set corresponding to the first feature set and a first negative sample feature set corresponding to the first feature set, and generate sample feature pairs of the first model representing the first positive sample feature set and the first negative sample feature set; a training unit, configured to take the sample images in the sample image set as input, and take the second feature set corresponding to the sample images in the input sample image set as output, to train the second model, and obtain a feature extraction model, wherein the second model is a student model to be trained, and the loss function of the second model is constructed based on the sample feature pairs of the first model and the sample feature pairs of the second model, and the sample feature pairs of the second model are used to represent the positive sample feature set in the second feature set and the negative sample feature set in the second feature set.
[0008] According to a fourth aspect of the present disclosure, a device for generating information is provided, comprising: an image acquisition unit, configured to acquire a target image; an information generation unit, configured to input the target image into a pre-trained feature extraction model, and generate a feature set corresponding to the target image, wherein the feature extraction model is trained by a method such as any one of the embodiments of the above-mentioned model training method.
[0009] According to a fifth aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in any implementation manner of the first aspect or the second aspect.
[0010] According to a sixth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute a method as described in any implementation of the first aspect or the second aspect.
[0011] According to a seventh aspect of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the computer program implements the method described in any implementation manner of the first aspect or the second aspect.
[0012] According to the technology disclosed in the present invention, a sample image set is obtained, and the sample images in the sample image set are input into a pre-trained first model to obtain a first feature set corresponding to the sample images in the input sample image set, wherein the first model is a trained teacher model, and based on the first feature set, a first positive sample feature set corresponding to the first feature set and a first negative sample feature set corresponding to the first feature set are determined, and sample feature pairs of the first model representing the first positive sample feature set and the first negative sample feature set are generated, and the sample images in the sample image set are used as input, and the second feature set corresponding to the sample images in the input sample image set is used as output, and the second model is trained to obtain a feature extraction model, wherein the second model is a student model to be trained, and the loss function of the second model is constructed based on the sample feature pairs of the first model and the sample feature pairs of the second model, and the sample feature pairs of the second model are used to represent the positive sample feature sets in the second feature set and the negative sample feature sets in the second feature set, thereby realizing a knowledge distillation method for the relationship between positive and negative sample feature pairs and improving the distillation accuracy.
[0013] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0014] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.
[0015] Figure 1 is a schematic diagram of a first embodiment of a model training method according to the present disclosure;
[0016] Figure 2 is a scene diagram that can implement the model training method of the embodiment of the present disclosure;
[0017] Figure 3 is a schematic diagram of a second embodiment of a model training method according to the present disclosure;
[0018] Figure 4 is a schematic diagram of a first embodiment of a method for generating information according to the present disclosure;
[0019] Figure 5 is a structural schematic diagram of an embodiment of a model training device according to the present disclosure;
[0020] Figure 6is a structural schematic diagram of an embodiment of a device for generating information according to the present disclosure;
[0021] Figure 7 It is a block diagram of an electronic device used to implement the model training method of an embodiment of the present disclosure. DETAILED DESCRIPTION
[0022] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.
[0023] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present disclosure may be combined with each other. The present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0024] Figure 1 A schematic diagram 100 of a first embodiment of a model training method according to the present disclosure is shown. The model training method comprises the following steps:
[0025] Step 101: Obtain a sample image set.
[0026] In this embodiment, the execution entity (such as a server) can obtain a sample image set from other electronic devices or locally through a wired connection or a wireless connection. Among them, the sample image set may include a large number of sample images. The sample images may be images obtained by photographing targets of various categories. Generally, the sample images may be labeled sample images. It should be noted that the above-mentioned wireless connection methods may include but are not limited to 3G, 4G, 5G connections, WiFi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB (ultra wideband) connections, and other wireless connection methods currently known or to be developed in the future.
[0027] Step 102: input a sample image in the sample image set into a pre-trained first model to obtain a first feature set corresponding to the sample image in the input sample image set.
[0028] In this embodiment, for the sample images in the sample image set obtained in step 101, the execution subject can input the sample images in the sample image set into a pre-trained first model to obtain a first feature set corresponding to the sample images in the input sample image set. Among them, the first model is a trained teacher model. The teacher model can be a model obtained by pre-training any neural network that can be used for classification using a labeled sample set. Generally, the teacher model is a complex model with high accuracy and slow inference speed, and its ability is usually greater than that of the student model. The first feature set can characterize the correspondence between the labels of the sample images in the sample image set and the feature information of the corresponding sample images in the sample image set. The feature information of the image can be information used to characterize the features of the image, and the features of the image can be various basic elements of the image (such as color, shape, line, texture, etc.).
[0029] Step 103: Based on the first feature set, determine a first positive sample feature set corresponding to the first feature set and a first negative sample feature set corresponding to the first feature set, and generate sample feature pairs of a first model representing the first positive sample feature set and the first negative sample feature set.
[0030] In this embodiment, the execution subject can classify the obtained first feature set by using the sample feature division method, generate the first positive sample feature set corresponding to the first feature set representing the positive sample features and the first negative sample feature set corresponding to the first feature set representing the negative sample features, and then summarize the first positive sample feature set and the first negative sample feature set to generate sample feature pairs of the first model representing the first positive sample feature set and the first negative sample feature set. Among them, the positive sample feature can represent the features of sample images with high similarity belonging to the same category, and the negative sample feature can represent the features of sample images with low similarity that do not belong to the same category.
[0031] It should be noted that the sample feature division model can be a correspondence table pre-prepared by technicians based on a large number of statistical calculations to characterize the image feature information and the positive sample features and negative sample features; it can also be obtained by training using various existing logistic regression models (LR).
[0032] Step 104 , taking the sample images in the sample image set as input, taking the second feature set corresponding to the sample images in the input sample image set as output, training the second model, and obtaining a feature extraction model.
[0033] In this embodiment, the execution subject can use a machine learning algorithm to take a sample image in a sample image set as input, and take a second feature set corresponding to the sample image in the input sample image set as output, to train the second model and obtain a feature extraction model. Wherein, the second model is a student model to be trained, and the loss function of the second model is constructed based on the sample feature pairs of the first model and the sample feature pairs of the second model, and the sample feature pairs of the second model are used to characterize the positive sample feature set in the second feature set and the negative sample feature set in the second feature set. The number of features of the first feature set and the second feature set can be the same or different. The generation process of the positive sample feature set, the negative sample feature set and the sample feature pair of the second model is basically the same as that of the first model. The student model can be any neural network that can be used for classification, whether it is pre-trained or not. The student model is a streamlined and low-complexity model, and its ability is usually less than that of the teacher model. Here, supervised training is performed on the student model to obtain a feature extraction model. Usually, the trained student model can be directly used as a feature extraction model. When receiving an image to be recognized, the trained student model is directly used to extract features or recognize the image to be recognized, and the teacher model is no longer used, thereby improving the image recognition speed.
[0034] It should be noted that the above-mentioned execution body may store a pre-trained first model, a second model and a feature extraction model, the network architecture of each model is pre-defined, and each model may be, for example, a data table or a calculation formula, etc. This embodiment does not limit this aspect. The above-mentioned machine learning algorithm is a well-known technology that is currently widely studied and applied, and will not be described in detail here.
[0035] For ease of understanding, a scenario in which the model training method of the embodiment of the present disclosure can be implemented is provided. Figure 2, the model training method 200 of this embodiment runs in the server 201. The server 201 first obtains a sample image set 202, wherein the sample image set includes sample images with labeled labels, and then the server 201 inputs the sample images in the sample image set into the pre-trained first model to obtain a first feature set 203 corresponding to the sample images in the input sample image set, and then the server 201 determines the first positive sample feature set corresponding to the first feature set and the first negative sample feature set corresponding to the first feature set based on the first feature set, and generates a sample feature pair 204 of the first model representing the first positive sample feature set and the first negative sample feature set, and finally the server 201 takes the sample images in the sample image set as input and the second feature set corresponding to the sample images in the input sample image set as output, trains the second model, and obtains a feature extraction model 205, wherein the first model is a trained teacher model, the second model is a student model to be trained, and the loss function of the second model is constructed based on the sample feature pairs of the first model and the sample feature pairs of the second model, and the sample feature pairs of the second model are used to represent the positive sample feature set in the second feature set and the negative sample feature set in the second feature set.
[0036] The model training method provided by the above-mentioned embodiment of the present disclosure adopts the method of obtaining a sample image set, inputting the sample images in the sample image set into a pre-trained first model, and obtaining a first feature set corresponding to the sample images in the input sample image set, wherein the first model is a trained teacher model, and based on the first feature set, a first positive sample feature set corresponding to the first feature set and a first negative sample feature set corresponding to the first feature set are determined, and sample feature pairs of the first model representing the first positive sample feature set and the first negative sample feature set are generated, and the sample images in the sample image set are taken as input, and the second feature set corresponding to the sample images in the input sample image set is taken as output, and the second model is trained. The training is performed to obtain a feature extraction model, wherein the second model is the student model to be trained, and the loss function of the second model is constructed based on the sample feature pairs of the first model and the sample feature pairs of the second model. The sample feature pairs of the second model are used to characterize the positive sample feature set in the second feature set and the negative sample feature set in the second feature set, which solves the problem of lack of intra-class and inter-class information constraints in the existing distillation technology, and realizes a knowledge distillation method for the relationship between positive and negative sample feature pairs, so that the relationship between the positive and negative sample feature pairs output by the student model is as close as possible to the relationship between the positive and negative sample feature pairs output by the teacher model, so as to achieve the purpose of small enough intra-class distance and large enough inter-class distance, thereby improving the distillation accuracy.
[0037] Further references Figure 3 , which shows a schematic diagram 300 of a second embodiment of a model training method. The process of the method includes the following steps:
[0038] Step 301: Obtain a sample image set.
[0039] Step 302: input a sample image in the sample image set into a pre-trained first model to obtain a first feature set corresponding to the sample image in the input sample image set.
[0040] Step 303 , based on the first feature set, sample images in the input sample image set are divided into samples to obtain a first positive sample set corresponding to the sample images in the sample image set and a first negative sample set corresponding to the sample images in the sample image set.
[0041] In this embodiment, the execution subject can analyze the first feature set, and divide the sample images in the input sample image set based on the analysis result to obtain a first positive sample set representing positive samples corresponding to the sample images in the sample image set and a first negative sample set representing negative samples corresponding to the sample images in the sample image set. Among them, the positive sample can represent samples belonging to a certain category, and the negative sample can represent samples that do not belong to a certain category. For example, if the image recognition of the letter A is to be performed, the sample of the letter A is a positive sample, and the sample that is not the letter A is a negative sample.
[0042] In some optional implementations of the present embodiment, based on the first feature set, sample images in the input sample image set are divided into samples to obtain a first positive sample set corresponding to the sample images in the sample image set and a first negative sample set corresponding to the sample images in the sample image set, including: analyzing the first feature set to generate a similarity matrix corresponding to the first feature set, the similarity matrix represents the similarity of each sample image in the input sample image set; according to the similarity matrix and the similarity threshold, sample images in the input sample image set are divided into samples to obtain a first positive sample set corresponding to the sample images in the sample image set and a first negative sample set corresponding to the sample images in the sample image set, the positive sample set represents samples whose similarity values in the similarity matrix are greater than the similarity threshold, and the negative sample set represents samples whose similarity values in the similarity matrix are not greater than the similarity threshold. An efficient and accurate positive and negative sample division method is realized.
[0043] Step 304: Select a first feature set according to the first positive sample set and the first negative sample set to obtain a first positive sample feature set corresponding to the first positive sample set and a first negative sample feature set corresponding to the first negative sample set.
[0044] In this embodiment, the execution subject may select the first feature set according to the first positive sample set and the first negative sample set obtained in step 303, and obtain the first positive sample feature set corresponding to the first positive sample set and the first negative sample feature set corresponding to the first negative sample set. The first positive sample feature set is a set of features of each positive sample, and the first negative sample feature set is a set of features of each negative sample.
[0045] Step 305 , merging the first positive sample feature set and the first negative sample feature set to generate sample feature pairs of a first model that characterize the first positive sample feature set and the first negative sample feature set.
[0046] In this embodiment, the execution entity may merge the first positive sample feature set and the first negative sample feature set according to the sample feature pair generation rule to generate a sample feature pair of the first model representing the first positive sample feature set and the first negative sample feature set.
[0047] Step 306: input the sample images in the sample image set into the second model to obtain a second feature set corresponding to the sample images in the input sample image set.
[0048] In this embodiment, the execution entity may input a sample image in the sample image set into the second model to obtain a second feature set corresponding to the input sample image in the sample image set.
[0049] Step 307: Based on the second feature set, determine a second positive sample feature set corresponding to the second feature set and a second negative sample feature set corresponding to the second feature set, and generate sample feature pairs of a second model representing the second positive sample feature set and the second negative sample feature set.
[0050] In this embodiment, the execution entity can classify the second feature set by using the sample feature division method, generate a second positive sample feature set corresponding to the second feature set that represents the positive sample features and a second negative sample feature set corresponding to the second feature set that represents the negative sample features, and then aggregate the second positive sample feature set and the second negative sample feature set to generate a sample feature pair of the second model that represents the second positive sample feature set and the second negative sample feature set.
[0051] In some optional implementations of the present embodiment, based on the second feature set, a second positive sample feature set corresponding to the second feature set and a second negative sample feature set corresponding to the second feature set are determined, and a sample feature pair of a second model representing the second positive sample feature set and the second negative sample feature set is generated, including: based on the second feature set, sample images in the input sample image set are divided into samples to obtain a second positive sample set corresponding to the sample images in the sample image set and a second negative sample set corresponding to the sample images in the sample image set; according to the second positive sample set and the second negative sample set, the second feature set is selected to obtain a second positive sample feature set corresponding to the second positive sample set and a second negative sample feature set corresponding to the second negative sample set; the second positive sample feature set and the second negative sample feature set are merged to generate a sample feature pair of a second model representing the second positive sample feature set and the second negative sample feature set. The generation of sample feature pairs is completed based on sample division, and another method for dividing positive and negative sample feature pairs is implemented as data support for subsequent model training.
[0052] In some optional implementations of this embodiment, based on the second feature set, sample images in the input sample image set are divided into samples to obtain a second positive sample set corresponding to the sample images in the sample image set and a second negative sample set corresponding to the sample images in the sample image set, including: analyzing the second feature set to generate a similarity matrix corresponding to the second feature set; and dividing the sample images in the input sample image set according to the similarity matrix and similarity threshold corresponding to the second feature set to obtain a second positive sample set corresponding to the sample images in the sample image set and a second negative sample set corresponding to the sample images in the sample image set. An efficient and accurate positive and negative sample division method is implemented, and the calculation of the loss function is accurate and reasonable by keeping consistent with the sample feature pairs of the first model.
[0053] Step 308: Calculate a loss function based on the sample feature pairs of the first model and the sample feature pairs of the second model.
[0054] In this embodiment, the execution entity can calculate the loss function based on the sample feature pairs of the first model generated in step 305 and the sample feature pairs of the second model generated in step 307. The loss function aims to make the distance of negative samples as far as possible and the distance of positive samples as close as possible, and to make the distance between the positive samples and negative samples of the second model close to the distance between the positive samples and negative samples of the first model. Here, the formula of the loss function can be expressed as: Among them, m is a constant, and e is a constant ≈ 2.71828. The model training is to make the relationship between the positive and negative sample feature pairs output by the student model as close as possible to the relationship between the positive and negative sample feature pairs output by the teacher model, so as to achieve the goal of small enough intra-class distance and large enough inter-class distance.
[0055] Step 309, back-propagating the loss function based on gradient descent, updating the parameters of the second model, and obtaining a feature extraction model.
[0056] In this embodiment, the execution entity can back-propagate the loss function based on gradient descent, update the parameters of the second model, and obtain the feature extraction model.
[0057] Typically, for each round of iterative training, the execution subject may first determine whether the loss function calculated in the iterative training is minimized. If the loss function is minimized, it means that the student model has converged, and the student model at this time can be used as an image recognition model. If the loss function is not minimized, it means that the student model has not yet converged. At this time, the loss function is back-propagated based on gradient descent, the parameters of the student model are updated, and the next round of iterative training is continued. Among them, the more rounds of iterative training, the higher the accuracy of the feature extraction model obtained by training.
[0058] In some optional implementations of this embodiment, when training the student model, the loss function used may be a cross entropy loss function, the learning rate used may be a cosine descent strategy, and the optimizer used may be a momentum gradient descent optimizer. When training the student model, the cross entropy loss function is used. The cross entropy loss function does not involve other hyperparameters (such as temperature parameters), and there is no need to spend manpower and material resources to adjust parameters, making iteration faster and easier.
[0059] In some optional implementations of this embodiment, the method further includes: performing image preprocessing on sample images in the sample image set, and adding the processed sample images to the sample image set, wherein the image preprocessing includes at least one of the following: image inversion, image compression, and contrast stretching. By preprocessing the image, irrelevant information in the image is eliminated, useful real information is restored, the detectability of relevant information is enhanced, and data is simplified to the maximum extent; by adding the processed sample images to the sample image set, the sample data can be expanded and the accuracy of model training can be improved.
[0060] In this embodiment, the specific operations of steps 301 and 302 are the same as Figure 1 The operations of steps 101 and 102 in the illustrated embodiment are substantially the same and will not be described in detail herein.
[0061] from Figure 3 It can be seen that Figure 1 Compared with the corresponding embodiments, the schematic diagram 300 of the model training method in this embodiment divides the training samples to obtain the positive and negative sample feature pairs after division, and calculates the loss function based on the sample feature pairs of the first model and the sample feature pairs of the second model. The loss function aims to make the distance between the negative samples as far as possible and the distance between the positive samples as close as possible, and to make the distance between the positive and negative samples of the second model close to the distance between the positive and negative samples of the first model. The loss function is back-propagated based on gradient descent, and the parameters of the second model are updated to obtain a feature extraction model, thereby realizing another knowledge distillation method for the relationship between positive and negative sample feature pairs, so that the relationship between the positive and negative sample feature pairs output by the student model is as close as possible to the relationship between the positive and negative sample feature pairs output by the teacher model, so as to achieve the purpose of small enough intra-class distance and large enough inter-class distance, thereby improving the distillation accuracy.
[0062] Further references Figure 4 , which shows a schematic diagram 400 of a first embodiment of a method for generating information according to the present disclosure. The method for generating information comprises the following steps:
[0063] Step 401, acquiring a target image.
[0064] In this embodiment, the execution subject (such as a server or a terminal device) can obtain the target image from other electronic devices or locally through a wired connection or a wireless connection.
[0065] Step 402: input the target image into a pre-trained feature extraction model to generate a feature set corresponding to the target image.
[0066] In this embodiment, the execution subject can input the target image acquired in step 401 into a pre-trained feature extraction model to generate a feature set corresponding to the target image. The feature extraction model is trained by a method such as any embodiment of the above-mentioned model training method.
[0067] from Figure 4 It can be seen that Figure 1 Compared with the corresponding embodiment, the process 400 of the method for generating information in this embodiment highlights the step of using the trained feature extraction model to generate a feature set corresponding to the target image. Therefore, the solution described in this embodiment can use a more accurate model to achieve targeted feature extraction of different types, levels, and depths.
[0068] Further references Figure 5 , as a response to the above Figures 1 to 3 The present disclosure provides an embodiment of a model training device, and the device embodiment is Figure 1 Corresponding to the method embodiment shown in FIG. 1 , in addition to the features described below, the device embodiment may also include Figure 1 The same or corresponding features of the method embodiment shown, and the generation of Figure 1 The method embodiment shown has the same or corresponding effects, and the device can be specifically applied to various electronic devices.
[0069] like Figure 5As shown, the model training device 500 of this embodiment includes: an acquisition unit 501, an input unit 502, a generation unit 503 and a training unit 504, wherein the acquisition unit is configured to acquire a sample image set, wherein the sample image set includes sample images with labeled labels; the input unit is configured to input the sample images in the sample image set into a pre-trained first model to obtain a first feature set corresponding to the sample images in the input sample image set, wherein the first model is a trained teacher model; the generation unit is configured to determine, based on the first feature set, a first positive sample feature set corresponding to the first feature set and a first feature set pair A first negative sample feature set corresponding to the first model is used to generate sample feature pairs of the first model that characterize the first positive sample feature set and the first negative sample feature set; a training unit is configured to take a sample image in the sample image set as input, and take a second feature set corresponding to the sample image in the input sample image set as output, train the second model to obtain a feature extraction model, wherein the second model is a student model to be trained, the loss function of the second model is constructed based on the sample feature pairs of the first model and the sample feature pairs of the second model, and the sample feature pairs of the second model are used to characterize the positive sample feature set in the second feature set and the negative sample feature set in the second feature set.
[0070] In this embodiment, the specific processing of the acquisition unit 501, the input unit 502, the generation unit 503 and the training unit 504 of the model training device 500 and the technical effects thereof can be referred to respectively. Figure 1 The relevant descriptions of steps 101 to 104 in the corresponding embodiment are not repeated here.
[0071] In some optional implementations of the present embodiment, the generation unit includes: a division module, configured to perform sample division on the sample images in the input sample image set based on the first feature set, to obtain a first positive sample set corresponding to the sample images in the sample image set and a first negative sample set corresponding to the sample images in the sample image set; a selection module, configured to select the first feature set according to the first positive sample set and the first negative sample set, to obtain a first positive sample feature set corresponding to the first positive sample set and a first negative sample feature set corresponding to the first negative sample set; and a merging module, configured to merge the first positive sample feature set and the first negative sample feature set to generate a sample feature pair of a first model that characterizes the first positive sample feature set and the first negative sample feature set.
[0072] In some optional implementations of the present embodiment, the division module is further configured to: analyze the first feature set to generate a similarity matrix corresponding to the first feature set; perform sample division on the sample images in the input sample image set according to the similarity matrix and the similarity threshold to obtain a first positive sample set corresponding to the sample images in the sample image set and a first negative sample set corresponding to the sample images in the sample image set, the positive sample set representing samples whose similarity values in the similarity matrix are greater than the similarity threshold, and the negative sample set representing samples whose similarity values in the similarity matrix are not greater than the similarity threshold.
[0073] In some optional implementations of the present embodiment, the training unit includes: an input module, configured to input a sample image in a sample image set into a second model to obtain a second feature set corresponding to the sample image in the input sample image set; a generation module, configured to determine, based on the second feature set, a second positive sample feature set corresponding to the second feature set and a second negative sample feature set corresponding to the second feature set, and generate sample feature pairs of the second model that characterize the second positive sample feature set and the second negative sample feature set; a calculation module, configured to calculate a loss function based on the sample feature pairs of the first model and the sample feature pairs of the second model; and an update module, configured to back-propagate the loss function based on gradient descent to update the parameters of the second model.
[0074] In some optional implementations of the present embodiment, the generation module includes: a division submodule, configured to perform sample division on the sample images in the input sample image set based on the second feature set, to obtain a second positive sample set corresponding to the sample images in the sample image set and a second negative sample set corresponding to the sample images in the sample image set; a selection submodule, configured to select the second feature set according to the second positive sample set and the second negative sample set, to obtain a second positive sample feature set corresponding to the second positive sample set and a second negative sample feature set corresponding to the second negative sample set; and a merging submodule, configured to merge the second positive sample feature set and the second negative sample feature set to generate a sample feature pair of a second model representing the second positive sample feature set and the second negative sample feature set.
[0075] In some optional implementations of the present embodiment, the division submodule is further configured to: analyze the second feature set to generate a similarity matrix corresponding to the second feature set; perform sample division on the sample images in the input sample image set according to the similarity matrix and the similarity threshold corresponding to the second feature set to obtain a second positive sample set corresponding to the sample images in the sample image set and a second negative sample set corresponding to the sample images in the sample image set.
[0076] In some optional implementations of this embodiment, the loss function aims to make the distance between the positive samples and the negative samples of the second model close to the distance between the positive samples and the negative samples of the first model.
[0077] In some optional implementations of this embodiment, the device also includes: a preprocessing unit, configured to perform image preprocessing on sample images in the sample image set, and add the processed sample images to the sample image set, wherein the image preprocessing includes at least one of the following: image inversion, image compression, and contrast stretching.
[0078] The above-mentioned embodiment of the present disclosure provides a model training device, which obtains a sample image set through an acquisition unit, and the input unit inputs the sample images in the sample image set into a pre-trained first model to obtain a first feature set corresponding to the sample images in the input sample image set. The generation unit determines, based on the first feature set, a first positive sample feature set corresponding to the first feature set and a first negative sample feature set corresponding to the first feature set, and generates sample feature pairs of the first model that characterize the first positive sample feature set and the first negative sample feature set. The training unit takes the sample images in the sample image set as input and the second feature set corresponding to the sample images in the input sample image set as output, trains the second model to obtain a feature extraction model, wherein the second model is a student model to be trained, and the loss function of the second model is constructed based on the sample feature pairs of the first model and the sample feature pairs of the second model, and the sample feature pairs of the second model are used to characterize the positive sample feature sets in the second feature set and the negative sample feature sets in the second feature set, which enriches the model training method based on the knowledge distillation technology, helps to achieve the purpose of small enough intra-class distance and large enough inter-class distance, and improves the distillation accuracy.
[0079] Continue to refer Figure 6 , as a response to the above Figure 4 The present disclosure provides an embodiment of a device for generating information. Figure 4 Corresponding to the method embodiment shown in FIG. 1 , in addition to the features described below, the device embodiment may also include Figure 4 The same or corresponding features of the method embodiment shown, and the generation of Figure 4 The method embodiment shown has the same or corresponding effects, and the device can be specifically applied to various electronic devices.
[0080] like Figure 6 As shown, the device 600 for generating information in this embodiment includes: an image acquisition unit 601 and an information generation unit 602, wherein the image acquisition unit is configured to acquire a target image; the information generation unit is configured to input the target image into a pre-trained feature extraction model to generate a feature set corresponding to the target image, wherein the feature extraction model is trained by a method such as any embodiment of the above-mentioned model training method.
[0081] In this embodiment, the specific processing of the image acquisition unit 601 and the information generation unit 602 of the device for generating information 600 and the technical effects thereof can be referred to in Figure 4 The relevant descriptions of step 401 to step 402 in the corresponding embodiment are not repeated here.
[0082] It is pointed out here that if the technical solution of the present disclosure involves the acquisition, storage and application of user personal information, it shall comply with the provisions of relevant laws and regulations and shall not violate public order and good morals.
[0083] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.
[0084] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.
[0085] like Figure 7 As shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.
[0086] A number of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a disk, an optical disk, etc.; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0087] The computing unit 701 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 701 performs the various methods and processes described above, such as method XXX. For example, in some embodiments, method XXX may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 708. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 700 via ROM 702 and / or communication unit 709. When the computer program is loaded into RAM 703 and executed by the computing unit 701, one or more steps of the method XXX described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform method XXX in any other appropriate manner (e.g., by means of firmware).
[0088] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0089] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0090] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
[0091] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0092] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.
[0093] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.
[0094] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.
[0095] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.
Claims
1. A model training method, comprising: Acquire a sample image set, wherein the sample image set includes sample images with labeled labels; Inputting a sample image in the sample image set into a pre-trained first model to obtain a first feature set corresponding to the input sample image in the sample image set, wherein the first model is a trained teacher model; Based on the first feature set, sample images in the sample image set are divided into samples to obtain a first positive sample set and a first negative sample set corresponding to the sample images in the sample set; based on the first positive sample set and the first negative sample set, the first feature set is selected to obtain a first positive sample feature set corresponding to the first positive sample set and a first negative sample feature set corresponding to the first negative sample set; the first positive sample feature set and the first negative sample feature set are combined to generate sample feature pairs of the first model; A sample image in the sample image set is taken as input, and a second feature set corresponding to the input sample image in the sample image set is taken as output, and the second model is trained to obtain a feature extraction model, wherein the second model is a student model to be trained, and the loss function of the second model is constructed based on the sample feature pairs of the first model and the sample feature pairs of the second model, and the sample feature pairs of the second model are used to characterize the positive sample feature set in the second feature set and the negative sample feature set in the second feature set.
2. The method according to claim 1, wherein: The step of performing sample division on the sample images in the input sample image set based on the first feature set to obtain a first positive sample set and a first negative sample set corresponding to the sample images in the sample image set includes: Analyzing the first feature set to generate a similarity matrix corresponding to the first feature set; According to the similarity matrix and the similarity threshold, sample division is performed on the sample images in the input sample image set to obtain a first positive sample set corresponding to the sample images in the sample image set and a first negative sample set corresponding to the sample images in the sample image set, wherein the positive sample set represents samples whose similarity values in the similarity matrix are greater than the similarity threshold, and the negative sample set represents samples whose similarity values in the similarity matrix are not greater than the similarity threshold.
3. The method according to claim 1 or 2, wherein: The step of taking the sample image in the sample image set as input, taking the second feature set corresponding to the sample image in the sample image set as output, and training the second model comprises: Inputting a sample image in the sample image set into the second model to obtain a second feature set corresponding to the input sample image in the sample image set; Based on the second feature set, determining a second positive sample feature set corresponding to the second feature set and a second negative sample feature set corresponding to the second feature set, and generating a sample feature pair of the second model representing the second positive sample feature set and the second negative sample feature set; Calculating a loss function based on the sample feature pairs of the first model and the sample feature pairs of the second model; The loss function is back-propagated based on gradient descent to update the parameters of the second model.
4. The method according to claim 3, wherein: The method of determining, based on the second feature set, a second positive sample feature set corresponding to the second feature set and a second negative sample feature set corresponding to the second feature set, and generating a sample feature pair of the second model representing the second positive sample feature set and the second negative sample feature set, comprises: Based on the second feature set, sample images in the input sample image set are divided into samples to obtain a second positive sample set corresponding to the sample images in the sample image set and a second negative sample set corresponding to the sample images in the sample image set; Selecting the second feature set according to the second positive sample set and the second negative sample set to obtain a second positive sample feature set corresponding to the second positive sample set and a second negative sample feature set corresponding to the second negative sample set; The second positive sample feature set and the second negative sample feature set are combined to generate sample feature pairs of the second model that characterize the second positive sample feature set and the second negative sample feature set.
5. The method according to claim 4, wherein: The method of performing sample division on the sample images in the input sample image set based on the second feature set to obtain a second positive sample set corresponding to the sample images in the sample image set and a second negative sample set corresponding to the sample images in the sample image set includes: Analyze the second feature set to generate a similarity matrix corresponding to the second feature set; According to the similarity matrix and similarity threshold corresponding to the second feature set, sample images in the input sample image set are divided into samples to obtain a second positive sample set corresponding to the sample images in the sample image set and a second negative sample set corresponding to the sample images in the sample image set.
6. The method according to claim 1, wherein: The loss function aims to make the distance between the positive samples and the negative samples of the second model close to the distance between the positive samples and the negative samples of the first model.
7. The method according to any one of claims 1 to 6, further comprising: Perform image preprocessing on the sample images in the sample image set, and add the processed sample images to the sample image set, wherein the image preprocessing includes at least one of the following: image inversion, image compression, and contrast stretching.
8. A method for generating information, comprising: Get the target image; The target image is input into a pre-trained feature extraction model to generate a feature set corresponding to the target image, wherein the feature extraction model is trained by the method according to any one of claims 1-7.
9. A model training device, comprising: An acquisition unit is configured to acquire a sample image set, wherein the sample image set includes sample images with labeled labels; An input unit is configured to input a sample image in the sample image set into a pre-trained first model to obtain a first feature set corresponding to the input sample image in the sample image set, wherein the first model is a trained teacher model; The generating unit includes: a dividing module configured to perform sample division on the sample images in the sample image set based on the first feature set to obtain a first positive sample set and a first negative sample set corresponding to the sample images in the sample set; a selecting module configured to select the first feature set according to the first positive sample set and the first negative sample set to obtain a first positive sample feature set corresponding to the first positive sample set and a first negative sample feature set corresponding to the first negative sample set; and a merging module configured to merge the first positive sample feature set and the first negative sample feature set to generate a sample feature pair of the first model; A training unit is configured to take a sample image in the sample image set as input, take a second feature set corresponding to the input sample image in the sample image set as output, train the second model to obtain a feature extraction model, wherein the second model is a student model to be trained, and the loss function of the second model is constructed based on the sample feature pairs of the first model and the sample feature pairs of the second model, and the sample feature pairs of the second model are used to characterize the positive sample feature set in the second feature set and the negative sample feature set in the second feature set.
10. The device according to claim 9, wherein: The partitioning module is further configured to: Analyzing the first feature set to generate a similarity matrix corresponding to the first feature set; According to the similarity matrix and the similarity threshold, sample division is performed on the sample images in the input sample image set to obtain a first positive sample set and a first negative sample set corresponding to the sample images in the sample image set, wherein the positive sample set represents samples in the similarity matrix whose similarity values are greater than the similarity threshold, and the negative sample set represents samples in the similarity matrix whose similarity values are not greater than the similarity threshold.
11. The device according to claim 9 or 10, wherein: The training unit comprises: An input module, configured to input a sample image in the sample image set into the second model to obtain a second feature set corresponding to the input sample image in the sample image set; A generating module is configured to determine, based on the second feature set, a second positive sample feature set corresponding to the second feature set and a second negative sample feature set corresponding to the second feature set, and generate a sample feature pair of the second model representing the second positive sample feature set and the second negative sample feature set; A calculation module, configured to calculate a loss function based on the sample feature pairs of the first model and the sample feature pairs of the second model; An updating module is configured to back-propagate the loss function based on gradient descent to update the parameters of the second model.
12. The device according to claim 11, wherein The generating module comprises: a division submodule, configured to perform sample division on the sample images in the input sample image set based on the second feature set, to obtain a second positive sample set corresponding to the sample images in the sample image set and a second negative sample set corresponding to the sample images in the sample image set; a selection submodule, configured to select the second feature set according to the second positive sample set and the second negative sample set, to obtain a second positive sample feature set corresponding to the second positive sample set and a second negative sample feature set corresponding to the second negative sample set; The merging submodule is configured to merge the second positive sample feature set and the second negative sample feature set to generate sample feature pairs of the second model that characterize the second positive sample feature set and the second negative sample feature set.
13. The device according to claim 12, wherein: The partitioning submodule is further configured to: Analyze the second feature set to generate a similarity matrix corresponding to the second feature set; According to the similarity matrix and similarity threshold corresponding to the second feature set, sample images in the input sample image set are divided into samples to obtain a second positive sample set corresponding to the sample images in the sample image set and a second negative sample set corresponding to the sample images in the sample image set.
14. The device according to claim 9, wherein: The loss function aims to make the distance between the positive samples and the negative samples of the second model close to the distance between the positive samples and the negative samples of the first model.
15. The device according to any one of claims 9 to 14, further comprising: The preprocessing unit is configured to perform image preprocessing on the sample images in the sample image set and add the processed sample images to the sample image set, wherein the image preprocessing includes at least one of the following: image inversion, image compression, and contrast stretching.
16. A device for generating information, comprising: An image acquisition unit, configured to acquire a target image; The information generating unit is configured to input the target image into a pre-trained feature extraction model to generate a feature set corresponding to the target image, wherein the feature extraction model is trained by the method according to any one of claims 1-7.
17. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.
18. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-8.
19. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 8.
