Distillation learning method, device, computer equipment and storage medium

Through channel matching and parameter adjustment, the problem of large differences in the performance of student networks and teachers in traditional distillation learning is solved, and the accuracy of distillation learning effect and semantic segmentation is improved.

CN111898735BActive Publication Date: 2024-11-22刘丽
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202010674185.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2020-07-14
Publication Date
2024-11-22
Estimated Expiration
2040-07-14

AI Technical Summary

Technical Problem

In traditional distillation learning methods, the performance differences between student networks and teacher networks are large, resulting in poor distillation learning results.

Method used

By entering the target image into the teacher network and student network, obtaining the teacher feature map and student feature map, and performing channel matching, calculating the target loss value based on the matching results, and adjusting the student network parameters until the loss value is less than the preset value, so as to realize the teacher network Knowledge transfer to student networks.

Benefits of technology

It improves the distillation learning effect of student network on teacher networks, reduces the performance differences between student networks and teacher networks, and improves the accuracy of semantic segmentation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN111898735B_ABST
    Figure CN111898735B_ABST
Patent Text Reader

Abstract

The present application relates to a distillation learning method, device, computer equipment and storage medium. The method comprises: inputting a target image into a teacher network and a student network respectively, obtaining a teacher feature map output by the teacher network and a student feature map output by the student network; performing channel matching on the teacher feature map and the student feature map, obtaining a target loss value between the teacher feature map and the student feature map according to the matching result; adjusting the parameters in the student network according to the target loss value, and obtaining a target student network. The use of this method can improve the distillation learning effect of the student network on the teacher network and reduce the performance difference between the student network and the teacher network.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of machine learning, and in particular to a distillation learning method, apparatus, computer device and storage medium. Background Art

[0002] With the development of machine learning technology, distillation learning technology has emerged. Distillation learning adopts transfer learning, which uses the output of a pre-trained complex network model (teacher network, Teacher model) as a supervision signal to train another simple network model (student network, Student model) to obtain a student network with a simple structure and low computational complexity, and which has the knowledge of the teacher network.

[0003] In traditional technologies, most of them are carried out from three aspects: pixel-level distillation, relationship pair distillation, and discriminator-based global distillation. However, the distillation learning effect is poor, and the performance of the obtained student network is very different from that of the teacher network. Summary of the invention

[0004] Based on this, it is necessary to provide a distillation learning method, device, computer equipment and storage medium to address the above technical problems.

[0005] A distillation learning method, the method comprising:

[0006] Inputting the target image into the teacher network and the student network respectively, obtaining a teacher feature map output by the teacher network and a student feature map output by the student network;

[0007] Perform channel matching on the teacher feature map and the student feature map, and obtain a target loss value between the teacher feature map and the student feature map according to the matching result;

[0008] The parameters in the student network are adjusted according to the target loss value to obtain a target student network.

[0009] In one embodiment, performing channel matching on the teacher feature map and the student feature map includes:

[0010] Numbering the channels of the teacher characteristic graph and the student characteristic graph according to the channel type to obtain a teacher channel number and a student channel number; wherein the teacher channel number and the student channel number of the same channel type are the same;

[0011] The teacher channel number and the student channel number are traversed to match the numbers of the channels of the teacher feature map and the channels of the student feature map.

[0012] In one embodiment, the traversing the teacher channel number and the student channel number to match the number of the channel corresponding to the teacher feature map and the channel of the student feature map includes:

[0013] According to the order of the teacher channel numbers from small to large, the channels of each teacher characteristic map are numbered and matched with the channels of each student characteristic map in sequence according to the order of the student channel numbers from small to large.

[0014] In one embodiment, before obtaining the target loss value between the teacher feature map and the student feature map according to the matching result, the method includes:

[0015] Performing a softmax calculation according to the teacher feature map to obtain a first probability distribution;

[0016] Performing a softmax calculation according to the student feature map to obtain a second probability distribution;

[0017] Accordingly, the calculating the target loss value between the teacher feature map and the student feature map according to the matching result includes:

[0018] The target loss value is calculated according to the matching result, the first probability distribution, and the second probability distribution.

[0019] In one embodiment, calculating the target loss value according to the matching result, the first probability distribution, and the second probability distribution includes:

[0020] Obtain a first probability distribution of a channel-matched teacher feature map and a second probability distribution of a student feature map;

[0021] Calculate a first KL divergence between the teacher feature map and the student feature map according to the probability values ​​on the first probability distribution and the second probability distribution of the channel matching, and take the positive value of the first KL divergence as the first loss value;

[0022] Obtain a first probability distribution of a channel-mismatched teacher feature map and a second probability distribution of a student feature map;

[0023] Calculate the second KL divergence between the teacher feature map and the student feature map according to the probability values ​​on the first probability distribution and the second probability distribution of channel mismatch, and take the negative value of the second KL divergence as the second loss value;

[0024] The target loss value is obtained according to the first loss value and the second loss value.

[0025] In one embodiment, obtaining the target loss value according to the first loss value and the second loss value includes:

[0026] The first loss value is added to the second loss value to obtain the target loss value.

[0027] In one embodiment, adjusting the parameters in the student network according to the target loss value to obtain the target student network includes:

[0028] The parameters in the student network are adjusted according to the target loss value until the target loss value obtained by the student network after the parameter adjustment is less than the preset loss value, and the student network after the parameter adjustment is used as the target student network.

[0029] A distillation learning device, comprising:

[0030] A feature output module, used to input the target image into the teacher network and the student network respectively, to obtain a teacher feature map output by the teacher network and a student feature map output by the student network;

[0031] A channel matching module, used for performing channel matching on the teacher feature map and the student feature map, and obtaining a target loss value between the teacher feature map and the student feature map according to the matching result;

[0032] A parameter adjustment module is used to adjust the parameters in the student network according to the target loss value to obtain a target student network.

[0033] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0034] Inputting the target image into the teacher network and the student network respectively, obtaining a teacher feature map output by the teacher network and a student feature map output by the student network;

[0035] Perform channel matching on the teacher feature map and the student feature map, and obtain a target loss value between the teacher feature map and the student feature map according to the matching result;

[0036] The parameters in the student network are adjusted according to the target loss value to obtain a target student network.

[0037] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0038] Inputting the target image into the teacher network and the student network respectively, obtaining a teacher feature map output by the teacher network and a student feature map output by the student network;

[0039] Perform channel matching on the teacher feature map and the student feature map, and obtain a target loss value between the teacher feature map and the student feature map according to the matching result;

[0040] The parameters in the student network are adjusted according to the target loss value to obtain a target student network.

[0041] The above-mentioned distillation learning method, device, computer device and storage medium obtain the teacher feature map output by different channels in the teacher network and the student feature map output by different channels in the student network by inputting the target image into the teacher network and the student network respectively. Since the response distribution on the teacher feature map and the student feature map is consistent with the target distribution of the corresponding semantics in the input target image, the channel-matched teacher feature map can be directly used to train the student network to achieve the knowledge transfer of the teacher network to the student network, improve the distillation learning effect of the student network on the teacher network, and reduce the performance difference between the student network and the teacher network. The computer device specifically matches the teacher feature map and the student feature map through channels, obtains the target loss value between the teacher feature map and the student feature map according to the matching result; and adjusts the parameters in the student network according to the target loss value to obtain the target student network. The response distribution on the teacher feature map and the student feature map can intuitively display the structural information on the target image. The high response corresponds to the semantic target area extracted by the current channel, that is, the foreground area, and the low response corresponds to other targets (other semantic targets and background), that is, the background area. The success of knowledge distillation from the teacher network to the student network requires ensuring that the student network learns more foreground knowledge and less background knowledge. The channel-level teacher feature map in the teacher network is used to align the student feature map with the corresponding distribution of the foreground and background. During the alignment process, the parameters in the student network are continuously adjusted to reduce the performance difference between the student network and the teacher network, so as to obtain a target learning network that has learned the superior performance of the teacher network, thereby improving the accuracy of semantic segmentation of the target student network. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 Schematic diagram of a flow chart of a distillation learning method in one embodiment;

[0043] Figure 2 A schematic diagram of a process of channel matching in one embodiment;

[0044] Figure 3 A schematic diagram of a process for obtaining a target loss value in one embodiment;

[0045] Figure 4 A schematic diagram of a process for obtaining a target loss value in another embodiment;

[0046] Figure 5 is a flow chart of a distillation learning method in another embodiment;

[0047] Figure 6 A schematic diagram of an application of a distillation learning method in one embodiment;

[0048] Figure 7 is a structural block diagram of a distillation learning device in one embodiment;

[0049] Figure 8 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0051] In one embodiment, Figure 1 As shown, a distillation learning method is provided. This embodiment uses the method applied to a terminal as an example. It can be understood that the method can also be applied to a server, and can also be applied to a system including a terminal and a server, and is implemented through the interaction between the terminal and the server. Among them, the terminal can be but is not limited to various personal computers, laptops, smart phones, tablet computers and portable wearable devices, and the server can be implemented with an independent server or a server cluster composed of multiple servers. In this embodiment, the method includes the following steps:

[0052] S110, inputting the target image into the teacher network and the student network respectively, obtaining a teacher feature map output by the teacher network and a student feature map output by the student network.

[0053] Among them, the teacher network uses resnet101-pspnet as the initial network, and the student network uses resnet18-pspnet as the initial network. Mainstream datasets such as cityscape or Pascal Voc are used as training samples, and the network models for semantic segmentation are obtained by inputting the above initial networks for training.

[0054] Specifically, the computer device inputs the target image into the teacher network to obtain the final teacher feature map output by the teacher network for semantic segmentation, and inputs the target image into the student network to obtain the final student feature map output by the student network for semantic segmentation. The computer device extracts a two-dimensional slice of each channel from the final teacher feature map as the teacher feature map obtained by the corresponding channel, and similarly extracts a two-dimensional slice of each channel from the final student feature map as the student feature map obtained by the corresponding channel.

[0055] S120, performing channel matching on the teacher feature map and the student feature map, and obtaining a target loss value between the teacher feature map and the student feature map according to the matching result.

[0056] Among them, the target loss value can be used to characterize the overall difference between the teacher feature map and the student feature map obtained in each channel between the teacher network and the student network. The larger the target loss value, the greater the overall difference between the teacher feature map and the student feature map.

[0057] Specifically, the computer device matches the channel type of the teacher feature map and the student feature map, and obtains the target loss value between the teacher feature map and the student feature map according to the matching result. For example, the loss value between the teacher feature map and the student feature map with matching channel types and the loss value between the teacher feature map and the student feature map with mismatched channel types are calculated, and then the overall target loss value between the teacher feature map and the student feature map is determined based on the above two loss values.

[0058] S130. Adjust the parameters in the student network according to the target loss value to obtain the target student network.

[0059] Specifically, the computer device adjusts the parameters in the student network according to the obtained target loss value, so that the target loss value obtained by the student network after parameter adjustment continues to decrease, so as to obtain the target student network.

[0060] In this embodiment, the computer device inputs the target image into the teacher network and the student network respectively, and obtains the teacher feature map output by different channels in the teacher network and the student feature map output by different channels in the student network. Since the response distribution on the teacher feature map and the student feature map is consistent with the target distribution of the corresponding semantics in the input target image, the channel-matched teacher feature map can be directly used to train the student network to achieve knowledge transfer from the teacher network to the student network, improve the distillation learning effect of the student network on the teacher network, and reduce the performance difference between the student network and the teacher network.

[0061] The computer device specifically matches the teacher feature map and the student feature map through channels, obtains the target loss value between the teacher feature map and the student feature map according to the matching result, and adjusts the parameters in the student network according to the target loss value to obtain the target student network. The response distribution on the teacher feature map and the student feature map can intuitively show the structural information on the target image. The high response corresponds to the semantic target area extracted by the current channel, that is, the foreground area, and the low response corresponds to other targets (other semantic targets and background), that is, the background area. The success of the knowledge distillation from the teacher network to the student network requires ensuring that the student network learns more foreground knowledge and less background knowledge. The channel-level teacher feature map in the teacher network is used to align the corresponding distribution of the foreground and background of the student feature map. During the alignment process, the parameters in the student network are continuously adjusted to reduce the performance difference between the student network and the teacher network, so as to obtain the target learning network that has learned the superior performance of the teacher network, and improve the accuracy of the semantic segmentation of the target student network.

[0062] In one embodiment, in order to improve the efficiency of channel matching, as Figure 2 As shown, S120 includes:

[0063] S210. Number the channels in the teacher characteristic graph and the student characteristic graph according to the channel type to obtain the teacher channel number and the student channel number.

[0064] Among them, the teacher channel number and student channel number of the same channel type are the same.

[0065] Specifically, the computer device numbers the channels of the teacher feature map and the channels of the student feature map according to the channel type, and numbers the channels of the same channel type between the teacher feature map and the student feature map as the same number, thereby obtaining the teacher channel number and the student channel number. For example, the teacher network and the student network are network models for semantic segmentation of people, cats, and dogs. The computer device numbers the channel of the extracted feature map of people as 1, the channel of the extracted feature map of cats as 2, and the channel of the extracted feature map of dogs as 3, thereby obtaining the teacher channel number c=1, 2, 3 and the student channel number j=1, 2, 3.

[0066] S220, traverse the teacher channel number and the student channel number to match the number of the channel of the teacher feature map and the channel of the student feature map.

[0067] If the channel numbers match, the channels match; if the channel numbers do not match, the channels do not match.

[0068] Specifically, the computer device matches the channels of each teacher characteristic graph with the channels of each student characteristic graph in the order of the teacher channel numbers from small to large, and in the order of the student channel numbers from small to large. For example, the computer device matches the channels of the teacher channel number c=1 with the channels of the student channel numbers j=1, 2, 3 in sequence, and then matches the channels of the teacher channel number c=2 with the channels of the student channel numbers j=1, 2, 3 in sequence, and then matches the channels of the teacher channel number c=3 with the channels of the student channel numbers j=1, 2, 3 in sequence.

[0069] Furthermore, the number matching process also includes:

[0070] After the computer device sequentially matches the channel with the teacher channel number c with the channel with the student channel number j, it determines whether c is equal to the preset teacher channel number C. If not, the channel with the teacher channel number c+ is sequentially matched with the channel with the student channel number j; if so, the number matching stops.

[0071] At the same time, the computer device determines whether the student channel number j matched with the channel of the teacher channel number c is equal to the preset teacher channel number C. If not, the channel of the teacher channel number c is matched with the channel of the student channel number j+; if so, the channel of the teacher channel number c+ is matched with the channel of the student channel number j in sequence.

[0072] In this embodiment, the computer device numbers the channels in the teacher characteristic map and the student characteristic map, and assigns the same number to the channels of the same channel type, thereby utilizing number matching to achieve channel matching between the teacher characteristic map and the student characteristic map. The number matching process is simple and convenient, thereby improving the efficiency of channel matching between the teacher characteristic map and the student characteristic map.

[0073] In one embodiment, a softmax calculation may be performed on the feature map to convert the response distribution into a probability distribution, thereby obtaining the loss value between the feature maps. Figure 3 As shown, before obtaining the target loss value between the teacher feature map and the student feature map according to the matching result, it includes:

[0074] S310, performing softmax calculation according to the teacher feature map to obtain a first probability distribution.

[0075] S320, performing softmax calculation according to the student feature map to obtain a second probability distribution.

[0076] The softmax calculation is to convert the digital matrix of the response distribution in the feature map into a probability distribution through the softmax formula (1).

[0077]

[0078] Among them, i is the value of the i-th number in the digital matrix, i∈j.

[0079] Specifically, the computer device performs softmax calculation on the digital matrices of the response distribution in the teacher feature map and the student feature map using softmax formula (1), and correspondingly obtains the first probability distribution and the second probability distribution.

[0080] Accordingly, S130 includes:

[0081] The target loss value is calculated according to the matching result, the first probability distribution and the second probability distribution.

[0082] Specifically, the computer device calculates the loss value between the channel-matched teacher feature map and the student feature map based on the first probability distribution of the channel-matched teacher feature map and the second probability distribution of the student feature map. The computer device then calculates the loss value between the channel-mismatched teacher feature map and the student feature map based on the first probability distribution of the channel-mismatched teacher feature map and the second probability distribution of the student feature map. The computer device determines the target loss value of the teacher feature map and the student feature map as a whole by combining the loss value obtained by the channel matching and the loss value obtained by the channel mismatch.

[0083] In this embodiment, the computer device performs softmax calculation based on the response distribution on the teacher characteristic map and the student characteristic map, and obtains the first probability distribution and the second probability distribution accordingly, so as to convert the response distribution on the teacher characteristic map and the student characteristic map into a probability distribution, thereby providing a data foundation for calculating the target loss value between the teacher characteristic map and the student characteristic map.

[0084] In one embodiment, KL (Kullback-Leibler divergence) divergence, i.e., relative entropy, can be used as the loss value between the teacher feature map and the student feature map. Figure 4 As shown, the above calculation of the target loss value based on the matching result, the first probability distribution and the second probability distribution includes:

[0085] S410, obtaining a first probability distribution of a channel-matched teacher feature map and a second probability distribution of a student feature map.

[0086] S420, calculating the first KL divergence between the teacher feature map and the student feature map according to the probability values ​​on the first probability distribution and the second probability distribution of channel matching, and taking the positive value of the first KL divergence as the first loss value.

[0087] S430, obtaining a first probability distribution of a channel-mismatched teacher feature map and a second probability distribution of a student feature map.

[0088] S440, calculating the second KL divergence between the teacher feature map and the student feature map according to the probability values ​​on the first probability distribution and the second probability distribution of channel mismatch, and taking the negative value of the second KL divergence as the second loss value.

[0089] S450: Obtain a target loss value according to the first loss value and the second loss value.

[0090] Among them, KL divergence D KL It can be obtained by using KL formula (2) according to the probability distribution p of the real event and the probability distribution q of the fitting event. In this embodiment, the real event is the teacher feature map obtained by the teacher network, and the fitting event is the student feature map obtained by the student network.

[0091]

[0092] Among them, p(x i ) is the probability value of the real event, p(x i ) is the probability value of the fitting event.

[0093] Specifically, the computer device uses KL divergence formula (2) to calculate the first KL divergence between the channel-matched teacher feature map and the student feature map according to the first probability distribution of the channel-matched teacher feature map and the second probability distribution of the student feature map, and calculates the first KL divergence between the channel-mismatched teacher feature map and the student feature map according to the first probability distribution of the channel-mismatched teacher feature map and the second probability distribution of the student feature map. The computer device further takes the positive value of the first KL divergence as the first loss value, takes the negative value of the second KL divergence as the second loss value, and adds the first loss value to the second loss value to obtain the target loss value.

[0094] In this embodiment, the purpose of the teacher network training the student network is to reduce the target loss function obtained by adjusting the parameters in the student network. In the above method, the computer device takes the positive value of the first K divergence obtained according to the channel matching as the first loss value, takes the positive value of the second K divergence obtained according to the channel mismatch as the second loss value, and then adds the first loss value to the second loss value as the target loss value. In this way, in the process of adjusting the parameters in the student network to reduce the target loss function obtained, the first loss value can be reduced to reduce the difference between the channel matching teacher feature map and the student feature map, and the second loss value can be increased to increase the difference between the channel mismatching teacher feature map and the student feature map, so that the student network can learn more prospects and at least less background knowledge, so that the target student network finally obtained can learn the superior performance of the teacher network, improve the performance of the target student network, and perform semantic segmentation The accuracy.

[0095] In one embodiment, when the target loss value reaches a preset loss value, the parameter adjustment can be stopped to obtain the target student network, and S130 includes:

[0096] The parameters in the student network are adjusted according to the target loss value until the target loss value obtained by the student network after the parameter adjustment is less than the preset loss value, and the student network after the parameter adjustment is used as the target student network.

[0097] Specifically, the computer device adjusts the parameters in the student network by gradient updating according to the obtained target loss value until the target loss value obtained by the student network after the parameter adjustment is less than the preset loss value, then the parameter adjustment can be stopped, and the student network after the parameter adjustment is used as the target student network, and the training is completed. After the training is completed, the data set can be used again to verify the performance of the obtained target student network.

[0098] In this embodiment, the computer device shortens the training time and improves the training efficiency while ensuring the network performance of the target student by presetting the loss value.

[0099] In a specific embodiment, Figure 5 As shown, a distillation learning method is provided, including:

[0100] S501. Input the target image into the teacher network and the student network respectively to obtain a teacher feature map output by the teacher network and a student feature map output by the student network.

[0101] S502. Number the channels of the teacher characteristic graph and the student characteristic graph according to the channel type to obtain the teacher channel number and the student channel number; wherein the teacher channel number and the student channel number of the same channel type are the same.

[0102] S503, in the order of the teacher channel numbers from small to large, the channels of each teacher feature map are numbered and matched with the channels of each student feature map in sequence in the order of the student channel numbers from small to large.

[0103] S504: Perform softmax calculation according to the teacher feature map to obtain a first probability distribution.

[0104] S505. Perform softmax calculation according to the student feature map to obtain a second probability distribution.

[0105] S506: Obtain a first probability distribution of a channel-matched teacher feature map and a second probability distribution of a student feature map.

[0106] S507. Calculate the first KL divergence between the teacher feature map and the student feature map according to the probability values ​​on the first probability distribution and the second probability distribution of channel matching, and take the positive value of the first KL divergence as the first loss value.

[0107] S508. Obtain a first probability distribution of a channel-mismatched teacher feature map and a second probability distribution of a student feature map.

[0108] S509, calculating the second KL divergence between the teacher feature map and the student feature map according to the probability values ​​on the first probability distribution and the second probability distribution of channel mismatch, and taking the negative value of the second KL divergence as the second loss value.

[0109] S510. Add the first loss value to the second loss value to obtain a target loss value.

[0110] S511. Adjust the parameters in the student network according to the target loss value until the target loss value obtained by the student network after the parameter adjustment is less than the preset loss value, and use the student network after the parameter adjustment as the target student network.

[0111] Combination Figure 6 , the computer device inputs the target image into the teacher network and the student network respectively, and after multiple feature extractions, outputs the final teacher feature map and the final student feature map for semantic segmentation. The computer device extracts the two-dimensional slice of each channel in the final teacher feature map as the teacher feature map obtained by the corresponding channel, and similarly obtains the student feature map. The computer device then numbers the channels of the teacher feature map and the student feature map, uses the numbers to match the channels, takes the sum of the positive value of the first KL divergence between the channel-matched teacher feature map and the student feature map and the negative value of the second KL divergence between the channel-unmatched teacher feature map and the student feature map, and obtains the target loss value. The computer device then adjusts the parameters in the student network according to the target loss value until the target loss value obtained by the student network after the parameter adjustment is less than the preset loss value, obtains the target student network, and the training ends.

[0112] In this embodiment, the computer device uses the above method to align the foreground and background distributions in the student network with weaker performance with the foreground and background distributions in the teacher network with superior performance. Based on the comparison of the channel-level teacher feature map and the student feature map, the student network learns more foreground and at least less background knowledge, improves the distillation learning effect of the student network on the teacher network, reduces the performance difference between the student network and the teacher network, and obtains the target student network that learns the superior performance of the teacher network, thereby improving the performance of the target student network and the accuracy of semantic segmentation.

[0113] It should be understood that although Figure 1-5The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1-5 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0114] In one embodiment, Figure 7 As shown, a distillation learning device is provided, including: a feature output module 701, a channel matching module 702 and a parameter adjustment module 703, wherein:

[0115] The feature output module 701 is used to input the target image into the teacher network and the student network respectively, and obtain the teacher feature map output by the teacher network and the student feature map output by the student network; the channel matching module 702 is used to perform channel matching on the teacher feature map and the student feature map, and obtain the target loss value between the teacher feature map and the student feature map according to the matching result; the parameter adjustment module 703 is used to adjust the parameters in the student network according to the target loss value to obtain the target student network.

[0116] In one embodiment, the channel matching module 702 is specifically used for:

[0117] The channels of the teacher characteristic map and the student characteristic map are numbered respectively according to the channel type to obtain the teacher channel number and the student channel number; wherein the teacher channel number and the student channel number of the same channel type are the same; the teacher channel number and the student channel number are traversed to match the numbers of the channels of the teacher characteristic map and the channels of the student characteristic map.

[0118] In one embodiment, the channel matching module 702 is specifically used for:

[0119] According to the order of the teacher channel numbers from small to large, the channels of each teacher characteristic map are numbered and matched with the channels of each student characteristic map in sequence according to the order of the student channel numbers from small to large.

[0120] In one of the embodiments, the device further includes: a probability calculation module; the probability calculation module is used to perform softmax calculation according to the teacher feature map to obtain a first probability distribution; and perform softmax calculation according to the student feature map to obtain a second probability distribution.

[0121] Correspondingly, the channel matching module 702 is specifically used to calculate the target loss value according to the matching result, the first probability distribution and the second probability distribution.

[0122] In one embodiment, the channel matching module 702 is specifically used for:

[0123] Obtain a first probability distribution of a channel-matched teacher feature map, and a second probability distribution of a student feature map; calculate a first KL divergence between the teacher feature map and the student feature map according to the probability values ​​on the first probability distribution and the second probability distribution of the channel matching, and take the positive value of the first KL divergence as the first loss value; obtain a first probability distribution of a channel-unmatched teacher feature map, and a second probability distribution of the student feature map; calculate a second KL divergence between the teacher feature map and the student feature map according to the probability values ​​on the first probability distribution and the second probability distribution of the channel mismatch, and take the negative value of the second KL divergence as the second loss value; obtain the target loss value according to the first loss value and the second loss value.

[0124] In one embodiment, the channel matching module 702 is specifically used for:

[0125] The first loss value is added to the second loss value to obtain the target loss value.

[0126] In one embodiment, the parameter adjustment module 703 is specifically used to:

[0127] The parameters in the student network are adjusted according to the target loss value until the target loss value obtained by the student network after the parameter adjustment is less than the preset loss value, and the student network after the parameter adjustment is used as the target student network.

[0128] For the specific definition of the distillation learning device, please refer to the definition of the distillation learning method above, which will not be repeated here. Each module in the above-mentioned distillation learning device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0129] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 8As shown. The computer device includes a processor, a memory and a network interface connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store distillation learning data. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a distillation learning method is implemented.

[0130] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0131] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the computer program, the following steps are implemented:

[0132] The target image is input into the teacher network and the student network respectively to obtain a teacher feature map output by the teacher network and a student feature map output by the student network; the teacher feature map and the student feature map are channel matched, and a target loss value between the teacher feature map and the student feature map is obtained according to the matching result; the parameters in the student network are adjusted according to the target loss value to obtain a target student network.

[0133] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0134] The channels of the teacher characteristic map and the student characteristic map are numbered respectively according to the channel type to obtain the teacher channel number and the student channel number; wherein the teacher channel number and the student channel number of the same channel type are the same; the teacher channel number and the student channel number are traversed to match the numbers of the channels of the teacher characteristic map and the channels of the student characteristic map.

[0135] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0136] According to the order of the teacher channel numbers from small to large, the channels of each teacher characteristic map are numbered and matched with the channels of each student characteristic map in sequence according to the order of the student channel numbers from small to large.

[0137] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0138] A softmax calculation is performed based on the teacher feature map to obtain a first probability distribution; a softmax calculation is performed based on the student feature map to obtain a second probability distribution; and the target loss value is calculated based on the matching result, the first probability distribution, and the second probability distribution.

[0139] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0140] Obtain a first probability distribution of a channel-matched teacher feature map, and a second probability distribution of a student feature map; calculate a first KL divergence between the teacher feature map and the student feature map according to the probability values ​​on the first probability distribution and the second probability distribution of the channel matching, and take the positive value of the first KL divergence as the first loss value; obtain a first probability distribution of a channel-unmatched teacher feature map, and a second probability distribution of the student feature map; calculate a second KL divergence between the teacher feature map and the student feature map according to the probability values ​​on the first probability distribution and the second probability distribution of the channel mismatch, and take the negative value of the second KL divergence as the second loss value; obtain the target loss value according to the first loss value and the second loss value.

[0141] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0142] The first loss value is added to the second loss value to obtain the target loss value.

[0143] In one embodiment, when the processor executes the computer program, the processor further implements the following steps:

[0144] The parameters in the student network are adjusted according to the target loss value until the target loss value obtained by the student network after the parameter adjustment is less than the preset loss value, and the student network after the parameter adjustment is used as the target student network.

[0145] In one embodiment, a computer readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

[0146] The target image is input into the teacher network and the student network respectively to obtain a teacher feature map output by the teacher network and a student feature map output by the student network; the teacher feature map and the student feature map are channel matched, and a target loss value between the teacher feature map and the student feature map is obtained according to the matching result; the parameters in the student network are adjusted according to the target loss value to obtain a target student network.

[0147] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0148] The channels of the teacher characteristic map and the student characteristic map are numbered respectively according to the channel type to obtain the teacher channel number and the student channel number; wherein the teacher channel number and the student channel number of the same channel type are the same; the teacher channel number and the student channel number are traversed to match the numbers of the channels of the teacher characteristic map and the channels of the student characteristic map.

[0149] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0150] According to the order of the teacher channel numbers from small to large, the channels of each teacher characteristic map are numbered and matched with the channels of each student characteristic map in sequence according to the order of the student channel numbers from small to large.

[0151] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0152] A softmax calculation is performed based on the teacher feature map to obtain a first probability distribution; a softmax calculation is performed based on the student feature map to obtain a second probability distribution; and the target loss value is calculated based on the matching result, the first probability distribution, and the second probability distribution.

[0153] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0154] Obtain a first probability distribution of a channel-matched teacher feature map, and a second probability distribution of a student feature map; calculate a first KL divergence between the teacher feature map and the student feature map according to the probability values ​​on the first probability distribution and the second probability distribution of the channel matching, and take the positive value of the first KL divergence as the first loss value; obtain a first probability distribution of a channel-unmatched teacher feature map, and a second probability distribution of the student feature map; calculate a second KL divergence between the teacher feature map and the student feature map according to the probability values ​​on the first probability distribution and the second probability distribution of the channel mismatch, and take the negative value of the second KL divergence as the second loss value; obtain the target loss value according to the first loss value and the second loss value.

[0155] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0156] The first loss value is added to the second loss value to obtain the target loss value.

[0157] In one embodiment, when the computer program is executed by a processor, the following steps are also implemented:

[0158] The parameters in the student network are adjusted according to the target loss value until the target loss value obtained by the student network after the parameter adjustment is less than the preset loss value, and the student network after the parameter adjustment is used as the target student network.

[0159] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0160] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0161] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the invention patent. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the attached claims.

Claims

1. A distillation learning method, characterized in that: The method comprises: Inputting the target image into the teacher network and the student network respectively, obtaining a teacher feature map output by the teacher network and a student feature map output by the student network; Performing a softmax calculation according to the teacher feature map to obtain a first probability distribution; Performing a softmax calculation according to the student feature map to obtain a second probability distribution; Perform channel matching on the teacher feature map and the student feature map, and obtain a target loss value between the teacher feature map and the student feature map according to the matching result; Adjust the parameters in the student network according to the target loss value to obtain a target student network; Wherein, obtaining the target loss value between the teacher feature map and the student feature map according to the matching result includes: Obtain a first probability distribution of a channel-matched teacher feature map and a second probability distribution of a student feature map; Calculate a first KL divergence between the teacher feature map and the student feature map according to the probability values ​​on the first probability distribution and the second probability distribution of the channel matching, and take the positive value of the first KL divergence as the first loss value; Obtain a first probability distribution of a channel-mismatched teacher feature map and a second probability distribution of a student feature map; Calculate the second KL divergence between the teacher feature map and the student feature map according to the probability values ​​on the first probability distribution and the second probability distribution of channel mismatch, and take the negative value of the second KL divergence as the second loss value; The first loss value is added to the second loss value to obtain the target loss value.

2. The method according to claim 1, characterized in that The channel matching of the teacher feature map and the student feature map comprises: Numbering the channels of the teacher characteristic graph and the student characteristic graph according to the channel type to obtain a teacher channel number and a student channel number; wherein the teacher channel number and the student channel number of the same channel type are the same; The teacher channel number and the student channel number are traversed to match the numbers of the channels of the teacher feature map and the channels of the student feature map.

3. The method according to claim 2, characterized in that The traversing the teacher channel number and the student channel number to match the number of the channel corresponding to the teacher feature map and the channel of the student feature map includes: According to the order of the teacher channel numbers from small to large, the channels of each teacher characteristic map are numbered and matched with the channels of each student characteristic map in sequence according to the order of the student channel numbers from small to large.

4. The method according to claim 1, characterized in that: The step of adjusting the parameters in the student network according to the target loss value to obtain a target student network includes: The parameters in the student network are adjusted according to the target loss value until the target loss value obtained by the student network after the parameter adjustment is less than the preset loss value, and the student network after the parameter adjustment is used as the target student network.

5. A distillation learning device, characterized in that: The device comprises: A feature output module, used to input the target image into the teacher network and the student network respectively, to obtain a teacher feature map output by the teacher network and a student feature map output by the student network; A probability calculation module, used to perform softmax calculation according to the teacher feature map to obtain a first probability distribution; and perform softmax calculation according to the student feature map to obtain a second probability distribution; A channel matching module, used for performing channel matching on the teacher feature map and the student feature map, and obtaining a target loss value between the teacher feature map and the student feature map according to the matching result; A parameter adjustment module, used for adjusting the parameters in the student network according to the target loss value to obtain a target student network; Among them, the channel matching module is specifically used to obtain a first probability distribution of a channel-matched teacher feature map, and a second probability distribution of a student feature map; calculate the first KL divergence between the teacher feature map and the student feature map according to the probability values ​​on the first probability distribution and the second probability distribution of the channel matching, and take the positive value of the first KL divergence as the first loss value; obtain the first probability distribution of the channel-unmatched teacher feature map, and the second probability distribution of the student feature map; calculate the second KL divergence between the teacher feature map and the student feature map according to the probability values ​​on the first probability distribution and the second probability distribution of the channel mismatch, and take the negative value of the second KL divergence as the second loss value; add the first loss value to the second loss value to obtain the target loss value.

6. The device according to claim 5, characterized in that The channel matching module is specifically used to number the channels of the teacher characteristic map and the student characteristic map according to the channel type to obtain the teacher channel number and the student channel number; wherein the teacher channel number and the student channel number of the same channel type are the same; and traverse the teacher channel number and the student channel number to match the numbers of the channels of the teacher characteristic map and the channels of the student characteristic map.

7. The device according to claim 6, characterized in that The channel matching module is specifically used to match the channels of each teacher feature map with the channels of each student feature map in the order of the teacher channel numbers from small to large, and then in the order of the student channel numbers from small to large.

8. The device according to claim 5, characterized in that The parameter adjustment module is specifically used to adjust the parameters in the student network according to the target loss value until the target loss value obtained by the student network after the parameter adjustment is less than the preset loss value, and use the student network after the parameter adjustment as the target student network.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 4 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.