A target network model acquisition method and device based on fusion features
By fusing feature maps and adjusting parameters of teacher-student network models, the problem of deploying large neural networks on edge devices is solved, and a target network model with excellent performance and small number of parameters is realized, which is suitable for miniaturized products.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-03-15
- Publication Date
- 2026-03-20
AI Technical Summary
Existing large-scale neural network models are difficult to deploy on edge devices or devices with limited computing resources, making product deployment difficult.
By acquiring the teacher network model and the student network model, feature maps are extracted from the training dataset, processed, and fused. The multilayer distillation loss is determined, and the parameters of the student network model are adjusted to obtain the target network model.
It reduces the complexity and number of parameters in the network model, improves network performance and generalization, and enables its application in miniaturized products.
Smart Images
Figure CN116258871B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present application relates to the technical field of artificial intelligence algorithm, and particularly relates to a target network model acquisition method and device based on fusion features. BACKGROUND
[0002] With the continuous development of computer vision technology, the performance of deep learning has been greatly improved, which makes the neural network have outstanding effect on tasks such as target detection, image classification, semantic segmentation, and human pose estimation. The technology in this field is applied in various fields of life, such as automatic driving, mask detection, face recognition, living body detection, image deblurring, etc. In recent years, with the improvement of performance, the parameter quantity and the calculation quantity of many excellent models have also increased. This makes it difficult to deploy the model on edge devices or devices with limited computing resources, resulting in the challenge that powerful neural networks are difficult to land products.
[0003] Therefore, how to reduce the complexity of the network model so that it can be applied on small-sized products and ensure its performance is a technical problem to be solved at present. SUMMARY
[0004] In view of the above problems, the present application provides a target network model acquisition method and device based on fusion features to overcome the above problems or at least partially solve the above problems.
[0005] In a first aspect, the present application provides a target network model acquisition method based on fusion features, comprising:
[0006] obtaining a teacher network model, a student network model, and a training data set, the parameter quantity of the student network model being less than the parameter quantity of the teacher network model;
[0007] inputting the training data set into the student network model to extract a student last layer feature map, and inputting the training data set into the teacher network model to extract a teacher each layer feature map;
[0008] processing the student last layer feature map and the teacher each layer feature map respectively to obtain a student last layer target feature map and a teacher each layer target feature map, the size and the number of channels of the student last layer target feature map being equal to the size and the number of channels of the teacher each layer target feature map;
[0009] fusing the student last layer target feature map with the teacher each layer target feature map respectively to obtain a fusion feature set;
[0010] determining a multi-layer distillation loss based on the fusion feature set;
[0011] Based on the multi-layer distillation loss, the parameters of the student network model are adjusted to obtain a target network model.
[0012] Further, the student final layer feature map and the teacher each layer feature map are processed respectively to obtain a student final layer target feature map and a teacher each layer target feature map, the size and the number of channels of the student final layer target feature map are equal to the size and the number of channels of the teacher each layer target feature map, and the student final layer target feature map comprises:
[0013] The size of the student final layer feature map and the size of the teacher each layer feature map are compared respectively, and the size of the student final layer feature map is equal to the size of the teacher each layer feature map through pooling processing;
[0014] The student final layer feature map and the teacher each layer feature map are processed by a 1x1 convolution layer respectively, so that the number of channels of the student final layer feature map is equal to the number of channels of the teacher each layer feature map;
[0015] The student final layer target feature map and the teacher each layer target feature map are obtained.
[0016] Further, the student final layer target feature map and the teacher each layer target feature map are fused respectively to obtain a fusion feature set, comprising:
[0017] The student final layer target feature map and the teacher each layer target feature map are spliced respectively to obtain a spliced feature set;
[0018] Based on the spliced feature set, the student final layer target feature map and the teacher each layer target feature map, a fusion feature set is obtained.
[0019] Further, based on the spliced feature set, the student final layer target feature map and the teacher each layer target feature map, a fusion feature set is obtained, comprising:
[0020] Any two spliced feature maps are extracted from the spliced feature set, one spliced feature map is subjected to matrix multiplication operation with the student final layer target feature map to obtain a student multiplication feature map, and another spliced feature map is subjected to matrix multiplication operation with the teacher each layer target feature map to obtain a teacher each layer multiplication feature map;
[0021] The student multiplication feature map and the teacher each layer multiplication feature map are subjected to matrix addition operation respectively to obtain a fusion feature set.
[0022] Further, the student multiplication feature map and the teacher each layer multiplication feature map are subjected to matrix addition operation respectively to obtain a fusion feature set, comprising:
[0023] Based on the number of cycles of training, a fusion coefficient is determined;
[0024] Based on the fusion coefficient, the student multiplication feature map and the teacher multiplication feature map of each layer are subjected to matrix addition operation respectively to obtain a fusion feature set.
[0025] Further, the fusion coefficient is determined based on the training cycle number, including:
[0026] The training period is determined based on the training cycle number;
[0027] The fusion coefficient is determined based on the training period, wherein when the training period is the early stage, the fusion coefficient is determined as 0, when the training period is the middle stage, the fusion coefficient is determined as wherein EPOCH is the training number, and when the training period is the later stage, the fusion coefficient is determined as 1.
[0028] Further, the multi-layer distillation loss is determined based on the fusion feature set, including:
[0029] The single-layer distillation loss is determined based on the fusion feature set, specifically in the following manner:
[0030]
[0031] wherein Loss i is any single-layer distillation loss, is any fusion feature in the fusion feature set, is any feature in the student feature set except the student last-layer feature map, 1≤j≤m-1, m is the serial number corresponding to the last functional block in the student network model, and MSE is the mean square error loss calculation;
[0032] The multi-layer distillation loss is obtained through iterative calculation based on the single-layer distillation loss, specifically in the following manner:
[0033]
[0034] wherein L CFKD is the multi-layer distillation loss.
[0035] In a second aspect, the present application further provides a target network model acquisition device based on fusion features, comprising:
[0036] An acquisition module is configured to acquire a teacher network model, a student network model and a training data set, wherein the parameter amount of the student network model is less than that of the teacher network model;
[0037] An extraction module is configured to input the training data set into the student network model, extract a student last-layer feature map, input the training data set into the teacher network model, and extract teacher feature maps of each layer;
[0038] a first obtaining module, configured to process the student last-layer feature map and the teacher each-layer feature map respectively to obtain a student last-layer target feature map and a teacher each-layer target feature map, wherein the size and the number of channels of the student last-layer target feature map are equal to the size and the number of channels of the teacher each-layer target feature map;
[0039] a fusion module, configured to fuse the student last-layer target feature map with the teacher each-layer target feature map respectively to obtain a fusion feature set;
[0040] a determination module, configured to determine a multi-layer distillation loss based on the fusion feature set;
[0041] a second obtaining module, configured to adjust parameters of the student network model based on the multi-layer distillation loss to obtain a target network model.
[0042] In a third aspect, the present application further provides a computer device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the method steps of the first aspect when executing the program.
[0043] In a fourth aspect, the present application further provides a computer readable storage medium, having a computer program stored thereon, characterized in that the program is executable on a processor to implement the method steps of the first aspect.
[0044] The one or more technical solutions in the embodiments of the present application have at least the following technical effects or advantages:
[0045] The application provides a target network model acquisition method based on fusion features, comprising the following steps: acquiring a teacher network model, a student network model and a training data set, the parameter quantity of the student network model being less than that of the teacher network model; inputting the training data set into the student network model to extract a student end layer feature map, and inputting the training data set into the teacher network model to extract teacher layer feature maps; processing the student end layer feature map and the teacher layer feature maps respectively to obtain a student end layer target feature map and teacher layer target feature maps, the size and the number of channels of the student end layer target feature map being equal to those of the teacher layer target feature maps; fusing the student end layer target feature map with the teacher layer target feature maps respectively to obtain a fusion feature set; determining a multi-layer distillation loss based on the fusion feature set; and adjusting the parameters of the student network model based on the multi-layer distillation loss to obtain a target network model, wherein the fusion features are obtained by fusing the features of the student network model and the teacher network model, the fusion features are transmitted to the small student network model, the rich fusion features greatly improve the network performance and the network generalization of the student network model, and then the target network model with small memory and parameter quantity and excellent performance is obtained, so that the target network model can be applied to small products. BRIEF DESCRIPTION OF DRAWINGS
[0046] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of preferred embodiments, and are not intended to limit the scope of the application. Furthermore, the same reference numerals are used throughout the several views of the drawings to denote the same or similar parts. In the drawings:
[0047] Figure 1 A step flowchart of the target network model acquisition method based on fusion features in the embodiment of the application is shown;
[0048] Figure 2 A schematic diagram of the teacher network model and the student network model in the embodiment of the application is shown;
[0049] Figure 3 A schematic diagram of the fusion process in the embodiment of the application is shown;
[0050] Figure 4 A schematic diagram of the dynamic feature fusion function in the embodiment of the application is shown;
[0051] Figure 5 A schematic diagram of the combination of the student multiplication feature map and the dynamic fusion function in the embodiment of the application is shown;
[0052] Figure 6 A schematic diagram of the application of the teacher-student cross-layer fusion feature knowledge distillation algorithm to a convolutional neural network in the embodiment of the application is shown;
[0053] Figure 7 A structural schematic diagram of a target network model acquisition device based on fusion features in an embodiment of the present application is shown.
[0054] Figure 8 A structural schematic diagram of a computer device for implementing a target network model acquisition method based on fusion features in an embodiment of the present application is shown. DETAILED DESCRIPTION
[0055] Exemplary embodiments of the present disclosure will be described in greater detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be accurately conveyed to those skilled in the art.
[0056] Embodiment One
[0057] An embodiment of the present application provides a target network model acquisition method based on fusion features, as shown in the accompanying drawings, comprising: Figure 1
[0058] S101, a teacher network model, a student network model and a training data set are acquired, the parameter quantity of the student network model being less than that of the teacher network model;
[0059] S102, the training data set is input into the student network model to extract a student end layer feature map, and the training data set is input into the teacher network model to extract teacher layer feature maps;
[0060] S103, the student end layer feature map and the teacher layer feature maps are respectively processed to obtain a student end layer target feature map and target layer feature maps, the size and the number of channels of the student end layer target feature map being equal to those of the teacher target layer feature maps;
[0061] S104, the student end layer target feature map is fused with the teacher target layer feature maps to obtain a fusion feature set;
[0062] S105, based on the fusion feature set, a multi-layer distillation loss is determined;
[0063] S106, based on the multi-layer distillation loss, the parameters of the student network model are adjusted to obtain a target network model.
[0064] First, in S101, the teacher network model is a trained model, and the student network model is an untrained model. The teacher network model comprises a plurality of functional blocks (Blocks), and each functional block (Block) comprises a plurality of structure layers (layers). The student network model has a smaller parameter quantity.
[0065] The training data set is preprocessed, including flipping, scaling, and normalizing the pictures in the training data set.
[0066] Next, the training data set is input into the student network model to extract the student last layer feature map, and the training data set is input into the teacher network model to extract the teacher layer feature map.
[0067] The training data set can be selected as pictures x, x∈32*32*128, where 32*32 represents the picture size, and 128 represents the number of pictures.
[0068] The training data set is input into the student network model and the teacher network model to extract the student last layer feature map and the teacher layer feature map.
[0069] Specifically, as shown in Figure 2 The teacher network model is more complex than the student network model. The corresponding feature map obtained from each functional block (Block) 201 of the teacher network model is the teacher layer feature map, and the corresponding feature map obtained from the last functional block (Block) 202 of the student network model is the student last layer feature map.
[0070] Let represent the teacher layer feature map, Let represent the student layer feature map, n represents the serial number corresponding to the last layer of the teacher network model, and m represents the serial number corresponding to the last layer of the student network model.
[0071] Therefore, the obtained student last layer feature map is and the teacher layer feature map is
[0072] Next, the teacher layer feature map and the student last layer feature map are processed, S103 is performed, the student last layer feature map and the teacher layer feature map are processed to obtain the student last layer target feature map and the teacher layer target feature map, and the size and the number of channels of the student last layer target feature map are equal to the size and the number of channels of the teacher layer target feature map.
[0073] Because the student last layer feature map and the teacher layer feature map have inconsistent channel numbers and inconsistent sizes, they need to be adjusted to be consistent for later processing.
[0074] Specifically, the size of the student last layer feature map and the size of each teacher layer feature map are compared respectively, and the size of the student last layer feature map is equal to the size of each teacher layer feature map through pooling processing; the student last layer feature map and each teacher layer feature map are processed by a 1x1 convolution layer respectively, so that the number of channels of the student last layer feature map is equal to the number of channels of each teacher layer feature map, and the student last layer target feature map and each teacher layer target feature map are obtained.
[0075] When the size of the student last layer feature map is greater than the size of any teacher target layer feature map, the student last layer feature map is specifically processed by pooling, so that the size of the student last layer feature map is equal to the size of the teacher target layer feature map, otherwise, the teacher target layer feature map is processed by pooling; when the number of channels of the student last layer feature map is greater than or less than the number of channels of any teacher target layer feature map, the student last layer feature map and the teacher target layer feature map are processed by a 1x1 convolution layer respectively, so that the number of channels of the student last layer feature map is equal to the number of channels of each teacher layer feature map, and the student last layer target feature map and each teacher layer target feature map are obtained through the above processing.
[0076] The obtained teacher layer target feature map is represented as The obtained student last layer target feature map is represented as
[0077] The specific calculation formula is as follows:
[0078]
[0079] Wherein, W is a weight matrix, and the weight matrix corresponding to each layer feature map is different, and b is a bias.
[0080] Then, S104 is executed to fuse the student last layer target feature map and the teacher layer target feature map to obtain a fused feature set.
[0081] The student last layer target feature map is spliced with each teacher layer target feature map respectively to obtain a spliced feature set; and a fused feature set is obtained based on the spliced feature set, the student last layer target feature map and the teacher layer target feature map.
[0082] The fusion process in S104 is specifically to generate a fused feature set through a fusion function formula, that is:
[0083]
[0084] Specifically, as shown in Figure 3 , the student last layer target feature map is spliced with each teacher layer target feature map through a splicing operation to obtain a spliced feature set 301.
[0085] Next, based on the spliced feature set 301 and the student's final target feature map... and teacher target feature maps at each level The fused feature set is obtained by extracting any two spliced feature maps f from the spliced feature set 301. ′ , splice a feature map f ′ Student's final target feature map Perform matrix multiplication to obtain the student multiplication feature map. Use another spliced feature map f ′ Corresponding to the target feature maps of teachers at each level Perform matrix multiplication to obtain the multiplication feature maps of each level of the teacher. Student multiplication characteristic graph Multiplication characteristic diagrams for different levels of teachers Perform matrix addition operations separately to obtain the fused feature set.
[0086] Specifically, when performing matrix addition, the fusion coefficient S is determined based on the number of training iterations. Based on this fusion coefficient, the student multiplication feature map and the teacher's multiplication feature maps at each layer are subjected to matrix addition to obtain the fused feature set.
[0087] The steps to determine the fusion coefficient are as follows:
[0088] The training period is determined based on the number of training iterations; the fusion coefficient is then determined based on the training period, with the fusion coefficient set to 0 in the early training phase and to a certain value in the middle training phase. Here, EPOCH represents the number of training iterations, and the fusion coefficient is set to 1 when the training period is in its later stages.
[0089] The fusion coefficient is determined using the following formula:
[0090]
[0091] After determining the fusion coefficient, the result of multiplying the student's multiplication feature map by the fusion coefficient is then added to the teacher's multiplication feature maps at each level.
[0092] Specifically, such as Figure 4 The diagram shown is a schematic of the dynamic fusion function. Figure 5 The diagram shown below illustrates the combination of the student multiplication feature map and the dynamic fusion function:
[0093] In the early stages of training, the fusion coefficient is set to 0. Multiplying the student multiplication feature map by 0 results in 0. Therefore, after performing matrix addition operations with the teacher's multiplication feature maps at each level, the resulting fusion feature set is the teacher's multiplication feature map at each level.
[0094] This is because the performance of the student network model is poor in the early training stage, so only the feature output of the teacher network is learned.
[0095] In the middle of the training, the student network model has certain prediction ability, so the student multiplication feature map is multiplied by the fusion coefficient corresponding to the period, the fusion coefficient is in the range of [0.9, 1], and then the matrix addition operation is performed with the multiplication feature map of each layer of the teacher to obtain the fusion feature set. The purpose is not to completely suppress the feature fusion of the student deep layer feature and the feature of each layer of the teacher network model.
[0096] In the later training stage, the performance gap between the student network model and the teacher network model is small, so the student multiplication feature map is multiplied by the fusion coefficient corresponding to the period, that is, the fusion coefficient is 1, and then the matrix addition operation is performed with the multiplication feature map of each layer of the teacher to obtain the fusion feature set.
[0097] Finally, S105 is executed to determine the multi-layer distillation loss based on the fusion feature set.
[0098] The fusion feature set is taken as a standard feature set, and the features in the student feature set except the student last layer feature map are respectively calculated with the fusion feature set.
[0099] Specifically, based on the fusion feature set, a single-layer distillation loss is determined, which is specifically determined in the following manner:
[0100]
[0101] Wherein, Loss i is any single-layer distillation loss, is any fusion feature in the fusion feature set, is a feature in the student feature set except the student last layer feature map, 1≤j≤m-1, m is the serial number corresponding to the last layer function block in the student network model, and MSE is the mean square error loss calculation.
[0102] Then, based on the single-layer distillation loss, the multi-layer distillation loss is obtained through iterative calculation, which is specifically obtained in the following manner:
[0103]
[0104] Wherein, L CFKD is the multi-layer distillation loss.
[0105] Finally, S106 is executed to adjust the parameters of the student network model based on the multi-layer distillation loss to obtain a target network model.
[0106] In addition to obtaining the multi-layer distillation loss, the cross-entropy loss L CEand a probability loss L KL Based on the multi-layer distillation loss, the cross-entropy loss and the probability loss, a network loss sum is obtained:
[0107] L total = aL KL + (1-a)L CE + bL CFKD
[0108] Wherein, L total is the network loss sum, and a and b are hyperparameters for adjusting each method.
[0109] The cross-entropy loss L CE is used to represent the error between the final output of the student network model and the true value. The probability loss L KL is a KL divergence function, which is used to transmit the final predicted probability value.
[0110] Finally, based on the network loss sum, the student network model is trained by using the principle of gradient descent to adjust the parameters of the student network model, so as to obtain a target network model. The target network model is an optimized student network model.
[0111] As Figure 6 shown is a schematic diagram of a teacher-student cross-layer fusion feature knowledge distillation algorithm applied to a convolutional neural network.
[0112] The target network model can also be verified, specifically the calculation amount, parameter amount and accuracy of the target network model are verified. The calculation amount is the number of addition and multiplication operations in the process of the target network model, and is usually represented by FLOPs (floating point operation number), which is mainly used to measure the complexity of the model. The parameter amount is mainly the number of weights in the convolutional layer and the fully connected layer of the target network model. The accuracy can be divided into top-1 accuracy and top-5 accuracy, top-1 (accuracy) is the accuracy of the target network model when the predicted maximum class probability of the sample is the same as the true class, and top-5 (accuracy) is the accuracy of the target network model when the true class is contained in the top five probability values of the predicted value of the sample. The target network model is verified through the above indexes.
[0113] One or more technical solutions in the embodiments of the present application have at least the following technical effects or advantages:
[0114] The application provides a target network model acquisition method based on fusion features, comprising the following steps: acquiring a teacher network model, a student network model and a training data set, the parameter quantity of the student network model being less than that of the teacher network model; inputting the training data set into the student network model to extract a student last-layer feature map, and inputting the training data set into the teacher network model to extract teacher-layer feature maps; processing the student last-layer feature map and the teacher-layer feature maps respectively to obtain a student last-layer target feature map and teacher-layer target feature maps, the size and the number of channels of the student last-layer target feature map being equal to those of the teacher-layer target feature maps; fusing the student last-layer target feature map with the teacher-layer target feature maps respectively to obtain a fusion feature set; determining a multi-layer distillation loss based on the fusion feature set; and adjusting the parameters of the student network model based on the multi-layer distillation loss to obtain a target network model, wherein the fusion features are obtained by fusing the features of the student network model and the teacher network model, the fusion features are transmitted to the small student network model, the rich fusion features greatly improve the network performance and the network generalization of the student network model, and then the target network model with small memory and parameter quantity and excellent performance is obtained, so that the target network model can be applied to small products.
[0115] In the application, CIFAR100 is used as the data set, ResNet32x4 with a memory size of about 25M is used as the teacher network model, ShuffleNetV2 with a memory size of 5.33M is used as the student network model, and the compressed target network model can achieve a 100-class object recognition rate of 75.86% under the condition of occupying about 5.33M of memory.
[0116] Embodiment two
[0117] Based on the same inventive concept, the application also provides a target network model acquisition device based on fusion features, as shown in Figure 7 The device comprises:
[0118] The acquisition module 701 is configured to acquire a teacher network model, a student network model and a training data set, and the parameter quantity of the student network model is less than that of the teacher network model.
[0119] The extraction module 702 is configured to input the training data set into the student network model to extract a student last-layer feature map, and input the training data set into the teacher network model to extract teacher-layer feature maps.
[0120] The first obtaining module 703 is configured to process the student last-layer feature map and the teacher-layer feature maps respectively to obtain a student last-layer target feature map and teacher-layer target feature maps, and the size and the number of channels of the student last-layer target feature map are equal to those of the teacher-layer target feature maps.
[0121] fusing the student last-layer target feature map with each of the teacher layer target feature maps to obtain a fused feature set;
[0122] determining a multi-layer distillation loss based on the fused feature set;
[0123] adjusting parameters of the student network model based on the multi-layer distillation loss to obtain a target network model.
[0124] In an optional implementation, the first obtaining module 703 is configured to:
[0125] comparing sizes of the student last-layer feature map and the teacher layer feature maps respectively, and performing pooling processing to make the size of the student last-layer feature map equal to the size of the teacher layer feature maps;
[0126] performing 1x1 convolution layer processing on the student last-layer feature map and the teacher layer feature maps respectively to make the number of channels of the student last-layer feature map equal to the number of channels of the teacher layer feature maps;
[0127] obtaining the student last-layer target feature map and the teacher layer target feature maps.
[0128] In an optional implementation, the fusing module 704 includes:
[0129] a splicing unit configured to splice the student last-layer target feature map with each of the teacher layer target feature maps to obtain a spliced feature set;
[0130] an obtaining unit configured to obtain the fused feature set based on the spliced feature set, the student last-layer target feature map, and the teacher layer target feature maps.
[0131] In an optional implementation, the obtaining unit includes:
[0132] a first operation sub-unit configured to extract any two spliced feature maps from the spliced feature set, perform matrix multiplication operation on one spliced feature map and the student last-layer target feature map to obtain a student multiplication feature map, and perform matrix multiplication operation on the other spliced feature map and each of the teacher layer target feature maps to obtain teacher layer multiplication feature maps;
[0133] a second operation sub-unit configured to perform matrix addition operation on the student multiplication feature map and the teacher layer multiplication feature maps to obtain the fused feature set.
[0134] In an optional implementation, the second operation sub-unit is configured to:
[0135] determine a fusion coefficient based on the number of training cycles;
[0136] perform matrix addition operations on the student multiplication feature map and the teacher multiplication feature map of each layer based on the fusion coefficient to obtain a fusion feature set.
[0137] In an optional implementation, the second operation subunit is configured to:
[0138] determine a training period based on the number of training cycles;
[0139] determine the fusion coefficient based on the training period, wherein when the training period is an early stage, the fusion coefficient is determined to be 0, when the training period is a middle stage, the fusion coefficient is determined to be wherein EPOCH is the number of training cycles, and when the training period is a late stage, the fusion coefficient is determined to be 1.
[0140] In an optional implementation, the determining module 705 is configured to:
[0141] determine a single-layer distillation loss based on the fusion feature set, specifically in the following manner:
[0142]
[0143] wherein Loss i is any single-layer distillation loss, is any fusion feature in the fusion feature set, is any feature in the student feature set except the last layer feature map of the student, 1≤j≤m-1, m is the serial number corresponding to the last functional block in the student network model, and MSE is the mean square error loss calculation.
[0144] obtain a multi-layer distillation loss through iterative calculation based on the single-layer distillation loss, specifically in the following manner:
[0145]
[0146] wherein L CFKD is the multi-layer distillation loss.
[0147] Embodiment Three
[0148] Based on the same inventive concept, the embodiments of the present application provide a computer device, as shown in Figure 8 the memory 804, the processor 802, and a computer program stored in the memory 804 and executable on the processor 802, wherein the processor 802 implements the steps of the above-mentioned fusion feature-based target network model acquisition method when executing the program.
[0149] wherein, inFigure 8 In particular embodiments, a bus architecture (represented by bus 800) can include any number of interconnecting buses and bridges, and the bus 800 can link various circuitry including one or more processors represented by processor 802, and memory represented by memory 804. The bus 800 can also link various other circuitry, such as peripheral devices, voltage regulators, and power management circuitry, all of which are well known in the art, and therefore, not described further. Bus interface 806 provides an interface between the bus 800 and receiver 801 and transmitter 803. Receiver 801 and transmitter 803 can be the same device, i.e., a transceiver, providing a unit for communicating with various other apparatus over a transmission medium. The processor 802 is responsible for managing the bus 800 and carrying out instructions. The memory 804 can be used for storing data used by the processor 802 in executing operational instructions.
[0150] Embodiment Four
[0151] Based on the same inventive concept, the embodiments of the present application provide a computer readable storage medium, having stored thereon a computer program, which, when executed by a processor, implements the steps of the above-mentioned method for obtaining a target network model based on fusion features.
[0152] The algorithms and displays presented herein are not inherently related to any particular computer, virtual system, or other apparatus. Various general purpose systems can be used with programs in accordance with the teachings herein, or it can prove convenient to construct more specialized apparatus to perform the required method steps. The required structure for a variety of these systems will be apparent from the description above. In addition, the present application is not intended to be limited to any particular programming language. It will be appreciated that there are many programming languages that can be used to implement the teachings herein, and any such programming language can be used in connection with the various aspects of the present application.
[0153] In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the application can be practiced without these specific details. In some instances, well-known methods, structures and techniques have not been described in detail in order to avoid obscuring the understanding of this description.
[0154] Similarly, it is to be understood that the embodiments of the present application can be placed into practice notwithstanding modifications to form yet further embodiments of the present application. As such, the terms and expressions of the foregoing description are used solely by way of the example thereof, but to the extent possible no limitation is intended to the details of the construction described herein other than as described in the claims. In this manner, the embodiments of the present application as described herein are susceptible to modifications and alternative forms known to those skilled in the art. It is, therefore, desired to be protected in the broadest scope of the appended claims as they can be interpreted to cover the subject matter of the above description.
[0155] Those skilled in the art can appreciate that modules in the apparatus in the embodiments can be adaptively changed and disposed in one or more apparatuses different from the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and further can be divided into multiple sub-modules or sub-units or sub-components. Any combination of all the features disclosed in the specification (including the accompanying claims, abstract and drawings), and any method or apparatus of all the processes or units disclosed in the specification as such can be adopted in any combination, except that at least some of such features and / or processes or units are mutually exclusive. Unless explicitly stated otherwise, each feature disclosed in the specification (including the accompanying claims, abstract and drawings) can be replaced by an alternative feature providing the same, equivalent or similar function.
[0156] Furthermore, those skilled in the art will appreciate that different embodiments of the application have different features and that not all embodiments of the application will exhibit all of the features described or claimed. It is therefore intended that the scope of the application extend to all features and embodiments disclosed or claimed herein, alone or in any combination with one another.
[0157] The various component embodiments of the present application can be implemented in hardware, or as software modules running in one or more processors, or in combinations thereof. As will be appreciated by one skilled in the art, microprocessors or digital signal processors (DSPs) can be used to implement some or all of the functionality of some or all of the components of the fusion feature based target network model acquisition apparatus and computer device according to the embodiments of the present application in practice. The present application can also be implemented as a program (e.g., computer program and computer program product) for executing any or all of the methods described herein on a device or apparatus (e.g., computer). Such program(s) can be stored on a computer readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, or provided on a carrier medium, or in any other form.
[0158] It should be noted that the above-mentioned embodiments illustrate rather than limit the application, and that one skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. The word 'comprising' does not exclude the presence of elements or steps other than those listed in a claim. The word 'a' or 'an' preceding an element does not exclude the presence of a plurality of such elements. The application can be implemented by means of both hardware and software, and any combination thereof. In a unit claim, several devices can be listed with a conjunction like 'or', but it is to be understood that each of these devices can be implemented by its own hardware item. The use of the word 'at least' followed by a list of one or more members does not preclude the existence of additional such members. The word 'first' or'second' does not have any order of magnitude.
Claims
1. A method for obtaining a target network model based on fusion features, characterized in that, include: Obtain a teacher network model, a student network model, and a training dataset. The number of parameters in the student network model is less than the number of parameters in the teacher network model. The training dataset specifically consists of images and labels. The training dataset is input into the student network model to extract the student's last layer feature map, and the training dataset is input into the teacher network model to extract the teacher's feature maps at each layer. The student's final layer feature map and the teacher's feature maps at each layer are processed separately to obtain the student's final layer target feature map and the teacher's target feature maps at each layer. The size and number of channels of the student's final layer target feature map are equal to the size and number of channels of the teacher's target feature maps at each layer. The student's final layer target feature map is fused with the teacher's target feature maps at each layer to obtain a fused feature set; Based on the fused feature set, the multilayer distillation loss is determined, including: Based on the fused feature set, the single-layer distillation loss is determined, specifically as follows: ； in, For any single-layer distillation loss, For any fusion feature in the fusion feature set, The features in each layer of the student feature set are those excluding the student's last layer feature map. , This refers to the sequence number of the last functional block in the student network model. Calculation of mean squared error loss; Based on the single-layer distillation loss, the multi-layer distillation loss is obtained through iterative calculation, specifically as follows: ； in, This is due to losses from multiple distillation processes; Based on the multi-layer distillation loss, the parameters of the student network model are adjusted to obtain the target network model.
2. The method as described in claim 1, characterized in that, The process of processing the student's final layer feature map and the teacher's feature maps at each layer to obtain the student's final layer target feature map and the teacher's target feature maps at each layer, wherein the size and number of channels of the student's final layer target feature map are equal to the size and number of channels of the teacher's target feature maps at each layer, includes: The size of the student's last layer feature map and the size of each layer feature map of the teacher are compared respectively, and pooling is used to make the size of the student's last layer feature map equal to the size of each layer feature map of the teacher. The student's final feature map and the teacher's feature maps at each layer are processed by a 1×1 convolutional layer, so that the number of channels in the student's final feature map is equal to the number of channels in the teacher's feature maps at each layer. The student's final target feature map and the teacher's target feature maps at each level are obtained.
3. The method as described in claim 1, characterized in that, The step of fusing the student's final-layer target feature map with the teacher's target feature maps at each layer to obtain a fused feature set includes: The student's final layer target feature map is concatenated with the teacher's target feature maps at each layer to obtain a concatenated feature set; Based on the spliced feature set, the student's final target feature map, and the teacher's target feature maps at each level, a fused feature set is obtained.
4. The method as described in claim 3, characterized in that, The fused feature set obtained based on the spliced feature set, the student's final target feature map, and the teacher's target feature maps at each level includes: Extract any two spliced feature maps from the spliced feature set, and perform a matrix multiplication operation between one spliced feature map and the student's last layer target feature map to obtain the student multiplication feature map. Perform a matrix multiplication operation between the other spliced feature map and the teacher's target feature maps at each layer to obtain the teacher's multiplication feature maps at each layer. The student multiplication feature map and the teacher's multiplication feature maps at each level are subjected to matrix addition operations to obtain a fused feature set.
5. The method as described in claim 4, characterized in that, The process of performing matrix addition operations on the student multiplication feature map and the teacher's multiplication feature maps at each level to obtain a fused feature set includes: The fusion coefficient is determined based on the number of training iterations. Based on the fusion coefficient, the student multiplication feature map and the teacher's multiplication feature maps at each level are subjected to matrix addition operations to obtain the fused feature set.
6. The method as described in claim 5, characterized in that, The determination of the fusion coefficient based on the number of training iterations includes: The training period is determined based on the number of training cycles. The fusion coefficient is determined based on the training period. Specifically, the fusion coefficient is set to 0 during the early training period and 1% during the middle training period. ,in, The number of training iterations is used, and the fusion coefficient is set to 1 during the later stages of training.
7. A target network model acquisition device based on fusion features, characterized in that, include: The acquisition module is used to acquire the teacher network model, the student network model, and the training dataset. The number of parameters in the student network model is less than the number of parameters in the teacher network model. The training dataset specifically consists of images and labels. The extraction module is used to input the training dataset into the student network model to extract the student's last layer feature map, and to input the training dataset into the teacher network model to extract the teacher's feature maps at each layer. The first obtaining module is used to process the student's final layer feature map and the teacher's feature maps at each layer respectively to obtain the student's final layer target feature map and the teacher's target feature maps at each layer. The size and number of channels of the student's final layer target feature map are equal to the size and number of channels of the teacher's target feature maps at each layer. The fusion module is used to fuse the student's final layer target feature map with the teacher's target feature maps at each layer to obtain a fused feature set; The determination module is used to determine the multilayer distillation loss based on the fused feature set, wherein the determination module is used to: Based on the fused feature set, the single-layer distillation loss is determined, specifically as follows: ； in, For any single-layer distillation loss, For any fusion feature in the fusion feature set, The features in each layer of the student feature set are those excluding the student's last layer feature map. , This refers to the sequence number of the last functional block in the student network model. Calculation of mean squared error loss; Based on the single-layer distillation loss, the multi-layer distillation loss is obtained through iterative calculation, specifically as follows: ； in, This is due to losses from multiple distillation processes; The second obtaining module is used to adjust the parameters of the student network model based on the multilayer distillation loss to obtain the target network model.
8. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method as described in any one of claims 1 to 6.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the method as described in any one of claims 1 to 6.