Inter-class similarity knowledge distillation method and model based on feature prototype projection

Abstract

The application discloses an inter-class similarity knowledge distillation method and model based on feature prototype projection, and the method comprises the following steps: taking the weighted average value of the intra-class features coded by a teacher network as a feature prototype after L2 norm standardization; using the feature prototype to respectively perform projection mapping calculation on teacher features and student features, and correspondingly obtaining inter-class cosine similarity of the teacher features and inter-class cosine similarity of the student features; taking the inter-class cosine similarity of the teacher features as expected knowledge, and constructing a mean square error loss function to perform knowledge distillation and guide the student network to learn the inter-class similarity distribution. Compared with other distillation methods, the method can bring the largest amplitude of recognition performance improvement, and guarantee the real-time performance and accuracy of system application.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of plankton recognition and knowledge distillation, and in particular to a plankton inter-class similarity knowledge distillation method based on feature prototype projection and a related network model. BACKGROUND

[0002] A neural network model with a large number of parameters can generally achieve higher recognition accuracy, but in most industrial application scenarios, due to the limitation of computing resources and cost of funds, it is difficult for a large model to be deployed and applied in ordinary hardware devices. A small model has fewer parameters and faster inference speed, and is more suitable for engineering deployment than a large model, but the recognition accuracy of a small model is relatively low. Knowledge distillation is an effective method to improve the recognition accuracy of a small model, and its core idea is to extract the information learned by a "teacher" network (usually a large model with better recognition performance) as expected knowledge to a "student" network to help the small model improve recognition accuracy. As shown in Figure 1 The general framework diagram of the knowledge distillation method is shown in Figure 1, and the knowledge distillation method mainly includes intermediate layer feature or feature relationship distillation and classification probability distillation.

[0003] Knowledge distillation method based on classification probability distribution (KD): Hinton et al. proposed a knowledge distillation method based on probability distribution output in the document "Hinton G, Vinyals O, Dean J. Distilling the Knowledge in a Neural Network[J]. Computer Science, 2015, 14(7):38-39", using the classification probability as a soft label, and using an annealing temperature coefficient to smooth the probability distribution, and then using the KL divergence between the smoothed result and the probability distribution output by the classification layer of the "student" network as one of the loss functions, formula (1). In addition, the conventional hard label, i.e. the CE cross-entropy loss between the classification probability output by the "student" network and the true label, is also needed, and the weighted sum of the two losses is used as the total loss function, formula (2). This method involves two hyperparameters: the annealing temperature coefficient and the proportion coefficient α of soft and hard labels. In the distillation experiment, the parameters need to be adjusted for different situations to make the network achieve better results.

[0004] (1)

[0005] (2)

[0006] Since the weight a of the method and the temperature coefficient are artificially selected, which has great subjectivity, so the distillation performance is easily affected by subjectivity and finally affects the recognition accuracy of the student network.

[0007] FitNet: Romero et al. proposed to distill the feature maps of the intermediate layers in the network in the document “Romero A, Ballas N, Kahou S E, et al. FitNets: Hints for Thin Deep Nets [C] / / In International Conference on Learning Representations, 2015. ICLR 2015”, which takes the intermediate layer features of the teacher network as expected knowledge. However, since the dimensions of the teacher features and the dimensions of the student features are different, the student features need to be mapped to a space consistent with the dimensions of the teacher features to obtain new student features; and the total loss function of this method also needs the participation of the cross-entropy loss function CE, and its total loss function is Therefore, this method also needs to select the weight a artificially, so the training process is still affected by human factors.

[0008] SP: Tung et al. in the document “Tung F, Mori G. Similarity-Preserving Knowledge Distillation [C] / / International Conference on Computer Vision, 2019. ICCV 2019” believed that the similarity between different input sequences after network coding should be consistent in both the teacher network feature space and the student network feature space, so they proposed a distillation method based on the similarity between individual features to guide the network to learn the similarity between the teacher space features. The advantage of this method is that it does not need to map the student network features to a space consistent with the dimensions of the teacher network features. However, the total loss function of this method also needs the participation of the cross-entropy loss function CE, and its total loss function is Therefore, this method also needs to select the weight a artificially, so the training process is still affected by human factors.

[0009] Since the in-situ monitoring of plankton requires real-time processing of data, the algorithm for target recognition needs to consider both accuracy and real-time performance, thus requiring the neural network model used in the target recognition part to be both highly accurate and lightweight. SUMMARY

[0010] In view of this, the present application proposes a plankton inter-class similarity knowledge distillation method based on feature prototype projection (PPD), which extracts and migrates the intra-class and inter-class similarity information learned by the "teacher" model on plankton samples to a "student" model with smaller parameter quantity and faster reasoning speed, so as to improve the recognition accuracy of the "student" model for plankton.

[0011] A feature prototype projection-based inter-class similarity knowledge distillation method, comprising the following steps: S1, performing L2 norm standardization on the weighted average value of the intra-class features coded by the teacher network to obtain a feature prototype; S2, using the feature prototype to respectively project and map the teacher features and the student features, and correspondingly obtaining the inter-class cosine similarity of the teacher features and the inter-class cosine similarity of the student features; S3, taking the inter-class cosine similarity of the teacher features as expected knowledge, constructing a mean square error loss function to perform knowledge distillation, and guiding the student network to learn the inter-class similarity distribution.

[0012] Further, step S1 specifically comprises: S11, using the teacher network to extract features of the sample, and the extracted feature vectors collectively constitute a teacher feature space; S12, for each class in the teacher feature space, summing and averaging the intra-class features to obtain the average features of the corresponding class; S13, performing L2 norm standardization on the average features of the corresponding class to normalize the vectors to the range of [-1, 1] to obtain the feature prototype of the corresponding class.

[0013] Further, step S2 specifically comprises: S21, concatenating the feature prototypes of all classes by rows to obtain a feature prototype matrix; S22, using the teacher network and the student network to extract features of the sample respectively to obtain the teacher features and the student features; S23, the process of performing inner product operation on the teacher features and the student features after L2 norm standardization and the feature prototype matrix is recorded as a mapping function , and the teacher features and the student features are projected into the class space through the mapping function respectively, and correspondingly the inter-class cosine similarity of the teacher features and the inter-class cosine similarity of the student features are obtained.

[0014] Further, step S2 further comprises: performing After standardization by norm, the inner product is taken with the feature prototype to obtain the cosine similarity between the teacher features and the feature prototype; the student features are then processed... After standardization by norm, the inner product is taken with the feature prototype to obtain the cosine similarity between student features and feature prototypes; the mapping function is defined based on the calculation expressions for the cosine similarity between teacher features and feature prototypes, and between student features and feature prototypes. .

[0015] Furthermore, teacher characteristics and student characteristics are respectively processed through mapping functions. The steps for projecting onto the category space include:

[0016]

[0017]

[0018] in, , These represent the inter-class cosine similarity of teacher characteristics and the inter-class cosine similarity of student characteristics, respectively. , These are teacher characteristics and student characteristics, C T Represents the feature prototype matrix C transpose, Represents the teacher's feature vector The j-th element in Represents student feature vectors The j-th element in; D Teacher feature vector The total number of elements in the vector is equal to the student feature vector. The total number of elements in the array.

[0019] Furthermore, step S3 specifically includes: calculating and The mean squared error loss between the two is used to backpropagate the gradient to the student network, update the network parameters of the student network, and guide the student network to learn the similarity between individual samples and each category under the teacher network encoding.

[0020] Furthermore, the mean squared error loss function is:

[0021]

[0022] Where N is the total number of samples.

[0023] The application further provides an inter-class similarity knowledge distillation model based on feature prototype projection, comprising a teacher encoder, a student encoder, a feature prototype construction module, a teacher feature mapping module, a student feature mapping module and a loss function construction module; the output of the teacher encoder and the output of the feature prototype construction module are both connected to the input of the teacher feature mapping module, and the output of the student encoder and the output of the feature prototype construction module are both connected to the input of the student feature mapping module; the output of the teacher feature mapping module and the output of the student feature mapping module are both connected to the input of the loss function construction module; the teacher encoder and the student encoder respectively receive sample input and perform feature extraction, and correspondingly obtain teacher features and student features; the feature prototype construction module is used for calculating the weighted average of the intra-class features of the teacher network coding and performing L2 norm standardization to obtain a feature prototype; the teacher feature mapping module is used for performing projection mapping calculation on the teacher features by using the feature prototype to obtain the inter-class cosine similarity of the teacher features; the student feature mapping module is used for performing projection mapping calculation on the student features by using the feature prototype to obtain the inter-class cosine similarity of the student features; and the loss function construction module is used for taking the inter-class cosine similarity of the teacher features as expected knowledge to construct a mean square error loss function for knowledge distillation to guide the student network to learn the inter-class similarity distribution.

[0024] The application further provides a computer readable storage medium, which stores a computer program, and the computer program can realize the steps of the foregoing method when executed by a processor.

[0025] The application has the advantages that the feature prototype is used as the clustering center, the influence of sample data that is incorrectly classified by humans or has unclear biological features in the plankton classification training set on the recognition performance of the network can be avoided, only one plankton inter-class similarity loss function is used, the plankton classification probability loss calculation does not need to be additionally increased, the weight coefficient between the loss functions does not need to be set, and the distillation performance is not affected by manual adjustment of the weight coefficient. The intra-class and inter-class similarity information learned by the teacher network on the plankton samples is extracted and migrated to the student network which has smaller parameters and faster reasoning speed, and it is verified through a contrast experiment that the method can bring the largest amplitude of plankton recognition performance improvement compared with other distillation methods, and the real-time performance and accuracy of the system application are ensured. BRIEF DESCRIPTION OF DRAWINGS

[0026] Figure 1 It is a general framework diagram of the knowledge distillation method.

[0027] Figure 2 It is a PCA dimension reduction visualization effect diagram of the South China Sea plankton teacher feature space.

[0028] Figure 3 The inter-class similarity matrix heat map of the individual characteristics of the South China Sea plankton and the characteristic prototypes.

[0029] Figure 4 The workflow diagram of the plankton inter-class similarity knowledge distillation method based on characteristic prototype projection proposed in the embodiment of the present application. DETAILED DESCRIPTION

[0030] The present application will be further described below in conjunction with the drawings and specific embodiments.

[0031] ① A lightweight network model with high recognition accuracy is required on the plankton monitoring equipment, but the lightweight model easy to deploy often has low accuracy due to the small number of parameters; ② The knowledge distillation method commonly used to improve the recognition accuracy of the lightweight model often needs to manually set the weight coefficient between the loss functions, which introduces too many subjective factors; ③ The data with artificial error classification and the data with unclear biological characteristics in the training set will affect the recognition performance of the network. In view of the above three problems, the present application embodiment proposes a plankton inter-class similarity knowledge distillation method based on characteristic prototype projection, so that the lightweight network model for deployment can also obtain high accuracy.

[0032] Figure 4 The workflow diagram of the plankton inter-class similarity knowledge distillation method based on characteristic prototype projection (hereinafter referred to as "PPD" method) proposed in the embodiment of the present application. As shown in Figure 4 The PPD method of the present application embodiment includes: taking the weighted average value of the intra-class features coded by the teacher network (teacher encoder) as the characteristic prototype after L2 norm standardization, using the characteristic prototype to respectively project and map the teacher features and the student features to calculate the corresponding inter-class cosine similarity, taking the inter-class cosine similarity matrix of the teacher as the expected knowledge, constructing the mean square error loss function to distill the knowledge, guiding the student network to learn the inter-class similarity distribution, thereby improving the recognition accuracy of the student network. Here, the loss function plays a role in guiding the student network, because the neural network is guided to optimize the model with the goal of minimizing the loss function, and the present application introduces a loss function with teacher network information (inter-class cosine similarity given by the teacher network) in the training of the student network. The student network can constantly approach the teacher network with the goal of minimizing this loss function, that is, try to output a result close to the inter-class cosine similarity of the teacher network, so as to learn the distribution of the inter-class cosine similarity.

[0033] The work of the neural network can be understood as mapping the data sample space to a high-dimensional feature expression space . Suppose there are samples in total, categories constitute the sample space , class label , each class contains k =1,2,…, K samples, i.e., the class label = 1 contains M 1 sample, the class label = 2 contains M 2 samples, and so on, the class label = K contains M K samples, thus . The encoder of the neural network processes the input samples i =1,2,…, N to obtain the corresponding features . The cosine similarity between any two features in the sample space after L2 norm normalization is , and the calculation formula is shown in equation 3.1:

[0034] (3.1)

[0035] where is the cosine similarity between any two samples and .

[0036] In the classification task, ideally, the feature vectors of different classes are orthogonal, i.e., the cosine similarity between two feature vectors ; the feature vectors of the same class are in the same direction, i.e., the cosine similarity between vectors . The goal of the network is to distinguish the features of different classes, i.e., to minimize the inter-class similarity as much as possible while maximizing the intra-class similarity as much as possible.

[0037] In the feature space, the similarity of the feature vectors of the same class is close enough, and the similarity between the feature vectors of different classes is far enough. The average value of the feature vectors of the same class is taken as the mathematical expectation feature to represent the class, and after the norm normalization, it is called the feature prototype of the class.

[0038] Suppose there are a total of N samples to form the sample space , where there are classes, and the class label , k =1,2,…, K ​​​) indicates category label The teacher sample subspace composed of the samples, teacher sample subspace Includes One sample, The teacher network is fully trained using CE loss (cross-entropy loss), resulting in a teacher encoder denoted as... Given sufficient data labels, the teacher network should have adequate ability to distinguish feature categories. Therefore, it will pass through the teacher encoder. The features obtained from the mapping are weighted and averaged along each dimension, and the resulting average features are then processed... Using norm standardization as the feature prototype for this category aligns with statistical principles. Furthermore, using the average feature... Norm standardization has another advantage as a feature prototype: it can avoid interference from individual outliers and reduce the damage of noisy data to classification performance.

[0039] Because the next step only requires the teacher encoder As a feature extractor, gradient backpropagation updates are unnecessary; therefore, the parameters of the teacher network can be fixed during the feature extraction stage. A teacher encoder is used. Feature extraction is performed on all samples (Equation 3.2):

[0040] (3.2)

[0041] in, t i These refer to the sample features extracted by the teacher encoder, known as teacher features. The extracted feature vectors collectively constitute the teacher feature space. ,in Indicates category label The category feature subspace composed of features (the upper right corner here) T (Representing "teachers"). Setting The number of features is , subspace The average feature is obtained by summing all the features within the range and taking the average value. (Equation 3.3), with As The category center.

[0042] (3.3)

[0043] right conduct Norm standardization normalizes a vector to a normal form. Within the range, the corresponding feature prototypes are obtained. (Form 3.4):

[0044] (Form 3.4):

[0045] In the above formula means the jth element in the vector, D is the total number of elements in the vector.

[0046] Concatenate all the feature prototypes of each category by row (the "row hang" of the matrix) to obtain the feature prototype matrix (Form 3.5):

[0047] (Form 3.5):

[0048] After the teacher feature is norm-normalized, it is done with the feature prototype inner product, to obtain the cosine similarity of and (Form 3.6):

[0049] (Form 3.6): The student encoder is

[0050] , and the sample goes through the student encoder to obtain the student feature (Form 3.7):

[0051] (Form 3.7): After the student feature

[0052] is norm-normalized, it is done with the feature prototype inner product, to obtain the cosine similarity of and (Form 3.8):

[0053] (Form 3.8): Based on the cosine similarity between the above teacher feature and the feature prototype

[0054] and the cosine similarity between the student feature and the feature prototype (Form 3.8): ​​​The calculation expression and matrix multiplication can be used to derive: Teacher characteristics With all feature prototypes Cosine similarity can be used go through The result after norm standardization and the feature prototype matrix The dot product operation is performed to obtain the result in one step. Similarly, student characteristics... With all feature prototypes Cosine similarity can be used go through The result after norm standardization and the feature prototype matrix The result is obtained by performing a dot product operation in one step. Here, the "teacher characteristics / student characteristics" are processed... L The result after 2-norm standardization and the feature prototype matrix Performing a dot product operation yields the teacher features / student features and all feature prototypes in one step. The process of "cosine similarity" is denoted as the mapping function. Teacher characteristics and student characteristics After passing through mapping functions respectively Projecting onto the category space yields the corresponding inter-class similarity. (Equation 3.9) and (Equation 3.10):

[0055] (3.9)

[0056] (3.10)

[0057] in, , These represent the inter-class cosine similarity of teacher characteristics and the inter-class cosine similarity of student characteristics, respectively. C T Represents the feature prototype matrix C transpose, Represents the teacher's feature vector The j-th element in Represents student feature vectors The j-th element in D It is both a teacher feature vector The total number of elements in the vector is also the student feature vector. The total number of elements in the array.

[0058] calculate and The mean square error loss between the student network and the teacher network is calculated, and the gradient of the student network is returned to update the network parameters, so that the student network learns the similarity between the individual sample and each class under the encoding of the teacher network. The expression is shown as 3.11:

[0059] (3.11)

[0060] The method provided by the embodiment of the application can be used for research and development of a target recognition model in a plankton underwater optical imager, so that the model has less occupied memory and faster processing speed, thereby improving the real-time performance of the imager and reducing the requirement for hardware devices of the imager. A data set used for training the model usually needs high precision, but the plankton has various morphologies, the imaging scale spans microns and centimeters, and due to the influence of underwater imaging conditions, the imaging effect is not particularly ideal, so the process of artificially making the data set is inevitably prone to the problem of false classification. The application has strong resistance to "noise data" in the data set, and can greatly reduce the damage of noise to the network performance. Therefore, the application has strong applicability to a data set containing certain noise.

[0061] In addition, the application can also be used in many target detection scenes in the medical field, such as lung nodule detection and fundus lesion detection. In the development process of a special medical instrument, real-time performance and precision are certainly important indicators, so the precision and lightweight of the recognition model are certainly one of the research focuses. The application can reduce the influence of artificially false classified data on the network performance while making the network lightweight, thereby bringing the maximum performance improvement to the network, and thus has certain reference significance.

[0062] The effectiveness of the application is verified through a specific example.

[0063] The spatial features of the plankton samples in the South China Sea The teacher feature space obtained after encoding by the teacher network The principal component analysis method (PCA) is used to reduce the features from 1024 dimensions to 2 dimensions, and the visualization effect of the reduced feature space is shown in Figure 2 It can be seen that the features of the same class are basically clustered together, and there is a certain distance between the classes, but there are individual outliers. Most of these outliers are misclassified during artificial library building, or the biological features of the samples are not clear. The five-point star represents the average feature in the class, and the average feature of each class is located at the center of the corresponding clustering cluster, so the average feature is used to The norm is normalized as a feature prototype, which meets the mathematical expectation, avoids the interference of outliers, and reduces the damage of noise data to the network classification performance.

[0064] Figure 3 is the inter-class cosine similarity matrix heat map of the individual characteristics of plankton after projection of the feature prototype, 0-11 on the left represents the corresponding features of each sample extracted from class 0-class 11, 0-11 at the bottom represents the feature prototype of class 0-class 11, and all the numbers on the heat map represent the inter-class similarity between the individual characteristics and the feature prototype, for example, 0.12 in the first column of the 0th row represents the inter-class similarity between the individual characteristics of class 0 and the feature prototype of class 1. It can be seen that the similarity between the individual characteristics and the feature prototype of the same class is the largest, and the similarity with other classes is smaller, and these similarities are not absolutely 0 or 1. In fact, the physical forms of plankton of different classes are indeed similar and different, so compared with the hard label of 0 or 1 used in ordinary classification training, the soft label of guiding the network to learn the inter-class feature similarity is more in line with the physical reality and can improve the recognition and generalization performance of the network.

[0065] The South China Sea plankton classification dataset (30,000 samples, 12 plankton classes) is used as experimental data. Swin-B (Swin Transformer Base) is selected as the teacher network, which has the highest recognition accuracy of 94.34% on the South China Sea plankton dataset. The parameter quantity of Swin-B is 87M, and the inference speed is 26 samples per second. Since each class feature prototype is obtained by L2 norm standardization of the weighted average of the intra-class features output by the "teacher" network, its dimension is consistent with the output feature dimension of the "teacher" network. When calculating the inter-class similarity between the student feature and the feature prototype, the inner product operation of the vector is included, which requires the dimension of the student feature to be consistent with the feature prototype, that is, consistent with the output feature dimension of the "teacher" network. In this experiment, five deep neural network models with different parameter quantities and different architectures are selected as student networks. Among them, the output feature dimension of ShNet V2 is consistent with that of Swin-B, so no special treatment is needed, while the feature vector dimensions output by other "student" networks are different from that of Swin-B. Therefore, in order to complete the similarity calculation operation between the "student" individual feature and the feature prototype, the "student" network output feature needs to be projected into a space with the same dimension as the "teacher" network output feature. Therefore, a multi-layer perceptron structure (MLP) is added to the other four "student" networks in this experiment, that is, a fully connected layer and a GELU activation function are added to complete the dimension conversion. The experimental results are shown in Table 1.

[0066] Table 1 Comparison of plankton classification performance of five knowledge distillation methods

[0067]

[0068] The neural networks with five different parameter magnitudes and architectures are used as student networks, and the improvement effects of the PPD method and other three knowledge distillation methods on the plankton recognition performance are compared. In Table 1, the first column CE represents the method of using the standard cross-entropy loss function for classification training, without using the knowledge distillation method for auxiliary training. Here, the recognition accuracy reached by the CE training is used as the baseline reference. The accuracy improvement amplitude of the method proposed in the application on the four student networks reaches the optimal effect or the effect of being flat with other distillation methods.

[0069] ShNet V2 (ShuffleNet V2) is improved from 92.35% to 93.13%, which exceeds the 93.02% obtained by the ordinary training of resnet 50, and the parameter amount is only one twentieth of res50, and the inference speed is four times that of res50. Swin-T (SwinTransformer Tiny) is improved from 93.82% to 94.21%, which is slightly lower than the "teacher" network Swin-B, but the parameter amount is only one third of Swin-B, and the inference speed is three times that of Swin-B. From the above experimental results, it can be known that the method of the application can use the knowledge of a large model to improve the recognition performance of a small model, and then improve the inference speed and reduce the deployment difficulty of the model.

[0070] In addition, another embodiment of the present application also proposes an inter-class similarity knowledge distillation model based on feature prototype projection, comprising: a teacher encoder, a student encoder, a feature prototype construction module, a teacher feature mapping module, a student feature mapping module, and a loss function construction module. The output of the teacher encoder and the output of the feature prototype construction module are both connected to the input of the teacher feature mapping module, and the output of the student encoder and the output of the feature prototype construction module are both connected to the input of the student feature mapping module; the output of the teacher feature mapping module and the output of the student feature mapping module are simultaneously connected to the input of the loss function construction module; the teacher encoder and the student encoder respectively receive sample input and perform feature extraction, and correspondingly obtain teacher features and student features; the feature prototype construction module is used to calculate the weighted average of the intra-class features of the teacher network coding and is normalized by L2 norm to obtain a feature prototype; the teacher feature mapping module is used to project and map the teacher features using the feature prototype to obtain the inter-class cosine similarity of the teacher features; the student feature mapping module is used to project and map the student features using the feature prototype to obtain the inter-class cosine similarity of the student features; and the loss function construction module is used to take the inter-class cosine similarity of the teacher features as expected knowledge, and construct a mean square error loss function for knowledge distillation to guide the student network to learn the inter-class similarity distribution.

[0071] Furthermore, another embodiment of the present application also proposes a computer readable storage medium having a computer program stored thereon, which, when executed by a processor, can implement the steps of the inter-class similarity knowledge distillation method based on feature prototype projection described in the foregoing embodiments of the present application. The present application can be in the form of a computer program product implemented on one or more computer usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer usable program code.

[0072] The above is a further detailed description of the present application in combination with specific preferred embodiments, and the specific implementation of the present application should not be limited to these descriptions. For those skilled in the art to which the present application belongs, without departing from the concept of the present application, a number of equivalent alternatives or obvious variations can be made, and the same performance or use should be regarded as belonging to the protection scope of the present application.

Claims

1. A knowledge distillation method for inter-class similarity of planktonic organisms based on feature prototype projection, used for planktonic organism identification, characterized in that, Includes the following steps: S1. Use a teacher network to extract features from planktonic samples. The extracted feature vectors together constitute the teacher feature space. For each planktonic category in the teacher feature space, sum the features within the category and take the average value to obtain the average feature of the corresponding planktonic category. Standardize the average feature of the corresponding planktonic category using the L2 norm to obtain the feature prototype of the corresponding planktonic category. S2. Concatenate the feature prototypes of all planktonic categories row-wise to obtain a feature prototype matrix; extract features from planktonic samples using teacher and student networks respectively, obtaining corresponding teacher and student features; the process of standardizing the teacher and student features using the L2 norm and then performing an inner product operation with the feature prototype matrix is ​​denoted as the mapping function. Teacher characteristics and student characteristics are respectively processed by mapping functions Projecting onto the planktonic category space yields the cosine similarity between planktonic categories for teacher characteristics and the cosine similarity between planktonic categories for student characteristics. S3. Using the cosine similarity between planktonic classes based on teacher characteristics as expected knowledge, a mean squared error loss function is constructed to distill the similarity knowledge between planktonic classes, guiding students to learn about the similarity distribution between planktonic classes online.

2. The method for distilling knowledge of inter-class planktonic similarity as described in claim 1, characterized in that, In step S1, the average features of the corresponding plankton category are standardized using the L2 norm, and the vector is normalized to the range of [-1,1] to obtain the feature prototype of the corresponding plankton category.

3. The method for distilling knowledge about the similarity between planktonic organisms as described in claim 1, characterized in that, Step S2 also includes: Teacher characteristics After standardization by norm, the inner product is taken with the feature prototype to obtain the inter-class cosine similarity between the teacher feature and the feature prototype; Student characteristics After standardization by norm, the inner product is taken with the feature prototype to obtain the cosine similarity between the student features and the feature prototypes; The mapping function is defined based on the calculation expressions for the cosine similarity between planktonic classes and between teacher characteristics and their prototypes, as well as the cosine similarity between student characteristics and their prototypes. .

4. The method for distilling knowledge of inter-class planktonic similarity as described in claim 1, characterized in that, Teacher characteristics and student characteristics are respectively processed by mapping functions The steps involved in projecting onto the planktonic category space include: in, , The cosine similarity between planktonic groups represents teacher characteristics, and the cosine similarity between planktonic groups represents student characteristics, respectively. , These are teacher characteristics and student characteristics, C T Represents the feature prototype matrix C transpose, Represents the teacher's feature vector The j-th element in Represents student feature vectors The j-th element in; D Teacher feature vector The total number of elements in the vector is equal to the student feature vector. The total number of elements in the array.

5. The method for distilling knowledge of inter-class planktonic similarity as described in claim 4, characterized in that, Step S3 specifically includes: calculate and The mean squared error loss between the two is calculated, and gradient backpropagation is performed on the student network to update the network parameters of the student network, guiding the student network to learn the similarity between individual planktonic samples and various planktonic categories under the teacher network encoding.

6. The method for distilling knowledge of inter-class similarity among planktonic organisms as described in claim 5, characterized in that, The mean squared error loss function is: Where N is the total number of planktonic samples.

7. A knowledge distillation device for inter-class similarity of planktonic organisms based on feature prototype projection, characterized in that, include: The module includes a teacher encoder, a student encoder, a feature prototype construction module, a teacher feature mapping module, a student feature mapping module, and a loss function construction module. The outputs of the teacher encoder and the feature prototype construction module are both connected to the input of the teacher feature mapping module, and the outputs of the student encoder and the feature prototype construction module are both connected to the input of the student feature mapping module; the outputs of the teacher feature mapping module and the student feature mapping module are simultaneously connected to the input of the loss function construction module. The teacher encoder and the student encoder respectively receive planktonic sample input and perform feature extraction to obtain teacher features and student features respectively. The feature prototype construction module is used to calculate the weighted average of the intraclass features of planktonic organisms encoded by the teacher network and then standardize it using the L2 norm to obtain the feature prototype. The teacher feature mapping module is used to perform projection mapping calculation on teacher features using feature prototypes to obtain the cosine similarity between planktonic groups of teacher features. The student feature mapping module is used to perform projection mapping calculation on student features using feature prototypes to obtain the cosine similarity between planktonic groups of student features. The loss function construction module is used to construct a mean squared error loss function for distilling knowledge of the similarity between plankton classes, using the cosine similarity between plankton classes based on teacher features as expected knowledge, so as to guide students to learn the similarity distribution between plankton classes online.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program can perform the steps of the method according to any one of claims 1-6.

Images