A fish identification method, system, device and medium based on knowledge distillation

Abstract

The present invention discloses a fish identification method, system, device and medium based on knowledge distillation, and relates to the field of image recognition technology. The method comprises: obtaining sample data; constructing a knowledge distillation network model; extracting sample features; constructing an enhanced knowledge distillation loss; constructing a contrast loss based on similarity; constructing a total loss function; training a student network model; obtaining real-time fish images; and inputting the fish images into the student network model, which outputs classification results. The present invention performs enhanced knowledge distillation by splicing logits and features generated from converted data and normal data. Then, in a self-supervised task, key sample pairs are mined and the similarity relationship of cross samples is defined by the intrinsic structure of the data without relying on external labels or supervision, and knowledge transfer is achieved through contrastive learning.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a fish recognition method, system, device and medium based on knowledge distillation. Background Art

[0002] Deep neural networks have made remarkable progress in recent decades and have revolutionized the field of computer vision. Tasks such as image classification, object detection, and semantic segmentation have achieved impressive results and have been successfully applied to various practical scenarios. However, the pursuit of excellent performance has led to the continuous expansion of the size of neural networks, which limits the widespread application of these high-performance models on mobile devices. To address this challenge, knowledge distillation has emerged as an effective technique that transfers knowledge from a large teacher model to a smaller student model. This innovative approach provides a solution for deploying high-performance models on resource-constrained mobile devices.

[0003] The goal of knowledge distillation is to generate a soft target for the teacher model to guide the training of the student model by increasing the final softmax temperature. This guidance process involves minimizing the Kullback-Leibler divergence between the teacher and student outputs. Even though the teacher model may assign probabilities to the wrong categories, the existence of the relative probabilities of these wrong answers provides very rich information for generalized training patterns. The knowledge hidden in these secondary probabilities is often referred to as "dark knowledge". However, this highly abstract concept of dark knowledge ignores a large part of the complex information encoded in the hidden layers. Therefore, subsequent research naturally proposed transferring feature maps or their more subtle information between the intermediate layers of the teacher and the student to provide more refined guidance to the student. Although methods based on logical knowledge and feature knowledge have achieved considerable success, they are highly task-specific because they all originate from supervised tasks, usually normal classification tasks. This specificity may hinder the network from exploring richer feature representations.

[0004] To mine non-task-specific knowledge as a supplement to supervised classification knowledge. SSKD combines self-supervised tasks with knowledge distillation for the first time by adding lightweight auxiliary modules to the teacher's backbone, applying auxiliary tasks to the teacher, updating the auxiliary modules when the backbone is frozen, and then extracting the corresponding self-supervised signals from the auxiliary modules for distillation. DOGO learns self-supervised visual representations that are not specific to classification tasks by contrasting semantically similar (positive) and dissimilar (negative) data sample pairs, so that similar data sample pairs achieve maximum consistency through contrastive loss in the latent space. HSSAKD proposes a rotation-based self-supervised enhancement task and further transfers rich knowledge from hierarchical feature maps using well-designed auxiliary branches. Inspired by instance discrimination in self-supervised learning, EGA models instance-to-instance relationships through a graph formula in the feature embedding space and extracts teacher knowledge into the student network by adjusting the teacher graph and the student network.

[0005] In summary, the purpose of the present invention is to solve the problem that current knowledge distillation methods mainly focus on transferring task-specific knowledge from supervised tasks, such as logits and features, while ignoring the valuable information hidden in the cross-sample similarities between teacher and student models. Its effect in fish image recognition and classification needs to be improved. Summary of the invention

[0006] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.

[0007] In view of the problems existing in the above-mentioned existing fish identification method, system, equipment and medium based on knowledge distillation, the present invention is proposed.

[0008] Therefore, the purpose of the present invention is to provide a fish identification method, system, device and medium based on knowledge distillation, which aims to solve the problem that current knowledge distillation methods mainly focus on transferring task-specific knowledge from supervised tasks, such as logits and features, while ignoring the valuable information hidden in the cross-sample similarities of teacher and student models.

[0009] In order to solve the above technical problems, the present invention provides the following technical solutions:

[0010] A fish identification method based on knowledge distillation, the method comprising the following steps:

[0011] S1: Obtain sample data; wherein the sample data includes: image sample data and label data;

[0012] S2: construct a knowledge distillation network model; wherein the knowledge distillation network model includes a teacher network model and a student network model, both of which use convolutional neural networks, and the teacher network model is pre-trained;

[0013] S3: Extract sample features; the preprocessed fish images are called common fish image samples, and the preprocessed and data-enhanced fish images are called converted fish image samples;

[0014] The normal and converted fish image sample data are input into the pre-trained teacher network model and the untrained student network model respectively, the convolution layer of the teacher network model outputs the normal and converted first intermediate feature maps, the fully connected layer of the teacher network model outputs the normal and converted first logits features, the convolution layer of the student network model outputs the normal and converted second intermediate feature maps, and the fully connected layer of the student network model outputs the normal and converted second logits features;

[0015] S4: Construct enhanced knowledge distillation loss; concatenate the ordinary and converted first logits features to generate enhanced first logits features, concatenate the ordinary and converted second logits features to generate enhanced second logits features, and construct enhanced logits knowledge distillation loss;

[0016] The ordinary and transformed first intermediate features are concatenated to generate enhanced first intermediate features, and the ordinary and transformed second intermediate features are concatenated to generate enhanced second intermediate features, and the enhanced feature knowledge distillation loss is constructed. The enhanced logits knowledge distillation loss and the enhanced feature knowledge distillation loss are added to obtain the enhanced knowledge distillation loss;

[0017] S5: Construct similarity-based contrast loss;

[0018] Based on the converted first logits and the converted second logits, the cross-sample similarity relationship is defined, a template strategy is constructed according to the difference between the two, a cosine decay strategy is constructed according to the similarity strength of the samples, and the template strategy and the cosine decay strategy are used to optimize the cross-sample similarity relationship to obtain the contrast loss;

[0019] S6: construct a total loss function; wherein the total loss includes the enhanced knowledge distillation loss and the similarity-based contrast loss;

[0020] S7: training student network model;

[0021] Use the total loss function calculated in step S6 to train the student network model, and back-propagate to update the parameters of the student network model to obtain a mature student network model;

[0022] S8: Obtain real-time fish images; input the fish images into the student network model, and the student network model outputs classification results.

[0023] As a preferred solution of the fish identification method based on knowledge distillation of the present invention, in which: in the S1 process, the acquired fish image sample data is preprocessed and data enhanced, the preprocessing includes cropping, scaling and rotation, and the data enhancement includes color jittering, grayscale conversion and Gaussian blur.

[0024] As a preferred solution of the fish identification method based on knowledge distillation of the present invention, in which: in S4, the knowledge distillation loss is and The specific construction process is:

[0025] a. Enhanced logits knowledge distillation loss The expression is:

[0026] = = CE ;

[0027] Among them, s and t represent students and teachers respectively. and is a hyperparameter used to balance the weights of the two losses. and It means that the enhanced first logits and the enhanced second logits generate a prediction vector through the softmax function. represents the prediction vector generated by the ordinary second logits through the softmax function, CE represents the cross entropy function, KLD represents the KL divergence function, represents the true label, T represents a temperature hyperparameter;

[0028] b. Enhanced feature knowledge distillation loss The expression is:

[0029] = MSE ;

[0030] Among them, s and t represent students and teachers respectively, b represents the batch size, and for the i-th sample in the batch, the corresponding Represents the first The features of the layer, MSE represents the mean square error function, Represents the self-attention weight of the i-th sample.

[0031] As a preferred solution of the fish identification method based on knowledge distillation of the present invention, in which: in S4, when calculating the self-attention weight of the i-th sample The specific method is:

[0032] First, calculate the similarity matrix between teachers and students and ,in;

[0033] =R( )·R ; =R( )·R ;

[0034] Among them, R(·) represents the deformation operation of the feature, Represents the transpose operation of the matrix, and then passes through the multi-layer perceptron MLP to obtain the vector and vector :

[0035] =MLPQ ; =MLPQ ;

[0036] Then the vector and vector Through the softmax function F(·), the self-attention weights of all samples in the batch can be obtained ;

[0037] Self-attention weight of the i-th sample in the batch The calculation process is as follows:

[0038] =F( · )= .

[0039] As a preferred solution of the fish identification method based on knowledge distillation of the present invention, in which: in the process of S5, when constructing the comparison loss function based on similarity, the specific method is:

[0040] a. Divide the positive and negative sets into As a decision boundary Whether it belongs to the positive set or the negative set, the formula is as follows:

[0041]

[0042] in, represents a positive set, represents a negative set;

[0043] b. Use abbreviated symbols = , = ,Will and The relationship between is called cross-sample similarity, and knowledge is transferred through contrastive learning;

[0044] From the main set and negative set Mining sample pairs of various difficulties, namely difficult positive sample pairs and positive and negative sample pairs , medium positive and negative sample pairs and negative sample pairs Simple positive sample to negative sample pair ,The detailed mining process is as follows:

[0045] = , = ,

[0046] = , = ,

[0047] = , = ;

[0048] c. Template strategy, reallocation during optimization and The weight in the template is calculated by taking the teacher network as the main guide of the student, and the formula is defined as follows:

[0049] ;

[0050] in, Indicates zero-cutoff operation, the purpose is to ensure and non-negative;

[0051] d. Cosine decay strategy: In the optimization process, the proportion of samples with different difficulties is dynamically adjusted according to the sample difficulty. This is achieved by using cosine decay. The cosine decay weight β is calculated. The formula is defined as follows:

[0052]

[0053] Among them, the weight , and They are used to control the weights of simple, medium, and difficult sample pairs respectively. Indicates the total number of training epochs, Indicates the current number of training epochs, and k is used to control the rate of cosine decay;

[0054] e. Optimize the contrast loss by using the template weight α and the cosine decay weight β, and obtain the similarity-based contrast loss as follows:

[0055] =

[0056] in, is a temperature hyperparameter.

[0057] As a preferred solution of the fish identification method based on knowledge distillation of the present invention, wherein: in step S6, when constructing the total loss function, the expression of the total loss function Ltotal is:

[0058] ;

[0059] in, and represents the hyperparameter of the balanced loss term, represents the enhanced logit knowledge distillation loss, represents the enhanced feature knowledge distillation loss, and What is called enhancement is just the distillation loss, is the comparison loss.

[0060] As a preferred solution of the fish identification method based on knowledge distillation of the present invention, in step S7, when updating the parameters of the student network model, the specific method is:

[0061] Use the step total loss function to train the student network model, back propagate to calculate the gradient, and update the network parameters. The parameters are = - ;

[0062] in, represents the parameters of the student network model, represents the learning rate, Represents the total loss function Compute the gradients of the student network parameters.

[0063] A system applied to the above-mentioned fish identification method based on knowledge distillation, the system comprising:

[0064] A fish image sample data acquisition module is used to acquire fish image sample data and label data;

[0065] The knowledge distillation network model construction module is used to construct the knowledge distillation network model. The knowledge distillation network model includes a teacher network model and a student network model. Both the teacher network model and the student network model use convolutional neural networks, and the teacher network model is pre-trained.

[0066] Sample feature extraction module, the preprocessed fish images are called ordinary fish image samples, and the preprocessed and data enhanced fish image samples are called converted fish image samples. The ordinary and converted fish image sample data are input into the pre-trained teacher network model and the untrained student network model respectively. The convolution layer of the teacher network model outputs the ordinary and converted first intermediate feature maps, the fully connected layer of the teacher network model outputs the ordinary and converted first logits features, the convolution layer of the student network model outputs the ordinary and converted second intermediate feature maps, and the fully connected layer of the student network model outputs the ordinary and converted second logits features;

[0067] An enhanced knowledge distillation loss construction module is used to concatenate ordinary and converted first logits features to generate enhanced first logits features, concatenate ordinary and converted second logits features to generate enhanced second logits features, and construct an enhanced logits knowledge distillation loss. Concatenate ordinary and converted first intermediate features to generate enhanced first intermediate features, concatenate ordinary and converted second intermediate features to generate enhanced second intermediate features, and construct an enhanced feature knowledge distillation loss. The enhanced logits knowledge distillation loss and the enhanced feature knowledge distillation loss are added to obtain the enhanced knowledge distillation loss;

[0068] A similarity-based contrast loss building module is used to define a cross-sample similarity relationship based on the transformed first logits and the transformed second logits, build a template strategy based on the difference between the two, build a cosine decay strategy based on the similarity strength of the samples, and use the template strategy and the cosine decay strategy to optimize the cross-sample similarity relationship to obtain the contrast loss;

[0069] The total loss function building module is used to build the total loss function. The total loss includes enhanced knowledge distillation loss and contrast loss.

[0070] The student network model training module is used to train the student network model using the total loss function, and back-propagate to update the parameters of the student network model to obtain a mature student network model;

[0071] The fish image real-time classification module is used to obtain real-time fish images and input the fish images into the student network model, and the student network model outputs the classification results.

[0072] The present invention also discloses an electronic device comprising:

[0073] one or more processors;

[0074] A storage device for storing one or more programs;

[0075] When one or more programs are executed by one or more processors, the one or more processors implement the above method.

[0076] The present invention also discloses a computer-readable medium on which a computer program is stored. When the program is executed by a processor, any of the above methods is implemented.

[0077] Beneficial effects of the present invention:

[0078] 1. In the supervised task, the present invention performs enhanced knowledge distillation by concatenating logits and features generated from the transformed data and normal data. Then, in the self-supervised task, it mines key sample pairs and defines the similarity relationship of cross samples through the intrinsic structure of the data without relying on external labels or supervision, and realizes knowledge transfer through contrastive learning.

[0079] 2. In order to optimize the similarity relationship of cross samples, the present invention introduces a template strategy to moderately optimize the sample pairs, and uses a cosine decay strategy to dynamically adjust the weights of sample proportions of different similarity strengths, thereby optimizing the entire training process. The cross sample similarity knowledge extracted from the self-supervised task can be used as a supplement to the supervised task to achieve better knowledge transfer and improve the effect of fish image recognition and classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0080] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. Among them:

[0081] Figure 1 This is a flow chart of a fish identification method based on knowledge distillation proposed in the present invention. DETAILED DESCRIPTION

[0082] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.

[0083] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0084] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.

[0085] Secondly, the present invention is described in detail with reference to the schematic diagram. When describing the embodiments of the present invention in detail, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0086] Example 1, reference Figure 1 , as an embodiment of the present invention, provides a fish identification method based on knowledge distillation, the method comprising the following steps:

[0087] S1: Obtain sample data; wherein the sample data includes: image sample data and label data;

[0088] Specifically, the acquired fish image sample data is preprocessed and data enhanced. The preprocessing includes cropping, scaling and rotation, and the data enhancement includes color jittering, grayscale conversion and Gaussian blur.

[0089] S2: construct a knowledge distillation network model; wherein the knowledge distillation network model includes a teacher network model and a student network model, both of which use convolutional neural networks, and the teacher network model is pre-trained;

[0090] S3: Extract sample features; the preprocessed fish images are called common fish image samples, and the preprocessed and data-enhanced fish images are called converted fish image samples;

[0091] The normal and converted fish image sample data are input into the pre-trained teacher network model and the untrained student network model respectively, the convolution layer of the teacher network model outputs the normal and converted first intermediate feature maps, the fully connected layer of the teacher network model outputs the normal and converted first logits features, the convolution layer of the student network model outputs the normal and converted second intermediate feature maps, and the fully connected layer of the student network model outputs the normal and converted second logits features;

[0092] S4: Construct enhanced knowledge distillation loss; concatenate the ordinary and converted first logits features to generate enhanced first logits features, concatenate the ordinary and converted second logits features to generate enhanced second logits features, and construct enhanced logits knowledge distillation loss;

[0093] The ordinary and transformed first intermediate features are concatenated to generate enhanced first intermediate features, and the ordinary and transformed second intermediate features are concatenated to generate enhanced second intermediate features, and the enhanced feature knowledge distillation loss is constructed. The enhanced logits knowledge distillation loss and the enhanced feature knowledge distillation loss are added to obtain the enhanced knowledge distillation loss;

[0094] Specifically, the knowledge distillation loss is and The specific construction process is:

[0095] a. Enhanced logits knowledge distillation loss The expression is:

[0096] = = CE ;

[0097] Among them, s and t represent students and teachers respectively. and is a hyperparameter used to balance the weights of the two losses. and It means that the enhanced first logits and the enhanced second logits generate a prediction vector through the softmax function. represents the prediction vector generated by the ordinary second logits through the softmax function, CE represents the cross entropy function, KLD represents the KL divergence function, represents the true label, T represents a temperature hyperparameter;

[0098] b. Enhanced feature knowledge distillation loss The expression is:

[0099] = MSE ;

[0100] Among them, s and t represent students and teachers respectively, b represents the batch size, and for the i-th sample in the batch, the corresponding Represents the first The features of the layer, MSE represents the mean square error function, Represents the self-attention weight of the i-th sample.

[0101] In addition, when calculating the self-attention weight of the i-th sample The specific method is:

[0102] First, calculate the similarity matrix between teachers and students and ,in;

[0103] =R( )·R ; =R( )·R ;

[0104] Among them, R(·) represents the deformation operation of the feature, Represents the transpose operation of the matrix, and then passes through the multi-layer perceptron MLP to obtain the vector and vector :

[0105] =MLPQ ; =MLPQ ;

[0106] Then the vector and vector vector and vector Through the softmax function F(·), the self-attention weights of all samples in the batch can be obtained ;

[0107] Self-attention weight of the i-th sample in the batch The calculation process is as follows:

[0108] =F( · )= .

[0109] S5: Construct similarity-based contrast loss;

[0110] Based on the converted first logits and the converted second logits, the cross-sample similarity relationship is defined, a template strategy is constructed according to the difference between the two, a cosine decay strategy is constructed according to the similarity strength of the samples, and the template strategy and the cosine decay strategy are used to optimize the cross-sample similarity relationship to obtain the contrast loss;

[0111] Specifically, when constructing a similarity-based contrast loss function, the specific method is:

[0112] a. Divide the positive and negative sets into As a decision boundary Whether it belongs to the positive set or the negative set, the formula is as follows:

[0113]

[0114] in, represents a positive set, represents a negative set;

[0115] b. Use abbreviated symbols = , = ,Will and The relationship between is called cross-sample similarity, and knowledge is transferred through contrastive learning;

[0116] From the main set and negative set Mining sample pairs of various difficulties, namely difficult positive sample pairs and positive and negative sample pairs , medium positive and negative sample pairs and negative sample pairs Simple positive sample to negative sample pair ,The detailed mining process is as follows:

[0117] = , = ,

[0118] = , = ,

[0119] = , = ;

[0120] c. Template strategy, reallocation during optimization and The weight in the template is calculated by taking the teacher network as the main guide of the student, and the formula is defined as follows:

[0121] ;

[0122] in, Indicates zero-cutoff operation, the purpose is to ensure and non-negative;

[0123] d. Cosine decay strategy: In the optimization process, the proportion of samples with different difficulties is dynamically adjusted according to the sample difficulty. This is achieved by using cosine decay. The cosine decay weight β is calculated. The formula is defined as follows:

[0124]

[0125] Among them, the weight , and They are used to control the weights of simple, medium, and difficult sample pairs respectively. Indicates the total number of training epochs, Indicates the current number of training epochs, and k is used to control the rate of cosine decay;

[0126] e. Optimize the contrast loss by using the template weight α and the cosine decay weight β, and obtain the similarity-based contrast loss as follows:

[0127] =

[0128] in, is a temperature hyperparameter.

[0129] S6: construct a total loss function; wherein the total loss includes the enhanced knowledge distillation loss and the similarity-based contrast loss;

[0130] Specifically, when constructing the total loss function, the expression of the total loss function Ltotal is:

[0131] ;

[0132] in, and represents the hyperparameter of the balanced loss term, represents the enhanced logit knowledge distillation loss, represents the enhanced feature knowledge distillation loss, and What is called enhancement is just the distillation loss, is the comparison loss.

[0133] S7: training student network model;

[0134] The total loss function calculated by step S6 is used to train the student network model, and back-propagation is performed to update the parameters of the student network model to obtain a mature student network model.

[0135] Specifically, when updating the parameters of the student network model, the specific method is:

[0136] Use the step total loss function to train the student network model, back propagate to calculate the gradient, and update the network parameters. The parameters are = - ;

[0137] in, represents the parameters of the student network model, represents the learning rate, Represents the total loss function Calculate the gradient of the student network parameters;

[0138] S8: Obtain real-time fish images; input the fish images into the student network model, and the student network model outputs classification results.

[0139] Embodiment 2, a system applied to the above-mentioned fish identification method based on knowledge distillation, the system comprising:

[0140] A fish image sample data acquisition module is used to acquire fish image sample data and label data;

[0141] The knowledge distillation network model construction module is used to construct the knowledge distillation network model. The knowledge distillation network model includes a teacher network model and a student network model. Both the teacher network model and the student network model use convolutional neural networks, and the teacher network model is pre-trained.

[0142] Sample feature extraction module, the preprocessed fish images are called ordinary fish image samples, and the preprocessed and data enhanced fish image samples are called converted fish image samples. The ordinary and converted fish image sample data are input into the pre-trained teacher network model and the untrained student network model respectively. The convolution layer of the teacher network model outputs the ordinary and converted first intermediate feature maps, the fully connected layer of the teacher network model outputs the ordinary and converted first logits features, the convolution layer of the student network model outputs the ordinary and converted second intermediate feature maps, and the fully connected layer of the student network model outputs the ordinary and converted second logits features;

[0143] An enhanced knowledge distillation loss construction module is used to concatenate ordinary and converted first logits features to generate enhanced first logits features, concatenate ordinary and converted second logits features to generate enhanced second logits features, and construct an enhanced logits knowledge distillation loss. Concatenate ordinary and converted first intermediate features to generate enhanced first intermediate features, concatenate ordinary and converted second intermediate features to generate enhanced second intermediate features, and construct an enhanced feature knowledge distillation loss. The enhanced logits knowledge distillation loss and the enhanced feature knowledge distillation loss are added to obtain the enhanced knowledge distillation loss;

[0144] A similarity-based contrast loss building module is used to define a cross-sample similarity relationship based on the transformed first logits and the transformed second logits, build a template strategy based on the difference between the two, build a cosine decay strategy based on the similarity strength of the samples, and use the template strategy and the cosine decay strategy to optimize the cross-sample similarity relationship to obtain the contrast loss;

[0145] The total loss function building module is used to build the total loss function. The total loss includes enhanced knowledge distillation loss and contrast loss.

[0146] The student network model training module is used to train the student network model using the total loss function, and back-propagate to update the parameters of the student network model to obtain a mature student network model;

[0147] The fish image real-time classification module is used to obtain real-time fish images and input the fish images into the student network model, and the student network model outputs the classification results.

[0148] In addition, the present invention also discloses an electronic device capable of running the system and a computer-readable medium, the device comprising: one or more processors;

[0149] A storage device for storing one or more programs;

[0150] When the above program is executed by one or more processors, the one or more processors implement the above-mentioned fish identification method based on knowledge distillation.

[0151] As for computer readable medium, readable medium can be readable signal medium or readable storage medium. The readable storage medium can be, for example, but not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or components, or any combination thereof. More specific examples (non-exhaustive list) of readable storage medium include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0152] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the exemplary embodiments described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the embodiment of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the embodiment of the present invention.

[0153] In summary, the present invention performs enhanced knowledge distillation in the supervised task by splicing logits and features generated from converted data and normal data. Then in the self-supervised task, key sample pairs are mined and the similarity relationship of cross samples is defined by the intrinsic structure of the data without relying on external labels or supervision, and knowledge transfer is achieved through contrastive learning. In addition, in order to optimize the similarity relationship of cross samples, a template strategy is introduced to moderately optimize the sample pairs, and a cosine decay strategy is used to dynamically adjust the weights of the sample ratios of different similarity intensities, thereby optimizing the entire training process. The cross-sample similarity knowledge extracted by the self-supervised task can be used as a supplement to the supervised task to achieve better knowledge transfer and improve the effect of fish image recognition and classification.

[0154] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A fish identification method based on knowledge distillation, characterized in that: The method comprises the following steps: S1: Obtain sample data; wherein the sample data includes: image sample data and label data; S2: construct a knowledge distillation network model; wherein the knowledge distillation network model includes a teacher network model and a student network model, both of which use convolutional neural networks, and the teacher network model is pre-trained; S3: Extract sample features; the preprocessed fish images are called common fish image samples, and the preprocessed and data-enhanced fish images are called converted fish image samples; The normal and converted fish image sample data are input into the pre-trained teacher network model and the untrained student network model respectively, the convolution layer of the teacher network model outputs the normal and converted first intermediate feature maps, the fully connected layer of the teacher network model outputs the normal and converted first logits features, the convolution layer of the student network model outputs the normal and converted second intermediate feature maps, and the fully connected layer of the student network model outputs the normal and converted second logits features; S4: Construct enhanced knowledge distillation loss; concatenate the ordinary and converted first logits features to generate enhanced first logits features, concatenate the ordinary and converted second logits features to generate enhanced second logits features, and construct enhanced logits knowledge distillation loss; The ordinary and converted first intermediate features are concatenated to generate enhanced first intermediate features, the ordinary and converted second intermediate features are concatenated to generate enhanced second intermediate features, an enhanced feature knowledge distillation loss is constructed, and the enhanced logits knowledge distillation loss and the enhanced feature knowledge distillation loss are added to obtain the enhanced knowledge distillation loss; S5: Construct similarity-based contrast loss; Based on the converted first logits and the converted second logits, the cross-sample similarity relationship is defined, a template strategy is constructed according to the difference between the two, a cosine decay strategy is constructed according to the similarity strength of the samples, and the template strategy and the cosine decay strategy are used to optimize the cross-sample similarity relationship to obtain the contrast loss; The specific method is: a. Divide the positive and negative sets into As a decision boundary Whether it belongs to the positive set or the negative set, the formula is as follows: in, represents a positive set, represents a negative set; b. Use abbreviated symbols = , = ,Will and The relationship between is called cross-sample similarity, and knowledge is transferred through contrastive learning; From the main set and negative set Mining sample pairs of various difficulties, namely difficult positive sample pairs and negative sample pairs , medium positive and negative pairs Simple positive sample to negative sample pair ,The detailed mining process is as follows: = ， = ， = ， = ， = ， = ； c. Template strategy, reallocation during optimization and The weight in the template is calculated by taking the teacher network as the main guide of the student, and the formula is defined as follows: ； in, Indicates zero-cutoff operation, the purpose is to ensure and non-negative; d. Cosine decay strategy: In the optimization process, the proportion of samples with different difficulties is dynamically adjusted according to the sample difficulty. This is achieved by using cosine decay. The cosine decay weight β is calculated. The formula is defined as follows: Among them, the weight , and They are used to control the weights of simple, medium, and difficult sample pairs respectively. Indicates the total number of training epochs, Indicates the current number of training epochs, and k is used to control the rate of cosine decay; e. Optimize the contrast loss by using the template weight α and the cosine decay weight β, and obtain the similarity-based contrast loss as follows: = in, is a temperature hyperparameter; S6: construct a total loss function; wherein the total loss includes the enhanced knowledge distillation loss and the similarity-based contrast loss; S7: training student network model; The total loss function calculated in step S6 is used to train the student network model, and back-propagation is performed to update the parameters of the student network model to obtain a mature student network model; S8: Obtain real-time fish images; input the fish images into the student network model, and the student network model outputs classification results.

2. The fish identification method based on knowledge distillation according to claim 1, characterized in that: In the S1 process, the acquired fish image sample data is preprocessed and data enhanced, the preprocessing includes cropping, scaling and rotation, and the data enhancement includes color jittering, grayscale conversion and Gaussian blur.

3. The fish identification method, system, device and medium based on knowledge distillation according to claim 2, characterized in that: In the S4 process, the knowledge distillation loss is and The specific construction process is: a. Enhanced logits knowledge distillation loss The expression is: = = THIS ; Among them, s and t represent students and teachers respectively. and is a hyperparameter used to balance the weights of the two losses. and It means that the enhanced first logits and the enhanced second logits generate a prediction vector through the softmax function. represents the prediction vector generated by the ordinary second logits through the softmax function, CE represents the cross entropy function, KLD represents the KL divergence function, represents the true label, T represents a temperature hyperparameter; b. Enhanced feature knowledge distillation loss The expression is: = ·MSE ; Among them, s and t represent students and teachers respectively, b represents the batch size, and for the i-th sample in the batch, the corresponding Represents the first The features of the layer, MSE represents the mean square error function, Represents the self-attention weight of the i-th sample.

4. The fish identification method based on knowledge distillation according to claim 3 is characterized in that: In the S4 process, when calculating the self-attention weight of the i-th sample The specific method is: First, calculate the similarity matrix between teachers and students and ,in; =R( )·R ; =R( )·R ; Among them, R(·) represents the deformation operation of the feature, Represents the transpose operation of the matrix, and then passes through the multi-layer perceptron MLP to obtain the vector and vector : =MLPQ ; =MLPQ ; Then the vector and vector Through the softmax function F(·), the self-attention weights of all samples in the batch are obtained ; Self-attention weight of the i-th sample in the batch The calculation process is as follows: =F（ · ）= 5. The fish identification method based on knowledge distillation according to claim 1, characterized in that: In step S6, when constructing the total loss function, the expression of the total loss function Ltotal is: ; in, and represents the hyperparameter of the balanced loss term, represents the enhanced logit knowledge distillation loss, represents the enhanced feature knowledge distillation loss, and Together they are called the enhanced knowledge distillation loss, is the comparison loss.

6. The fish identification method based on knowledge distillation according to claim 1, characterized in that: In step S7, when updating the parameters of the student network model, the specific method is: Use the step total loss function to train the student network model, back propagate to calculate the gradient, and update the network parameters. The parameters are = - ; in, represents the parameters of the student network model, represents the learning rate, Represents the total loss function Compute the gradients of the student network parameters.

7. A system applied to the fish identification method based on knowledge distillation according to any one of claims 1 to 6, characterized in that: The system includes: A fish image sample data acquisition module is used to acquire fish image sample data and label data; The knowledge distillation network model construction module is used to construct the knowledge distillation network model. The knowledge distillation network model includes a teacher network model and a student network model. Both the teacher network model and the student network model use convolutional neural networks, and the teacher network model is pre-trained. Sample feature extraction module, the preprocessed fish image is called the ordinary fish image sample, the preprocessed and data enhanced fish image sample is called the converted fish image sample, the ordinary and converted fish image sample data are respectively input into the pre-trained teacher network model and the untrained student network model, the convolution layer of the teacher network model outputs the ordinary and converted first intermediate feature map, the fully connected layer of the teacher network model outputs the ordinary and converted first logits features, the convolution layer of the student network model outputs the ordinary and converted second intermediate feature map, and the fully connected layer of the student network model outputs the ordinary and converted second logits features; an enhanced knowledge distillation loss construction module, for concatenating ordinary and converted first logits features to generate enhanced first logits features, concatenating ordinary and converted second logits features to generate enhanced second logits features, constructing enhanced logits knowledge distillation loss, concatenating ordinary and converted first intermediate features to generate enhanced first intermediate features, concatenating ordinary and converted second intermediate features to generate enhanced second intermediate features, constructing enhanced feature knowledge distillation loss, and adding enhanced logits knowledge distillation loss and enhanced feature knowledge distillation loss to obtain enhanced knowledge distillation loss; A similarity-based contrast loss building module is used to define a cross-sample similarity relationship based on the transformed first logits and the transformed second logits, build a template strategy based on the difference between the two, build a cosine decay strategy based on the similarity strength of the samples, and use the template strategy and the cosine decay strategy to optimize the cross-sample similarity relationship to obtain the contrast loss; The total loss function building module is used to build the total loss function. The total loss includes enhanced knowledge distillation loss and contrast loss. The student network model training module is used to train the student network model using the total loss function, and back-propagate to update the parameters of the student network model to obtain a mature student network model; The fish image real-time classification module is used to obtain real-time fish images and input the fish images into the student network model, and the student network model outputs the classification results.

8. An electronic device, characterized in that: include: one or more processors; A storage device for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images