Microalgae image classification method and related device based on feature calibration Transformer

Abstract

The present invention belongs to the technical field of image classification in computer vision and discloses a microalgae image classification method and related apparatus based on a feature-calibrated Transformer. The microalgae image classification method comprises the following steps: obtaining a microalgae image to be classified; and classifying the microalgae using a pre-trained deep image classification network based on the microalgae image to obtain a classification result. The technical solution disclosed in the present invention, which uses a pre-trained deep image classification network for classification, can solve the technical problem of high misclassification caused by insufficient capture of salient features of microalgae images in the prior art. The method utilizes a feature calibration module to increase attention to salient information of microalgae targets, and proposes a feature calibration attention module to model long-range dependencies involving local information fusion and salient feature enhancement, thereby improving the classification accuracy of microalgae images.

Description

Technical Field

[0001] The present invention belongs to the technical field of image classification in computer vision, and in particular relates to a microalgae image classification method based on feature-calibrated Transformer and related devices. Background Art

[0002] Microalgae image classification involves classifying microscopic algae in microscopic images. It's a key step in marine ecology for monitoring and predicting the marine ecosystem. Explanation: The complex marine environment, with its numerous environmental disturbances (e.g., pixel size, focus distance, camera angle, environmental contamination, and external light reflection), makes it difficult to distinguish microalgae in captured images, hindering the accuracy of microalgae image classification. Furthermore, microalgae exhibit both inter-class similarities and intra-class differences in morphology, making it even more difficult to determine their class. In summary, microalgae image classification is a highly challenging task.

[0003] At present, the existing microalgae image classification methods can be divided into two categories according to the technical process and development stage, namely traditional microalgae image classification and microalgae image classification based on deep learning; among them, traditional microalgae image classification is mainly based on manual classification, manual extraction of low-level features (such as edges, colors, textures, etc.) combined with machine learning classification algorithms for classification. This solution relies on the participation of a large number of professional researchers with knowledge of marine phytoplankton classification, with high labor costs and is not suitable for large-scale microalgae image data. Moreover, with the increasing number of microalgae images obtained by plankton imagers (such as the Imaging Flow Cytobot flow cytometer used for microalgae imaging), manual classification has become more difficult and less feasible; in contrast, microalgae image classification based on deep learning completes the feature extraction and classification process through a deep image classification network to achieve end-to-end automatic classification, with high classification accuracy, efficiency and reliability.

[0004] Existing deep learning-based image classification methods are mainly based on convolutional neural networks (CNNs) and vision transformers (ViTs). Convolutional neural networks use the prior knowledge of image translation invariance to aggregate pixel information layer by layer through convolution operations to effectively learn microalgae features. Although this scheme has high classification efficiency and reliability, it faces complex underwater environments with external interference and blurred and ghosting images. Directly using a general deep image classification network cannot effectively focus on and extract microalgae image features that are difficult to distinguish, and cannot capture the salient information of microalgae images well. Therefore, establishing an efficient visual cognitive computing module is crucial to enhancing the capture of salient information in complex underwater environments and improving the accuracy of microalgae image classification (Top1Accuracy). Furthermore, the visual Transformer models long-range dependencies based on the Multi-Head Self-Attention (MHSA) mechanism. In the classification tasks of existing multi-stage visual Transformers, downsampling is often performed stage by stage to finally output feature maps for classification. However, the multi-head self-attention calculation often directly calculates the pre-attention matrix of query, key, and value formed by the linear mapping of the feature map, without considering the information fusion of the neighbor matrix, which is not conducive to capturing the correlation between local spatial information. In addition, the pre-attention matrix of query, key, and value formed by the linear mapping of the feature map does not consider effective feature calibration, which is not conducive to accurately locating the salient information of the target to be classified. Therefore, a feature calibration attention mechanism that considers local information fusion and salient feature enhancement is proposed, which is very necessary to improve the classification accuracy of the visual Transformer in microalgae images. Summary of the Invention

[0005] The present invention aims to provide a method and related apparatus for microalgae image classification based on a feature-calibrated Transformer to address one or more of the aforementioned technical issues. The present invention discloses a method for microalgae image classification based on a feature-calibrated Transformer. This method, which utilizes a pre-trained deep image classification network for classification, addresses the existing technical issue of high misclassification rates caused by insufficient capture of salient features in microalgae images. Specifically, the method utilizes a feature calibration module to enhance attention to salient information of microalgae targets, and proposes a feature calibration attention module to model long-range dependencies involving local information fusion and salient feature enhancement, thereby improving the classification accuracy of microalgae images.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides a microalgae image classification method based on feature-calibrated Transformer, comprising the following steps:

[0008] Acquiring an image of microalgae to be classified;

[0009] Based on the microalgae image to be classified, using a pre-trained deep image classification network to classify the microalgae to obtain a classification result;

[0010] in,

[0011] The deep image classification network includes stage 1, stage 2, stage 3, stage 4 and a linear classification head connected in sequence; the stage 1 includes a 4-fold patch embedding, a plurality of encoder modules and a feature calibration encoder module connected in series; the stage 2, the stage 3 and the stage 4 each include a 2-fold patch embedding, a plurality of encoder modules and a feature calibration encoder module connected in series;

[0012] The feature calibration encoder module includes a convolutional position encoding, a first LayerNorm layer normalization operation, a feature calibration attention module, a second LayerNorm layer normalization operation and a feedforward neural network layer connected in series, and the input of the first LayerNorm layer normalization operation is added to the output of the feature calibration attention module, and the input of the second LayerNorm layer normalization operation is added to the output of the feedforward neural network through a skip connection layer;

[0013] The feature calibration attention module includes a feature calibration module, a multi-head self-attention module, a convolution relative position encoding module and a depth-separable convolution module; wherein, the feature calibration module is used to input the query Q, key K, value V matrix formed by the normalization operation output feature map of the first LayerNorm layer through linear mapping, and output the query Q after feature calibration. * , key K * , value V * Matrix; The multi-head self-attention module is used to input the query Q after feature calibration * , key K * , value V * matrix, outputs the feature map of the multi-head self-attention module; the convolution relative position encoding module is used to input the query Q after feature calibration * , value V *Matrix, output convolution relative position encoding module output matrix; the depth separable convolution module is used to input the query Q, key K, value V matrix formed by linear mapping the output feature map of the first LayerNorm layer normalization operation, and output the local matrix of one-fifth of the channel, and output the output feature map of the depth separable convolution module; the feature calibration attention module output feature map is the sum of the output feature map of the multi-head self-attention module and the output matrix of the convolution relative position encoding module, and is spliced ​​with the output feature map of the depth separable convolution module in the channel dimension to form a feature map.

[0014] A further improvement of the method of the present invention is that the feature calibration module includes a backbone network and a branch network; the output of the feature calibration module is the point-by-point product of the backbone network output and the branch network output, and the obtained grouped feature map is spliced ​​by channel; wherein, the input of the backbone network and the branch network is a query Q, key K, and value V matrix of a group of 8 channels, the operation of the backbone network is identity mapping, and the operation of the branch network is depth-separable convolution, LayerNorm layer normalization, and sigmoid activation function operation.

[0015] A further improvement of the method of the present invention is that, in the deep image classification network,

[0016] The 4x patch embedding is used to input a microalgae image and output a 4x downsampled feature map;

[0017] The 2x patch embedding is used to input the output feature map of the feature calibration encoder module of the previous stage, and output a 2x downsampled feature map;

[0018] The encoder module is used to input a 4x or 2x downsampled feature map, and the output encoder module outputs a feature map;

[0019] The feature calibration encoder module is used to input the encoder module output feature map, and output the feature calibration encoder module output feature map.

[0020] A further improvement of the method of the present invention is that in the feature calibration encoder module,

[0021] The convolution position encoding is used to input the output feature map of the encoder module, and the convolution position encoding output feature map is output;

[0022] The first LayerNorm layer normalization operation is used to input the convolution position encoding output feature map, and output the first LayerNorm layer normalization operation output feature map;

[0023] The feature calibration attention module is used to input the output feature map of the first LayerNorm layer normalization operation and output the output feature map of the feature calibration attention module;

[0024] The second LayerNorm layer normalization operation is used to input the sum of the feature calibration attention module output feature map and the convolution position encoding output feature map, and output the second LayerNorm layer normalization operation output feature map;

[0025] The feedforward neural network layer is used to input the normalization operation output feature map of the second LayerNorm layer and output the feedforward neural network output feature map;

[0026] The feature calibration encoder module output feature map is the sum of the feedforward neural network output feature map and the second LayerNorm layer normalization operation input feature map.

[0027] A further improvement of the method of the present invention is that the encoder module includes a convolutional position encoding, a first LayerNorm layer normalization operation, a multi-head self-attention module, a second LayerNorm layer normalization operation and a feedforward neural network layer connected in series in sequence, and the input of the first LayerNorm layer normalization operation is added to the output of the multi-head self-attention module, and the input of the second LayerNorm layer normalization operation is added to the output of the feedforward neural network through a jump connection layer.

[0028] A further improvement of the method of the present invention is that the feature calibration module is represented as:

[0029]

[0030] Where Q i , K i 、V i are the local matrices of Q, K, and V after channel segmentation, i = 1, 2, ..., n, where n is the number of channel segmentations; Q, K, and V are the query, key, and value matrices formed by linear mapping of image tokens, respectively; Q after feature calibration i , K i 、V i ; o is the point-by-point product; σ(·) is the Sigmoid activation function; LN(·) is the LayerNorm layer normalization; DWC 3×3 is a depthwise separable convolution operation with a filter size of 3×3.

[0031] A further improvement of the method of the present invention is that in the feature calibration attention module,

[0032] The multi-head self-attention module is,

[0033]

[0034] Where Att is the output feature map of the multi-head self-attention module; Q * , K * 、V * Q i , K i 、V i The query, key, and value matrices are formed after feature calibration and channel concatenation; d is the scaling factor; Softmax(·) is the exponential normalization function; (·) T Represents matrix transpose;

[0035] The convolutional relative position encoding module is,

[0036] ReP=Q * oConv 3，5，7 (V * ),

[0037] Where ReP is the output matrix of the convolutional relative position encoding module; Conv 3，5，7 Perform convolution operations with filter sizes of 3×3, 5×5, and 7×7 for each channel;

[0038] The output feature map ConvAtt of the multi-head self-attention module with convolutional relative position encoding is,

[0039]

[0040] In a second aspect, the present invention provides a microalgae image classification system based on feature-calibrated Transformer, comprising:

[0041] An image acquisition module, used for acquiring images of microalgae to be classified;

[0042] A classification module, configured to classify the microalgae based on the microalgae image to be classified using a pre-trained deep image classification network to obtain a classification result;

[0043] in,

[0044] The deep image classification network includes stage 1, stage 2, stage 3, stage 4 and a linear classification head connected in sequence; the stage 1 includes a 4-fold patch embedding, a plurality of encoder modules and a feature calibration encoder module connected in series; the stage 2, the stage 3 and the stage 4 each include a 2-fold patch embedding, a plurality of encoder modules and a feature calibration encoder module connected in series;

[0045] The feature calibration encoder module includes a convolutional position encoding, a first LayerNorm layer normalization operation, a feature calibration attention module, a second LayerNorm layer normalization operation and a feedforward neural network layer connected in series, and the input of the first LayerNorm layer normalization operation is added to the output of the feature calibration attention module, and the input of the second LayerNorm layer normalization operation is added to the output of the feedforward neural network through a skip connection layer;

[0046] The feature calibration attention module includes a feature calibration module, a multi-head self-attention module, a convolution relative position encoding module and a depth-separable convolution module; wherein, the feature calibration module is used to input the query Q, key K, value V matrix formed by the normalization operation output feature map of the first LayerNorm layer through linear mapping, and output the query Q after feature calibration. * , key K * , value V * Matrix; The multi-head self-attention module is used to input the query Q after feature calibration * , key K * , value V * matrix, outputs the feature map of the multi-head self-attention module; the convolution relative position encoding module is used to input the query Q after feature calibration * , value V * Matrix, output convolution relative position encoding module output matrix; the depth separable convolution module is used to input the query Q, key K, value V matrix formed by linear mapping the output feature map of the first LayerNorm layer normalization operation, and output the local matrix of one-fifth of the channel, and output the output feature map of the depth separable convolution module; the feature calibration attention module output feature map is the sum of the output feature map of the multi-head self-attention module and the output matrix of the convolution relative position encoding module, and is spliced ​​with the output feature map of the depth separable convolution module in the channel dimension to form a feature map.

[0047] In a third aspect of the present invention, an electronic device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the microalgae image classification method based on feature-calibrated Transformer as described in any one of the first aspects of the present invention is implemented.

[0048] In a fourth aspect, the present invention provides a non-transitory computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method for classifying microalgae images based on a feature-calibrated Transformer as described in any one of the first aspects of the present invention is implemented.

[0049] Compared with the prior art, the present invention has the following beneficial effects:

[0050] The present invention provides a microalgae image classification method based on feature calibration transformer, which uses a pre-trained deep image classification network for classification, and can solve the technical problem of high misclassification caused by insufficient capture of salient features of microalgae images in the prior art. Specifically, the deep image classification network disclosed in the present invention includes four stages and one linear classification head. Each stage includes a patch embedding, an encoder module, and a feature calibration encoder module; wherein the feature calibration encoder module includes a feature calibration attention module; the feature calibration attention module includes a feature calibration module; wherein the feature calibration module is used to batch the feature map by channel, fuse the neighbor space information of the local channel, and further calculate the pixel weight to capture the target salient information; the feature calibration attention module is used to model the long-range dependency relationship of local information enhancement; the feature calibration encoder module is used to encode the attention map and output it to the classification head to obtain the classification result. In summary, in response to the problem that the prior art does not consider the salient features of microalgae images enough, the microalgae image classification method provided by the present invention fully considers the local information fusion and salient feature enhancement mechanism, which can effectively improve the accuracy of microalgae image classification and reduce misclassification. The method of the present invention is based on the visual Transformer framework and introduces visual cognitive computing into the field of automatic classification, which can effectively improve the accuracy of microalgae image classification. In addition, each module is simple to implement, has no excessive dependencies, and has strong applicability. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below; obviously, the drawings described below are some embodiments of the present invention, and for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0052] Figure 1 This is a flow chart of a microalgae image classification method based on feature-calibrated Transformer provided by an embodiment of the present invention.

[0053] Figure 2 Schematic diagram of the structure of a deep image classification network in an embodiment of the present invention.

[0054] Figure 3 2 is a schematic structural diagram of a feature calibration encoder module in an embodiment of the present invention.

[0055] Figure 4 2 is a schematic diagram of the structure of the feature calibration attention module in an embodiment of the present invention.

[0056] Figure 5 2 is a schematic structural diagram of a feature calibration module in an embodiment of the present invention.

[0057] Figure 6 : is a schematic diagram of microalgae image classification results in a specific embodiment of the present invention; wherein, Figure 6 (a) is a schematic diagram of the classification results of the Asterionellopsis microalgae image sample. Figure 6 (b) is a schematic diagram of the classification results of the microalgae image sample of the G_delicatula_detritus category. Figure 6 (c) is a schematic diagram of the classification results of the Chrysochromulina microalgae image sample. Figure 6 (d) is a schematic diagram of the classification results of the Chaetoceros class microalgae image samples. Figure 6 (e) is a schematic diagram of the classification results of the Chaetoceros_didymus_flagellate microalgae image sample. Figure 6 (f) is a schematic diagram of the classification results of the Cerataulina_flagellate microalgae image sample. Figure 6 (g) is a schematic diagram of the classification results of the Prorocentrum microalgae image sample. Figure 6 Middle (h) is a schematic diagram of the classification results of Coscinodiscus microalgae image samples. Figure 6 (i) is a schematic diagram of the classification results of Dactyliosolen category microalgae image samples.

[0058] Figure 7 Schematic diagram of a microalgae image classification system based on feature-calibrated Transformer provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the drawings in the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0060] See also Figure 1 and Figure 2 In an embodiment of the present invention, a microalgae image classification method based on feature calibration Transformer is provided, comprising the following steps:

[0061] Step 1: Acquire a microalgae image to be classified. For example, the microalgae image to be classified may be an RGB image (i.e., a color image) or an L image (i.e., a grayscale image) of microalgae taken underwater, and the image contains the microalgae target to be classified.

[0062] Step 2: Based on the microalgae image to be classified, a pre-trained deep image classification network is used to classify the microalgae to obtain a classification result; the classification result is output in the form of category and probability;

[0063] Among them, Figure 2 As shown, the deep image classification network includes stage 1, stage 2, stage 3, stage 4 and a linear classification head connected in series; wherein, the stage 1 includes 4-fold patch embedding, several encoder modules and a feature calibration encoder module connected in series; the stages 2, 3 and 4 all include 2-fold patch embedding, several encoder modules and a feature calibration encoder module connected in series; explanatory, the 4-fold patch embedding is used to input the microalgae image and output a 4-fold downsampled feature map; the 2-fold patch embedding is used to input the output feature map of the feature calibration encoder module of the previous stage and output a 2-fold downsampled feature map; the encoder module is used to input the 4-fold Or 2 times down-sampled feature map, output encoder module output feature map; the feature calibration encoder module is used to input the encoder module output feature map, and output the feature calibration encoder module output feature map; further specific example explanatory, the 4 times patch embedding performs 4 times down-sampling on the input microalgae image, and the downsampling operation adopts 2 convolution operations with a filter size of 3×3 and a step size of 2, and performs 1 depth-separable convolution operation with a filter size of 3×3 and a step size of 1; the 2 times patch embedding performs 2 times down-sampling on the output feature map of the feature calibration encoder module, and the downsampling operation adopts 1 depth-separable convolution operation with a filter size of 3×3 and a step size of 2.

[0064] Among them, Figure 3As shown, the feature calibration encoder module includes convolution position encoding, a first LayerNorm layer normalization operation, a feature calibration attention module, a second LayerNorm layer normalization operation and a feedforward neural network layer, and the input of the first LayerNorm layer normalization operation is added to the output of the feature calibration attention module, and the input of the second LayerNorm layer normalization operation is added to the output of the feedforward neural network through a jump connection layer; specifically, the convolution position encoding is used to input the output feature map of the encoder module and output the convolution position encoding output feature map; the first LayerNorm layer normalization operation is used to input the convolution position encoding output feature map and output the first LayerNorm layer normalization operation. The feature calibration attention module is used to input the output feature map of the first LayerNorm layer normalization operation and output the feature calibration attention module output feature map; the second LayerNorm layer normalization operation is used to input the sum of the feature calibration attention module output feature map and the convolution position encoding output feature map, and output the second LayerNorm layer normalization operation output feature map; the feedforward neural network layer is used to input the output feature map of the second LayerNorm layer normalization operation and output the feedforward neural network output feature map; the feature calibration encoder module output feature map is the sum of the feedforward neural network output feature map and the second LayerNorm layer normalization operation input feature map;

[0065] Among them, Figure 4 As shown, the feature calibration attention module includes a feature calibration module, a multi-head self-attention module, a convolution relative position encoding module and a depth-separable convolution module; wherein, the feature calibration module is used to input the query Q, key K, value V matrix formed by the first LayerNorm layer normalization operation output feature map through linear mapping, and output the query Q after feature calibration. * , key K * , value V * Matrix; the multi-head self-attention module inputs the feature-calibrated query Q * , key K * , value V * Matrix, output multi-head self-attention module output feature map; convolution relative position encoding module is used to input feature-calibrated query Q * , value V *Matrix, output convolution relative position encoding module output matrix; the depth separable convolution module is used to input the query Q, key K, value V matrix formed by linear mapping of the output feature map of the first LayerNorm layer normalization operation, and output the local matrix of one-fifth of the channel, and output the output feature map of the depth separable convolution module; the output feature map of the feature calibration attention module is the sum of the output feature map of the multi-head self-attention module and the output matrix of the convolution relative position encoding module, and is spliced ​​with the output feature map of the depth separable convolution module in the channel dimension to form a feature map.

[0066] In one embodiment of the present invention, the encoder module removes the feature calibration module on the basis of the feature calibration encoder module; specifically, the encoder module includes a convolution position encoding, a first LayerNorm layer normalization operation, a multi-head self-attention module, a second LayerNorm layer normalization operation and a feedforward neural network layer, and adds the first LayerNorm layer normalized input and the multi-head self-attention module output, and adds the second LayerNorm layer normalized input and the feedforward neural network output through a jump connection layer; wherein, the convolution position encoding is used to input 4 times patch embedding or 2 times patch embedding, and output the convolution position encoding output feature map; the first LayerNorm layer normalization operation is used to input the convolution position encoding output Output feature map, output the first LayerNorm layer normalization operation output feature map; the multi-head self-attention module is used to input the first LayerNorm layer normalization operation output feature map, output the multi-head self-attention module output feature map; the second LayerNorm layer normalization operation is used to input the sum of the multi-head self-attention module output feature map and the convolution position encoding output feature map, output the second LayerNorm layer normalization operation output feature map; the feedforward neural network layer is used to input the second LayerNorm layer normalization operation output feature map, output the feedforward neural network output feature map; the encoder module output feature map is the sum of the feedforward neural network output feature map and the second LayerNorm layer normalization operation input feature map.

[0067] In one embodiment of the present invention, Figure 5 As shown, the feature calibration module groups the matrices of query Q, key K, and value V that occupy four-fifths of the channels into groups of 8 channels, and performs feature calibration operations on each group; the feature calibration module includes a backbone network and a branch network, and the module output is the point-by-point product of the backbone network output and the branch network output, and the obtained grouped feature map is spliced ​​by channel; wherein, the input of the backbone network and the branch network is the query Q, key K, and value V matrices in a group of 8 channels, the operation of the backbone network is identity mapping, and the operation of the branch network is depth-separable convolution, LayerNorm layer normalization, and sigmoid activation function operation.

[0068] In the embodiment of the present invention, cognitive computing technology based on visual transformers is applied to the field of automatic recognition. A new feature calibration module is proposed for local information fusion and salient feature enhancement, and a feature calibration attention module is designed to model the long-range dependency of local information enhancement. Based on this, an image classification network is constructed to solve the microalgae image classification task. Further explanation: the feature calibration module disclosed in the embodiment of the present invention is based on basic convolution and matrix operations, and the input and output data formats are consistent, without excessive dependence, which is convenient for application in various deep image classification models and can be extended to other image recognition tasks such as target detection and semantic segmentation, with broad application prospects. In summary, the microalgae image classification method based on feature calibration transformer provided by the embodiment of the present invention uses a visual cognitive method based on feature calibration to improve the attention to microalgae target features, and proposes a feature calibration attention module to further effectively model the long-range dependency of local information enhancement, which can realize salient feature extraction and improve the accuracy of microalgae image classification. Therefore, the technical solution disclosed in the embodiment of the present invention can solve the technical problems of microalgae image misclassification caused by insufficient local information modeling and insufficient consideration of salient information in the above-mentioned prior art.

[0069] In one embodiment of the present invention, the feature calibration module obtains a weight matrix of local weighted fusion from the local matrices that occupy four-fifths of the channels of the Q, K, and V matrices in groups, and multiplies the local matrix by the weight matrix point by point to obtain a feature map after feature calibration;

[0070] Among them, the feature calibration module is expressed as:

[0071]

[0072]

[0073] Where Q i , K i 、V i are the local matrices of Q, K, and V after channel segmentation, i = 1, 2, ..., n, where n is the number of channel segmentations; Q, K, and V are the query, key, and value matrices formed by linear mapping of image tokens, respectively; Q after feature calibration i , K i 、V i ; o is the point-by-point product; σ(·) is the Sigmoid activation function; LN(·) is the LayerNorm layer normalization; DWC 3×3 is a depthwise separable convolution operation with a filter size of 3×3.

[0074] In one embodiment of the present invention, the feature calibration attention module includes a feature calibration module, a multi-head self-attention module, a convolutional relative position encoding module and a depth-wise separable convolution module;

[0075] Among them, the multi-head self-attention module is:

[0076]

[0077] Where Att is the output feature map of the multi-head self-attention module; Q * , K * 、V * Q i , K i 、V i (i=1,2,...,n) is the query, key, and value matrix formed after feature calibration and channel concatenation; d is the scaling factor; Softmax(·) is the exponential normalization function; (·) T Represents matrix transpose;

[0078] Among them, the convolution relative position encoding module is:

[0079] ReP=Q * oConv 3，5，7 (V * ),

[0080] Where ReP is the output matrix of the convolutional relative position encoding module, Conv 3，5，7 Perform convolution operations with filter sizes of 3×3, 5×5, and 7×7 for each channel;

[0081] The output feature map of the multi-head self-attention module with convolutional relative position encoding is composed of the output feature map of the multi-head self-attention module and the output matrix of the convolutional relative position encoding module:

[0082]

[0083] Where ConvAtt represents the output feature map of the multi-head self-attention module with convolutional relative position encoding.

[0084] In one embodiment of the present invention, the training process of the deep image classification network includes:

[0085] 1) Data preprocessing: The microalgae images in the training and validation sets were scaled to 224 × 224 and converted into tensors.

[0086] 2) Data loading and batch processing: Based on the set batch size of 64, the training set and validation set data are loaded in batches;

[0087] 3) Model initialization: Initialize the defined feature-calibrated Transformer optimizer and cross-entropy loss function;

[0088] 4) Model training: For each training cycle (epoch), the current batch of training set images is fed into the network via forward propagation to obtain predictions. The cross-entropy loss function is used to calculate the difference between the predictions and the actual labels. The error backpropagation algorithm is used to calculate the gradient and update the network parameters to minimize the loss.

[0089] 5) Model Validation: After each training cycle, the model performance is evaluated by calculating the cross entropy loss and accuracy on the validation set.

[0090] 6) Model saving: If the performance on the validation set improves, save the current state of the model.

[0091] In a specific embodiment of the present invention, the classification process specifically includes the following steps:

[0092] Step 1: Construct a dataset of microalgae images taken in underwater environments. The collected images must contain three color channels (RGB) (i.e., color images) or a single color channel (L) (i.e., grayscale images) and include corresponding manual classification results. The dataset must have a length and width pixel size range of 50 to 1000. The dataset must be randomly divided into a training set and a validation set. The collected images must be transferred to a computer that executes the algorithm.

[0093] Step 2: Build an image classification network, including the sequentially connected 4x or 2x patch embedding, encoder module, and feature calibration encoder module;

[0094] The deep image classification network consists of four stages. The patch embedding operation in each stage downsamples the image by 4 or 2 times. The four stages contain 2, 2, 11, and 3 encoder modules and 1, 1, 1, and 1 feature calibration encoder modules respectively. The 4x patch embedding module contains the following two convolution operations with a stride of 2 and a filter size of 3×3 and one depthwise separable convolution operation with a stride of 1 and a filter size of 3×3:

[0095]

[0096] Where X1 is the input microalgae image of the deep image classification network, is the feature map output by the convolution operation, Output feature map of depth-separable convolution, Conv 3×3 For convolution operations with a filter size of 3×3, DWC 3×3It is a depth-wise separable convolution operation with a filter size of 3×3; after each convolution operation, batch normalization and Hardwish activation function processing are performed; the two convolution operations downsample the resolution of the input microalgae image by 4 times and expand the number of channels of the input microalgae image to 40; the depth-wise separable convolution operation does not change the image resolution and number of channels; the 2x patch embedding module of stages 2, 3, and 4 contains a depth-wise separable convolution operation with a stride of 2 and a filter size of 3×3, which downsamples the input feature map resolution by 2 times and expands the number of input feature map channels to 80, 160, and 160 respectively.

[0097] In an embodiment of the present invention, the attention module in the feature calibration encoder module is a feature calibration attention module, and the attention module in the encoder module is a multi-head self-attention module formed by removing the feature calibration module and processing with batch normalization and Hardwish activation function;

[0098] Among them, the convolution position encoding operation is:

[0099]

[0100] Where X2 is the 4x patch embedding or 2x patch embedding or the output feature map of the encoder module, Output feature map for convolution position encoding, Conv 3×3 is a convolution operation with a filter size of 3×3.

[0101] In an embodiment of the present invention, the feature calibration attention module linearly maps the input feature map of the feature calibration attention module into a query Q, key K, and value V matrix through linear mapping; wherein, the matrices occupying one-fifth of the channels of the query Q, key K, and value V matrices are connected in the channel dimension and a depth-separable convolution operation is performed, the filter size is 3×3, the step size is 1, and the number of output feature map channels is reduced to one-third while maintaining the spatial resolution unchanged; wherein, the pre-attention branch matrix occupying four-fifths of the channels of the query Q, key K, and value V matrices is divided according to the number of channels 8, and for each local Q i , K i 、V i After feature calibration (i=1,2,...,n), the Splicing to form Q * , K * 、V * , and perform the following multi-head self-attention calculation:

[0102]

[0103] Where Att is the output feature map of the multi-head self-attention module, Q *, K * 、V * Q i , K i 、V i (i=1,2,...,n) is the query, key, and value matrix formed after feature calibration and channel concatenation. d is the scaling factor, Softmax(·) is the exponential normalization function, and (·) T Represents matrix transpose;

[0104] Among them, the convolution relative position encoding module is:

[0105] ReP=Q * oConv 3，5，7 (V * ),

[0106] Where ReP is the output matrix of the convolutional relative position encoding module, Conv 3，5，7 Perform convolution operations with filter sizes of 3×3, 5×5, and 7×7 for each channel;

[0107] The output feature map of the multi-head self-attention module with convolutional relative position encoding is composed of the output feature map of the multi-head self-attention module and the output matrix of the convolutional relative position encoding module:

[0108]

[0109] Where ConvAtt represents the output feature map of the multi-head self-attention module with convolutional relative position encoding; ConvAtt is concatenated with the branch matrix formed by one-fifth of the query Q, key K, and value V in the channel dimension and linearly mapped to form the output feature map of the feature calibration attention module.

[0110] In the embodiment of the present invention, the feature calibration module inputs the local Q i , K i 、V i (i=1,2,...,n), and output the local Among them, the feature calibration module is:

[0111]

[0112]

[0113] Where Q i , K i 、V i (i=1,2,...,n) are the local matrices of Q, K, and V after channel segmentation, n is the number of channel segmentations, Q, K, and V are the pre-attention branches of the query, key, and value matrices formed by the linear mapping of image tokens, Q after feature calibration i , K i 、V i , o is the point-by-point product, σ(·) is the Sigmoid activation function, LN(·) is the LayerNorm layer normalization, DWC 3×3 It is a depth-separable convolution operation with a filter size of 3×3; the local Splicing in the channel dimension to obtain Q, K, V matrices for feature calibration * , K * 、V * matrix.

[0114] In the embodiment of the present invention, the classification module uses a linear classification head to generate target categories; the linear classification head is a commonly used module in the field of image classification.

[0115] Step 3: Data preprocessing. Before inputting into the network training, each image is first scaled to 224 pixels in both width and height. Then, a portion of the image is randomly cropped and scaled to 224 pixels in both width and height. Next, a random left-right flip is performed at a ratio of 50%. Finally, the image pixels are normalized from 0-255 to the range of 0-1, so that the image size is 224×224 pixels.

[0116] Step 4: Training process: After the image is input into the network, it undergoes four stages of feature extraction to obtain the fourth stage feature map, and then outputs the microalgae classification result through the linear classification head module;

[0117] In addition, illustratively, in each step of training, back propagation is performed starting from the loss function value; the AdamW optimizer is used to optimize the network parameters according to the gradient information obtained by back propagation, thereby guiding the neural network to achieve accurate image classification results based on the input image.

[0118] The present invention applies visual cognitive computing technology to the field of automatic monitoring, and newly proposes a local feature calibration module, which is applied to the self-attention calculation under the visual Transformer framework, thereby forming a multi-head self-attention module based on feature calibration, and constructing a microalgae image classification network to solve the task of automatic classification of microalgae images. In response to the problem that the existing technology is insufficient in extracting significant target features, the present invention fully considers the aggregation of neighbor information and the enhancement of local feature weights, which can effectively improve the accuracy of microalgae image classification and reduce misclassification. The feature calibration module proposed in the present invention is based on basic convolution and matrix operations, and the input and output data formats are consistent, without excessive dependence, and is easy to apply to various deep image classification models. It can also be extended to other image recognition tasks, such as target detection and semantic segmentation, and has broad application prospects.

[0119] In the following specific embodiments of the present invention: the microalgae image dataset used contains a total of 41,862 microalgae L images (i.e., grayscale images) taken in situ in the ocean in 79 categories, with image width and height dimensions of 50-1000 pixels; it is randomly divided into a training set consisting of 33,517 images and a validation set consisting of 8,345 images, respectively. The images are first scaled to 224×224 pixels in the preprocessing stage, and the L images (i.e., grayscale images) are converted into RGB images (i.e., color images); the image classification network is initialized by random initialization in the embodiment of the present invention; the operating environment is a computer with a framework such as PyTorch, which can read given images and complete the construction and training of the model of this method. The training time of the embodiment of the present invention on a Gold6626R@2.90GHz CPU, 8G memory and NVIDIAGeForceRTX3090 GPU is about 9 hours.

[0120] In an embodiment of the present invention, the specific implementation steps include: first setting relevant training parameters, setting the optimizer used for network update in the present invention to the AdamW optimizer, setting its momentum value to 0.05, the initial learning rate to 0.001, and the weight decay coefficient to 0.00001. The learning rate adjustment strategy is set to linear preheating and cosine simulated annealing, and the preheating period is 10. The deep image classification network of the embodiment of the present invention consists of a feature extraction module comprising 4 stages and a linear classification head module. The 4 stages of the feature extraction module respectively include 3, 3, 12, and 4 basic visual cognitive calculation modules; the input of the network is an RGB image (i.e., a color image), and the texture and abstract semantic information of the microalgae image are extracted by the encoder module and the feature calibration encoder module. The feature map output by it undergoes multiple stages of feature extraction and is finally passed to the linear classification head module to obtain a classification result image containing the target category and probability. Next, when using the divided data set for network training, 64 pictures are randomly selected from the training set each time to input the network, and the parameters are updated using the selected AdamW optimizer. The training is completed after 100 rounds of iteration on the data set. Finally, the images in the validation set are input into the trained network for classification, and the results of the embodiment of the invention are obtained, such as Figure 6 As shown in the figure, the microalgae categories and their probabilities given in the examples indicate that the proposed method achieves good microalgae image classification results. In all nine microalgae image samples, the microalgae categories were correctly assigned with high probabilities, demonstrating the effectiveness of the present invention in accurately classifying microalgae images.

[0121] In summary, an embodiment of the present invention discloses a microalgae image classification method based on a feature calibration transformer, wherein the feature calibration module of the method is used for local information fusion and significant feature enhancement; and the feature calibration attention module is used to achieve long-distance dependency modeling for local information enhancement. The method of the present invention introduces visual cognitive computing into the field of automatic classification, which can effectively improve the accuracy of microalgae image classification, and each module is simple to implement, without excessive dependence, and has strong applicability. In the microalgae image classification method based on a feature calibration transformer provided by the present invention, the implementation result is the calibration of the target species in the image, which can solve the technical problem of misclassification caused by insufficient capture of significant features of microalgae images in the above-mentioned prior art.

[0122] The following are device embodiments of the present invention, which can be used to perform the method embodiments of the present invention. For details not disclosed in the device embodiments, please refer to the method embodiments of the present invention.

[0123] See also Figure 7 In an embodiment of the present invention, a microalgae image classification system based on feature-calibrated Transformer is provided, comprising:

[0124] An image acquisition module, used for acquiring images of microalgae to be classified;

[0125] A classification module, configured to classify the microalgae based on the microalgae image to be classified using a pre-trained deep image classification network to obtain a classification result;

[0126] in,

[0127] The deep image classification network includes stage 1, stage 2, stage 3, stage 4 and a linear classification head connected in sequence; the stage 1 includes a 4-fold patch embedding, a plurality of encoder modules and a feature calibration encoder module connected in series; the stage 2, the stage 3 and the stage 4 each include a 2-fold patch embedding, a plurality of encoder modules and a feature calibration encoder module connected in series;

[0128] The feature calibration encoder module includes a convolutional position encoding, a first LayerNorm layer normalization operation, a feature calibration attention module, a second LayerNorm layer normalization operation and a feedforward neural network layer connected in series, and the input of the first LayerNorm layer normalization operation is added to the output of the feature calibration attention module, and the input of the second LayerNorm layer normalization operation is added to the output of the feedforward neural network through a skip connection layer;

[0129] The feature calibration attention module includes a feature calibration module, a multi-head self-attention module, a convolution relative position encoding module and a depth-separable convolution module; wherein, the feature calibration module is used to input the query Q, key K, value V matrix formed by the normalization operation output feature map of the first LayerNorm layer through linear mapping, and output the query Q after feature calibration. * , key K * , value V * Matrix; The multi-head self-attention module is used to input the query Q after feature calibration * , key K * , value V * matrix, outputs the feature map of the multi-head self-attention module; the convolution relative position encoding module is used to input the query Q after feature calibration * , value V * Matrix, output convolution relative position encoding module output matrix; the depth separable convolution module is used to input the query Q, key K, value V matrix formed by linear mapping the output feature map of the first LayerNorm layer normalization operation, and output the local matrix of one-fifth of the channel, and output the output feature map of the depth separable convolution module; the feature calibration attention module output feature map is the sum of the output feature map of the multi-head self-attention module and the output matrix of the convolution relative position encoding module, and is spliced ​​with the output feature map of the depth separable convolution module in the channel dimension to form a feature map.

[0130] In one embodiment of the present invention, a computer device is provided, comprising a processor and a memory, wherein the memory is used to store a computer program, the computer program including program instructions, and the processor is used to execute the program instructions stored in the computer storage medium. The processor may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing core and control core of the terminal, which is suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions in the computer storage medium to implement the corresponding method flow or corresponding function; the processor described in the embodiment of the present invention can be used to perform the operation of the microalgae image classification method based on feature calibration transformer.

[0131] In one embodiment of the present invention, a storage medium is provided, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device for storing programs and data. It is understandable that the computer-readable storage medium here can include both built-in storage media in the computer device and, of course, extended storage media supported by the computer device. The computer-readable storage medium provides a storage space, which stores the operating system of the terminal. In addition, one or more instructions suitable for being loaded and executed by the processor are also stored in the storage space. These instructions can be one or more computer programs (including program codes). It should be noted that the computer-readable storage medium here can be a high-speed RAM memory or a non-volatile memory, such as at least one disk memory. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the corresponding steps of the microalgae image classification method based on feature-calibrated Transformer in the above embodiment.

[0132] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0133] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0134] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0135] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0136] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not depart from the spirit and scope of the present invention should be covered by the scope of protection of the claims of the present invention.

Claims

1. A microalgae image classification method based on feature calibration Transformer, characterized in that: The following steps are involved: Acquiring an image of microalgae to be classified; Based on the microalgae image to be classified, using a pre-trained deep image classification network to classify the microalgae to obtain a classification result; in, The deep image classification network includes stage 1, stage 2, stage 3, stage 4 and a linear classification head connected in sequence; the stage 1 includes a 4-fold patch embedding, a plurality of encoder modules and a feature calibration encoder module connected in series; the stage 2, the stage 3 and the stage 4 each include a 2-fold patch embedding, a plurality of encoder modules and a feature calibration encoder module connected in series; The feature calibration encoder module includes a convolutional position encoding, a first LayerNorm layer normalization operation, a feature calibration attention module, a second LayerNorm layer normalization operation and a feedforward neural network layer connected in series, and the input of the first LayerNorm layer normalization operation is added to the output of the feature calibration attention module, and the input of the second LayerNorm layer normalization operation is added to the output of the feedforward neural network through a skip connection layer; The feature calibration attention module includes a feature calibration module, a multi-head self-attention module, a convolution relative position encoding module and a depth-separable convolution module; wherein, the feature calibration module is used to input the query Q, key K, value V matrix formed by the normalization operation output feature map of the first LayerNorm layer through linear mapping, and output the query Q after feature calibration. * , key K * , value V * Matrix; The multi-head self-attention module is used to input the query Q after feature calibration * , key K * , value V * matrix, outputs the feature map of the multi-head self-attention module; the convolution relative position encoding module is used to input the query Q after feature calibration * , value V * Matrix, output convolution relative position encoding module output matrix; the depth separable convolution module is used to input the query Q, key K, value V matrix formed by linear mapping the output feature map of the first LayerNorm layer normalization operation, and output the local matrix of one-fifth of the channel, and output the output feature map of the depth separable convolution module; the feature calibration attention module output feature map is the sum of the output feature map of the multi-head self-attention module and the output matrix of the convolution relative position encoding module, and is spliced ​​with the output feature map of the depth separable convolution module in the channel dimension to form a feature map.

2. The microalgae image classification method based on feature calibration Transformer according to claim 1, characterized in that: The feature calibration module includes a backbone network and a branch network; the output of the feature calibration module is the point-by-point product of the backbone network output and the branch network output, and the obtained grouped feature map is spliced ​​by channel; wherein, the input of the backbone network and the branch network is a query Q, key K, and value V matrix of a group of 8 channels, the operation of the backbone network is identity mapping, and the operation of the branch network is depth-separable convolution, LayerNorm layer normalization, and sigmoid activation function operation.

3. The microalgae image classification method based on feature calibration Transformer according to claim 1, characterized in that: In the deep image classification network, The 4x patch embedding is used to input a microalgae image and output a 4x downsampled feature map; The 2x patch embedding is used to input the output feature map of the feature calibration encoder module of the previous stage, and output a 2x downsampled feature map; The encoder module is used to input a 4x or 2x downsampled feature map, and the output encoder module outputs a feature map; The feature calibration encoder module is used to input the encoder module output feature map, and output the feature calibration encoder module output feature map.

4. The microalgae image classification method based on feature calibration Transformer according to claim 1, characterized in that: In the feature calibration encoder module, The convolution position encoding is used to input the output feature map of the encoder module, and the convolution position encoding output feature map is output; The first LayerNorm layer normalization operation is used to input the convolution position encoding output feature map, and output the first LayerNorm layer normalization operation output feature map; The feature calibration attention module is used to input the output feature map of the first LayerNorm layer normalization operation and output the output feature map of the feature calibration attention module; The second LayerNorm layer normalization operation is used to input the sum of the feature calibration attention module output feature map and the convolution position encoding output feature map, and output the second LayerNorm layer normalization operation output feature map; The feedforward neural network layer is used to input the normalization operation output feature map of the second LayerNorm layer and output the feedforward neural network output feature map; The feature calibration encoder module output feature map is the sum of the feedforward neural network output feature map and the second LayerNorm layer normalization operation input feature map.

5. The microalgae image classification method based on feature calibration Transformer according to claim 1, characterized in that: The encoder module includes a convolutional position code, a first LayerNorm layer normalization operation, a multi-head self-attention module, a second LayerNorm layer normalization operation and a feedforward neural network layer connected in series in sequence, and the input of the first LayerNorm layer normalization operation is added to the output of the multi-head self-attention module, and the input of the second LayerNorm layer normalization operation is added to the output of the feedforward neural network through a jump connection layer.

6. The microalgae image classification method based on feature calibration Transformer according to claim 1, characterized in that: The feature calibration module is represented as, Where Q i , K i 、V i are the local matrices of Q, K, and V after channel segmentation, i = 1, 2, ..., n, where n is the number of channel segmentations; Q, K, and V are the query, key, and value matrices formed by linear mapping of image tokens, respectively; Q after feature calibration i , K i 、V i ; o is the point-by-point product; σ(·) is the Sigmoid activation function; LN(·) is the LayerNorm layer normalization; DWC 3×3 is a depthwise separable convolution operation with a filter size of 3×3.

7. The microalgae image classification method based on feature calibration Transformer according to claim 1, characterized in that: In the feature calibration attention module, The multi-head self-attention module is, Where Att is the output feature map of the multi-head self-attention module; Q * , K * 、V * Q i , K i 、V i The query, key, and value matrices are formed after feature calibration and channel concatenation; d is the scaling factor; Softmax(·) is the exponential normalization function; (·) T Represents matrix transpose; The convolutional relative position encoding module is, ReP=Q * o Conv 3，5，7 (V * ), Where ReP is the output matrix of the convolutional relative position encoding module; Conv 3，5，7 Perform convolution operations with filter sizes of 3×3, 5×5, and 7×7 for each channel; The output feature map ConvAtt of the multi-head self-attention module with convolutional relative position encoding is, 8. A microalgae image classification system based on feature calibration Transformer, characterized in that: include: An image acquisition module, used for acquiring images of microalgae to be classified; A classification module, configured to classify the microalgae based on the microalgae image to be classified using a pre-trained deep image classification network to obtain a classification result; in, The deep image classification network includes stage 1, stage 2, stage 3, stage 4 and a linear classification head connected in sequence; the stage 1 includes a 4-fold patch embedding, a plurality of encoder modules and a feature calibration encoder module connected in series; the stage 2, the stage 3 and the stage 4 each include a 2-fold patch embedding, a plurality of encoder modules and a feature calibration encoder module connected in series; The feature calibration encoder module includes a convolutional position encoding, a first LayerNorm layer normalization operation, a feature calibration attention module, a second LayerNorm layer normalization operation and a feedforward neural network layer connected in series, and the input of the first LayerNorm layer normalization operation is added to the output of the feature calibration attention module, and the input of the second LayerNorm layer normalization operation is added to the output of the feedforward neural network through a skip connection layer; The feature calibration attention module includes a feature calibration module, a multi-head self-attention module, a convolution relative position encoding module and a depth-separable convolution module; wherein, the feature calibration module is used to input the query Q, key K, value V matrix formed by the normalization operation output feature map of the first LayerNorm layer through linear mapping, and output the query Q after feature calibration. * , key K * , value V * Matrix; The multi-head self-attention module is used to input the query Q after feature calibration * , key K * , value V * matrix, outputs the feature map of the multi-head self-attention module; the convolution relative position encoding module is used to input the query Q after feature calibration * , value V * Matrix, output convolution relative position encoding module output matrix; the depth separable convolution module is used to input the query Q, key K, value V matrix formed by linear mapping the output feature map of the first LayerNorm layer normalization operation, and output the local matrix of one-fifth of the channel, and output the output feature map of the depth separable convolution module; the feature calibration attention module output feature map is the sum of the output feature map of the multi-head self-attention module and the output matrix of the convolution relative position encoding module, and is spliced ​​with the output feature map of the depth separable convolution module in the channel dimension to form a feature map.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the microalgae image classification method based on feature-calibrated Transformer according to any one of claims 1 to 7 is implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the microalgae image classification method based on feature-calibrated Transformer is implemented according to any one of claims 1 to 7.

Images