Diptera insect classification and identification method based on neural network hybrid model

Abstract

The invention discloses a diptera insect classification method based on a neural network hybrid model, and the method comprises the following steps: constructing a diptera insect image data set, and marking an insect target; performing data enhancement on the constructed data set to obtain a complete data set, and dividing the complete data set into a training set, a verification set and a test set in proportion; constructing a diptera insect classification network based on an improved mobile vision transformer layer; initializing the diptera insect classification network of the improved mobile vision transformer layer, setting training parameters, and training the diptera insect classification network based on the improved mobile vision transformer layer by adopting the training set to obtain a trained diptera insect classification network; and inputting the test set image into the trained diptera insect classification network to realize classification of diptera insects. In combination with a convolutional local feature extraction capability and a self-attention global feature modeling capability, the model can efficiently extract deep semantic information, and classification of diptera insect images is realized.

Description

Technical Field

[0001] The invention belongs to the technical field of dipteran insect image classification, and in particular relates to a dipteran insect classification method based on a neural network hybrid model. Background Art

[0002] With the increasing global transportation and international trade, a large number of alien species have been introduced into various countries via shipping, causing serious harm to local communities. Among these alien species, Diptera insects are particularly unique. Their small size makes them difficult to identify, posing challenges for customs authorities in their detection. Furthermore, Diptera insects often carry viruses and easily spread diseases, posing serious risks to local ecosystems, agriculture, and human health. Therefore, accurately identifying whether an alien Diptera insect is an invasive species and preventing its entry into the country is crucial.

[0003] At present, the classification of Diptera targets in laboratory environments mainly faces the following difficulties: (1) The appearance features of the same Diptera may vary greatly due to the influence of different angles and postures, which puts higher demands on the generalization ability of the classification model; (2) Diptera of different categories often have similar appearance features, which easily leads to classification confusion and poses challenges to the model's feature extraction and detail attention capabilities. In response to the above difficulties, this paper proposes the following solutions: By designing a random grid mask data augmentation technology in the training phase to generate diverse input samples, the model's adaptability to Diptera targets at different angles and postures is improved, and the model's generalization ability is enhanced; a learnable parallel attention mechanism is designed to guide the model to focus on the key detail features of the insects and reduce the interference of background or other irrelevant information, thereby improving the feature extraction ability and classification accuracy of the classification model; a lightweight multi-head attention module is designed and combined with a self-feature fusion strategy to reduce the model's computational overhead while improving feature extraction efficiency. By combining the above methods, this paper designs an efficient Diptera classification method, aiming to solve the problems of angle variation and appearance similarity in the Diptera classification task in laboratory environment, and provide a more accurate and efficient solution for Diptera classification. Summary of the Invention

[0004] In order to solve the above problems, the technical solution adopted by the present invention is: a method for classifying Diptera insects based on a neural network hybrid model, comprising the following steps:

[0005] S1: Construct a Diptera insect image dataset and annotate insect targets;

[0006] S2: Perform data augmentation on the constructed dataset to obtain a complete dataset, which is then divided into training, validation, and test sets in proportion.

[0007] S3: Constructing a Diptera insect classification network based on an improved mobile visual transformer layer;

[0008] S4: Initialize the diptera insect classification network based on the improved mobile visual transformer layer, set training parameters, and use the training set to train the diptera insect classification network based on the improved mobile visual transformer layer to obtain a trained diptera insect classification network;

[0009] S5: Input the test set images into the trained Diptera classification network to achieve Diptera classification.

[0010] Furthermore, the process of performing data enhancement on the constructed dataset to obtain a complete dataset is as follows:

[0011] Preprocess the images in the data set to obtain preprocessed images;

[0012] The preprocessed image is masked using a random grid masking method.

[0013] Furthermore, the process of performing mask processing on the pre-processed image using a random grid mask processing method is as follows:

[0014] The input image I with dimensions W×H×3 is divided into (H×W) / (r×r) patches. Based on the divided grid, four adjacent patches are set as a group, with a total of (H×W) / 4(r×r) groups.

[0015] In each group, k patches are randomly masked, which means that k×25% of the entire image is masked out;

[0016] The size of the grid can be set by adjusting the r parameter to perform mask operations on different datasets.

[0017] Furthermore, the insect classification network of the improved mobile visual transformer layer includes:

[0018] Convolutional layer, used to extract local features of Diptera insects;

[0019] A first convolutional neural sub-network, configured to further extract local features from the local features output by the convolutional layer;

[0020] A first parallel channel attention module is used to dynamically guide the network to focus on key information of Diptera insects based on the local features extracted by the convolutional neural sub-network;

[0021] A second convolutional neural sub-network is used to dynamically guide the first parallel channel attention module to focus on key information of Diptera insects and perform local feature extraction;

[0022] A second parallel channel attention module is used to dynamically guide the network to focus on key information of Diptera insects based on the local features extracted by the second convolutional neural network;

[0023] The third convolutional neural sub-network is used to dynamically guide the second parallel channel attention module to focus on key information of Diptera insects and perform local feature extraction;

[0024] a fourth convolutional neural sub-network, configured to further extract local features from the local features extracted by the third convolutional neural sub-network;

[0025] a fifth convolutional neural sub-network, configured to further extract local features from the local features extracted by the fourth convolutional neural sub-network;

[0026] The classification subnetwork is used to classify Diptera insects based on the local features extracted by the fifth convolutional neural subnetwork.

[0027] Furthermore, the first parallel channel attention module and the second parallel channel attention module have the same structure; the process of the first parallel channel attention module dynamically guiding the network to focus on the key information of Diptera insects based on the extracted local features is as follows:

[0028] For the input feature map Where C, H and W represent the number of channels, height and width respectively; first, pooling is performed in two ways along the spatial dimension to obtain the global average pooling and global max pooling Where C is the number of channels, 1×1 represents height and width, that is, the C channel of size 1×1 represents the output vector, which represents the maximum and average value of each channel of the feature map in the spatial dimension;

[0029] Secondly, the outputs of global average pooling and global maximum pooling are weighted by a trainable weight vector w to obtain the weighted result A;

[0030] The weighted result A is normalized by the Sigmoid activation function to obtain the attention coefficient A of each channel out ;

[0031] The obtained channel attention coefficient A out Applied to each channel of the original feature map, weighted, to obtain the feature map after attention guidance.

[0032] Furthermore, the third convolutional neural sub-network, the fourth convolutional neural sub-network and the fifth convolutional neural sub-network each include a mobile visual transformer layer;

[0033] The mobile vision transformer layer includes a lightweight self-feature fusion multi-head self-attention module;

[0034] The lightweight self-feature fusion multi-head self-attention module includes:

[0035] The self-feature fusion module is used to fusion the input features x∈R H×W×C , after a linear mapping layer, the mutual interaction combination between global features is obtained, and then the SiLU activation function is used for nonlinear transformation and enhances the expression ability of the network;

[0036] A lightweight multi-head self-attention module. Within the lightweight multi-head self-attention module, grouped convolutional mapping divides the input channels into multiple groups, and convolution operations are performed simultaneously and in parallel, significantly improving computational efficiency and shortening inference time.

[0037] The self-feature fusion module and the lightweight multi-head self-attention module are connected sequentially.

[0038] Furthermore, the self-feature fusion module is expressed as:

[0039] z=x+φ(Wx)

[0040] Where: z is the output of the self-feature fusion module, φ is the SiLU activation function, and W is the weight of the linear layer;

[0041] The SiLU function is expressed as:

[0042]

[0043] Furthermore, the lightweight multi-head self-attention module is represented as using a grouped convolution map with a convolution kernel size of 1×1 to replace the linear map in the original multi-head attention. For the input feature x∈R H×W×C , using 1×1 group convolution to perform convolution mapping, the query Q, key K and value V are expressed as:

[0044] Q = Conv1 × 1_g(X)

[0045] K = Conv1 × 1_g(X)

[0046] V = Conv1 × 1_g(X)

[0047] Among them: the group number parameter is g, Conv1×1_g is a 1×1 group convolution, and the input feature x∈R H×W×C Divided into The convolution within the group shares parameters, and the number of parameters can be reduced by setting a reasonable parameter g;

[0048] Thus we get Q, K, V∈R L×CMatrix for attention score calculation:

[0049]

[0050] Where: d k The vector feature dimension C is K, and the expression of the softmax function is as follows:

[0051]

[0052] Where: x i express

[0053] The i-th element of a row in the matrix represents the relevance score of a query with the i-th key, which is normalized to the attention weight through softmax.

[0054] Furthermore, the classification subnetwork uses an improved weighted cross entropy loss function as the classification loss function;

[0055] The expression of the improved weighted cross entropy loss function is as follows:

[0056]

[0057] Among them, y i is the true category, that is, only y corresponding to the true category i =1, the rest are 0; is the predicted probability of the model.

[0058] Furthermore, the process of initializing the dipteran insect classification network based on the improved mobile visual transformer layer, setting training parameters, and using a training set to train the dipteran insect classification network based on the improved mobile visual transformer layer to obtain a trained dipteran insect classification network is as follows:

[0059] S41: Initialize the backbone Mobile ViT--xxs using pre-trained weights on the ImageNet image dataset;

[0060] S42: During network training, an adaptive moment estimation optimizer with weight decay is used as the optimizer, and the weight decay is set to 0.0001;

[0061] S43: The initial learning rate is set to 0.0002, the batch size is set to 64, and a total of 150 batches are trained for the Diptera classification network.

[0062] The present invention provides a method for classifying Diptera insects based on a neural network hybrid model, which has the following advantages:

[0063] (1) By combining the local feature extraction capability of convolution and the global feature modeling capability of self-attention, the model can efficiently extract deep semantic information and realize the classification of Diptera insect images.

[0064] (2) A new data augmentation method is designed to perform grid mask operations with appropriate probabilities on the input images, making the training input data more diverse and enhancing the generalization ability of the model.

[0065] (3) To strengthen the network’s focus on key features, this application designs a learnable parallel attention module. This module learns the importance of each channel and dynamically adjusts its weight, enabling the model to more accurately distinguish subtle morphological features between different species of Diptera insects, thereby improving classification accuracy.

[0066] (4) To solve the problem of excessive computational overhead of the model, a lightweight multi-head attention module with a self-feature fusion mechanism is designed. This module can reduce computational overhead and accelerate model training. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0068] Figure 1 This is a flow chart of a method for classifying Diptera insect images based on a neural network hybrid model according to an embodiment of the present invention;

[0069] Figure 2 These are sample images from a Diptera insect image dataset according to an embodiment of the present invention; (a) is Lucilia forcifolia, (b) is Chrysomelidae macrocephala, (c) is Aedes aegypti, (d) is Housefly, (e) is Carrion Fly, (f) Alicia gigantea, (g) Lucilia luminifera, (h) Lucilia shenyangensis, (i) Lucilia sericata, (j) Sarcophagia ruficollis, (k) Flower Fly, (i) Aedes aegypti, (m) Aedes aegypti, (n) Culex pipiens, and (o) Anopheles sinensis.

[0070] Figure 3 A schematic diagram of an improved Mobile ViT network according to an embodiment of the present invention;

[0071] Figure 4 Schematic diagram of a data enhancement process according to an embodiment of the present invention;

[0072] Figure 5 is a schematic diagram of an attention module according to an embodiment of the present invention;

[0073] Figure 6 Schematic diagram of a lightweight self-feature fusion module according to an embodiment of the present invention.

[0074] Figure 7 Schematic diagram of training and validation curves of an embodiment of the present invention.

[0075] Figure 8 Schematic diagram of classification results according to an embodiment of the present invention. DETAILED DESCRIPTION

[0076] It should be noted that, unless there is any conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0077] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. The following description of at least one exemplary embodiment is actually only illustrative and is in no way intended to limit the present invention and its application or use. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0078] like Figure 1 As shown, a method for classifying Diptera insects based on a neural network hybrid model includes the following steps:

[0079] S1: Construct a Diptera insect image dataset and annotate insect targets;

[0080] S2: Perform data augmentation on the constructed dataset to obtain a complete dataset, which is then divided into training, validation, and test sets in a ratio of 6:2:2.

[0081] S3: Constructing a Diptera insect classification network based on an improved mobile visual transformer layer;

[0082] S4: Initialize the diptera insect classification network based on the improved mobile visual transformer layer, set training parameters, and use the training set to train the diptera insect classification network based on the improved mobile visual transformer layer to obtain a trained diptera insect classification network;

[0083] S5: Input the test set images into the trained Diptera classification network to achieve Diptera classification.

[0084] Steps S1 / S2 / S3 / S4 / S5 are performed sequentially;

[0085] Figure 2 These are sample images from a Diptera insect image dataset according to an embodiment of the present invention; (a) is Lucilia forcifolia, (b) is Chrysomelidae macrocephala, (c) is Aedes aegypti, (d) is Housefly, (e) is Carrion Fly, (f) Alicia gigantea, (g) Lucilia luminifera, (h) Lucilia shenyangensis, (i) Lucilia sericata, (j) Sarcophagia ruficollis, (k) Flower Fly, (i) Aedes aegypti, (m) Aedes aegypti, (n) Culex pipiens, and (o) Anopheles sinensis.

[0086] The process of constructing a Diptera insect image dataset and labeling insect targets is as follows:

[0087] S11: A dataset of Diptera insects, mainly flies and mosquitoes, was collected from the China Customs Laboratory. The original high-resolution images were downsampled to 224×224 pixels to complete the dataset construction.

[0088] S12: Use the labeling tool LabelImg to label the Diptera insects in the dataset and obtain a labeling file with categories.

[0089] Furthermore, the process of performing data enhancement on the constructed dataset to obtain a complete dataset is as follows:

[0090] Preprocessing the images in the data set to obtain preprocessed images; the preprocessing uses random flipping, random rotation, or random scaling to process the dipteran insect images;

[0091] The preprocessed image is masked using a random grid masking method.

[0092] Furthermore, the process of performing mask processing on the pre-processed image using a random grid mask processing method is as follows:

[0093] The input image I with dimensions W×H×3 is divided into (H×W) / (r×r) patches. Based on the divided grid, four adjacent patches are set as a group, with a total of (H×W) / 4(r×r) groups.

[0094] In each group, k patches are randomly masked. For the entire image, k×25% of the area is masked out; the range of k is 0-4, and 1 is the optimal value.

[0095] The size of the grid can be set by adjusting the r parameter to perform mask operations on different datasets.

[0096] The diversity of input images is increased through mask operations to improve generalization ability; in the convolutional network, a parallel channel attention module (Learnable Max AvgPool Attention, LMAP) is designed to focus on key information to enhance local feature extraction; in the Transformer network, a lightweight self-feature fusion transformer module is proposed to achieve efficient multi-head attention and reduce computational overhead.

[0097] The insect classification network with an improved mobile visual transformer layer uses a hybrid neural network, Mobile ViT, as a feature extraction network. A learnable parallel attention module is designed and added to the MobileNetV2 (MV2) in the shallow convolutional layer of the network. A learnable parallel channel attention module is designed after the feature output of the last layer of the feature extraction network MV2 to dynamically guide the network to focus on key information and improve the ability to extract key features.

[0098] The Mobile Vision Transformer layer is improved by incorporating MV2. MV2 in the MobileViT architecture is the inverted residual block of MobileNetV2, responsible for local feature extraction and downsampling. Stacking and self-attention modules achieve efficient feature extraction and global information modeling. The MV2 layer significantly reduces the number of parameters and computational overhead by stacking depthwise separable convolutions and layer-by-layer convolutions, while also extracting efficient local features. The self-attention layer, by stacking multiple self-attention modules, captures the global semantic information of the image, enhancing the model's adaptability to complex backgrounds and varying angles.

[0099] After selecting the last feature maps {C1, C2, C3} of the three inverse residual blocks of convolutional layer MV2 in the shallow convolutional layer of the network, a learnable parallel attention mechanism is designed and added. Through the above process, the attention mechanism is introduced.

[0100] Figure 3 A schematic diagram of an improved Mobile ViT network according to an embodiment of the present invention;

[0101] Figure 4 Schematic diagram of a data enhancement process according to an embodiment of the present invention;

[0102] Figure 5 is a schematic diagram of an attention module according to an embodiment of the present invention;

[0103] Figure 6 Schematic diagram of a lightweight self-feature fusion module according to an embodiment of the present invention.

[0104] Furthermore, the insect classification network of the improved mobile visual transformer layer includes:

[0105] Convolutional layer, used to extract local features of Diptera insects;

[0106] A first convolutional neural sub-network, configured to further extract local features from the local features output by the convolutional layer;

[0107] A first parallel channel attention module is used to dynamically guide the network to focus on key information of Diptera insects based on the local features extracted by the convolutional neural sub-network;

[0108] A second convolutional neural sub-network is used to dynamically guide the first parallel channel attention module to focus on key information of Diptera insects and perform local feature extraction;

[0109] A second parallel channel attention module is used to dynamically guide the network to focus on key information of Diptera insects based on the local features extracted by the second convolutional neural network;

[0110] The third convolutional neural sub-network is used to dynamically guide the second parallel channel attention module to focus on key information of Diptera insects and perform local feature extraction;

[0111] a fourth convolutional neural sub-network, configured to further extract local features from the local features extracted by the third convolutional neural sub-network;

[0112] a fifth convolutional neural sub-network, configured to further extract local features from the local features extracted by the fourth convolutional neural sub-network;

[0113] The classification subnetwork is used to classify Diptera insects based on the local features extracted by the fifth convolutional neural subnetwork.

[0114] Furthermore, the first parallel channel attention module and the second parallel channel attention module have the same structure;

[0115] The first parallel channel attention module dynamically guides the network to focus on key information of Diptera insects based on the extracted local features as follows:

[0116] For the input feature map Where C, H, and W represent the number of channels, height, and width, respectively. First, pooling is performed in two ways along the spatial dimension to obtain Global Average Pooling (GAP). and Global Max Pooling (GMP) Where C is the number of channels 1×1 represents height and width, that is, C channels of size 1×1 (representing the output vector, representing the maximum and average values of each channel of the feature map in the spatial dimension)

[0117] Secondly, the outputs of global average pooling and global maximum pooling are weighted by a trainable weight vector w to obtain the weighted result A;

[0118] The weighted result A is normalized by the Sigmoid activation function to obtain the attention coefficient A of each channel out ;

[0119] The sigmoid activation function can be expressed as:

[0120]

[0121] The obtained channel attention coefficient A out Applied to each channel of the original feature map, weighted, to obtain the feature map after attention guidance.

[0122] Furthermore, the third convolutional neural sub-network, the fourth convolutional neural sub-network and the fifth convolutional neural sub-network each include a mobile visual transformer layer, MV2 and a Mobile ViT block; the mobile visual transformer layer, MV2 and MobileViT block are connected in sequence; The mobile vision transformer layer includes a lightweight self-feature fusion multi-head self-attention module; it also includes a feedforward network, layer normalization and residual connection; The lightweight self-feature fusion multi-head self-attention module includes: Self-feature fusion module is used to fusion the input features , after a linear mapping layer, the mutual interaction combination between global features is obtained, and then the SiLU activation function is used for nonlinear transformation and enhances the expression ability of the network; A lightweight multi-head self-attention module. Within this lightweight multi-head self-attention module, grouped convolutional mapping divides the input channels into multiple groups, performing convolution operations simultaneously and in parallel. This significantly improves computational efficiency and reduces inference time, making it suitable for resource-constrained scenarios. This module leverages the independent processing capabilities of each group and taps into the potential of parallel computing, significantly increasing model training speed while maintaining prediction accuracy. Grouped convolutional mapping offers significant advantages in terms of parameters. By dividing the input data into independent groups for convolution operations, the number of model parameters is effectively reduced, and overall complexity can be precisely controlled based on the number of groups. This parameter reduction not only reduces the computational burden of training and inference, but also lowers memory usage, making the model more suitable for efficient operation in resource-constrained environments. Furthermore, grouped convolutional mapping introduces a regularization effect. By limiting parameter sharing within groups, it effectively suppresses overfitting during training, improving the model's adaptability and stability to unknown data samples, enabling it to better adapt to unseen data.

[0123] The self-feature fusion module and the lightweight multi-head self-attention module are connected sequentially.

[0124] Design a lightweight multi-head attention module and combine it with a self-feature fusion module to reduce the computational overhead of the Diptera insect classification model while improving feature extraction efficiency;

[0125] Furthermore: the self-feature fusion module is expressed as:

[0126] z=x+φ(Wx)

[0127] Where: z is the output of the self-feature fusion module, φ is the SiLU activation function, and W is the weight of the linear layer;

[0128] The SiLU function is expressed as:

[0129]

[0130] The lightweight multi-head self-attention module is expressed as using a grouped convolution mapping of the convolution kernel size to replace the linear mapping in the original multi-head attention, so as to reduce the number of parameters while strengthening the information exchange between channels in the group. This is beneficial for generating features such as Q, K, and V, because Q, K, and V play an important role in the self-attention mechanism. For the input feature x∈R H×W×C , using 1×1 group convolution to perform convolution mapping to obtain Q, K, V, expressed as: query (Query, Q), key (Key, K) and value (Value, V)

[0131] Q = Conv1 × 1_g(X)

[0132] K = Conv1 × 1_g(X)

[0133] V = Conv1 × 1_g(X)

[0134] Among them: the group number parameter is g, Conv1×1_g is a 1×1 group convolution, and the input feature x∈R H×W×C Divided into The convolution parameters within the group are shared. By setting a reasonable parameter g, the number of parameters can be effectively reduced.

[0135] Thus we get Q, K, V∈R L×C Matrix for attention score calculation: query (Query, Q), key (Key, K) and value (Value, V)

[0136]

[0137] Where: d k The vector feature dimension C is K, and the softmax function is:

[0138]

[0139] Where: x i express

[0140] The i-th element of a row in the matrix represents the relevance score of a query with the i-th key, which is normalized to the attention weight through softmax.

[0141] The classification subnetwork uses an improved weighted cross entropy loss function as the classification loss function;

[0142] The expression of the improved weighted cross entropy loss function is as follows:

[0143]

[0144] Among them, y i is the true category, that is, only y corresponding to the true category i =1, the rest are 0; is the predicted probability of the model.

[0145] The process of initializing the dipteran insect classification network based on the improved mobile visual transformer layer, setting training parameters, and using a training set to train the dipteran insect classification network based on the improved mobile visual transformer layer to obtain a trained dipteran insect classification network is as follows:

[0146] S41: Initialize the backbone Mobile ViT-XXS model using pre-trained weights on ImageNet (ImageNet is a large-scale image dataset commonly used for training and evaluating deep learning models);

[0147] S42: During network training, an adaptive moment estimation optimizer with weight decay is used as the optimizer, and the weight decay is set to 0.0001;

[0148] S43: The initial learning rate is set to 0.0002, the batch size is set to 64, and a total of 150 batches are trained for the Diptera classification network.

[0149] Based on the classification prediction results, ablation and comparative experiments are conducted as follows:

[0150] S51: Accuracy is defined as follows:

[0151]

[0152] Where True Positive (TP), True Negative (TN), False Positive (FP) and False Negative (FN) are the number of true positive, true lunar, false positive and false negative samples respectively.

[0153] S52: Classification and discrimination are performed based on the output probability of the softmax function;

[0154] S53: Visualize the training curve of the insect classification network model; Figure 7 As shown;

[0155] S54: Output the classification results of Diptera insects; the classification results are as follows: Figure 8 As shown, the blue font is the true category of Diptera insects, and the red font is the predicted category.

[0156] S55: Conduct an ablation experiment on the Diptera insect classification network model. The ablation experiment of the model is shown in Table 1. The experimental baseline model is Mobile ViT-XXS. The ablation module includes the random mask module (A), the attention module (B), and the lightweight self-feature fusion multi-head self-attention module (C).

[0157] Table 1: Ablation experiments of the model

[0158]

[0159] S56: Model comparison experiment. The model comparison experiment is shown in Table 2, comparing the accuracy, number of parameters, and number of floating-point operations of the models.

[0160] Table 2: Comparative experiments of models

[0161]

[0162] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for classifying Diptera insects based on a neural network hybrid model, characterized in that: The following steps are involved: S1: Construct a Diptera insect image dataset and annotate insect targets; S2: Perform data augmentation on the constructed dataset to obtain a complete dataset, which is then divided into training, validation, and test sets in proportion. S3: Constructing a Diptera insect classification network based on an improved mobile visual transformer layer; S4: Initialize the diptera insect classification network based on the improved mobile visual transformer layer, set training parameters, and use the training set to train the diptera insect classification network based on the improved mobile visual transformer layer to obtain a trained diptera insect classification network; S5: Input the test set images into the trained Diptera classification network to achieve Diptera classification.

2. A method for classifying Diptera insects based on a neural network hybrid model according to claim 1, characterized in that: The process of performing data enhancement on the constructed dataset to obtain a complete dataset is as follows: Preprocess the images in the data set to obtain preprocessed images; The preprocessed image is masked using a random grid masking method.

3. A method for classifying Diptera insects based on a neural network hybrid model according to claim 2, characterized in that: The process of masking the pre-processed image using the random grid masking method is as follows: The input image I with dimensions W×H×3 is divided into (H×W) / (r×r) patches. Based on the divided grid, four adjacent patches are set as a group, with a total of (H×W) / 4(r×r) groups. In each group, k patches are randomly masked, which means that k×25% of the entire image is masked out; The size of the grid can be set by adjusting the r parameter to perform mask operations on different datasets.

4. The method for classifying Diptera insect images based on a neural network hybrid model according to claim 1, wherein: The insect classification network with improved mobile vision transformer layer includes: Convolutional layer, used to extract local features of Diptera insects; A first convolutional neural sub-network, configured to further extract local features from the local features output by the convolutional layer; A first parallel channel attention module is used to dynamically guide the network to focus on key information of Diptera insects based on the local features extracted by the convolutional neural sub-network; A second convolutional neural sub-network is used to dynamically guide the first parallel channel attention module to focus on key information of Diptera insects and perform local feature extraction; A second parallel channel attention module is used to dynamically guide the network to focus on key information of Diptera insects based on the local features extracted by the second convolutional neural network; The third convolutional neural sub-network is used to dynamically guide the second parallel channel attention module to focus on key information of Diptera insects and perform local feature extraction; a fourth convolutional neural sub-network, configured to further extract local features from the local features extracted by the third convolutional neural sub-network; a fifth convolutional neural sub-network, configured to further extract local features from the local features extracted by the fourth convolutional neural sub-network; The classification subnetwork is used to classify Diptera insects based on the local features extracted by the fifth convolutional neural subnetwork.

5. The method for classifying Diptera insect images based on a neural network hybrid model according to claim 3, wherein: The first parallel channel attention module has the same structure as the second parallel channel attention module; the process of the first parallel channel attention module dynamically guiding the network to focus on the key information of Diptera insects based on the extracted local features is as follows: For the input feature map Where C, H and W represent the number of channels, height and width respectively; first, pooling is performed in two ways along the spatial dimension to obtain the global average pooling and global max pooling Where C is the number of channels, 1×1 represents height and width, that is, the C channel of size 1×1 represents the output vector, which represents the maximum and average value of each channel of the feature map in the spatial dimension; Secondly, the outputs of global average pooling and global maximum pooling are weighted by a trainable weight vector w to obtain the weighted result A; The weighted result A is normalized by the Sigmoid activation function to obtain the attention coefficient A of each channel out ; The obtained channel attention coefficient A out Applied to each channel of the original feature map, weighted, to obtain the feature map after attention guidance.

6. The method for classifying Diptera insect images based on a neural network hybrid model according to claim 3, wherein: The third convolutional neural sub-network, the fourth convolutional neural sub-network and the fifth convolutional neural sub-network each include a mobile visual transformer layer; The mobile vision transformer layer includes a lightweight self-feature fusion multi-head self-attention module; The lightweight self-feature fusion multi-head self-attention module includes: The self-feature fusion module is used to fusion the input features x∈R H×W×C , after a linear mapping layer, the mutual interaction combination between global features is obtained, and then the SiLU activation function is used for nonlinear transformation and enhances the expression ability of the network; A lightweight multi-head self-attention module. Within the lightweight multi-head self-attention module, grouped convolutional mapping divides the input channels into multiple groups, and convolution operations are performed simultaneously and in parallel, significantly improving computational efficiency and shortening inference time. The self-feature fusion module and the lightweight multi-head self-attention module are connected sequentially.

7. The method for classifying Diptera insect images based on a neural network hybrid model according to claim 5, wherein: The self-feature fusion module is expressed as: z=x+φ(Wx) Where: z is the output of the self-feature fusion module, φ is the SiLU activation function, and W is the weight of the linear layer; The SiLU function is expressed as:

8. The method for classifying Diptera insect images based on a neural network hybrid model according to claim 5, wherein: The lightweight multi-head self-attention module is represented as using a grouped convolution map with a convolution kernel size of 1×1 to replace the linear map in the original multi-head attention. For the input feature x∈R H×W×C , using 1×1 group convolution to perform convolution mapping, the query Q, key K and value V are expressed as: Q = Conv1 × 1_g(X) K = Conv1 × 1_g(X) V = Conv1 × 1_g(X) Among them: the group number parameter is g, Conv1×1_g is a 1×1 group convolution, and the input feature x∈R H×W×C Divided into The convolution within the group shares parameters, and the number of parameters can be reduced by setting a reasonable parameter g; Thus we get Q, K, V∈R L×C Matrix for attention score calculation: Where: d k The vector feature dimension C is K, and the expression of the softmax function is as follows: Where: x i express The i-th element of a row in the matrix represents the relevance score of a query with the i-th key, which is normalized to the attention weight through softmax.

9. The method for classifying Diptera insect images based on a neural network hybrid model according to claim 3, wherein: The classification subnetwork uses an improved weighted cross entropy loss function as the classification loss function; The expression of the improved weighted cross entropy loss function is as follows: Among them, y i is the true category, that is, only y corresponding to the true category i =1, the rest are 0; is the predicted probability of the model.

10. The method for classifying Diptera insect images based on a neural network hybrid model according to claim 1, wherein: The process of initializing the dipteran insect classification network based on the improved mobile visual transformer layer, setting training parameters, and using a training set to train the dipteran insect classification network based on the improved mobile visual transformer layer to obtain a trained dipteran insect classification network is as follows: S41: Initialize the backbone Mobile ViT--xxs using pre-trained weights on the ImageNet image dataset; S42: During network training, an adaptive moment estimation optimizer with weight decay is used as the optimizer, and the weight decay is set to 0.0001; S43: The initial learning rate is set to 0.0002, the batch size is set to 64, and a total of 150 batches are trained for the Diptera classification network.

Images