A method for behavior recognition of diptera based on a 3D convolution network model

Abstract

The method for recognizing the behavior of tephritid insects based on the 3D convolutional neural network model is named Y-NLR3D behavior recognition model. In the model, the Yolov5n network is used as the target detection head of the model, so that the rapid tracking and positioning of the target can be realized, and the problems of small size, rapid movement and difficult positioning of tephritid insects are solved; in the latter part of the model, the Resnet3D network added with a non-local attention module is used for behavior analysis of tephritid insects. The Y-NLR3D model proposed in the application can not only complete the tracking and positioning of tephritid insects, but also has a higher accuracy compared with other insect behavior recognition methods.

Description

Technical Field

[0001] This invention relates to a method for recognizing insect behavior, specifically a method for recognizing insect behavior based on a 3D convolutional network model. Background Technology

[0002] In the agricultural and forestry fields, behavioral analysis of fruit flies is an important research area. Currently, the main method used is to manually observe and record insect behavioral characteristics to build insect behavioral atlases. However, most researchers using this method need to analyze video frame by frame to observe and record the start and end times of each insect behavior and count the frequency of each behavior. This method is not only time-consuming and labor-intensive, with high labor costs, but the error of manual statistical analysis also increases with the working hours of the staff.

[0003] To address the aforementioned problems associated with manual observation and recording of insect behavior, computer vision technology is currently used to study insect behavior, which to some extent overcomes the shortcomings of manual insect behavior identification. Existing methods using computer vision to study insect behavior, such as the invention patent with authorization announcement number CN111476119B, disclose an insect behavior recognition method and device based on spatiotemporal context. This method requires using a video spatiotemporal context feature extraction algorithm to extract the action and behavioral features of insect targets in the video data stream, and then using manual recognition or a 2D convolutional neural network structure to train a recognition model. However, the computational efficiency of the spatiotemporal context feature extraction algorithm is very slow and low, and it suffers from large recognition errors due to motion noise (local, irrelevant actions can interfere with the results).

[0004] Furthermore, fruit flies are characterized by their small size and high speed, making it difficult to track and locate targets and resulting in low accuracy in behavior recognition when using existing computer vision techniques to study insect behavior. Therefore, it is necessary to develop a new method for insect behavior recognition to address these problems. Summary of the Invention

[0005] The purpose of this invention is to provide an insect behavior recognition method based on a 3D convolutional network model, so as to solve the problems of low efficiency and large recognition error in existing insect behavior recognition methods.

[0006] The technical solution of this invention is:

[0007] A method for insect behavior recognition based on a 3D convolutional network model, the specific steps of which are as follows:

[0008] 1. Data Preprocessing

[0009] The data preprocessing module is divided into two parts: insect behavior dataset acquisition and dataset preprocessing.

[0010] 1.1 Dataset Acquisition;

[0011] 1) Data source:

[0012] 2) Selection of training data:

[0013] 3) Data annotation:

[0014] 1.2 Dataset preprocessing;

[0015] The dataset obtained in step 3) is divided into a training set and a validation set in an 8:2 ratio; the training set is used to train the parameters of the 3D convolutional neural network model, and the validation set is used to check the state and convergence of the model during the training process.

[0016] 2. Model training;

[0017] Model training is the core step of this invention. This invention proposes a general behavior recognition model for fruit flies based on a 3D convolutional neural network, which is the Y-NLR3D (Yolov5 combinate Non-Local-based Resnet3D) fruit fly behavior recognition model.

[0018] 3. Result output;

[0019] In the results output module, the Grad-CAM heatmap display algorithm was added to display the identified insect behavior using a heatmap.

[0020] 4. Use of the Y-NLR3D behavior recognition model for fruit flies.

[0021] Advantages of this invention:

[0022] This invention proposes a method for recognizing the behavior of fruit flies (Aegilops spp.) based on a 3D convolutional neural network model, named the Y-NLR3D behavior recognition model. In this model, a Yolov5n network is used as the target detection head, enabling rapid tracking and localization of the target, thus solving the problems of the small size, rapid movement, and difficulty in localization of fruit flies. In the latter part of the model, this invention uses a ResNet3D network with added non-local attention modules to analyze the behavior of fruit flies. The proposed Y-NLR3D model not only successfully tracks and localizes fruit flies but also achieves higher accuracy compared to other insect behavior recognition methods (such as those based on spatiotemporal context). Attached Figure Description

[0023] Figure 1 This is a detailed flowchart of the present invention;

[0024] Figure 2 This is an architectural diagram of the Y-NLR3D fruit fly behavior recognition model of the present invention.

[0025] Figure 3 This is a diagram showing the target detection module's recognition effect when the method described in this application is used to target the citrus fruit fly.

[0026] Figure 4 This is a schematic diagram of the structure of the Non-local attention module of the present invention;

[0027] Figure 5 This image shows the target detection module's recognition effect when the method described in this application is used to target the citrus fruit fly. Detailed Implementation

[0028] The specific steps of this insect behavior recognition method based on a 3D convolutional network model are as follows:

[0029] 1. Data Preprocessing

[0030] The data preprocessing module is divided into two parts: insect behavior dataset acquisition and dataset preprocessing.

[0031] 1.1 Dataset Acquisition;

[0032] 1) Data source: Videos of various behaviors of fruit fly insects were taken manually from top to bottom at a fixed angle during the experiment. The fruit fly insects could move freely in the shooting area. The video resolution was 1920*1080 and the frame rate was 25 frames / second.

[0033] 2) Selection of training data: The various behaviors of the fruit fly insects in the video dataset of fruit fly insects captured in step 1) are divided and the dataset of each behavior of the fruit fly insects is separated, that is, each video contains only one behavior of the fruit fly insect.

[0034] 3) Data annotation: Create a folder for each behavior of the fruit fly family insects. The folder contains the video dataset of the corresponding behavior of the fruit fly family insects. The folder name is the label of a behavior, and all the videos in the folder are the video dataset of the corresponding labeled behavior.

[0035] 1.2 Dataset preprocessing;

[0036] The dataset obtained in step 3) is divided into a training set and a validation set in an 8:2 ratio; the training set is used to train the parameters of the 3D convolutional neural network model, and the validation set is used to check the state and convergence of the model during the training process.

[0037] 2. Model training;

[0038] Model training is the core step of this invention. This invention proposes a general 3D convolutional neural network-based behavior recognition model for fruit flies (Y-NLR3D, Yolov5 combinate Non-Local-based ResNet3D) (model structure is attached to the specification). Figure 2 );

[0039] In the dataset of step 3), insects account for a very small proportion of the entire video image; the rest is useless noise data. If the entire video data is sent to the network for training, it will not only affect the training efficiency of the model and cause the network model to converge slowly, but it will also cause the model to learn other useless features. The Y-NLR3D fruit fly insect behavior recognition model proposed in this invention is divided into an object detection module and an action recognition module (see the appendix of the specification). Figure 2 );

[0040] The target detection module is responsible for tracking and locating insect targets, thereby eliminating the influence of irrelevant noise. The behavior recognition module is responsible for recognizing the behavior of insect targets.

[0041] 1) Target detection module;

[0042] The object detection module consists of a Yolov5n network model, which is divided into four parts: input, backbone, neck, and head. The input part primarily preprocesses the image input to the model, reducing its size to the required input size for the backbone (640x640). The backbone mainly performs feature extraction. The CBS, C3, and SPPF modules constitute the backbone structure. The CBS module first performs a convolution operation on the input features, then passes them through a cascaded BN (batch normalization) layer to prevent overfitting, and finally outputs the result after passing through the siLU activation function. The internal structure of the C3 module differs slightly from that in the backbone and head. The C3 module has two branches: one branch passes through a CBS module and performs residual operations; the other branch only passes through a CBS module to adjust the number of channels. Finally, the two branches perform a concat feature fusion operation. The C3 module, using residual operations, has stronger feature extraction capabilities. Unlike the backbone, in the neck section, both branches of the C3 module only pass through the CBS module, only the number of CBS modules in the two branches differs. SPPF is an improvement on the SPP pyramid structure feature extraction module in YOLOv3, enabling SPPF to pay more attention to the spatial information of the image. This module can not only merge local and global feature information to solve the problem of the target detection object being too small, but also improve computational efficiency.

[0043] The Neck section, located between the backbone network and the head output, primarily fuses the output features from different layers, resulting in better feature extraction and increased model robustness. This part mainly utilizes FPN and PAN feature fusion mechanisms to achieve this. After passing through the Neck section, the network generates three feature maps of different sizes. These three feature maps are then fed into the head section, where the YOLOv5 model generates corresponding object detection boxes and class labels. The object detection module using the YOLOv5n object detection model effectively solves the problem of tracking and locating fruit flies.

[0044] 2) Behavior recognition module;

[0045] The behavior recognition module primarily consists of a ResNet3D network model with an added Non-local attention module. This is because ResNet3D alone is not very effective at recognizing the actions of small targets like insects. The addition of the Non-local attention module increases the sensitivity of the ResNet3D network structure to the subtle movement features of insects, making the entire behavior recognition network module more capable of capturing these subtle movements. The Non-local attention structure is as follows: Figure 4 As shown:

[0046] The mathematical definition of a non-local network module is shown in formula (1):

[0047] Z = W z y+X#(1)

[0048] Z represents the output of the non-local operator, and y on the right side of equation (1) represents the computation result within the non-local operator, given by equation (2); W z It is a weight matrix aligned with x, which guarantees W z The output shape of ×y is consistent with that of x; x represents the feature map input of the Non-local module, which is connected to W via residual connections. z ×y are added together. The residual connection structure design ensures that the non-local network module can be embedded into any network model without changing the original network framework;

[0049]

[0050]

[0051] In formula (2), X and y represent the input and output signals, respectively, and their shapes and sizes are consistent. i represents a position index in the feature signal (which can be time, space, or spatiotemporal information). This indicates all locations where the characteristic signal needs to be calculated. The overall representation is the output signal calculated from the feature signals at all locations, where j represents the index of all possible locations of the feature signals. This represents all possible locations of the characteristic signal. The function f(x) i ,x j ) indicates position X i and X j The degree of correlation. A univariate function g(x) j Primarily used for position X j Signal encoding is performed. Information from the entire feature map is extracted by traversing all positions j (non-local). C(X) mainly performs normalization on the calculated result.

[0052] The bivariate function calculates the position X in the feature map using equation (3). i and X j The correlation;

[0053] f(x i ,x j )=θ(x i ) T φx j )#(3)

[0054] In the above equation, θ(x) i ) = W θ x i ,φx j ) = W φ x j W θ and W φ It is the weight matrix, θ(x) i ) and φx j ) are different branch outputs. In this relational function, C(X) = N in formula (2), where N represents X in the input signal X. i The number of elements. Here, N is used for normalization primarily to simplify gradient calculations within the module;

[0055] The input signal X received by the non-local structure is the size of the feature map tensor. For example, the input tensor in the figure is T×H×W×C, where T, H, W, and C represent the length of the time series (e.g., the frame sequence in a video, arranged in chronological order) as T, the feature map height as H, the feature map width as W, and the number of channels as C of the feature map, respectively (see the appendix in the specification). Figure 4 );

[0056] The input signal first passes through three branches, each representing a function θ(x) i (orange box), φx j (green box), g(x) j (Blue box). Each of these three branches first performs a 1×1×1 convolution operation, reducing the number of channels in the input feature signal to half its original size—a dimensionality reduction operation—thereby reducing computational cost. Then, these three branches perform two matrix multiplications (see the appendix in the manual). Figure 5 The two matrix multiplication operations are as shown in formulas (2) and (3). 1 / N represents the normalization operation. After the two matrix multiplication operations and the 1×1×1 convolution upscaling operation, the number of channels in the feature map will become the same as the number of channels in the input, and finally a residual connection will be made with the input signal X.

[0057] The training and validation sets separated in step 3) are fed into the model together. The training set is mainly used to train the model's parameters, while the validation set is mainly used to select the best-performing model after training on the training set. The model is then used to predict on the validation set data, and the accuracy is recorded. The parameters corresponding to the best-performing model are then selected, i.e., used to adjust the model parameters. Therefore, the validation set can be used to adjust the model's hyperparameters and to perform a preliminary evaluation of the model's capabilities.

[0058] After the Y-NLR3D fruit fly behavior recognition model completes one training iteration, its accuracy needs to be evaluated. If the accuracy evaluation result does not meet expectations, and the training data scale is sufficient, the model hyperparameters, including the learning rate, dropout value, and batch size, can be adjusted to proceed to the next model training process (see the instruction manual appendix). Figure 1 The expected value can be freely determined. In this application, the expected value is set to 90%. After continuous iterative training, a Y-NLR3D fruit fly behavior recognition model that meets the expectations is selected.

[0059] 3. Result output;

[0060] In the results output module, the Grad-CAM heatmap display algorithm was added to display the identified insect behaviors using heatmaps. The calculation formula for Grad-CAM is shown in (4):

[0061]

[0062] Where A represents a feature layer, generally referring to the feature layer output by the last convolutional layer; k represents the k-th channel in feature layer A; and c represents the class. k This represents the data of channel K in feature layer A. Representative targeting A k The weighting. Regarding... The calculation is given by formula (5):

[0063]

[0064] Among them, y c This represents the network's predicted score for category c. Let Z represent the data at coordinate ij in channel K of feature layer A, and Z represent the product of the width and height of the feature layer. It can be seen from formula (2) that... It involves predicting the predicted score y for category c. cBackpropagation is performed, and the gradient information backpropagated to feature layer A is used to calculate the importance of each channel K in feature layer A. Next, the data for each channel of feature layer A is weighted and summed using α, and finally, the ReLU activation function is used to obtain the Grad-CAM heatmap data. After the model training is completed in step 2, the feature layer data after the ReLU activation function is input into the Grad-CAM heatmap visualization algorithm to display the behavior of fruit flies.

[0065] 4. Application of the Y-NLR3D behavioral recognition model for fruit flies:

[0066] 4.1 Collect the data set of fruit flies (Fungidae) and process the data set as described in step 1;

[0067] 4.2 Model Training. The data processed in step 4.1 was fed into the Y-NLR3D fruit fly behavior recognition model for training. During training, the initial learning rate was 0.001, the Adam algorithm was used for learning rate decay, and the number of epochs was 50.

[0068] 4.3 Evaluate the accuracy of the model from step 4.2. If the accuracy is lower than the expected value, retrain the model by adjusting hyperparameters including the learning rate, dropout value, and batch size. After iterative training, select the model that meets the expectations; the expected value can be freely defined, and in this application, the expected value is 90%. 4.4 Input the novel insect behavior video into the model selected in step 4.3 for prediction, and display the prediction results and heatmap.

[0069] To verify the correctness of this application, the applicant applied this method to the citrus fruit fly, and the specific process is as follows:

[0070] The insect behavior recognition method based on the 3D convolutional network model, when recognizing the behavior of the citrus fruit fly, has the following specific steps:

[0071] 1. Data Preprocessing

[0072] The data preprocessing module is divided into two parts: insect behavior dataset acquisition and dataset preprocessing.

[0073] 1.1 Dataset Acquisition;

[0074] 1) Data source: Videos of various behaviors of the citrus fruit fly were filmed from a fixed angle from top to bottom during the experiment. The citrus fruit fly could move freely in the filming area. The video resolution was 1920*1080 and the frame rate was 25 frames / second.

[0075] 2) Selection of training data: The various behaviors of fruit fly insects in the video dataset of citrus fruit fly captured in step 1) are divided and the dataset of each behavior of citrus fruit fly is separated, that is, each video contains only one behavior of fruit fly insect.

[0076] 3) Data labeling: Create a folder for each behavior of the citrus fruit fly, and store the video dataset of the corresponding behavior in the folder; the folder name is the label of a behavior, and all the videos in the folder are the video dataset of the corresponding labeled behavior;

[0077] 1.2 Dataset preprocessing;

[0078] The dataset obtained in step 3) is divided into a training set and a validation set in an 8:2 ratio; the training set is used to train the parameters of the 3D convolutional neural network model, and the validation set is used to check the state and convergence of the model during the training process.

[0079] 2. Model training;

[0080] Model training is the core step of this invention. This invention proposes a general 3D convolutional neural network-based behavior recognition model for fruit flies (Y-NLR3D, Yolov5 combinate Non-Local-based ResNet3D) (model structure is attached to the specification). Figure 2 ).

[0081] The Y-NLR3D fruit fly behavior recognition model proposed in this invention consists of an object detection module and an action recognition module (see appendix to the specification). Figure 2 ).

[0082] The target detection module is responsible for tracking and locating insect targets, thereby eliminating the influence of irrelevant noise. The behavior recognition module is responsible for recognizing the behavior of insect targets.

[0083] 1) Target detection module;

[0084] The object detection module consists of a Yolov5n network model, which is divided into four parts: input, backbone, neck, and head. The input part primarily preprocesses the image input to the model, reducing its size to the required input size for the backbone (640x640). The backbone mainly performs feature extraction. The CBS, C3, and SPPF modules constitute the backbone structure. The CBS module first performs a convolution operation on the input features, then passes them through a cascaded BN (batch normalization) layer to prevent overfitting, and finally outputs the result after passing through the siLU activation function. The internal structure of the C3 module differs slightly from that in the backbone and head. The C3 module has two branches: one branch passes through a CBS module and performs residual operations; the other branch only passes through a CBS module to adjust the number of channels. Finally, the two branches perform a concat feature fusion operation. The C3 module, using residual operations, has stronger feature extraction capabilities. Unlike the backbone, in the neck section, both branches of the C3 module only pass through the CBS module, only the number of CBS modules in the two branches differs. SPPF is an improvement on the SPP pyramid structure feature extraction module in YOLOv3, enabling SPPF to pay more attention to the spatial information of the image. This module can not only merge local and global feature information to solve the problem of the target detection object being too small, but also improve computational efficiency.

[0085] The Neck section, located between the backbone network and the head output, primarily fuses the output features from different layers, resulting in better feature extraction and increased model robustness. This part mainly utilizes FPN and PAN feature fusion mechanisms to achieve this. After passing through the Neck section, the network generates three feature maps of different sizes. These three feature maps are then fed into the head section, where the YOLOv5 model generates corresponding bounding boxes and class labels. The object detection module using the YOLOv5n object detection model effectively solves the problem of tracking and locating fruit flies.

[0086] 2) Behavior recognition module;

[0087] The behavior recognition module mainly consists of a ResNet3D network model with an added non-local attention module. This increased sensitivity makes the entire behavior recognition network module more capable of capturing subtle insect movements. The non-local attention structure is as follows: Figure 4As shown: The mathematical definition of a Non-local network module is given by formula (1):

[0088] Z = W z y+X#(1)

[0089] Z represents the output of the non-local operator, and y on the right side of equation (1) represents the computation result within the non-local operator, given by equation (2); W z It is a weight matrix aligned with x, which guarantees W z The output shape of ×y is consistent with that of x; x represents the feature map input of the Non-local module, which is connected to W via residual connections. z ×y are added together. The residual connection structure design ensures that the non-local network module can be embedded into any network model without changing the original network framework;

[0090]

[0091]

[0092] In formula (2), X and y represent the input and output signals, respectively, and their shapes and sizes are consistent. i represents a position index in the feature signal (which can be time, space, or spatiotemporal information). This indicates all locations where the characteristic signal needs to be calculated. The overall representation is the output signal calculated from the feature signals at all locations, where j represents the index of all possible locations of the feature signals. This represents all possible locations of the characteristic signal. The function f(x) i ,x j ) indicates position X i and X j The degree of correlation. A univariate function g(x) j Primarily used for position X j Signal encoding is performed. Information from the entire feature map is extracted by traversing all positions j (non-local). C(X) mainly performs normalization on the calculated result.

[0093] The bivariate function calculates the position X in the feature map using equation (3). i and X j The correlation;

[0094] f(x i ,x j )=θ(x i ) T φx j )#(3)

[0095] In the above equation, θ(x) i ) = W θ x i ,φx j ) = W φ x j W θ and W φ It is the weight matrix, θ(x) i ) and φx j ) are different branch outputs. In this relational function, C(X) = N in formula (2), where N represents X in the input signal X. i The number of elements. Here, N is used for normalization primarily to simplify gradient calculations within the module;

[0096] The input signal X received by the non-local structure is the size of the feature map tensor. For example, the input tensor in the figure is T×H×W×C, where T, H, W, and C represent the length of the time series (e.g., the frame sequence in a video, arranged in chronological order) as T, the feature map height as H, the feature map width as W, and the number of channels as C of the feature map, respectively (see the appendix in the specification). Figure 4 );

[0097] The input signal first passes through three branches, each representing a function θ(x) i (orange box), φx j (green box), g(x) j (Blue box). Each of these three branches first performs a 1×1×1 convolution operation, reducing the number of channels in the input feature signal to half its original size—a dimensionality reduction operation—thereby reducing computational cost. Then, these three branches perform two matrix multiplications (see the appendix in the manual). Figure 5 The two matrix multiplication operations are as shown in formulas (2) and (3). 1 / N represents the normalization operation. After the two matrix multiplication operations and the 1×1×1 convolution upscaling operation, the number of channels in the feature map will become the same as the number of channels in the input, and finally a residual connection will be made with the input signal X.

[0098] The training and validation sets separated in step 3) are fed into the model together. The training set is mainly used to train the model's parameters, while the validation set is mainly used to select the best-performing model after training on the training set. The model is then used to predict on the validation set data, and the accuracy is recorded. The parameters corresponding to the best-performing model are then selected, i.e., used to adjust the model parameters. Therefore, the validation set can be used to adjust the model's hyperparameters and to perform a preliminary evaluation of the model's capabilities.

[0099] After the Y-NLR3D fruit fly behavior recognition model completes one training iteration, its accuracy needs to be evaluated. If the accuracy evaluation result does not meet expectations, and the training data scale is sufficient, the model hyperparameters, including the learning rate, dropout value, and batch size, can be adjusted to proceed to the next model training process (see the instruction manual appendix). Figure 1 The expected value can be freely determined. In this application, the expected value is set to 90%. After continuous iterative training, a Y-NLR3D fruit fly behavior recognition model that meets the expectations is selected.

[0100] 3. Result output;

[0101] In the results output module, the Grad-CAM heatmap display algorithm was added to display the identified insect behaviors using heatmaps. The calculation formula for Grad-CAM is shown in (4):

[0102]

[0103] Where A represents a feature layer, generally referring to the feature layer output by the last convolutional layer; k represents the k-th channel in feature layer A; and c represents the class. k This represents the data of channel K in feature layer A. Representative targeting A k The weighting. Regarding... The calculation is given by formula (5):

[0104]

[0105] Among them, y c This represents the network's predicted score for category c. Let Z represent the data at coordinate ij in channel K of feature layer A, and Z represent the product of the width and height of the feature layer. It can be seen from formula (2) that... It involves predicting the predicted score y for category c. c Backpropagation is performed, and the gradient information backpropagated to feature layer A is used to calculate the importance of each channel K in feature layer A. Next, the data for each channel of feature layer A is weighted and summed using α, and finally, the ReLU activation function is used to obtain the Grad-CAM heatmap data. After the model training is completed in step 2, the feature layer data after the ReLU activation function is input into the Grad-CAM heatmap visualization algorithm to display the behavior of fruit flies.

[0106] 4. Application of the Y-NLR3D behavioral recognition model for fruit flies:

[0107] 4.1 Collect the data set of fruit flies (Fungidae) and process the data set as described in step 1;

[0108] 4.2 Model Training. The data processed in step 4.1 was fed into the Y-NLR3D fruit fly behavior recognition model for training. During training, the initial learning rate was 0.001, the Adam algorithm was used for learning rate decay, and the number of epochs was 50.

[0109] 4.3 Evaluate the accuracy of the model from step 4.2. If the accuracy is lower than the expected value, retrain the model by adjusting hyperparameters including the learning rate, dropout value, and batch size. After iterative training, select the model that meets the expectations; the expected value can be freely defined, and in this application, the expected value is 90%. 4.4 Input the new video of citrus fruit fly behavior into the model selected in step 4.3 for prediction, and display the prediction results and heatmap (see the appendix of the instruction manual). Figure 5 ).

[0110] This invention presents a method for recognizing the behavior of fruit flies (Aegilops spp.) based on a 3D convolutional neural network model, named the Y-NLR3D behavior recognition model. The model utilizes a Yolov5n network as the target detection head, enabling rapid tracking and localization of the target, thus addressing the challenges of the small size, rapid movement, and difficulty in locating fruit flies. In the latter part of the model, a ResNet3D network with an added non-local attention module is used for behavioral analysis of fruit flies. The proposed Y-NLR3D model not only successfully tracks and locates fruit flies but also achieves higher accuracy compared to other insect behavior recognition methods (such as those based on spatiotemporal context).

Claims

1. A method for insect behavior recognition based on a 3D convolutional network model, characterized in that: The specific steps of the insect behavior recognition method are as follows: An insect behavior dataset is acquired, the preprocessed insect behavior dataset is used to train a Y-NLR3D Diopsidae insect behavior recognition model, insect behavior of the Diopsidae family is recognized according to the trained Y-NLR3D Diopsidae insect behavior recognition model, and the recognized Diopsidae insect behavior is displayed using a Grad-CAM heat map display algorithm; The Y-NLR3D Diopsidae insect behavior recognition model comprises a target detection module and a behavior recognition module; The target detection module is composed of a Yolov5n network model and comprises four parts: Input, Backbone, Neck and Head. The images input into the model are preprocessed in the Input input part, and the size of the preprocessed images is changed to the input size required by the Backbone, i.e., 640x640. Feature extraction is performed in the Backbone part. The Backbone network backbone structure comprises three modules: CBS, C3 and SPPF. The CBS module first performs a convolution operation on the input feature information, then prevents overfitting through a concatenated BN layer, and finally outputs the result through a siLU activation function. The C3 module has two branches. One branch passes through a CBS module and then performs a residual operation. The other branch passes through a CBS module to adjust the number of channels. Finally, the two branches are subjected to concat feature fusion operation. The Neck part is located between the Backbone network and the head output. This part uses FPN and PAN feature fusion mechanisms to realize this operation. After passing through the Neck part, the network generates three feature maps of different sizes. After the three feature maps of different sizes are input into the Head part, the yolov5 model generates the corresponding target detection frame and the corresponding class label. The behavior recognition module consists of a ResNet3D network model with a non-local attention module added. The mathematical definition of the non-local network module is shown in formula (1): Z = W z y+X (1), Z represents the output of the non-local operator, and X represents the feature map input of the non-local module, which is connected to W in a residual manner. z ×y are added together, where y represents the result of the non-local calculation, and W z It is a weight matrix aligned with X, ensuring W z The ×y output shape remains consistent with the X shape. 2.The insect behavior recognition method based on a 3D convolution network model according to claim 1, characterized in that: The specific steps of the dataset acquisition are as follows: 1) Data source: videos of various behaviors of Diopsidae insects are shot from top to bottom at a fixed angle by manual operation. Diopsidae insects can freely move in the shooting area. The video resolution is 1920*1080, and the frame rate is 25 frames / second; 2) Selection of training data: various behaviors of Diopsidae insects in the shot video dataset are divided, and the dataset of each behavior of Diopsidae insects is separated, i.e., each video only contains one behavior of Diopsidae insects; 3) Data labeling: a folder is created for each behavior of Diopsidae insects, and the video dataset of the corresponding behavior of Diopsidae insects is stored in the folder. The folder name is the label of one behavior, and all the videos under the folder are the video dataset of the corresponding label behavior. 3.The insect behavior recognition method based on a 3D convolution network model according to claim 1, characterized in that: The calculation formula of the Grad-CAM is as follows: ； where A represents a certain feature layer; k represents the kth channel in the feature layer A, c represents a class, A k represents data of the channel k in the feature layer A; represents the weight of the feature layer A k for the class c, and the calculation formula is as follows: ； where y c represents the prediction score of the network for the class c, represents the data of the feature layer A at coordinates ij in channel k, and z represents the product of the width and height of the feature layer; The prediction score y c Backpropagation is performed, and then the gradient information backpropagated to the feature layer A is used to calculate the importance of each channel k of the feature layer A; then, the data of each channel of the feature layer A is weighted and summed, and finally the heat map data of Grad-CAM is obtained by using the ReLU activation function; after the model training is completed, the feature layer data after the ReLU activation function of the last layer of the model is input into the Grad-CAM heat map visualization algorithm, and the behavior of the Drosophilidae insects can be displayed.

4. The insect behavior recognition method based on a 3D convolution network model according to claim 1, characterized in that: The use steps of the Y-NLR3D Diopsidae insect behavior recognition model are as follows: 1) Diopsidae insect dataset collection and data processing; 2) Model training: the processed data is input into the Y-NLR3D Diopsidae insect behavior recognition model for training. In the training process, the initial learning rate is 0.001, the learning rate decay strategy uses the Adam algorithm, and the epoch training round is 50; the accuracy of the model is evaluated, if the evaluation result of the accuracy is less than the expected value, the model is retrained by adjusting the hyperparameters including learning rate, dropout value and batch size; after continuous iterative training, the model meeting the expectation is selected; the newly input insect behavior video is input into the selected model for prediction, and the prediction result and the heat map are displayed.

Images