Grain insect target identification and classification detection method and system
By uploading local model data for granary anomaly detection of local terminals to the cloud for optimization, the problems of inaccurate detection of grain insects and long model training time in the existing technology have been solved, and the accuracy of grain insect detection and the efficiency of model training have been improved.
Patent Information
- Application Number
- CN202510006217.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-02
- Publication Date
- 2025-05-02
AI Technical Summary
The existing technology cannot accurately and quickly detect grain insects in the granary and determine their species. The model training time is long, so the parameters cannot be optimized and adjusted in time, and the model update and iteration speed is slow.
By uploading the data of the local model of the granary exception detection of the local terminal to the cloud, the cloud optimizes the global model of the granary exception detection, and sends the optimized model data back to the local terminal to achieve rapid training and optimization of the local model.
It improves the accuracy of grain insect detection and category judgment, speeds up model training, and realizes timely optimization and update of the model.
Smart Images
Figure CN119919728A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of automatic grain insect detection, and in particular to a grain insect target recognition and classification detection method and system. Background Art
[0002] China ranks first in the world in grain production, and also ranks first in the world in rice and wheat production. Faced with such a huge task of grain storage, the prevention and control of grain pests (hereinafter referred to as grain insects) is also very important. According to conservative estimates, the annual grain loss caused by grain insects accounts for about 5% of the total reserves. Therefore, how to quickly find and identify grain insects in granaries is an important step in grain insect prevention and control. Existing technologies cannot accurately and quickly detect grain insects in granaries and determine their types. In addition, the existing detection model takes a long time to train, and it is impossible to optimize the adjustment parameters in time, and the model update iteration speed is slow. Summary of the invention
[0003] In order to at least partially solve the problems that the existing models cannot accurately and quickly detect grain insects and determine their types, and that the existing models have a long training optimization time and slow iteration, the present invention provides a grain insect target recognition and classification detection method and system, by uploading the data of the local model of grain silo anomaly detection of the local terminal to the cloud, and optimizing the global model of grain silo anomaly detection in the cloud, and sending the optimized global model data of grain silo anomaly detection back to the local local model of grain silo anomaly detection, the local terminal's grain silo anomaly detection local model can be quickly trained and optimized, thereby improving the accuracy of grain insect detection and category judgment.
[0004] In order to achieve the above object, the technical solution of the present invention is:
[0005] The first aspect of the present invention provides a method for grain insect target recognition and classification detection, comprising:
[0006] Step 1: Collect historical images of multiple granaries, input the multiple historical images into the granary anomaly detection local model of the local terminal corresponding to the granary, and obtain the types and locations of grain insects in the historical images;
[0007] Step 2: Train the local terminal's granary anomaly detection local model according to the grain insect types and grain insect locations in the historical images, and obtain multiple optimal local terminal's granary anomaly detection local model data for completing local training;
[0008] Step 3: uploading the data of multiple optimal granary anomaly detection local models and the historical images in multiple granaries to the cloud, and training the cloud-based granary anomaly detection global model using the data of multiple granary anomaly detection local models and the historical images in multiple granaries to obtain an updated cloud-based granary anomaly detection global model, which is used to complete the cloud-based granary anomaly detection global model training;
[0009] Step 4: Send the updated cloud-based granary anomaly detection global model data back to the local terminal, and update the local granary anomaly detection local model of the local terminal, so as to update the local granary anomaly detection local model of the local terminal according to the cloud-based granary anomaly detection global model trained on the cloud, so as to improve the detection accuracy;
[0010] Step 5: Input the real-time image in the granary into the updated local terminal's granary anomaly detection local model to obtain the type and location of grain insects in the real-time image, and complete grain insect identification and classification detection.
[0011] Furthermore, the granary anomaly detection local model and the granary anomaly detection global model on the cloud have the same structure, and all convolutions in the granary anomaly detection local model and the granary anomaly detection global model on the cloud are replaced by partial convolutions;
[0012] The local model of granary anomaly detection includes an abnormal image acquisition unit, an image enhancement unit and a HRNet network;
[0013] The abnormal image acquisition unit is used to extract the input historical images according to OPencv to obtain the grain insect abnormal image;
[0014] The image enhancement unit is used to enhance the grain insect abnormality image to obtain an enhanced grain insect abnormality image;
[0015] The HRNet network is used to process the enhanced grain insect abnormal images, obtain a historical grain insect classification image dataset, and complete grain insect recognition and classification detection.
[0016] Furthermore, the grain silo anomaly detection local model of the local terminal is trained according to the grain insect types and grain insect positions in the historical images, specifically including:
[0017] Use the VGG neural network graph to predict the features of the historical images and obtain the feature prediction image;
[0018] OPencv is used to extract the monopolar features of the feature prediction image to obtain the monopolar grain insect features, which is convenient for grain insect detection and identification;
[0019] The single-pole grain insect features are input into the improved UP-XLSTM network to obtain the predicted grain insect category;
[0020] According to the loss function, the accuracy of predicting grain insect categories is obtained. According to the value of the loss function and using LSkip, the OPencv parameters and the improved UP-XLSTM network parameters are dynamically adjusted to obtain the optimal OPencv parameters and the improved UP-XLSTM network, and the optimal local terminal granary anomaly detection local model is obtained, which is convenient for optimizing the local terminal granary anomaly detection local model according to the detection accuracy.
[0021] Furthermore, the improved UP-XLSTM network specifically includes a UP-XLSTM network and a GPT-CAM submodule;
[0022] The GPT-CAM submodule is arranged behind the CBAM in the UP-XLSTM network;
[0023] The GPT-CAM submodule includes a texture perception branch, a position sensitive branch and an adaptive fusion subunit;
[0024] The texture perception branch uses a directional convolution kernel group to convolve the output of CBAM to obtain a texture feature map, and convolves the texture feature maps in each direction to obtain texture perception features, which is convenient for focusing on the texture details of the grain insect shell and limb joints, and convolving from different directions to capture the multidirectionality of the texture;
[0025] The position-sensitive branch performs a pooling operation on the output space of CBAM to obtain a pooling feature map, and then convolves the pooling feature map to obtain a position weight map, so as to highlight key areas such as the grain insect head, abdomen, and leg gathering areas;
[0026] The adaptive fusion subunit is used to match the broadcast copy of the position weight map with the texture perception feature dimension to obtain the matching feature, and then calculate the fusion weight, and fuse the position weight map and the texture perception feature according to the fusion weight to obtain the fusion feature.
[0027] Furthermore, the texture perception branch is expressed by the following formula:
[0028] F t i =σ(K i *F),i∈{h,v,d}
[0029] F t =[F t h ,F t v ,F t d ]∈R H×W×3C
[0030] F t ′=conv1×1(F t)
[0031] Among them, F t i is the texture feature map in the i direction, σ is the activation function, K i is the convolution operation in the i direction, F is the output of CBAM, h is the horizontal direction, v is the vertical direction, d is the diagonal direction, and F t is the splicing feature, H is the height, W is the width, C is the number of channels, and F t ′ is the texture perception feature, and conv1×1 is a 1×1 convolution.
[0032] Furthermore, the position sensitive branch is expressed by the following formula:
[0033]
[0034] in, is the pooling feature map, pool is the pooling operation, F p is the position weight map, σ is the activation function, conv3×3 is the 3×3 convolution, and F is the output of CBAM.
[0035] Furthermore, the adaptive fusion subunit is expressed by the following formula:
[0036]
[0037] Among them, F out is the fusion feature, ω is the fusion weight, ⊙ is the dot product, To match features, MLP is a multi-layer perception mechanism and GAP is a global average pooling.
[0038] Furthermore, the loss function is expressed by the following formula:
[0039] L=L EIOU +L mIoU
[0040]
[0041] Among them, L is the loss function, L EIOU is the effective intersection-and-union ratio, L IOU is the intersection-over-union loss, L dis is the discriminator loss, L asp is an asymmetric loss, IOU is the intersection-over-union ratio, z is the center distance, e and e It are the center of the prediction box and the target box respectively, w and h are the width and height of the prediction box respectively, w gt and h gt are the width and height of the real frame, s w and h are the scale factors related to width and height, LmIoU is the average intersection-over-union ratio, m+1 is the total number of categories defined in the dataset, and q rr is the total number of true positives identified, q rp is the total number of false positives that are misidentified, q pr is the total number of false negative examples that were misidentified.
[0042] Furthermore, the cloud-based global granary anomaly detection model is trained using data from multiple granary anomaly detection local models and historical images in multiple granaries, specifically including:
[0043] Use the VGG neural network graph to predict the features of the historical images and obtain the feature prediction image;
[0044] Use OPencv to extract the monopolar features of the feature prediction image and obtain the monopolar grain insect features;
[0045] The HRNet network is used to extract the single-pole grain insect features and obtain multi-level grain insect features, which is convenient for grain insect detection and identification;
[0046] The multi-level grain insect features are input into the improved UP-XLSTM network to obtain the predicted grain insect category;
[0047] According to the loss function, the accuracy of predicting grain insect categories is obtained. According to the value of the loss function and using LSkip, the HRNet network parameters and the improved UP-XLSTM network parameters are dynamically adjusted to obtain the optimal HRNet network parameters and the improved UP-XLSTM network, and the updated cloud-based global model for grain silo anomaly detection is obtained, which is convenient for optimizing the cloud-based global model for grain silo anomaly detection.
[0048] The second aspect of the present invention provides a grain insect target recognition and classification detection system, comprising:
[0049] The granary anomaly detection local model module is used to collect historical images in multiple granaries, input the multiple historical images into the granary anomaly detection local model of the local terminal corresponding to the granary, and obtain the types and locations of grain insects in the historical images;
[0050] A local training module is used to train the local granary anomaly detection local model of the local terminal according to the grain insect types and grain insect locations in the historical images, and obtain multiple optimal local granary anomaly detection local model data of the local terminal for completing local training;
[0051] A cloud training module is used to upload data of multiple optimal local granary anomaly detection models and historical images of multiple granaries to the cloud, and the cloud-based global granary anomaly detection model is trained using the data of multiple local granary anomaly detection models and historical images of multiple granaries to obtain an updated cloud-based global granary anomaly detection model, which is used to complete the cloud-based global granary anomaly detection model training;
[0052] An update module is used to send the updated global model data of granary anomaly detection in the cloud back to the local terminal, and update the local model of granary anomaly detection in the local terminal to improve the detection accuracy;
[0053] The detection module is used to input the real-time image in the granary into the updated local terminal's granary anomaly detection local model, obtain the type and location of grain insects in the real-time image, and complete grain insect identification and classification detection.
[0054] Beneficial effects of the present invention:
[0055] (1) The present invention uploads the training data of the local terminal's grain silo anomaly detection local model to the cloud based on federated learning, and the cloud's grain silo anomaly detection global model is trained based on the relevant data, and the training results are sent back to the local terminal to update the local grain silo anomaly detection model of the terminal. This training method speeds up the model training speed and improves the accuracy of grain insect identification, detection and classification.
[0056] (2) A GPT-CAM submodule was added to the original UP-XLSTM network. The texture differences and position distribution differences of grain insects were captured through the texture perception branch and position sensitive branch, thereby improving the accuracy of classification and detection in complex environments. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] Figure 1 A flowchart of a grain insect target recognition and classification detection method provided in an embodiment of the present invention.
[0058] Figure 2 A schematic diagram of an abnormal image acquisition unit provided in an embodiment of the present invention.
[0059] Figure 3 A schematic diagram of an image enhancement unit provided in an embodiment of the present invention.
[0060] Figure 4 A schematic diagram of the HRNet network provided in an embodiment of the present invention.
[0061] Figure 5 A schematic diagram of a UI visualization interface displaying detection results provided in an embodiment of the present invention.
[0062] Figure 6A schematic diagram of real-time monitoring of grain insects provided by an embodiment of the present invention.
[0063] Figure 7 One of the schematic diagrams of training a local model for granary anomaly detection and a global model for granary anomaly detection provided in an embodiment of the present invention.
[0064] Figure 8 The second schematic diagram of the training of the local model and the global model for granary anomaly detection provided in the embodiment of the present invention.
[0065] Fig. 9 An architectural diagram of a grain insect target recognition and classification detection system provided in an embodiment of the present invention. DETAILED DESCRIPTION
[0066] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution in the embodiment of the present invention will be clearly described below in conjunction with the drawings in the embodiment of the present invention. Obviously, the described embodiment is a part of the embodiment of the present invention, not all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0067] Example 1
[0068] like Figure 1 As shown, a grain insect target recognition and classification detection method includes:
[0069] S101: Collect historical images of multiple granaries, input the multiple historical images into the granary anomaly detection local model of the local terminal corresponding to the granary, and obtain the types and locations of grain insects in the historical images.
[0070] Specifically, the historical images of the granary taken by the camera are found through the storage device of the camera in the granary. The local model of granary anomaly detection includes an abnormal image acquisition unit, an image enhancement unit and an HRNet network. Figure 2 As shown, the abnormal image acquisition unit uses ffmpeg to extract video frames from historical images at a preset time interval or fixed frame rate to obtain images, and uses LabelImg to annotate the images for easy training. OPencv is used to cut the extracted video frames to obtain grain insect abnormal images. The image cutting process is expressed by the following formula:
[0071] sub-img=img[y:y+h,x:x+w]
[0072] Where sub-img is the image subtraction operator, (x, y) is the coordinate of the upper left corner of the cut image, h is the height of the cut image, w is the width of the cut image, and img is the image segmentation operation.
[0073] Afterwards Figure 3 As shown, the image enhancement unit uses OPencv and calls library functions to perform RGB and HSV conversion on the grain insect abnormality image to obtain the RGB image and HSV image of the grain insect abnormality image.
[0074] Considering the dark light inside the granary, we choose to enhance the brightness of the RGB image and HSV image so that the subsequent steps can be carried out better. The brightness enhancement formula is expressed as follows:
[0075] H1=H0, S1=S0, V1=min(255,k×V0)
[0076] Among them, (H0, S0, V0) is the original pixel value, (H1, S1, V1) is the pixel value after brightness enhancement, and k is the brightness enhancement factor.
[0077] Perform filtering operation on the RGB image and HSV image after brightness enhancement. The filtering operation is expressed by the following formula:
[0078] R n =ω R ×R0
[0079] G n =ω G ×G0
[0080] B n =ω B ×B0
[0081] Among them, (R0, G0, B0) is the color value of the original pixel, (R n ,G n ,B n ) is the color value of the image pixel after filtering, ω R ,ω G and ω B are the weight coefficients for the red, green, and blue channels respectively.
[0082] The filtered image is processed by bilateral filtering to obtain an enhanced grain insect abnormality image.
[0083] The HRNet network is used to process the enhanced grain insect abnormal images to obtain the historical grain insect classification image dataset and complete grain insect recognition and classification detection.
[0084] Specifically, Figure 4The HRNet network shown includes HRNet convolutional layers, single-layer convolutional layers, pooling downsampling, multi-layer convolutional layer fusion, and up-down sampling feature fusion connected in sequence. The output of the multi-layer convolutional layer fusion is also input into the single-layer convolutional layer for convolution processing. The output of the single-layer convolutional layer is also input into the up-down sampling feature fusion for fusion, and finally the historical grain insect classification image dataset is obtained.
[0085] S102: training the local terminal's granary anomaly detection local model according to the grain insect types and grain insect positions in the historical images, and obtaining multiple optimal local terminal's granary anomaly detection local model data.
[0086] Specifically, during training, the local terminal's granary anomaly detection local model uses the VGG neural network graph to perform feature prediction on historical images to obtain a feature prediction image.
[0087] OPencv is used to extract the monopolar features of the feature prediction image and obtain the monopolar grain insect features.
[0088] The single-pole grain insect features are input into the improved UP-XLSTM network to obtain the predicted grain insect category.
[0089] According to the loss function and the labeled image, the accuracy of predicting the grain insect category is obtained. According to the value of the loss function and using LSkip, the OPencv parameters and the improved UP-XLSTM network parameters are dynamically adjusted to obtain the optimal OPencv parameters and the improved UP-XLSTM network, and the optimal local model for granary anomaly detection of the local terminal is obtained.
[0090] Preferably, all convolutions in the local model of granary anomaly detection are replaced by partial convolutions to reduce the number of parameters and speed up training.
[0091] S103: Upload data of multiple optimal local models for granary anomaly detection and historical images of multiple granaries to the cloud. The global model for granary anomaly detection on the cloud is trained using the data of multiple local models for granary anomaly detection and historical images of multiple granaries to obtain an updated global model for granary anomaly detection on the cloud.
[0092] Specifically, the cloud analyzes multiple uploaded model data to determine their common features and differences. If the differences are small, a baseline architecture can be selected to map the parameters of other models to this architecture. If the differences are large, it is necessary to use model conversion technology or extract key modules from the model for fusion to obtain a global model for granary anomaly detection on the cloud.
[0093] During training, the global model for detecting grain silos anomalies first uses the VGG neural network to predict the features of historical images and obtains a feature prediction image. OPencv is used to extract the single-pole features of the feature prediction image and obtain the single-pole grain insect features. Finally, the HRNet network is used to extract the single-pole grain insect features and obtain the multi-level grain insect features. The multi-level grain insect features are input into the improved UP-XLSTM network to obtain the predicted grain insect category.
[0094] According to the loss function and the labeled images, the accuracy of predicting grain insect categories is obtained. According to the value of the loss function and using LSkip, the HRNet network parameters and the improved UP-XLSTM network parameters are dynamically adjusted to obtain the optimal HRNet network parameters and the improved UP-XLSTM network, and the updated global model of granary anomaly detection in the cloud is obtained.
[0095] Preferably, all convolutions in the global model of granary anomaly detection are replaced by partial convolutions to reduce the number of parameters and speed up training.
[0096] S104: Send the updated granary anomaly detection global model data in the cloud back to the local terminal to update the granary anomaly detection local model of the local terminal.
[0097] Specifically, Figure 5 As shown, a UI visualization interface is established on the local terminal, and a local model for granary anomaly detection is updated on the local terminal to facilitate the display of grain insect anomaly detection.
[0098] S105: Input the real-time image in the granary into the updated local terminal's granary anomaly detection local model to obtain the type and location of grain insects in the real-time image, and complete grain insect identification and classification detection.
[0099] Specifically, Figure 6 As shown, the type and location of the grain insects are obtained based on the data of the camera that captures the grain insects, and the grain insect identification and classification detection are completed.
[0100] like Figure 7 and Figure 8 As shown, the present invention uploads the locally trained local model data of the granary anomaly detection of the local terminal to the global model of granary anomaly detection in the cloud through federated learning. The global model of granary anomaly detection in the cloud performs cloud training on multiple local model data of granary anomaly detection, and then sends it back to the local terminal to update the local model of granary anomaly detection, so as to facilitate the rapid training and updating of the model and ensure the accuracy of the local model of granary anomaly detection in identifying and detecting the types and locations of grain insects.
[0101] Example 2
[0102] Based on the above embodiments, the embodiments of the present invention provide an improved UP-XLSTM network, which specifically includes:
[0103] The improved UP-XLSTM network specifically includes the UP-XLSTM network and the GPT-CAM submodule (Grain Insect Texture-Position Coordinated Attention Mechanism). The GPT-CAM submodule is set after the CBAM in the UP-XLSTM network.
[0104] The GPT-CAM sub-module includes a texture sensing branch (TSB), a position sensitive branch (PSB) and an adaptive fusion module (AFM).
[0105] The texture perception branch uses the directional convolution kernel group to convolve the output of CBAM to obtain a texture feature map, and then convolves the texture feature maps in each direction to obtain texture perception features. The above process is expressed as follows:
[0106] F t i =σ(K i *F),i∈{h,v,d}
[0107] F t =[F t h ,F t v ,F t d ]∈R H×W×3C
[0108] F t ′=conv1×1(F t )
[0109] Among them, F t i is the texture feature map in the i direction, σ is the activation function, K i is the convolution operation in the i direction, F is the output of CBAM, h is the horizontal direction, v is the vertical direction, d is the diagonal direction, and F t is the splicing feature, H is the height, W is the width, C is the number of channels, and F t ′ is the texture perception feature, and conv1×1 is a 1×1 convolution.
[0110] The position-sensitive branch performs a spatial pooling operation on the output of CBAM to obtain a pooled feature map, and then convolves the pooled feature map to obtain a position weight map. The above process is expressed as follows:
[0111]
[0112] in, is the pooling feature map, pool is the pooling operation, F p is the position weight map, σ is the activation function, conv3×3 is the 3×3 convolution, and F is the output of CBAM.
[0113] The adaptive fusion subunit is used to match the broadcast copy of the position weight map with the texture perception feature dimension to obtain the matching feature, and then calculate the fusion weight, and fuse the position weight map and the texture perception feature according to the fusion weight to obtain the fusion feature. The above process is expressed by the following formula:
[0114]
[0115] Among them, F out is the fusion feature, ω is the fusion weight, ⊙ is the dot product, To match features, MLP is a multi-layer perception mechanism and GAP is a global average pooling.
[0116] In grain insect detection in granaries, grain insect texture and position are closely related and are key discriminant factors. Traditional attention is often focused on texture or position, and their synergy is easily overlooked. GPT-CAM innovatively integrates the two, mining the rich details of texture from multiple directions, locating key areas, and adaptively fusing and strengthening related features. For example, to distinguish between rice weevils and corn weevils larvae, the texture differences are subtle but the position distribution is different. This mechanism accurately captures features, helps the model learn unique patterns, improves classification and detection accuracy in complex environments, provides reliable technical support for precise storage management, and optimizes the basis for formulating grain insect prevention and control strategies.
[0117] Preferably, the OPencv parameters, HRNet network parameters and improved UP-XLSTM network are optimized through the loss function, and the feature fusion weights of the adaptive fusion sub-unit can also be dynamically adjusted. The loss function is expressed by the following formula:
[0118] L=L EIOU +L mIoU
[0119]
[0120] Among them, L is the loss function, L EIOU is the effective intersection-and-union ratio, L IOU is the intersection-over-union loss, L dis is the discriminator loss, L aspis an asymmetric loss, IOU is the intersection-over-union ratio, z is the center distance, e and e It are the center of the prediction box and the target box respectively, w and h are the width and height of the prediction box respectively, w gt and h gt are the width and height of the real frame, s w and h are the scale factors related to width and height, L mIoU is the average intersection-over-union ratio, m+1 is the total number of categories defined in the dataset, and q rr is the total number of true positives identified, q rp is the total number of false positives that are misidentified, q pr is the total number of false negative examples that were misidentified.
[0121] Example 3
[0122] Based on the above embodiments, Fig. 9 As shown, an embodiment of the present invention provides a grain insect target recognition and classification detection system, including:
[0123] The granary anomaly detection local model module is used to collect historical images in multiple granaries, and input the multiple historical images into the granary anomaly detection local model of the local terminal corresponding to the granary to obtain the types and locations of grain insects in the historical images.
[0124] The local training module is used to train the local terminal's granary anomaly detection local model according to the grain insect types and grain insect locations in the historical images, and obtain multiple optimal local terminal's granary anomaly detection local model data.
[0125] The cloud-based training module is used to upload data of multiple optimal local models for granary anomaly detection and historical images of multiple granaries to the cloud. The cloud-based global model for granary anomaly detection is trained using the data of multiple local models for granary anomaly detection and historical images of multiple granaries to obtain an updated cloud-based global model for granary anomaly detection.
[0126] The update module is used to send the updated global model data of granary anomaly detection in the cloud back to the local terminal to update the local model of granary anomaly detection in the local terminal.
[0127] The detection module is used to input the real-time image in the granary into the updated local terminal's granary anomaly detection local model, obtain the type and location of grain insects in the real-time image, and complete grain insect identification and classification detection.
[0128] It should be noted that the grain insect target recognition and classification detection system provided in the embodiment of the present invention is for realizing the above-mentioned grain insect target recognition and classification detection method. Its specific functions can be referred to the above-mentioned method embodiments and will not be repeated here.
[0129] The present invention uploads the training data of the local model of grain silo anomaly detection of the local terminal to the cloud based on federated learning, and the global model of grain silo anomaly detection in the cloud is trained according to the relevant data, and the training results are sent back to the local terminal for updating the local model of grain silo anomaly detection of the terminal. This training method speeds up the training speed of the model and improves the accuracy of grain insect identification, detection and classification. The GPT-CAM submodule is added to the original UP-XLSTM network, and the texture differences and position distribution differences of grain insects are captured through texture perception branches and position sensitive branches, which improves the accuracy of classification detection in complex environments.
[0130] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A method for identifying and classifying grain insects, characterized in that: include: Step 1: Collect historical images of multiple granaries, input the multiple historical images into the granary anomaly detection local model of the local terminal corresponding to the granary, and obtain the types and locations of grain insects in the historical images; Step 2: According to the grain insect types and grain insect locations in the historical images, the local terminal's grain silo anomaly detection local model is trained to obtain multiple optimal local terminal's grain silo anomaly detection local model data; Step 3: uploading data of multiple optimal granary anomaly detection local models and historical images of multiple granaries to the cloud, and training the cloud-based granary anomaly detection global model using the data of multiple granary anomaly detection local models and historical images of multiple granaries to obtain an updated cloud-based granary anomaly detection global model; Step 4: Send the updated granary anomaly detection global model data in the cloud back to the local terminal to update the granary anomaly detection local model in the local terminal; Step 5: Input the real-time image in the granary into the updated local terminal's granary anomaly detection local model to obtain the type and location of grain insects in the real-time image, and complete grain insect identification and classification detection.
2. A grain insect target recognition and classification detection method according to claim 1, characterized in that: The granary anomaly detection local model and the granary anomaly detection global model on the cloud have the same structure, and all convolutions in the granary anomaly detection local model and the granary anomaly detection global model on the cloud are replaced by partial convolutions; The local model of granary anomaly detection includes an abnormal image acquisition unit, an image enhancement unit and a HRNet network; The abnormal image acquisition unit is used to extract the input historical images according to OPencv to obtain the grain insect abnormal image; The image enhancement unit is used to enhance the grain insect abnormality image to obtain an enhanced grain insect abnormality image; The HRNet network is used to process the enhanced grain insect abnormal images, obtain a historical grain insect classification image dataset, and complete grain insect recognition and classification detection.
3. A grain insect target identification and classification detection method according to claim 2, characterized in that: The method of training the local granary anomaly detection local model of the local terminal according to the grain insect types and grain insect locations in the historical images specifically includes: Use the VGG neural network graph to predict the features of the historical images and obtain the feature prediction image; Use OPencv to extract the monopolar features of the feature prediction image and obtain the monopolar grain insect features; The single-pole grain insect features are input into the improved UP-XLSTM network to obtain the predicted grain insect category; According to the loss function, the accuracy of predicting grain insect categories is obtained. According to the value of the loss function and using LSkip, the OPencv parameters and the improved UP-XLSTM network parameters are dynamically adjusted to obtain the optimal OPencv parameters and the improved UP-XLSTM network, and the optimal local model for granary anomaly detection of the local terminal is obtained.
4. A grain insect target identification and classification detection method according to claim 3, characterized in that: The improved UP-XLSTM network specifically includes a UP-XLSTM network and a GPT-CAM submodule; The GPT-CAM submodule is arranged behind the CBAM in the UP-XLSTM network; The GPT-CAM submodule includes a texture perception branch, a position sensitive branch and an adaptive fusion subunit; The texture perception branch uses a directional convolution kernel group to convolve the output of CBAM to obtain a texture feature map, and concatenates the texture feature maps in each direction and performs a convolution operation to obtain a texture perception feature; The position sensitive branch performs a pooling operation on the output space of CBAM to obtain a pooling feature map, and then convolves the pooling feature map to obtain a position weight map; The adaptive fusion subunit is used to match the broadcast copy of the position weight map with the texture perception feature dimension to obtain the matching feature, and then calculate the fusion weight, and fuse the position weight map and the texture perception feature according to the fusion weight to obtain the fusion feature.
5. A grain insect target identification and classification detection method according to claim 4, characterized in that: The texture perception branch is expressed by the following formula: F t i =σ(K i *F),i∈{h,v,d} F t =[F t h ,F t v ,F t d ]∈R H×W×3C F t ′=conv1×1(F t ) Among them, F t i is the texture feature map in the i direction, σ is the activation function, K i is the convolution operation in the i direction, F is the output of CBAM, h is the horizontal direction, v is the vertical direction, d is the diagonal direction, and F t is the splicing feature, H is the height, W is the width, C is the number of channels, and F t ′ is the texture perception feature, and conv1×1 is a 1×1 convolution.
6. A grain insect target identification and classification detection method according to claim 5, characterized in that: The position-sensitive branch is expressed by the following formula: in, is the pooling feature map, pool is the pooling operation, F p is the position weight map, σ is the activation function, conv3×3 is the 3×3 convolution, and F is the output of CBAM.
7. A grain insect target identification and classification detection method according to claim 6 or 5, characterized in that: The adaptive fusion subunit is expressed by the following formula: Among them, F out is the fusion feature, ω is the fusion weight, ⊙ is the dot product, To match features, MLP is a multi-layer perception mechanism and GAP is a global average pooling.
8. A grain insect target identification and classification detection method according to claim 3, characterized in that: The loss function is expressed as follows: L=L EIOU +L mIoU Among them, L is the loss function, L EIOU is the effective intersection-union ratio, L IOU is the intersection-over-union loss, L dis is the discriminator loss, L asp is an asymmetric loss, IOU is the intersection-over-union ratio, z is the center distance, e and e It are the center of the prediction box and the target box respectively, w and h are the width and height of the prediction box respectively, w gt and h gt are the width and height of the real frame, s w and h are the scale factors related to width and height, L mIoU is the average intersection-over-union ratio, m+1 is the total number of categories defined in the dataset, and q rr is the total number of true positives identified, q rp is the total number of false positives that are misidentified, q pr is the total number of false negative examples that were misidentified.
9. A grain insect target recognition and classification detection method according to claim 3, characterized in that: The cloud-based granary anomaly detection global model is trained using data from multiple granary anomaly detection local models and historical images in multiple granaries, specifically including: Use the VGG neural network graph to predict the features of the historical images and obtain the feature prediction image; Use OPencv to extract the monopolar features of the feature prediction image and obtain the monopolar grain insect features; The HRNet network is used to extract the single-level grain insect features and obtain the multi-level grain insect features; The multi-level grain insect features are input into the improved UP-XLSTM network to obtain the predicted grain insect category; According to the loss function, the accuracy of predicting grain insect categories is obtained. According to the value of the loss function and using LSkip, the HRNet network parameters and the improved UP-XLSTM network parameters are dynamically adjusted to obtain the optimal HRNet network parameters and the improved UP-XLSTM network, and the updated global model of granary anomaly detection in the cloud is obtained.
10. A grain insect target recognition and classification detection system, characterized in that: include: The granary anomaly detection local model module is used to collect historical images in multiple granaries, input the multiple historical images into the granary anomaly detection local model of the local terminal corresponding to the granary, and obtain the types and locations of grain insects in the historical images; A local training module is used to train the local granary anomaly detection local model of the local terminal according to the grain insect types and grain insect positions in the historical images, and obtain multiple optimal local granary anomaly detection local model data of the local terminal; A cloud training module is used to upload data of multiple optimal granary anomaly detection local models and historical images of multiple granaries to the cloud, and the cloud-based granary anomaly detection global model is trained using the data of multiple granary anomaly detection local models and historical images of multiple granaries to obtain an updated cloud-based granary anomaly detection global model; An update module is used to send the updated granary anomaly detection global model data in the cloud back to the local terminal to update the granary anomaly detection local model in the local terminal; The detection module is used to input the real-time image in the granary into the updated local terminal's granary anomaly detection local model, obtain the type and location of grain insects in the real-time image, and complete grain insect identification and classification detection.