Disease and pest detection method and system based on dynamic scene

By extracting and processing the feature information of the initial images of pests and diseases and occlusion images in dynamic scenarios, an adaptive interaction model is built to identify the boundaries and categories of pests and diseases, the problems of missing features and misdetection of pest areas caused by occlusion in dynamic scenarios are solved, and the recognition accuracy and adaptability of pest detection are improved.

CN120107234AActive Publication Date: 2025-06-06WEIFANG UNIV OF SCI & TECH

Patent Information

Application Number
CN202510434377.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-04-08
Publication Date
2025-06-06
Estimated Expiration
2045-04-08

AI Technical Summary

Technical Problem

The prior art is difficult to accurately identify the missing features and misdetection problems caused by occlusion in pest detection in dynamic scenarios, and the recognition accuracy is reduced in multi-view situations.

Method used

Using a pest detection method based on dynamic scenes, by obtaining the initial pest and pest images in dynamic scenes, pre-processing and occlusion area detection, extracting feature information of the occlusion area, constructing a morphological correlation matrix and adaptive interaction model, identifying the pest boundaries and categories, and adjusting position prediction.

Benefits of technology

It improves the recognition accuracy of pest detection in dynamic scenarios, and can accurately identify pest and disease areas under occlusion and multi-view angles, improving the adaptability and efficiency of detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120107234A_ABST
    Figure CN120107234A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of disease and pest detection, in particular to a disease and pest detection method and system based on a dynamic scene, and the method comprises the steps: obtaining a plurality of disease and pest initial images and disease and pest occlusion images in the dynamic scene, extracting an occlusion region for disease and pest recognition according to the disease and pest initial images and the disease and pest occlusion images, according to the feature information of the occlusion area, setting a form incidence matrix of the occlusion area; identifying feature interaction information of the pest initial image and the pest occlusion image according to the form incidence matrix of the occlusion area, and forming a self-adaptive interaction model based on the feature interaction information; determining a boundary distribution probability and a category probability of the initial image of the pest and disease damage by using a self-adaptive interaction model; according to the boundary distribution probability and category probability of the pest initial image, using an approximate matrix to carry out position adjustment, and delineating a pest prediction position; accuracy and precision of disease and pest detection in a dynamic scene are realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of pest detection, and in particular to a pest detection method and system based on dynamic scenarios. Background Art

[0002] Crop pests and diseases have a significant impact on crop yield and quality. Large-scale, rapid, and high-precision monitoring and early warning of pests and diseases are the key to improving crop quality and yield. Traditional pest and disease detection methods mainly rely on manual observation and empirical judgment, or use static image processing technology for identification. However, in dynamic scenes, these methods often cannot meet the requirements of accuracy and real-time performance.

[0003] For example, Chinese patent publication number CN118624540A discloses a pest monitoring method based on remote sensing meteorological data collaboration, which relates to the field of pest monitoring technology, obtains remote sensing images of multiple sub-regions of the target area, and performs image preprocessing on the remote sensing images; uses pixel unit matching algorithm to identify sub-regions with pests and diseases according to the preprocessed remote sensing images of multiple sub-regions; obtains historical meteorological data of the pest sub-region in a fixed time period before the occurrence of pests and diseases and the spectral characteristic wavelength of the remote sensing image corresponding to the pest sub-region; establishes a pest outbreak prediction model based on historical meteorological data and the spectral characteristic wavelength of the remote sensing image, inputs the predicted meteorological data and the spectral characteristic wavelength of the remote sensing image of the sub-region to be tested into the pest outbreak prediction model, and performs pest monitoring. The prior art describes the spectral characteristics of possible onset of pests and diseases under time and meteorological correlation, and mainly wants to solve the recognition accuracy of pests and diseases under different lighting scenes caused by meteorology, but in this implementation mode, the occlusion existing on the image of the pest area is ignored, resulting in the problem of recognition accuracy when the lighting effect in the environment is removed.

[0004] For example, Chinese patent publication number CN116773550A discloses an agricultural intelligent monitoring system and method based on machine vision, the system includes: a module for acquiring an area to be monitored, for acquiring an area to be monitored in a farmland; a crop growth image acquisition module, for acquiring crop growth images in the area to be monitored; a pest and disease monitoring module, for performing pest and disease monitoring based on crop growth images and a preset pest and disease image library; wherein acquiring the area to be monitored in the farmland includes: establishing a pest and disease prediction map corresponding to the farmland; determining a plurality of predicted pest and disease locations and corresponding predicted pest and disease types from the pest and disease prediction map; based on a minimum encirclement delineation condition, delineating a minimum encirclement encircling part of the predicted pest and disease locations in the pest and disease prediction map; and using the encircled area of ​​the minimum encirclement as the area to be monitored. The existing technology describes the flight position and takes multiple samples according to these positions to predict the location of pests and diseases. However, this prediction method may ignore the relevant features in the current environment, resulting in predictions based only on variety information, historical growth environment information, fertilization history information, and irrigation history information. It ignores whether there are corresponding obstructions and hidden situations of pests and diseases in the description environment of various variety information, resulting in insufficient recognition of corresponding features of the basic data, affecting the accuracy and range of the final position prediction.

[0005] The existing technology describes the relative processing methods for light and pest locations, but for the pests and diseases that appear in these places, it is also necessary to identify the occlusion they exist in, and to obtain data through multiple sampling according to the different occlusions, in order to identify the occluded feature information, so as to improve the accuracy of prediction for pest and disease areas. Summary of the invention

[0006] In order to solve the above technical problems, the technical solution adopted by the present invention is: a pest and disease detection method based on dynamic scenes, comprising: S1, obtaining multiple pest and disease initial images and pest and disease occlusion images in a dynamic scene, and preprocessing the pest and disease initial images and pest and disease occlusion images.

[0007] S2, extracting the occluded area for pest and disease identification according to the preprocessed initial image of the pest and disease and the occluded image of the pest and disease, and setting the morphological association matrix corresponding to the occluded area according to the feature information of the occluded area.

[0008] S3, identifying the feature interaction information of the initial image of the pest and disease and the occluded image of the pest and disease according to the morphological association matrix of the occluded area, and forming an adaptive interaction model based on the feature interaction information.

[0009] S4, using the adaptive interaction model, identifies the corresponding pest boundaries and pest categories in the initial pest image, and determines the boundary distribution probability and category probability of the initial pest image.

[0010] S5, according to the boundary distribution probability and category probability of the initial image of the pests and diseases, the approximate matrix is ​​used to adjust the position and circle the predicted position of the pests and diseases.

[0011] A pest detection system based on dynamic scenes includes: an image acquisition module for acquiring an initial pest image and a pest occlusion image in a dynamic scene, and performing preprocessing.

[0012] The occluded area detection module is used to detect the occluded area in the pest and disease occluded image, extract the texture features and shape features of the occluded area, construct a multi-scale feature pyramid, and use the multi-scale feature pyramid to set the morphological association matrix corresponding to the occluded area.

[0013] Adaptive interaction module: It is used to identify the feature interaction information of the initial image of the pest and disease and the occluded image of the pest and disease according to the morphological association matrix of the occluded area, and to build an adaptive interaction model.

[0014] The pest and disease identification module is used to identify the pest and disease boundaries and categories in the initial pest and disease image using an adaptive interaction model, and calculate the boundary distribution probability and category probability.

[0015] The location prediction module is used to adjust the location based on the boundary distribution probability and category probability using an approximate matrix to delineate the predicted location of pests and diseases.

[0016] The beneficial effects of the present invention are as follows: 1. The present invention can identify the problem of missing features and false detection of pest areas due to occlusion in dynamic scenes by comprehensively processing the initial image of pests and diseases and the occluded image of pests and diseases; and by comparing according to the feature information existing in the occluded area, it can quickly identify the morphological associations existing in different pest and disease scenes, and associate these morphological associations with the dynamic scenes of pest and disease detection, so as to assist the system in timely discovering the existing pest and disease situations in constantly changing scenes, and improve the recognition accuracy of pests and diseases in different scenes.

[0017] 2. The present invention uses feature interaction information to process feature information existing in the initial image and the occluded image of pests and diseases, thereby realizing feature alignment and interaction optimization between multi-view images; to solve the problem that in multi-view situations, the accuracy of overall recognition and detection is reduced due to certain occluded areas, resulting in deviations in the measurement positions of pests and diseases when measuring the corresponding pests and diseases at different positions, and the inability to timely discover the occluded pests and diseases when the feature interaction information is incomplete; and after feature interaction, the interaction between features in different dimensions can be discovered, and these features can be comprehensively displayed to describe the actual distribution of pests and diseases in different scenarios.

[0018] 3. The present invention calculates the boundary distribution and categories of pests and diseases, evaluates the content output by the current adaptive interactive model, determines the accuracy of the current image processing, and the situation of the pests and diseases measured in the corresponding scene, and realizes accurate modeling and uncertainty quantification of the probability distribution of pest locations; to describe the uncertainty in dividing the boundaries of pests and diseases, so as to improve the relative accuracy and efficiency of the final identification of pests and diseases in the image, and to improve the adaptability of pest and disease detection in different scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The present invention is further described below in conjunction with the accompanying drawings and embodiments.

[0020] Figure 1 It is a flowchart of a pest and disease detection method based on dynamic scenarios.

[0021] Figure 2 The schematic diagram of the model is an adaptive interactive model of a pest and disease detection method based on dynamic scenarios.

[0022] Figure 3 The present invention is a flowchart of step S2 of a pest detection method based on dynamic scenarios.

[0023] Figure 4 The present invention is a schematic diagram of the effect of image recognition of a pest and disease detection method and system based on dynamic scenes.

[0024] Figure 5 The present invention is another schematic diagram of the effect of image recognition of a pest and disease detection method and system based on dynamic scenes.

[0025] Figure 6 It is a precision-confidence curve of a pest and disease detection method and system based on dynamic scenarios.

[0026] Figure 7 It is a pest and disease detection method and system F1 score-confidence curve based on dynamic scenes.

[0027] Figure 8 It is a recall-confidence curve of a pest and disease detection method and system based on dynamic scenarios.

[0028] Fig. 9 It is a precision-recall curve of a pest and disease detection method and system based on dynamic scenarios. DETAILED DESCRIPTION

[0029] The embodiments of the present invention are described in detail below. The embodiments described below are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. If no specific techniques or conditions are specified in the embodiments, the techniques or conditions described in the literature in the art or the product specifications are used.

[0030] See also Figure 1 , a pest detection method based on dynamic scenes, comprising: S1, obtaining multiple pest initial images and pest occlusion images in a dynamic scene, and preprocessing the pest initial images and pest occlusion images.

[0031] S2, extracting the occluded area for pest and disease identification according to the preprocessed initial image of the pest and disease and the occluded image of the pest and disease, and setting the morphological association matrix corresponding to the occluded area according to the feature information of the occluded area.

[0032] S3, identifying the feature interaction information of the initial image of the pest and disease and the occluded image of the pest and disease according to the morphological association matrix of the occluded area, and forming an adaptive interaction model based on the feature interaction information.

[0033] S4, using the adaptive interaction model, identifies the corresponding pest boundaries and pest categories in the initial pest image, and determines the boundary distribution probability and category probability of the initial pest image.

[0034] S5, according to the boundary distribution probability and category probability of the initial image of the pests and diseases, the approximate matrix is ​​used to adjust the position and circle the predicted position of the pests and diseases.

[0035] When identifying pests and diseases, the accuracy of identifying pests and diseases in dynamic scenes is easily affected by the environment, especially when there are obstructions or the environment changes. In this case, the original image of the pests and diseases can be obtained first, and image feature recognition can be performed. Then, the image with occlusions can be input to identify the pest and disease related information in the occluded area from the occluded image. This information will be used for subsequent completion processing and compared with the initial image form of the pests and diseases to determine whether the part being repaired and identified can be accurately identified.

[0036] In one embodiment of the present invention, Figure 2 As shown, a main model can be set to process the initial image of the pests and diseases, and then an auxiliary model can be set to process the occluded image of the pests and diseases, and after extracting the features of the occluded area, a convolutional autoencoder is used to complete the occluded area to describe key features such as the texture of the lesions and the morphology of the insect body. For example, the aphid cluster area occluded by the leaves may show the outline of the insect body or traces of secretions after restoration.

[0037] The features processed by the auxiliary model are interacted with the corresponding features of the main model to verify whether the completion method is normal, to improve the accuracy of the main model in identifying pests and diseases, to reduce the impact of the environment and occlusion during pest and disease identification, and finally to obtain the location of pests and diseases with comprehensive feature information, thereby improving the efficiency of processing in dynamic environments.

[0038] Figure 2 The first part represents the training stage of the main model and the auxiliary model, the second part represents the structure of the convolutional autoencoder, and the third part represents the process of the adaptive interaction model interacting the feature information of the occluded area.

[0039] When the adaptive interaction model interacts the features between branches, the weights are set according to the different selected features. For example, the information flow coefficient is set to represent the weight of the feature interaction between branches.

[0040] ;in, represents the information flow coefficient from branch x to branch y at layer k-1, represents the learnable weight of branch x at layer k-1, represents the learnable bias of branch x at layer k-1, Represents the output features of branch x at layer k-1.

[0041] Then, according to these weights and the features of the k-1 layer, the output features of branch y at the k layer are calculated.

[0042] ;in, represents the output feature of branch y at layer k, represents element-wise multiplication, represents element-wise addition, represents the Gaussian error linear unit activation function, represents the learnable weight of branch x at layer k-1, represents the learnable bias of branch x at layer k-1, represents the output feature of branch y at layer k-1, where Represents the output of the current adaptive interaction model at the corresponding feature layer to determine the specific iteration form of each feature layer in the feature interaction process to adapt to the changing data distribution and corresponding requirements.

[0043] Afterwards, according to the output features on each feature layer, the feature interaction information of the initial image of the pest and disease and the occluded image of the pest and disease can be understood, that is, the features extracted corresponding to the two images are obtained respectively, and these features are interacted in the form of feature layers; finally, according to the interacted data, the currently set model can identify the pest and disease boundaries and corresponding types appearing in the image according to these interacted data, so as to improve the recognition of pest and disease areas with occlusions in dynamic environments, thereby improving the accuracy of identifying pests and diseases.

[0044] In step S1, the pest initial image and the pest occlusion image are preprocessed by aligning continuous frames using GPS+IMU sensor data or an optical flow algorithm (such as RAFT), determining that the current pest initial image and the pest occlusion image describe the same area, adjusting the viewing angles of the pest initial image and the pest occlusion image to the same angle, and determining the occlusion area at the same angle.

[0045] Therefore, the implementation method of step S1 also includes: S11, respectively obtaining the acquisition time and coordinates of the initial pest image and the pest occlusion image. The initial pest image describes the pest image directly acquired, and the pest occlusion image is an image with random occlusion, and the pest occlusion image is consistent with the position described by the initial pest image.

[0046] S12, using the RAFT algorithm to calculate the dense optical flow field between the initial image of the pest and the occluded image of the pest and the disease, and obtain the optical flow displacement vector. When the RAFT algorithm calculates the dense optical flow field, the corresponding displacement between each pixel in the two images is calculated, and these generated displacements are set as the unique optical flow vector. After that, the pixels in the two images are mapped according to the calculated displacement, and the images are deformed and aligned to obtain two images with the same angle. At this time, the reaction mapping is to use the remap function of OpenCV to deform the image, and the identified displacement and corresponding coordinates are deformed to obtain two images with the same angle and target, and the occluded area in this image is analyzed.

[0047] S13, reverse mapping the pest occluded image according to the optical flow displacement vector to make the two images consistent in perspective.

[0048] At the same time, in the form of continuous frames, when the continuous frames express the same area, the corresponding initial image of the pest and disease and the occluded image of the pest and disease can also be processed to identify the occluded area and the type of pest and disease that may exist in the occluded area.

[0049] In one embodiment of the present invention, in addition to identifying the occluded area in step S2, a multi-scale feature pyramid is constructed through the pest and disease occlusion image to determine the features of each level, and the pest probability at each level is calculated to determine whether the current occluded area is an occluded area where pests exist.

[0050] In addition, it is also necessary to determine the different occlusion forms of the current occluded area under different insect bodies, as well as the differences between different insect bodies and background areas. That is, when extracting the feature information of the occluded area, it is also necessary to identify the distinguishing features corresponding to the insect body and the similarity measure between the features to obtain more information about the occluded area.

[0051] For example, the occluded areas have different sizes. The tissue texture of the occluded area is inferred through the adjacent unoccluded area, and the size of the occluded area or the pest range is further restricted or adjusted to obtain the corresponding association rules between the occluded objects and pests. At this time, the association between the occluded area under different occlusion conditions can be obtained, and the pest type of the occluded area can be further restricted based on these associations, so as to improve the accuracy of feature recognition of the repair position when performing feature repair on the image under the occluded area.

[0052] like Figure 3 As shown, the implementation method of step S2 also includes: S21, using the pre-processed pest initial image and the pest occlusion image to perform occlusion area detection, and determine the occlusion area in the pest occlusion image.

[0053] Generally speaking, the occluded area may include insect occlusion, leaf occlusion, soil occlusion, and light occlusion. According to the different types of occlusion, a multi-scale feature pyramid will be used to describe the performance of these features in the current occluded area, and identify the differences in these performances at different scales or levels. These feature differences will be used for subsequent pest probability identification to determine the probability of pests in the occluded area.

[0054] The identification method for detecting the occluded area in step S21 includes: performing pixel point difference on the preprocessed initial image of the pest and disease and the occluded image of the pest and disease, identifying the difference points during the pixel point difference, connecting the difference points according to the coordinate position, and setting the minimum bounding rectangle corresponding to the difference points after the connection as the occluded area. At this time, the occluded area that can exist at the current position under the normal image and the occluded image is mainly found by connecting the positions with differences according to their minimum bounding rectangles. After that, the features of the corresponding positions of the occluded area need to be identified and extracted to complete the feature filling and completion interaction of the occluded area.

[0055] S22, extracting texture features and shape features corresponding to the occluded area, and constructing a multi-scale feature pyramid, extracting features of the occluded area based on the multi-scale feature pyramid, and describing the difference features of the occluded area at each level; in the multi-scale feature pyramid, the bottom-level features represent the grayscale histogram of the texture of the occluded area, the middle-level features represent the context shape descriptors of the occluded area and the surrounding area, and the high-level features represent the semantic segmentation results of the occluded area, and the semantic segmentation results also represent the segmentation of the occluded area and the context, that is, the occluded area is divided and described using semantically related segmentation forms, such as segmentation based on leaves, segmentation based on insect bodies, etc., such as the spatial correlation of the corresponding insect bodies in the occluded area at the corresponding considered positions, or when a disease occurs, the identified parts of the insect body in the image, these contents are converted into feature identifiers, and are described using numbers as the high-level features sampled at this time.

[0056] That is, the bottom-level features of the multi-scale feature pyramid are represented by the texture features of the occluded area and its corresponding grayscale image. The middle-level features use shape features to describe the shape and difference distance of the occluded area. The high-level features upsample the image on the occluded area corresponding to the texture features and shape features, and then connect the sampled features in a top-down manner to complete the connection of the multi-scale feature pyramid.

[0057] The implementation method of step S22 includes: S221, based on the multi-scale feature pyramid corresponding to the occluded area, multi-scale difference feature extraction is performed, the difference features of each layer are calculated, and the cosine similarity difference and graph structure difference of each layer are determined; the difference feature at this time can be the local texture mutation found in the bottom layer, the abnormal edge direction detected in the middle layer, the pest semantic feature recognized in the high layer, etc., so as to find that there are differences in related pests in the current occluded area under the three currently established levels, and then the differences in cosine similarity and graph structure reflected in each layer are mapped with the difference feature of this description, and the data related to the difference feature after mapping is used as the difference feature of subsequent output to obtain relatively complete data about the difference feature.

[0058] At the same time, when the difference features are found to correspond, the features extracted from the occluded area at the corresponding level will be compared with the standard image in the historical data to obtain the cosine similarity differences and graph structure differences at each level. The graph structure difference means that the distribution positions of the difference features identified at each level at each level are different, and the sum of the Euclidean distances that differ is taken as the graph structure difference at this time.

[0059] S222, use the cosine similarity difference and graph structure difference of each level to map with the difference feature, determine the combination probability of the cosine similarity difference and graph structure difference and the difference feature of each level after mapping, and output the data corresponding to the corresponding combination probability as the difference feature of the occluded area at each level.

[0060] The total data currently processed includes multiple sets of multi-scale feature pyramids generated under multiple images to determine the difference characteristics of the data identified at this time; mapping the cosine similarity difference and the graph structure difference with the difference characteristics is to establish data mapping, which is convenient for finding the corresponding data later. The combined probability obtained later represents the conditional probability when the three conditions of cosine similarity difference, graph structure difference and difference characteristics are all met, to describe the probability of the difference feature appearing under the corresponding situation.

[0061] S23, according to the difference characteristics of the occluded area, the pest probability of each level is counted, the texture association between the current occluded area and the surrounding area under the corresponding pest probability is identified, and the morphological association matrix corresponding to the occluded area is obtained.

[0062] The probability of pests indicates the possibility of pests occurring in the occluded area. It reflects the risk level of pest threats in a specific area. For example, the area corresponding to the collected image is described according to the number of insects, leaf damage area, soil moisture and other features obtained in the image, so that the specific situation of pests in the corresponding area can be known when the location of the corresponding pests is circled, to determine whether the corresponding feature diffusion occurs, resulting in the need for further adjustment of the identified pest location. The probability of pests will be determined by statistics of each pest-related feature under the currently obtained differential features to determine the proportion of the number of pests in the total data volume when processing a large number of initial pest images and pest occluded images.

[0063] That is, the implementation method of step S23 also includes: S231, according to the difference characteristics of the occluded area, the pest probability of each level is calculated, and the conditional probability of the common occurrence of the data and pests at the corresponding level of the difference characteristics is described for the pest probability. For example, there is an abnormal edge trend in the middle level of the current difference characteristics. The proportion of the number of times the data corresponding to this abnormal edge trend has occurred in the total historical data can be used as the conditional probability of this middle level, that is, the number of data that meet the two conditions of abnormal edge trend in the middle level and pest identification in the total historical data. The pest probability of each level is combined using the weight of each level to obtain the statistical pest probability. At this time, the method of statistical pest probability is to perform weighted summation of the pest probability of each level to describe the overall pest probability, wherein the weights of the bottom, middle and high levels can be selected in the form of 0.3, 0.4, and 0.3 in turn, based on the data recorded by its morphological descriptor, and the features under these feature descriptions are combined for statistics.

[0064] S232, extracting local texture features under the probability of insect pests, and performing association according to the positions of the local texture features on the occluded area, and using the associated data as a morphological association matrix.

[0065] Local texture features can be extracted at the edges of occluded areas and other areas through Fourier transform, gray-level co-occurrence matrix, and local binary pattern, and the similarity between the corresponding local texture features at these positions is calculated. That is, after calculation using cosine similarity, the similarities of the corresponding areas are formed into a weighted distribution matrix, which is the morphological association matrix.

[0066] For example, when extracting differential features, the bottom layer detects local texture mutations (LBP value difference > 0.3), the middle layer detects abnormal edge trends (gradient direction histogram mismatch), and the high layer identifies the semantic features of pests (mouthparts are detected).

[0067] When calculating the hierarchical probability, the low-level probability is 0.65, the middle-level probability is 0.82, and the high-level probability is 0.91. The statistical pest probability (fusion probability) is 0.83. At this time, the hierarchical probability is large after combination, indicating that the current pest difference situation belongs to the common pest image style. Then further check its texture to obtain the specific pest feature information in the current image when there is an occluded area.

[0068] Texture association analysis: The similarity between the occluded area and the healthy area on the left is 0.25, and the similarity between the occluded area and the diseased area on the right is 0.68. At this time, the obtained weights will be set according to the similarity of the area calculated during texture association. That is, the ratio of the similarity existing in a certain area to the total similarity calculated at this time is used as the weight distribution on a certain part. These weight distributions are then output in sequence to form a matrix. This matrix represents the morphological association matrix corresponding to the occluded area.

[0069] In one embodiment of the present invention, step S3 also includes: S31, based on the morphological association matrix of the occluded area, identifying the feature association of the current covered area, and setting the spatial association matrix. After obtaining the morphological association matrix, the morphological association matrix represents the association between the occluded area and other surrounding areas, and then uses its spatial position to represent it according to these associations, and explains the location of the corresponding feature corresponding information under the spatial position. This information will be represented as a difference feature or a feature corresponding to the statistical pest area, and the features at this position are interacted using an adaptive interaction model to complete the feature interaction between the initial image of the pest and the occluded image of the pest and the disease. The spatial association matrix actually represents the corresponding data in the morphological association matrix using spatial position, and then performs feature interaction according to the spatial position.

[0070] S32, based on the spatial association matrix, identify the feature boundary of the feature interaction information, and perform feature interaction on the initial image of the pests and diseases and the occluded image of the pests and diseases in sequence according to the feature boundary. The feature boundary represents the feature information that needs to be interacted between the current initial image of the pests and diseases and the occluded image of the pests and diseases in the occluded area, and when describing the image or feature, identify the feature boundary described by the information, such as the boundary corresponding to the edge of the morphology, the region boundary and the descriptor corresponding to the morphological feature in the feature, and perform feature interaction in sequence according to the position and type represented by them, so as to complete the feature interaction under multiple feature layers. It is explained that each feature under the corresponding feature boundary continuously interacts the initial image of the pests and diseases and the occluded image of the pests and diseases according to the feature layer where it is located, so as to form a corresponding adaptive interaction control model. The data model formed at this time represents the integration of the relevant feature interaction content of the current collected initial image of the pests and diseases and the occluded image of the pests and diseases, which is used to improve the subsequent marking accuracy of the boundary and category of the corresponding pests and diseases identification under the feature interaction.

[0071] When using the adaptive interaction model, the feature interaction information of the initial image of the pest and the occluded image of the pest and the disease will interact through the features output by the main model and the auxiliary model, that is, the features of the main model are obtained through the initial image of the pest and the pest, and the features of the auxiliary model are obtained through the occluded image of the pest and the disease, to identify the range of pests and diseases in the occluded area under the feature interaction repair. That is, the feature interaction information is exchanged in the following form for the first time.

[0072] ;in, Represents the output feature of the main model at layer k. The output feature of the main model at feature layer k represents the output feature after the first exchange of feature interaction information is completed. This feature will be used as the input of layer k of the main model for corresponding calculations later. represents the output features of the auxiliary model at the k-1 layer, represents the output features of the main model at layer k-1, , They represent the learnable weights of the auxiliary model at the k-1 layer, , They respectively represent the learnable bias of the auxiliary model at the k-1 layer, and then the feature information of the auxiliary model is interacted with the main model. That is, after the feature information of the occluded area is interacted with the feature information in the initial image, CAE is used to reconstruct the part of the auxiliary model with the occluded area to extract the feature information for the second time. At this time, the feature information of the main model is also obtained to assist in the reconstruction of the part of the occluded area to obtain more feature information about the occluded area.

[0073] At this time, the output features of the auxiliary model at layer k are represented as follows.

[0074] ;in, Represents the output feature of the auxiliary model at layer k. The output feature of the auxiliary model at feature layer k represents the output feature after the second exchange of feature interaction information is completed. This feature will be used as the input of the auxiliary model layer k for corresponding calculations later. , They represent the learnable weights of the main model at the k-1 layer, , They respectively represent the learnable bias of the main model at the k-1 layer. According to the interaction of these contents, we can know the feature forms of the occluded area when the initial pest and disease image and the pest and disease occluded image are input, and whether there will be corresponding errors between the corresponding images when these features interact with each other, to describe the position of the pests and diseases under the occluded area when identifying the pest and disease related data that may exist in the occluded area.

[0075] In one embodiment of the present invention, in step S4, an adaptive interaction model is mainly used to output an intermediate bounding box and a predicted bounding box after feature interaction, and the uncertainty output by the current bounding box is determined according to the difference between the two bounding boxes, that is, the difference between the predicted coordinates of the bounding box and the original image is determined, and the calculated score is used as the boundary distribution probability, and the probability of an object appearing inside the bounding box is set to the corresponding category probability to determine the probability represented by the calibrated bounding box during the pest and disease treatment process. Among them, the intermediate bounding box represents the bounding box of the corresponding area of ​​the calibrated pest and disease output each time the model is iterated, and the predicted bounding box is an image that realizes the marking of the corresponding object in the pest and disease image, and these images are combined to obtain the currently identified boundary distribution probability and category probability.

[0076] Therefore, the implementation method of step S4 includes: according to the output result of the adaptive interaction model, obtaining multiple intermediate bounding boxes of the initial image of the pests and diseases after feature interaction, and forming a probability matrix according to the differences between the multiple intermediate bounding boxes and the predicted bounding box, and taking the average value of the probability matrix as the uncertainty matrix of the bounding box; according to the uncertainty matrix of the bounding box, determining the boundary distribution probability corresponding to the current initial image of the pests and diseases.

[0077] Get the object label corresponding to the middle bounding box, and set the category probability corresponding to the current initial image of pests and diseases according to the occurrence probability of the corresponding object label. At this time, the category probability is obtained in the same way as the comparison of the middle bounding box. At this time, a probability matrix related to the category is also constructed. After that, the average value corresponding to the probability matrix is ​​used to set a category uncertainty matrix, and then the elements in each matrix are solved to obtain the category probability.

[0078] It should be noted that the probability matrix obtained at this time will form multiple groups of probability matrices according to the difference between the intermediate bounding box and the predicted bounding box. Then, when obtaining the corresponding uncertainty matrix, the object label will indicate the sign of the pest corresponding to the insect body or other traces of the pest to explain what the main content of the pest image is at this time.

[0079] Assume that the difference between the intermediate bounding box and the predicted bounding box is represented by the coordinates of its bounding box prediction. The uncertainty matrix of the bounding box described at this time is The boundary distribution probability represented by each element in is expressed as shown below.

[0080] ;in, represents the boundary distribution probability of the jth coordinate of the i-th intermediate bounding box, represents the value of the jth coordinate of the i-th middle bounding box in the X-th enhanced image, Represents the value of the predicted bounding box corresponding to the jth coordinate of the i-th intermediate bounding box, represents the number of enhanced images, , the enhanced image represents multiple images generated by the adaptive interactive model based on the input initial image of the pest and disease when performing enhancement processing, and compared with the original image after annotation. These images can represent the corresponding dimensions of the current image processing; Represents an exponential function.

[0081] At the same time, the uncertainty matrix of the category The category probability represented by each element in can be expressed as follows.

[0082] ;in, represents the category probability of the nth object label of the i-th intermediate bounding box, It represents the probability of occurrence of the nth object label in the i-th middle bounding box in the X-th enhanced image. This probability indicates the occurrence of this object label in the bounding box in the corresponding enhanced image, which helps to determine whether the corresponding object label is correctly labeled. Represents the value of the predicted bounding box corresponding to the nth object label of the i-th intermediate bounding box.

[0083] Then, the two uncertainty matrices can be combined to form a unified uncertainty matrix M. ,in, It indicates that the two matrices are connected vertically, so as to obtain the corresponding effect on whether the bounding box predicted by the current model can achieve the estimated accuracy after feature interaction.

[0084] In one embodiment of the present invention, step S5 finally combines the boundary distribution probability and the category probability to find out the pest-related positions in the combined scenario of the two, thereby ultimately improving the recognition efficiency of pests and diseases under different occlusion conditions.

[0085] When using the approximate matrix corresponding to pests and diseases, the expression of the kernel matrix or the corresponding expression in the Fisher information matrix can be used to express the data after the boundary distribution probability and category probability are combined in the current scene, to evaluate whether the location of the pests and diseases described by the current output boundary distribution probability and category probability is accurate, and finally to identify the pest and disease location and corresponding signs with sufficient accuracy to complete the rapid processing of pest and disease related images.

[0086] Therefore, the implementation method of step S5 includes vertically connecting the boundary distribution probability and the category probability, converting the vertically connected data into an approximate matrix, obtaining the objective function of the approximate matrix, and judging whether the output of the approximate matrix meets the expected accuracy. If it meets the expected accuracy, the corresponding position is output as the pest and disease prediction position.

[0087] When converted into an approximate matrix, the data corresponding to the boundary distribution probability and the category probability form a diagonal matrix and an information matrix in turn. After a comprehensive comparison of the outputs of the diagonal matrix and the information matrix, it is determined whether the output of the current approximate matrix meets the expected content, and finally the predicted location of pests and diseases that meets the recognition accuracy is obtained.

[0088] The Fisher information matrix can be used when setting the information matrix, and its expression is shown below.

[0089] ;in, It represents the Fisher information matrix, which is a matrix that measures the stability of parameter estimation. It reflects the relationship between the variance of the parameter estimate and the true value of the parameter and is usually used to evaluate the accuracy of parameter estimation. represents the expectation operator, where Representing the data distribution means averaging all possible data samples I; this step ensures that the average amount of information on the entire data set is considered; represents the gradient vector, which represents the log-likelihood function with respect to the parameter The partial derivative of is a given parameter and data samples The uncertainty matrix The probability distribution of represents the transpose of the gradient vector; the calculation used here is the Fisher information matrix by calculating the log-likelihood function with respect to the parameter The product of the gradient vector and its transpose, and for all possible data samples The expected value is obtained; the inverse matrix of this matrix can be used to approximate the covariance matrix of parameter estimation, thereby giving the uncertainty of parameter estimation; therefore, the content calculated by the current boundary distribution probability and category probability under the approximate matrix represented by the information matrix is ​​evaluated, and whether the output bounding box position at this time can accurately identify the pests and diseases, thereby enabling accurate identification of the pests and diseases. The Fisher information matrix currently used is for scenarios that require position reasoning. This calculation method tends to be more like the Monte Carlo approximation method to repeatedly infer the location of the pests and diseases and the relative position deviation of the location after multiple sampling processes.

[0090] At the same time, the corresponding processing can also be completed using diagonal approximation methods, such as using a diagonal matrix and processing only based on the data on the diagonal to emphasize the boundaries of pest and disease identification in high dimensions under multidimensional processing, so as to achieve rapid output of pest and disease differences at corresponding positions.

[0091] The approximate matrix corresponding to its diagonal matrix is ​​expressed as shown below.

[0092] ;in, represents the approximate matrix corresponding to the diagonal matrix, Dependency parameters and the uncertainty matrix The loss function at this time will be identified by the comprehensive loss of the corresponding main model and auxiliary model according to the adaptive interaction model corresponding to the parameters and uncertainty matrix at this time, that is, the weighted value of the loss rate of the main model and the auxiliary model is used; represents a diagonal matrix; Represents the gradient vector, represents the loss function or uncertainty matrix About parameters The gradient of It also represents the transpose of the gradient vector. At this time, the diagonal matrix is ​​used to describe the similarity measure between the current output pest and disease positions during regression, so as to describe the evaluation of the recognition accuracy of the approximate matrix under the identification of the pest and disease positions and the continuous regression scenario, so as to find the actual position of the pest and disease output.

[0093] For the parameters used at this time It can represent the features or input values ​​of related data processed by the current model. After moving the average of the parameters, the data at the corresponding positions of the current boundary distribution probability and category probability are continuously adjusted to approximate the position of the matrix, so as to continuously regress the current prediction content and finally complete the sign of pest and disease prediction.

[0094] In the present invention, after obtaining the corresponding pest and disease location using an adaptive interactive model or the like, the output data is used to evaluate the accuracy of its existence. As shown in Figure 1, the following content can be used for evaluation.

[0095] Table 1. Output evaluation table

[0096]

[0097] In Table 1, the precision values ​​corresponding to Precision, Recall, mAP50, and mAP50-95 are used to describe the current image processing to illustrate the accuracy of the current actual processing and the recognition effect under this accuracy. AGIIN-MAF represents the model number used in the current processing, and compares the processing process with the effect of YOLO11 to illustrate the excellence of the current processing and highlight the advantages of AGIIN-MAF in practical applications.

[0098] like Figure 4-Figure 9As shown, the present invention also explains the image currently being segmented and the processing method. After performing feature interaction on the occluded area, the range of the recognized bounding box is eventually enlarged to include more textures and information about the pest characteristics. Based on this information, the corresponding pests and diseases can be quickly circled in dynamic scenes, and curves related to accuracy, confidence, etc. are used to illustrate the current effect, so as to achieve an effect description of the pest and disease treatment.

[0099] The present invention also provides a pest and disease detection system based on dynamic scenes, comprising: an image acquisition module, an occlusion area detection module, an adaptive interaction module, a pest and disease identification module and a position prediction module; wherein the output end of the image acquisition module is connected to the occlusion area detection module, the output end of the occlusion area detection module is connected to the adaptive interaction module, the output end of the adaptive interaction module is connected to the pest and disease identification module, and the output end of the pest and disease identification module is connected to the position prediction module.

[0100] The image acquisition module is used to acquire the initial image and the occluded image of the pests and diseases in the dynamic scene and perform preprocessing.

[0101] The occluded area detection module is used to detect the occluded area in the pest and disease occluded image, extract the texture features and shape features of the occluded area, construct a multi-scale feature pyramid, and use the multi-scale feature pyramid to set the morphological association matrix corresponding to the occluded area.

[0102] Adaptive interaction module: It is used to identify the feature interaction information of the initial image of the pest and disease and the occluded image of the pest and disease according to the morphological association matrix of the occluded area, and to build an adaptive interaction model.

[0103] The pest and disease identification module is used to identify the pest and disease boundaries and categories in the initial pest and disease image using an adaptive interaction model, and calculate the boundary distribution probability and category probability.

[0104] The location prediction module is used to adjust the location based on the boundary distribution probability and category probability using an approximate matrix to delineate the predicted location of pests and diseases.

[0105] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations on the present invention. A person skilled in the art may make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention and they are still covered by the protection scope of the present invention.

Claims

1. A method for detecting pests and diseases based on dynamic scenes, characterized in that: include: S1, obtaining multiple initial images of pests and diseases and occluded images of pests and diseases in a dynamic scene, and preprocessing the initial images of pests and diseases and the occluded images of pests and diseases; S2, extracting the occluded area for pest and disease identification according to the preprocessed initial image of the pest and disease and the occluded image of the pest and disease, and setting the morphological association matrix corresponding to the occluded area according to the characteristic information of the occluded area; S3, identifying the feature interaction information of the initial image of the pest and disease and the occluded image of the pest and disease according to the morphological association matrix of the occluded area, and forming an adaptive interaction model based on the feature interaction information; S4, using an adaptive interaction model, identifying the pest boundary and pest category corresponding to the pest initial image, and determining the boundary distribution probability and category probability of the pest initial image; S5, according to the boundary distribution probability and category probability of the initial image of the pests and diseases, the approximate matrix is ​​used to adjust the position and circle the predicted position of the pests and diseases.

2. A method for detecting pests and diseases based on dynamic scenes according to claim 1, characterized in that: The implementation of step S1 also includes: S11, respectively obtaining the acquisition time and coordinates of the initial image of the pest and disease and the occluded image of the pest and disease; S12, using the RAFT algorithm to calculate the dense optical flow field between the initial image of the pest and disease and the occluded image of the pest and disease, and obtain the optical flow displacement vector; S13, reverse mapping the pest occluded image according to the optical flow displacement vector to make the two images consistent in perspective.

3. The method for detecting pests and diseases based on dynamic scenes according to claim 1, characterized in that: The implementation of step S2 also includes: S21, using the preprocessed initial pest image and the pest occlusion image to perform occlusion area detection, and determine the occlusion area in the pest occlusion image; S22, extracting texture features and shape features corresponding to the occluded area, and constructing a multi-scale feature pyramid, extracting features of the occluded area based on the multi-scale feature pyramid, and describing the difference features of the occluded area at each level; S23, according to the difference characteristics of the occluded area, the pest probability of each level is counted, the texture association between the current occluded area and the surrounding area under the corresponding pest probability is identified, and the morphological association matrix corresponding to the occluded area is obtained.

4. The method for detecting pests and diseases based on dynamic scenes according to claim 3, characterized in that: The identification method of the occlusion area detection in step S21 includes: The preprocessed initial image of the pests and diseases and the occluded image of the pests and diseases are pixel-differentiated, the difference points are identified during the pixel difference, the difference points are connected according to the coordinate position, and the minimum bounding rectangle corresponding to the difference points after connection is set as the occluded area.

5. The method for detecting pests and diseases based on dynamic scenes according to claim 3, characterized in that: The implementation of step S22 includes: S221, based on the multi-scale feature pyramid corresponding to the occluded area, perform multi-scale difference feature extraction, calculate the difference features of each layer, and determine the cosine similarity difference and graph structure difference of each layer; S222, use the cosine similarity difference and graph structure difference of each level to map with the difference feature, determine the combination probability of the cosine similarity difference and graph structure difference and the difference feature of each level after mapping, and output the data corresponding to the corresponding combination probability as the difference feature of the occluded area at each level.

6. The method for detecting pests and diseases based on dynamic scenes according to claim 3, characterized in that: The implementation of step S23 also includes: S231, calculating the pest probability at each level according to the difference characteristics of the shielded area; S232, extracting local texture features under the probability of insect pests, and performing association according to the positions of the local texture features on the occluded area, and using the associated data as a morphological association matrix.

7. The method for detecting pests and diseases based on dynamic scenes according to claim 1, characterized in that: Step S3 also includes: S31, based on the morphological association matrix of the occluded area, identifying the feature association of the current covered area and setting the spatial association matrix; S32, based on the spatial correlation matrix, identifying the feature boundary of the feature interaction information, and performing feature interaction on the pest and disease initial image and the pest and disease occluded image in sequence according to the feature boundary.

8. The method for detecting pests and diseases based on dynamic scenes according to claim 1, characterized in that: The implementation of step S4 includes: According to the output results of the adaptive interaction model, multiple intermediate bounding boxes of the initial image of pests and diseases after feature interaction are obtained, and a probability matrix is ​​formed according to the differences between the multiple intermediate bounding boxes and the predicted bounding box, and the average value of the probability matrix is ​​taken as the uncertainty matrix of the bounding box; according to the uncertainty matrix of the bounding box, the boundary distribution probability corresponding to the current initial image of pests and diseases is determined; Get the corresponding object label in the middle bounding box, and set the category probability corresponding to the current pest and disease initial image according to the occurrence probability of the corresponding object label.

9. The method for detecting pests and diseases based on dynamic scenes according to claim 1, characterized in that: The implementation method of step S5 includes vertically connecting the boundary distribution probability and the category probability, converting the vertically connected data into an approximate matrix, obtaining the objective function of the approximate matrix, and judging whether the output of the approximate matrix meets the expected accuracy. If it meets the expected accuracy, the corresponding position is output as the predicted position of the pest and disease.

10. A pest detection system based on dynamic scenes, characterized in that: include: An image acquisition module is used to acquire the initial image and the occluded image of the pests and diseases in the dynamic scene and perform preprocessing; The occluded area detection module is used to detect the occluded area in the pest occluded image, extract the texture features and shape features of the occluded area, construct a multi-scale feature pyramid, and use the multi-scale feature pyramid to set the morphological association matrix corresponding to the occluded area; Adaptive interaction module: used to identify the feature interaction information of the initial image of the pest and disease and the occluded image of the pest and disease according to the morphological association matrix of the occluded area, and to build an adaptive interaction model; The pest and disease identification module is used to identify the pest and disease boundaries and categories in the initial pest and disease image using an adaptive interaction model, and calculate the boundary distribution probability and category probability; The location prediction module is used to adjust the location based on the boundary distribution probability and category probability using an approximate matrix to delineate the predicted location of pests and diseases.

Citation Information

Patent Citations

  • Agricultural intelligent monitoring system and method based on machine vision

    CN116773550A

  • Pest and disease monitoring method based on remote sensing meteorological data collaboration

    CN118624540A

  • Shielded face recognition method and device based on feature reconstruction and storage medium

    CN113989902A

  • Intelligent tomato disease and insect pest detection method based on image processing

    CN117974633A

  • Facade image analysis method and system based on completion and refinement network

    CN118397276A

Cited By

  • Inflammatory bowel disease curative effect prediction system based on multi-modal data

    CN120708939A

  • Face recognition detection method and device, computer equipment and program product

    CN121096006A