A dynamic scene-based pest detection method and system

Abstract

The application relates to the technical field of disease and pest detection, in particular to a disease and pest detection method and system based on a dynamic scene, which comprises the following steps: acquiring multiple disease and pest initial images and disease and pest occlusion images under a dynamic scene; according to the disease and pest initial images and the disease and pest occlusion images, extracting an occlusion area for disease and pest identification, and setting a shape correlation matrix of the occlusion area according to feature information of the occlusion area; according to the shape correlation matrix of the occlusion area, identifying feature interaction information of the disease and pest initial images and the disease and pest occlusion images, and forming an adaptive interaction model based on the feature interaction information; using the adaptive interaction model, determining boundary distribution probability and category probability of the disease and pest initial images; according to the boundary distribution probability and the category probability of the disease and pest initial images, using an approximate matrix to adjust a position, and circumscribing a disease and pest prediction position; and the accuracy and precision of disease and pest detection under a dynamic scene are realized.

Description

Technical Field

[0001] This invention relates to the field of pest and disease detection technology, specifically a pest and disease detection method and system based on dynamic scenarios. Background Technology

[0002] Crop diseases and pests significantly impact crop yield and quality. Large-scale, rapid, and high-precision monitoring and early warning of diseases and pests are crucial for improving crop quality and yield. Traditional methods for detecting diseases and pests mainly rely on manual observation and experience-based judgment, or on static image processing techniques for identification. However, in dynamic scenarios, these methods often fail to meet the requirements for accuracy and real-time performance.

[0003] For example, Chinese Patent Publication No. CN118624540A discloses a pest and disease monitoring method based on remote sensing meteorological data collaboration, which relates to the field of pest and disease monitoring technology. The method involves acquiring remote sensing images of multiple sub-regions of a target area and preprocessing these images. Based on the preprocessed images of the multiple sub-regions, a pixel unit matching algorithm is used to identify sub-regions with pests and diseases. Historical meteorological data of the pest and disease sub-regions within a fixed time period before the occurrence of pests and diseases, as well as the corresponding spectral characteristic wavelengths of the remote sensing images of the pest and disease sub-regions, are acquired. A pest and disease outbreak prediction model is established based on the historical meteorological data and the spectral characteristic wavelengths of the remote sensing images of the sub-regions to be monitored. The predicted meteorological data and the spectral characteristic wavelengths of the remote sensing images of the sub-regions to be monitored are input into the pest and disease outbreak prediction model for pest and disease monitoring. Existing technologies describe the spectral characteristics of pests and diseases that may occur under time and meteorological conditions, mainly aiming to solve the problem of pest and disease identification accuracy under different lighting scenarios caused by meteorological conditions. However, this implementation ignores the occlusion present in the images of pest and disease areas, resulting in identification accuracy problems even after removing the influence of ambient lighting.

[0004] For example, Chinese Patent Publication No. CN116773550A discloses an agricultural intelligent monitoring system and method based on machine vision. The system includes: a monitoring area acquisition module for acquiring the monitoring area in farmland; a crop growth image acquisition module for acquiring crop growth images in the monitoring area; and a pest and disease monitoring module for monitoring pests and diseases based on the crop growth images and a preset pest and disease image library. Acquiring the monitoring area in farmland includes: establishing a pest and disease prediction map corresponding to the farmland; determining multiple pest and disease prediction locations and corresponding pest and disease prediction types from the pest and disease prediction map; delineating a minimum bounding circle within the pest and disease prediction map based on the minimum bounding circle delineation condition; and using the area enclosed by the minimum bounding circle as the monitoring area. Existing technologies describe the location of pests and diseases during flight and then perform multiple samplings based on these locations to predict the location of pests and diseases. However, this prediction method has the problem of ignoring relevant features in the current environment. As a result, it only targets variety information, historical growth environment information, fertilization history information, and irrigation history information for prediction, ignoring whether pests and diseases are obscured or hidden in the environment described by various variety information, etc. This leads to insufficient identification of corresponding features in the basic data, affecting the accuracy and range of the final location prediction.

[0005] Existing technologies describe relative processing methods for light and pest locations. However, for pests and diseases appearing in these areas, it is necessary to identify the occlusion situation and obtain data through multiple samplings according to the different occlusions in order to identify the occluded feature information and improve the accuracy of pest and disease area prediction. Summary of the Invention

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: a method for detecting pests and diseases based on dynamic scenes, including: S1, acquiring multiple initial images of pests and diseases and occlusion images of pests and diseases in a dynamic scene, and preprocessing the initial images of pests and diseases and occlusion images of pests and diseases.

[0007] S2. Based on the preprocessed initial image of pests and diseases and the image of pests and diseases occlusion, extract the occlusion area for pest and disease identification, and set the morphological association matrix corresponding to the occlusion area according to the feature information of the occlusion area.

[0008] S3. Based on the morphological correlation matrix of the occluded area, identify the feature interaction information between the initial image of the pest and the image of the pest and disease occlusion, and form an adaptive interaction model based on the feature interaction information.

[0009] S4. Using an adaptive interaction model, identify the corresponding pest boundaries and pest categories in the initial pest image, and determine the boundary distribution probability and category probability of the initial pest image.

[0010] S5. Based on the boundary distribution probability and category probability of the initial image of pests and diseases, the approximate matrix is ​​used to adjust the position and delineate the predicted location of pests and diseases.

[0011] A pest and disease detection system based on dynamic scenes includes: an image acquisition module, used to acquire initial images of pests and diseases and images of pests and diseases occlusion in dynamic scenes, and to perform preprocessing.

[0012] The occlusion region detection module is used to detect occlusion regions in images occluded by pests and diseases, extract texture and shape features of the occluded regions, construct a multi-scale feature pyramid, and use the multi-scale feature pyramid to set the morphological correlation matrix corresponding to the occluded regions.

[0013] Adaptive Interaction Module: Based on the morphological correlation matrix of the occluded area, this module identifies the feature interaction information between the initial image of pests and diseases and the image occluded by pests and diseases, and constructs an adaptive interaction model.

[0014] The pest and disease identification module is used to identify the boundaries and categories of pests and diseases in the initial images using an adaptive interactive model, and to calculate the boundary distribution probability and category probability.

[0015] The location prediction module is used to adjust the location based on the boundary distribution probability and category probability using an approximate matrix to delineate the predicted location of pests and diseases.

[0016] The beneficial effects of this invention are as follows: First, by comprehensively processing the initial image of pests and diseases and the image of pests and diseases occlusion, this invention can identify the problem of missing features and false detection of pest areas caused by occlusion in dynamic scenes; and by comparing the feature information existing under the occluded area, it can quickly identify the morphological associations existing in different pest and disease scenarios, and associate these morphological associations with the dynamic scene of pest and disease detection, so as to help the system to timely detect existing pest and disease situations in constantly changing scenes and improve the recognition accuracy of pests and diseases in different scenarios.

[0017] Second, this invention processes feature information present in the initial image and the image of pests and diseases with occlusion by using feature interaction information, enabling feature alignment and interaction optimization between multi-view images. This addresses the issues of reduced overall detection accuracy due to occlusion in multi-view scenarios, resulting in deviations in pest and disease measurement positions when measuring corresponding pest and disease locations at different locations, and the inability to promptly detect occluded pests and diseases when feature interaction information is incomplete. Furthermore, after feature interaction, the invention reveals the interaction between features in different dimensions and comprehensively displays these features to describe the actual distribution of pests and diseases in different scenarios.

[0018] Third, this invention calculates the boundary distribution and category of pests and diseases, evaluates the output of the current adaptive interaction model, determines the accuracy of the current image processing, and assesses the measured pest and disease situation in the corresponding scenario. This achieves accurate modeling and uncertainty quantification of the probability distribution of pest location; it describes the uncertainty in dividing the boundaries of pests and diseases, thereby improving the relative accuracy and efficiency of the final identification of pests and diseases in the image, and enhancing the adaptability of pest and disease detection in different scenarios. Attached Figure Description

[0019] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0020] Figure 1 This is a flowchart illustrating a pest and disease detection method based on dynamic scenarios.

[0021] Figure 2 This is a schematic diagram of an adaptive interactive model for a pest and disease detection method based on dynamic scenarios.

[0022] Figure 3 This is a flowchart illustrating step S2 of a pest and disease detection method based on dynamic scenarios.

[0023] Figure 4 This is a schematic diagram illustrating the effect of image recognition in a pest and disease detection method and system based on dynamic scenes.

[0024] Figure 5 This is another schematic diagram illustrating the effect of image recognition in a pest and disease detection method and system based on dynamic scenes.

[0025] Figure 6 It is a precision-confidence curve of a pest and disease detection method and system based on dynamic scenarios.

[0026] Figure 7 It is an F1 score-confidence curve of a pest and disease detection method and system based on dynamic scenarios.

[0027] Figure 8 It is a recall-confidence curve of a pest and disease detection method and system based on dynamic scenarios.

[0028] Figure 9 It is a precision-recall curve of a pest and disease detection method and system based on dynamic scenarios. Detailed Implementation

[0029] The embodiments of the present invention are described in detail below. The embodiments described below are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention. Where specific techniques or conditions are not specified in the embodiments, they shall be performed in accordance with the techniques or conditions described in the literature in the art or in accordance with the product manual.

[0030] See Figure 1 A method for detecting pests and diseases based on dynamic scenes includes: S1, acquiring multiple initial images and occluded images of pests and diseases in a dynamic scene, and preprocessing the initial images and occluded images of pests and diseases.

[0031] S2. Based on the preprocessed initial image of pests and diseases and the image of pests and diseases occlusion, extract the occlusion area for pest and disease identification, and set the morphological association matrix corresponding to the occlusion area according to the feature information of the occlusion area.

[0032] S3. Based on the morphological correlation matrix of the occluded area, identify the feature interaction information between the initial image of the pest and the image of the pest and disease occlusion, and form an adaptive interaction model based on the feature interaction information.

[0033] S4. Using an adaptive interaction model, identify the corresponding pest boundaries and pest categories in the initial pest image, and determine the boundary distribution probability and category probability of the initial pest image.

[0034] S5. Based on the boundary distribution probability and category probability of the initial image of pests and diseases, the approximate matrix is ​​used to adjust the position and delineate the predicted location of pests and diseases.

[0035] When identifying pests and diseases, the accuracy of identification is easily affected by the environment in dynamic scenes, especially when there are obstructions or changes in the environment. In this case, the original image of the pests and diseases can be obtained first, and image feature recognition can be performed. Then, the image with obstruction can be input, and relevant information about pests and diseases in the obstructed area can be identified from the obstructed image. This information will be used for subsequent completion processing and compared with the original image of the pests and diseases to determine whether the parts of the pests and diseases being repaired and identified can be accurately identified.

[0036] In one embodiment of the present invention, such as Figure 2 As shown, a main model can be set up to process the initial image of pests and diseases, and then an auxiliary model can be set up to process the image of pests and diseases that are occluded. After extracting the features of the occluded areas, a convolutional autoencoder is used to complete the occluded areas to describe key features such as lesion texture and insect morphology that may be generated. For example, the aphid cluster area occluded by leaves may show the outline of the insect or traces of secretions after repair.

[0037] The features processed by the auxiliary model are interacted with the corresponding features of the main model to verify whether the completion method is normal, thereby improving the accuracy of the main model in identifying pests and diseases, reducing the impact of environment and occlusion on pest and disease identification, and finally obtaining the location of pests and diseases with comprehensive feature information, thus improving the efficiency of processing in dynamic environments.

[0038] Figure 2 The first part represents the training phase of the main model and the auxiliary model, the second part represents the structure of the convolutional autoencoder, and the third part represents the process by which the adaptive interactive model interacts with the feature information of the occluded region.

[0039] When the adaptive interaction model interacts with the features between branches, it will align and set weights according to the different features selected, such as setting an information flow coefficient to represent the weight of feature interaction between branches.

[0040] ;in, This represents the information flow coefficient from branch x to branch y at layer k-1. This represents the learnable weights of branch x at layer k-1. This represents the learnable bias of branch x at level k-1. This represents the output characteristics of branch x at layer k-1.

[0041] Then, based on these weights and the features of layer k-1, the output features of branch y at layer k are calculated.

[0042] ;in, This represents the output feature of branch y at layer k. This represents element-wise multiplication. This represents element-wise addition. Represents the Gaussian error linear unit activation function. This represents the learnable weights of branch x at layer k-1. This represents the learnable bias of branch x at level k-1. This represents the output feature of branch y at layer k-1, where This represents the output of the current adaptive interaction model at the corresponding feature layer, used to determine the specific iteration form of each feature layer during the feature interaction process, in order to adapt to the constantly changing data distribution and corresponding requirements.

[0043] Then, based on the output features of each feature layer, we can understand the feature interaction information between the initial image of the pest and the image of the pest occlusion. That is, we obtain the features extracted from the two images respectively, and interact these features in the form of feature layers. Finally, based on the data after interaction, the currently set model can identify the boundaries and corresponding types of pests and diseases in the image, thereby improving the identification of pest and disease areas with occlusion in dynamic environments and improving the accuracy of pest and disease identification.

[0044] In step S1, the preprocessing of the initial image of pests and diseases and the occluded image of pests and diseases can be carried out by: aligning consecutive frames with GPS+IMU sensor data or optical flow algorithms (such as RAFT), determining that the current description area of ​​the initial image of pests and diseases and the occluded image of pests and diseases is the same area, and adjusting the viewing angle of the initial image of pests and diseases and the occluded image of pests and diseases to the same angle, and determining the occlusion area at the same angle.

[0045] Therefore, the implementation of step S1 also includes: S11, obtaining the acquisition time and coordinates of the initial image of pests and diseases and the occluded image of pests and diseases, respectively. The initial image of pests and diseases describes the directly acquired image of pests and diseases, while the occluded image of pests and diseases is an image with random occlusion, and the location described by the occluded image of pests and diseases is consistent with that described by the initial image of pests and diseases.

[0046] S12, the RAFT algorithm is used to calculate the dense optical flow field between the initial image of the pest and the image of the pest occlusion, and the optical flow displacement vector is obtained. In calculating the dense optical flow field, the RAFT algorithm calculates the corresponding displacement between each pixel in the two images and sets these displacements as unique optical flow vectors. Then, the pixels in the two images are mapped according to the calculated displacements, and image deformation alignment is performed to obtain two images with the same angle. This reaction mapping uses OpenCV's remap function to perform image deformation, identifying the displacement and corresponding coordinates, and then deforming the image to obtain two images representing the same angle and target. The occlusion region within this image is then analyzed.

[0047] S13, reverse mapping of the disease and pest occlusion image based on the optical flow displacement vector to make the two images have the same viewpoint.

[0048] At the same time, by using consecutive frames, when consecutive frames represent the same area, the corresponding initial images of pests and diseases and images of pests and diseases occlusion can be processed to identify the occlusion area and the types of pests and diseases that may exist under the occlusion area.

[0049] In one embodiment of the present invention, in addition to identifying the occlusion area in step S2, a multi-scale feature pyramid is constructed using the pest occlusion image to determine the features at each level and calculate the pest probability at each level to determine whether the current occlusion area is an occlusion area with pests.

[0050] In addition, it is necessary to determine the different occlusion patterns of the current occlusion area under different insect bodies, as well as the differences between different insect bodies and the background area. That is, when extracting the feature information of the occlusion area, it is also necessary to identify the distinguishing features corresponding to the insect body and the similarity measure between the features to obtain more information about the occlusion area.

[0051] For example, if the occluded area has different sizes, the texture of the tissue in the occluded area can be inferred by using adjacent unoccluded areas. This further restricts or adjusts the size of the occluded area or the range of pests to obtain the corresponding association rules about the occluded object and the pests. At this time, the association of the occluded area under different occlusion conditions can be obtained, and the pest types in the occluded area can be further restricted based on these associations. This makes it easier to improve the accuracy of feature recognition of the repair location when performing feature repair on the image under the occluded area.

[0052] like Figure 3 As shown, the implementation of step S2 also includes: S21, using the preprocessed initial image of pests and diseases and the image of pests and diseases occlusion to perform occlusion area detection and determine the occlusion area in the image of pests and diseases occlusion.

[0053] Generally, the shaded area may include various types of shading, such as insect shading, leaf shading, soil shading, and light shading. Based on these different types of shading, a multi-scale feature pyramid is used to describe the performance of these features in the current shaded area and to identify the differences in these performances at different scales or levels. These feature differences are then used for subsequent pest probability identification to determine the probability of pests in the shaded area.

[0054] The identification method for occlusion region detection in step S21 includes: performing pixel-level difference analysis on the preprocessed initial image of pests and diseases and the image occluded by pests and diseases; identifying each difference point during pixel-level difference analysis; connecting each difference point according to its coordinate position; and setting the minimum bounding rectangle corresponding to each difference point as the occlusion region. This mainly involves connecting the positions with differences according to their minimum bounding rectangles to find the occlusion region that exists at the current position in both the normal image and the occluded image. Then, features at the corresponding positions of the occlusion region need to be identified and extracted to complete the feature filling and completion interaction of the occlusion region.

[0055] S22, extract the texture and shape features corresponding to the occluded area and construct a multi-scale feature pyramid. Based on the multi-scale feature pyramid, extract features from the occluded area to describe the differences in features at each level. In the multi-scale feature pyramid, the bottom layer features represent the grayscale histogram of the texture of the occluded area, the middle layer features represent the contextual shape descriptors of the occluded area and its surrounding areas, and the high layer features represent the semantic segmentation results of the occluded area. The semantic segmentation results also represent the segmentation of the occluded area with its context. That is, the occluded area is segmented using semantically related segmentation forms to describe the feature, such as segmentation based on leaves or segmentation based on insects. For example, the spatial relationship of the corresponding insect in the occluded area at its corresponding position, or the part of the insect identified in the image when disease occurs. These contents are converted into feature labels and described using numbers as the high-level features sampled at this time.

[0056] The multi-scale feature pyramid consists of a bottom layer of features representing the texture features of the occluded region and its corresponding grayscale image, a middle layer of features using shape features to describe the shape and distance of the occluded region, and a top layer of features which are upsampled from the texture and shape features and then connected in a top-down manner to complete the connection of the multi-scale feature pyramid.

[0057] The implementation of step S22 includes: S221, based on the multi-scale feature pyramid corresponding to the occluded area, multi-scale difference feature extraction is performed, the difference features of each layer are calculated, and the cosine similarity difference and graph structure difference of each layer are determined; the difference features at this time can be the discovery of local texture mutations at the bottom layer, the detection of abnormal edge directions at the middle layer, the identification of pest semantic features at the top layer, etc., thereby discovering the existence of relevant pest differences in the current occluded area under the three layers currently established. Then, the differences in cosine similarity and graph structure reflected at each layer are mapped to the difference features described therein, and the data related to the difference features after mapping are used as the difference features of the subsequent output to obtain more complete data about the difference features.

[0058] Simultaneously, when the difference features are found to correspond to the occluded area, the features extracted at the corresponding level are compared with the standard image in the historical data to obtain the cosine similarity difference and graph structure difference at each level. The graph structure difference means that the distribution position of the difference features identified at each level is calculated, and the sum of the Euclidean distances that are different is taken as the graph structure difference at this time.

[0059] S222 uses the cosine similarity difference and graph structure difference and difference features of each level for mapping, determines the combination probability of the cosine similarity difference and graph structure difference and difference features of each level after mapping, and outputs the data corresponding to the combination probability as the difference features of the occluded region at each level.

[0060] The current total data processed includes multiple sets of multi-scale feature pyramids generated from multiple images to determine the discrepancy features of the identified data. Mapping cosine similarity difference and graph structure difference to the discrepancy features establishes a data mapping, facilitating the subsequent identification of corresponding data. The resulting combined probability represents the conditional probability under the condition that cosine similarity difference, graph structure difference, and discrepancy feature are all satisfied, describing the probability of the discrepancy feature occurring in the corresponding situation.

[0061] S23. Based on the differences in the occluded areas, the probability of pests at each level is statistically analyzed, and the texture association between the current occluded area and the surrounding area under the corresponding pest probability is identified to obtain the morphological association matrix corresponding to the occluded area.

[0062] Pest probability represents the likelihood of pests occurring in occluded areas. It reflects the risk level of a specific area being threatened by pests. For example, it describes the area corresponding to the collected images based on features such as the number of insects, the area of ​​leaf damage, and soil moisture. This allows us to understand the specific situation of pest occurrence in the corresponding area after delineating the location of pests, and to determine whether there is a diffusion of the corresponding features, leading to further adjustments to the identified pest location. Pest probability is determined by statistically analyzing each pest-related feature under the currently obtained differential features, and then calculating the proportion of pests present in the total data volume when processing a large number of initial pest images and pest-occluded images.

[0063] The implementation of step S23 further includes: S231, calculating the pest probability at each level based on the difference characteristics of the occluded area. The pest probability describes the conditional probability of the data and pests co-occurring at the corresponding level for the difference feature. For example, if there is an abnormal edge trend in the middle level of the current difference feature, the proportion of the number of times the data corresponding to this abnormal edge trend has occurred with pests in the historical data can be used as the conditional probability of this middle level. That is, the number of data that meet both the conditions of an abnormal edge trend and pest identification in the middle level out of the total historical data. The pest probabilities at each level are combined using the weights of each level to obtain the statistical pest probability. At this time, the method of calculating the pest probability is to sum the pest probabilities at each level with weights to describe the overall pest probability. The weights of the bottom, middle, and top levels can be selected in the form of 0.3, 0.4, and 0.3 respectively. Based on the data recorded by the morphological descriptor, the features described by these features are combined for statistical analysis.

[0064] S232, extract local texture features under the probability of pest infestation, and associate them according to the position of the local texture features on the occluded area, and use the associated data as a morphological correlation matrix.

[0065] Local texture features can be extracted by using methods such as Fourier transform, gray-level co-occurrence matrix, and local binary pattern to extract the positions of occluded areas and other areas at their edges. The similarity between the corresponding local texture features at these positions can be calculated. After calculating using cosine similarity, the similarity of the corresponding areas is formed into a weighted distribution matrix, which is the morphological correlation matrix.

[0066] For example, during differential feature extraction, the bottom layer detects local texture mutations (LBP value difference > 0.3), the middle layer detects abnormal edge orientations (gradient direction histogram mismatch), and the top layer identifies pest semantic features (mouthparts are detected).

[0067] In the hierarchical probability calculation, the probability of the low layer is 0.65, the probability of the middle layer is 0.82, and the probability of the high layer is 0.91. The statistically calculated pest probability (fusion probability) is 0.83. At this point, the combined hierarchical probability is high, indicating that the current difference in pests belongs to a common pest image pattern. Further examination of its texture will reveal the specific pest features in the current image when occlusion areas exist.

[0068] Texture association analysis: Similarity between the occluded area and the healthy area on the left: 0.25, similarity between the occluded area and the lesion area on the right: 0.68. At this point, the weights obtained will be set according to the similarity of the regions calculated during texture association. That is, the ratio of the similarity existing in a certain region to the total similarity calculated at this time is used as the weight distribution of a certain part. Then, these weight distributions are output in sequence to form a matrix. This matrix will represent the morphological association matrix corresponding to the occluded area.

[0069] In one embodiment of the present invention, step S3 further includes: S31, identifying the feature association of the currently occluded area based on the morphological association matrix of the occluded area, and setting a spatial association matrix. After obtaining the morphological association matrix, which represents the association between the occluded area and other surrounding areas, these associations are then represented using their spatial locations, and the location of the corresponding feature information at that spatial location is described. This information is represented as differential features or features corresponding to statistically affected areas. The features at that location are then interacted using an adaptive interaction model to complete the feature interaction between the initial pest image and the pest occlusion image. The spatial association matrix is ​​essentially representing the data corresponding to the morphological association matrix using spatial locations, and then performing feature interaction according to the spatial locations.

[0070] S32, based on the spatial correlation matrix, identifies the feature boundaries of feature interaction information and sequentially performs feature interaction on the initial pest and disease image and the pest and disease occlusion image according to the feature boundaries. The feature boundaries represent the feature information that needs to be interacted on the occlusion area of ​​the current initial pest and disease image and the pest and disease occlusion image. When describing the image or feature, the feature boundaries described by these information are identified, such as the boundaries corresponding to the morphological edge parts, the region boundaries, and the information corresponding to the descriptors of the morphological features in the features. Feature interaction is then performed sequentially according to their represented positions and types, thereby completing feature interaction at multiple feature layers. This illustrates that each feature under the corresponding feature boundary continuously interacts with the initial pest and disease image and the pest and disease occlusion image according to its respective feature layer to form a corresponding adaptive interaction control model. The resulting data model represents the integration of relevant feature interaction content from the currently collected initial pest and disease image and the pest and disease occlusion image, used to improve the accuracy of subsequent labeling of the boundaries and categories of corresponding pests and diseases under feature interaction.

[0071] When using the adaptive interaction model, the feature interaction information between the initial image of pests and diseases and the image of pests and diseases occlusion is exchanged through the features output by the main model and the auxiliary model. That is, the features of the main model are obtained from the initial image of pests and diseases, and the features of the auxiliary model are obtained from the image of pests and diseases occlusion. This allows for the identification of the range of pests and diseases in the occluded areas under feature interaction repair. The feature interaction information in the first exchange is as follows.

[0072] ;in, This represents the output feature of the main model at layer k. This output feature of the main model at feature layer k represents the output feature after the first exchange of feature interaction information. This feature will be used as the input of the main model at layer k for corresponding calculations. This represents the output features of the auxiliary model at layer k-1. This represents the output features of the main model at layer k-1. , These represent the learnable weights of the auxiliary model at layer k-1. , These represent the learnable biases of the auxiliary model at layer k-1. Then, the feature information of the auxiliary model is interacted with the feature information in the initial image. After that, CAE is used to reconstruct the occluded part of the auxiliary model to extract the second feature information. At this time, the feature information of the main model is also obtained to assist in the reconstruction of the occluded part and to obtain more feature information about the occluded area.

[0073] The output features of the auxiliary model at layer k are represented as follows.

[0074] ;in, This represents the output feature of the auxiliary model at layer k. This output feature of the auxiliary model at feature layer k represents the output feature after the second exchange of feature interaction information. This feature will be used as the input of the auxiliary model at layer k for corresponding calculations. , These represent the learnable weights of the main model at layer k-1. , These represent the learnable biases of the main model at layer k-1. Based on the interaction of these contents, we can know the feature forms of the occluded region when the initial image of pests and diseases and the occluded image of pests and diseases are input, and whether there will be corresponding errors between the corresponding images when these features are interacted. This describes the location of pests and diseases under the occluded region when identifying pests and diseases related data that may exist in the occluded region.

[0075] In one embodiment of the present invention, step S4 mainly uses an adaptive interaction model to output intermediate bounding boxes and predicted bounding boxes after feature interaction. The uncertainty of the current bounding box output is determined based on the difference between the two bounding boxes, i.e., the difference between its predicted bounding box coordinates and the original image. The calculated score is used as the boundary distribution probability, and the probability of an object appearing inside this bounding box is set as the corresponding category probability to determine the probability represented by the labeled bounding box during pest and disease treatment. Here, the intermediate bounding box represents the bounding box of the corresponding pest and disease area output in each model iteration, and the predicted bounding box is the image that has labeled the corresponding object in the pest and disease image. Combining these images yields the currently identified boundary distribution probability and category probability.

[0076] Therefore, the implementation of step S4 includes: obtaining multiple intermediate bounding boxes of the initial image of pests and diseases after feature interaction based on the output of the adaptive interaction model, forming a probability matrix according to the differences between the multiple intermediate bounding boxes and the predicted bounding boxes, and taking the average value of the probability matrix as the uncertainty matrix of the bounding boxes; determining the boundary distribution probability corresponding to the current initial image of pests and diseases based on the uncertainty matrix of the bounding boxes.

[0077] Obtain the corresponding object labels within the middle bounding box, and set the category probability corresponding to the current pest and disease initial image according to the occurrence probability of the corresponding object labels. At this time, the method of obtaining the category probability is the same as the method of comparing the middle bounding box. At this time, a probability matrix related to the category will also be constructed. Then, after setting an uncertainty matrix of the category using the average value of the probability matrix, the elements in each matrix are solved to obtain the category probability.

[0078] It should be noted that the probability matrix obtained at this time will be composed of multiple probability matrices according to the difference between the intermediate bounding box and the predicted bounding box. Then, the corresponding uncertainty matrix will be calculated. The object label will represent the insect body or other insect traces corresponding to the pests and diseases, to indicate what the main content of the pest and disease image is at this time.

[0079] Assuming the difference between the intermediate bounding box and the predicted bounding box is represented by the coordinates of the predicted bounding box, the uncertainty matrix of the bounding box described in this case is... The boundary distribution probability represented by each element in the equation is expressed as follows.

[0080] ;in, Let represent the boundary distribution probability of the j-th coordinate of the i-th intermediate bounding box. This represents the value of the j-th coordinate of the i-th intermediate bounding box in the X-th augmented image. This represents the value on the predicted bounding box corresponding to the j-th coordinate of the i-th intermediate bounding box. Indicates the number of enhanced images. The enhanced image represents multiple images generated by the adaptive interactive model based on the initial image of the pests and diseases, which are compared with the original image after annotation. These images can represent the corresponding dimensions during the current image processing. This represents an exponential function.

[0081] Meanwhile, the uncertainty matrix regarding the category The probability of the class represented by each element can be expressed as shown below.

[0082] ;in, Let represent the class probability of the nth object label in the i-th middle bounding box. This represents the probability of the nth object label appearing in the i-th middle bounding box of the X-th augmented image. This probability indicates the likelihood of the object label appearing within the bounding box in the corresponding augmented image, and is used to help determine whether the corresponding object label is correctly labeled. This represents the value on the predicted bounding box corresponding to the nth object label of the i-th intermediate bounding box.

[0083] The two uncertain matrices can then be combined to form a unified uncertain matrix M. ,in, This indicates that two matrices are vertically connected, thus determining whether the predicted bounding box of the model can achieve the expected accuracy after feature interaction.

[0084] In one embodiment of the present invention, step S5 finally combines the boundary distribution probability and the category probability to find the pest-related location in the combined scenario, thereby improving the identification efficiency of pests under different occlusion conditions.

[0085] When using the approximate matrix corresponding to pests and diseases, the expression of the kernel matrix or the corresponding expression in the Fisher information matrix can be used to describe the data after combining the boundary distribution probability and category probability in the current scene. This allows for the evaluation of whether the location of pests and diseases described by the currently output boundary distribution probability and category probability is accurate. Finally, the location of pests and diseases and the corresponding markers with sufficient accuracy can be delineated to complete the rapid processing of images related to pests and diseases.

[0086] Therefore, the implementation of step S5 includes vertically connecting the boundary distribution probability and the category probability, converting the vertically connected data into an approximate matrix, obtaining the objective function of the approximate matrix, judging whether the output of the approximate matrix meets the expected accuracy, and outputting the corresponding position as the predicted position of pests and diseases when it meets the expected accuracy.

[0087] When transforming into an approximate matrix, the boundary distribution probability and the category probability corresponding to the data form a diagonal matrix and an information matrix in sequence. The outputs of the diagonal matrix and the information matrix are compared and then it is determined whether the output of the current approximate matrix meets the expected content. Finally, the predicted location of pests and diseases that meets the recognition accuracy is obtained.

[0088] When setting up the information matrix, the Fisher information matrix can be used, and its expression is shown below.

[0089] ;in, The Fisher information matrix is ​​a matrix that measures the stability of parameter estimates. It reflects the relationship between the variance of the parameter estimates and the true values ​​of the parameters and is usually used to evaluate the accuracy of parameter estimates. This represents the expectation operator, here Representing the data distribution means averaging over all possible data samples I; this step ensures that the average information content across the entire dataset is considered. Let represent the gradient vector, and let represent the log-likelihood function with respect to the parameters. The partial derivatives; here Is a given parameter and data samples Lower uncertainty matrix The probability distribution; This represents the transpose of the gradient vector; the calculations used here are based on the Fisher information matrix calculated by using the log-likelihood function with respect to the parameters. The product of the gradient vector and its transpose, and over all possible data samples. The expected value is obtained; the inverse of this matrix can be used to approximate the covariance matrix of the parameter estimation, thus giving the uncertainty of the parameter estimation; therefore, the evaluation assesses whether the content calculated by the current boundary distribution probability and class probability under the approximate matrix represented by the information matrix, and whether the output bounding box position can accurately identify pests and diseases, thereby achieving accurate identification of pests and diseases. The Fisher information matrix currently used is for scenarios requiring location inference. This calculation method tends to be a Monte Carlo approximation to infer the location of pests and diseases multiple times and the relative positional deviation of that location after multiple sampling processes.

[0090] Additionally, diagonal approximation can be used to complete the corresponding processing. For example, a diagonal matrix can be used to process only the data on the diagonal to emphasize the boundary of pest and disease identification in high-dimensional multidimensional processing, so as to achieve rapid output of pest and disease differences at corresponding locations.

[0091] The approximate matrix representation of its diagonal matrix is ​​shown below.

[0092] ;in, This represents the approximate matrix corresponding to the diagonal matrix. Indicates dependency parameters and uncertain matrix The loss function at this time will be identified by the combined loss of the main model and the auxiliary model according to the adaptive interaction model corresponding to the parameters and uncertainty matrix at this time, that is, by the weighted value of the loss rate of the main model and the auxiliary model. Represents a diagonal matrix; The gradient vector represents the loss function or uncertainty matrix. Regarding parameters The gradient, similarly It also represents the transpose of the gradient vector. Here, a diagonal matrix is ​​used to describe the similarity measure between the currently output pest and disease locations during regression. This describes the evaluation of the recognition accuracy of this approximation matrix in the regression scenario when identifying pest and disease locations, thereby finding the actual location of the output pest and disease.

[0093] For the parameters used at this time It can represent the input values ​​of features or related data that the current model is processing. After the parameters are moved averaged, the data at the corresponding positions of the current boundary distribution probability and category probability are continuously adjusted to approximate the position of the matrix, so as to continuously regress the current prediction content and finally complete the marker of pest and disease prediction.

[0094] In this invention, after obtaining the corresponding pest and disease locations using adaptive interactive models and other methods, the output data is used to evaluate the accuracy of their existence. As shown in Figure 1, the following methods can be used for evaluation.

[0095] Table 1. Output Evaluation Table

[0096]

[0097] Table 1 uses Precision, Recall, mAP50, and mAP50-95 to describe the precision values ​​used in the current image processing, illustrating the actual processing accuracy and the recognition performance at that precision. AGIIN-MAF represents the model number used in the current processing, and the processing results are compared with those of YOLO11 to demonstrate the superiority of the current processing and highlight the advantages of AGIIN-MAF in practical applications.

[0098] like Figures 4-9As shown, the present invention also describes the image segmentation and processing method. After feature interaction on the occluded area, the final recognized bounding box range becomes larger to include more texture and information about pest features. Based on this information, the corresponding pests can be quickly delineated in dynamic scenes. Curves related to accuracy and confidence are used to illustrate the current effect, thereby demonstrating the effectiveness of pest treatment.

[0099] The present invention also provides a pest and disease detection system based on dynamic scenes, comprising: an image acquisition module, an occlusion area detection module, an adaptive interaction module, a pest and disease identification module, and a location prediction module; wherein, the output end of the image acquisition module is connected to the occlusion area detection module, the output end of the occlusion area detection module is connected to the adaptive interaction module, the output end of the adaptive interaction module is connected to the pest and disease identification module, and the output end of the pest and disease identification module is connected to the location prediction module.

[0100] The image acquisition module is used to acquire initial images of pests and diseases and images of pests and diseases occlusion in dynamic scenes, and to perform preprocessing.

[0101] The occlusion region detection module is used to detect occlusion regions in images occluded by pests and diseases, extract texture and shape features of the occluded regions, construct a multi-scale feature pyramid, and use the multi-scale feature pyramid to set the morphological correlation matrix corresponding to the occluded regions.

[0102] Adaptive Interaction Module: Based on the morphological correlation matrix of the occluded area, this module identifies the feature interaction information between the initial image of pests and diseases and the image occluded by pests and diseases, and constructs an adaptive interaction model.

[0103] The pest and disease identification module is used to identify the boundaries and categories of pests and diseases in the initial images using an adaptive interactive model, and to calculate the boundary distribution probability and category probability.

[0104] The location prediction module is used to adjust the location based on the boundary distribution probability and category probability using an approximate matrix to delineate the predicted location of pests and diseases.

[0105] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention, which are still covered within the protection scope of the present invention.

Claims

1. A method for detecting pests and diseases based on dynamic scenarios, characterized in that, include: S1, acquire multiple initial images and occlusion images of pests and diseases in a dynamic scene, and preprocess the initial images and occlusion images of pests and diseases. S2, based on the preprocessed initial image of pests and diseases and the image of pests and diseases occlusion, extract the occlusion area of ​​pests and diseases for identification, and set the morphological association matrix corresponding to the occlusion area according to the feature information of the occlusion area; S21, use the preprocessed initial image of pests and diseases and the image of pests and diseases occlusion to detect the occlusion area and determine the occlusion area in the image of pests and diseases occlusion. S22, extract the texture and shape features corresponding to the occluded area, and construct a multi-scale feature pyramid. Based on the multi-scale feature pyramid, extract features from the occluded area to describe the differences in the occluded area at each level. S23. Based on the differences in the occluded areas, the probability of pests at each level is statistically analyzed, and the texture association between the current occluded area and the surrounding area under the corresponding pest probability is identified to obtain the morphological association matrix corresponding to the occluded area. S3. Based on the morphological correlation matrix of the occluded area, identify the feature interaction information of the initial image of the pest and disease and the image of the pest and disease occlusion, and form an adaptive interaction model based on the feature interaction information. S31, Based on the morphological association matrix of the occluded area, identify the feature association of the currently occluded area and set the spatial association matrix; S32, based on the spatial correlation matrix, identifies the feature boundaries of feature interaction information, and performs feature interaction on the initial image of pests and diseases and the image of pests and diseases occlusion in sequence according to the feature boundaries; S4. Using an adaptive interaction model, identify the corresponding pest boundaries and pest categories in the initial pest image, and determine the boundary distribution probability and category probability of the initial pest image. Based on the output of the adaptive interaction model, multiple intermediate bounding boxes of the initial image of pests and diseases after feature interaction are obtained. A probability matrix is ​​formed according to the differences between the multiple intermediate bounding boxes and the predicted bounding boxes, and the average value of the probability matrix is ​​taken as the uncertainty matrix of the bounding boxes. Based on the uncertainty matrix of the bounding boxes, the boundary distribution probability corresponding to the current initial image of pests and diseases is determined. The object labels corresponding to the intermediate bounding boxes are obtained, and the category probability corresponding to the current initial image of pests and diseases is set according to the occurrence probability of the corresponding object labels. S5. Based on the boundary distribution probability and category probability of the initial image of pests and diseases, the approximate matrix is ​​used to adjust the position and delineate the predicted location of pests and diseases.

2. The method for detecting pests and diseases based on dynamic scenes according to claim 1, characterized in that, The implementation of step S1 also includes: S11, acquire the acquisition time and coordinates of the initial image of pests and diseases and the image of pests and diseases occlusion, respectively; S12, use the RAFT algorithm to calculate the dense optical flow field between the initial image of pests and diseases and the image of pests and diseases occlusion, and obtain the optical flow displacement vector; S13, reverse mapping of the disease and pest occlusion image based on the optical flow displacement vector to make the two images have the same viewpoint.

3. The method for detecting pests and diseases based on dynamic scenes according to claim 2, characterized in that, The identification methods for occlusion region detection in step S21 include: The preprocessed initial image of pests and diseases and the image of pests and diseases with occlusion are subjected to pixel difference analysis. Each difference point is identified during pixel difference analysis, and each difference point is connected according to its coordinate position. The smallest bounding rectangle corresponding to each difference point after connection is set as the occlusion area.

4. The method for detecting pests and diseases based on dynamic scenes according to claim 1, characterized in that, The implementation methods of step S22 include: S221. Based on the multi-scale feature pyramid corresponding to the occluded region, multi-scale difference features are extracted, the difference features of each layer are calculated, and the cosine similarity difference and graph structure difference of each layer are determined. S222 uses the cosine similarity difference and graph structure difference and difference features of each level for mapping, determines the combination probability of the cosine similarity difference and graph structure difference and difference features of each level after mapping, and outputs the data corresponding to the combination probability as the difference features of the occluded region at each level.

5. The method for detecting pests and diseases based on dynamic scenes according to claim 1, characterized in that, The implementation of step S23 also includes: S231, Calculate the pest probability at each level based on the differences in the shading area; S232, extract local texture features under the probability of pest infestation, and associate them according to the position of the local texture features on the occluded area, and use the associated data as a morphological correlation matrix.

6. The method for detecting pests and diseases based on dynamic scenes according to claim 1, characterized in that, The implementation of step S5 includes vertically connecting the boundary distribution probability and the category probability, converting the vertically connected data into an approximate matrix, obtaining the objective function of the approximate matrix, judging whether the output of the approximate matrix meets the expected accuracy, and outputting the corresponding position as the predicted location of pests and diseases when it meets the expected accuracy.

7. A pest and disease detection system based on dynamic scenes, used to perform the steps of the pest and disease detection method based on dynamic scenes according to any one of claims 1-6, characterized in that, include: The image acquisition module is used to acquire initial images of pests and diseases and images of pests and diseases occlusion in dynamic scenes, and to perform preprocessing. The occlusion region detection module is used to detect occlusion regions in images occluded by pests and diseases, extract texture and shape features of the occlusion regions, construct a multi-scale feature pyramid, and use the multi-scale feature pyramid to set the morphological correlation matrix corresponding to the occlusion regions. Adaptive Interaction Module: Used to identify the feature interaction information of the initial image of pests and diseases and the image of pests and diseases occlusion based on the morphological correlation matrix of the occluded area, and to build an adaptive interaction model; The pest and disease identification module is used to identify the boundaries and categories of pests and diseases in the initial images of pests and diseases using an adaptive interactive model, and to calculate the boundary distribution probability and category probability. The location prediction module is used to adjust the location based on the boundary distribution probability and category probability using an approximate matrix to delineate the predicted location of pests and diseases.

Images