Agricultural pest and disease monitoring and early warning method and system based on image processing

Abstract

The present application relates to the technical field of pest and disease monitoring, and specifically discloses a method and system for monitoring and early warning of agricultural pests and diseases based on image processing, which continuously collects under-exposed, normally exposed, and over-exposed images of farmland crops, and aligns and combines the three into a high dynamic range image that fully retains the details of highlights and shadows. Subsequently, target detection is performed on the high dynamic range image to generate a binary mask map of the suspicious lesion area, and the binary mask map is further used to extract the lesion area images from the three original images with different exposures. By performing deep pest and disease feature extraction and multi-source feature dynamic compensation aggregation on the three groups of multi-source suspicious lesion images containing complementary information, a comprehensive and integrated representation of the pest and disease characteristics is formed, and precise identification of the pest and disease types is achieved. This method can improve the accuracy and robustness of agricultural pest and disease monitoring under complex natural lighting conditions through the comprehensive utilization of multi-exposure image data.

Description

Technical Field

[0001] The present application relates to the technical field of pest and disease monitoring, and more specifically, to an agricultural pest and disease monitoring and early warning method and system based on image processing. Background Art

[0002] Agriculture is the foundation of the national economy, and the stable production and supply of crops plays a vital role in ensuring food security and promoting socioeconomic development. However, during the growth process of crops, the frequent occurrence of various pests and diseases is one of the major threats to their yield and quality. Traditional pest and disease control relies primarily on on-site inspections and empirical judgment by farmers or agricultural technicians. This method is not only time-consuming and labor-intensive, but also highly subjective and often misses the optimal opportunity for prevention and control due to delayed detection. Therefore, it is necessary to establish an efficient agricultural pest and disease monitoring and early warning solution to achieve early detection and accurate identification of pests and diseases.

[0003] To overcome the drawbacks of traditional manual monitoring, numerous image processing-based agricultural pest monitoring methods have been proposed. These methods utilize cameras and other devices to capture crop images and then analyze and identify pest and disease features within these images using computer vision algorithms. However, in real-world farmland environments, lighting conditions are extremely complex and variable. Factors such as strong sunlight, cloud cover, and shadows between crop leaves or backlighting can result in significant brightness variations and noise in the captured images. This makes it difficult for the model to accurately locate lesions and limits its ability to extract subtle visual features of pests and diseases, significantly reducing the accuracy of pest and disease monitoring and identification.

[0004] Therefore, an optimized agricultural pest and disease monitoring and early warning method and system based on image processing is expected. Summary of the Invention

[0005] In order to solve the above technical problems, the present application is proposed. The embodiment of the present application provides an agricultural pest and disease monitoring and early warning method and system based on image processing, which continuously collects under-exposed, normally exposed and over-exposed images of farmland crops through a camera, and aligns and combines the three into high dynamic range image data that fully retains the highlights and shadow details. Subsequently, target detection is performed on the high dynamic range image data to generate a binary mask map of the suspicious lesion area, and the binary mask map is further used to extract the lesion area images from the three original images with different exposures. The three groups of multi-source suspicious lesion images containing complementary information are subjected to deep pest and disease feature extraction and multi-source feature dynamic compensation aggregation to form a comprehensive and integrated representation of the pest and disease characteristics, thereby achieving fine identification of the pest and disease types on this basis. This method can significantly improve the accuracy and robustness of agricultural pest and disease monitoring under complex natural lighting conditions through the comprehensive utilization of multi-exposure image data.

[0006] Accordingly, according to one aspect of the present application, a method for monitoring and early warning of agricultural pests and diseases based on image processing is provided, which includes:

[0007] Use a camera to continuously collect underexposed image data, normally exposed image data, and overexposed image data of farmland crops;

[0008] Performing image alignment on the underexposed image data, the normally exposed image data, and the overexposed image data, and then arranging them according to channel dimensions into high dynamic range image data;

[0009] Performing pest and disease target detection on the high dynamic range image data to obtain a binary mask image of a suspicious lesion area;

[0010] Based on the binary mask image of the suspicious lesion area, extracting an underexposed image of the farmland crop lesion area, a normally exposed image of the farmland crop lesion area, and an overexposed image of the farmland crop lesion area from the high dynamic range image data;

[0011] The underexposed image of the farmland crop lesion area, the normally exposed image of the farmland crop lesion area and the overexposed image of the farmland crop lesion area are subjected to pest and disease type identification based on dynamic compensation aggregation of multi-source information to obtain a pest and disease type identification result.

[0012] According to another aspect of the present application, there is provided an agricultural pest and disease monitoring and early warning system based on image processing, which includes:

[0013] A multi-exposure image acquisition module is used to continuously acquire under-exposure image data, normal-exposure image data, and over-exposure image data of farmland crops using a camera;

[0014] an image alignment and combination module, configured to align the underexposed image data, the normally exposed image data, and the overexposed image data, and then arrange them into high dynamic range image data according to channel dimensions;

[0015] a pest and disease target detection module, configured to perform pest and disease target detection on the high dynamic range image data to obtain a binary mask image of a suspicious lesion area;

[0016] a suspicious lesion region extraction module, configured to extract an underexposed image of the farmland crop lesion region, a normally exposed image of the farmland crop lesion region, and an overexposed image of the farmland crop lesion region from the high dynamic range image data based on the binary mask map of the suspicious lesion region;

[0017] The pest and disease type identification module is used to identify the pest and disease type based on dynamic compensation aggregation of multi-source information on the underexposed image of the farmland crop disease area, the normally exposed image of the farmland crop disease area and the overexposed image of the farmland crop disease area to obtain the identification result of the pest and disease type.

[0018] Compared with the existing technology, the agricultural pest and disease monitoring and early warning method and system based on image processing provided by the present application continuously collects under-exposed, normally exposed and over-exposed images of farmland crops through a camera, and aligns and combines the three into high dynamic range image data that fully retains the details of highlights and shadows. Subsequently, target detection is performed on the high dynamic range image data to generate a binary mask map of the suspicious lesion area, and the binary mask map is further used to extract the lesion area images from the three original images with different exposures. The three groups of multi-source suspicious lesion images containing complementary information are subjected to deep pest and disease feature extraction and multi-source feature dynamic compensation aggregation to form a comprehensive and integrated representation of the pest and disease characteristics, thereby achieving fine identification of the pest and disease types on this basis. This method can significantly improve the accuracy and robustness of agricultural pest and disease monitoring under complex natural lighting conditions through the comprehensive utilization of multi-exposure image data. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The above and other purposes, features, and advantages of the present application will become more apparent through a more detailed description of the embodiments of the present application in conjunction with the accompanying drawings. The accompanying drawings are intended to provide a further understanding of the embodiments of the present application and constitute a part of the specification. Together with the embodiments of the present application, they are used to explain the present application and do not constitute a limitation of the present application. In the drawings, the same reference numerals generally represent the same components or steps.

[0020] Figure 1 Flowchart of an agricultural pest and disease monitoring and early warning method based on image processing according to an embodiment of the present application.

[0021] Figure 2 Schematic diagram of data flow of the agricultural pest and disease monitoring and early warning method based on image processing according to an embodiment of the present application.

[0022] Figure 3 This is a flowchart of step S2 in the agricultural pest and disease monitoring and early warning method based on image processing according to an embodiment of the present application.

[0023] Figure 4 This is a flowchart of step S5 in the agricultural pest and disease monitoring and early warning method based on image processing according to an embodiment of the present application.

[0024] Figure 5 This is a flowchart of step S52 in the agricultural pest and disease monitoring and early warning method based on image processing according to an embodiment of the present application.

[0025] Figure 6 This is a block diagram of an agricultural pest and disease monitoring and early warning system based on image processing according to an embodiment of the present application. DETAILED DESCRIPTION

[0026] Below, the exemplary embodiments according to the present application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application, and it should be understood that the present application is not limited to the exemplary embodiments described herein.

[0027] Figure 1 Flowchart of an agricultural pest and disease monitoring and early warning method based on image processing according to an embodiment of the present application. Figure 2 Schematic diagram of data flow of agricultural pest monitoring and early warning method based on image processing according to an embodiment of the present application. Figure 1 and Figure 2 As shown, the agricultural pest and disease monitoring and early warning method based on image processing according to the embodiment of the present application includes the following steps: S1, using a camera to continuously collect underexposed image data, normally exposed image data and overexposed image data of farmland crops; S2, aligning the underexposed image data, the normally exposed image data and the overexposed image data and arranging them into high dynamic range image data according to the channel dimension; S3, performing pest and disease target detection on the high dynamic range image data to obtain a binary mask map of the suspicious lesion area; S4, based on the binary mask map of the suspicious lesion area, extracting underexposed images of the farmland crop lesion area, normally exposed images of the farmland crop lesion area and overexposed images of the farmland crop lesion area from the high dynamic range image data; S5, performing pest and disease type identification based on multi-source information dynamic compensation aggregation on the underexposed image of the farmland crop lesion area, the normally exposed image of the farmland crop lesion area and the overexposed image of the farmland crop lesion area to obtain a pest and disease type identification result.

[0028] In the above-mentioned agricultural pest and disease monitoring and early warning method based on image processing, the step S1 uses a camera to continuously collect underexposed image data, normally exposed image data and overexposed image data of farmland crops. It should be understood that in actual farmland operation scenes, the lighting conditions are affected by many uncontrollable factors such as weather, time, cloud cover and crop shadows themselves, and present extremely complex and changeable characteristics. Using the traditional single exposure method for image acquisition can easily lead to local overexposure or underexposure of the image, resulting in the loss of the texture details of the lesions in the highlight area or the contour information of the disease in the dark area, thereby seriously restricting the performance of the subsequent image analysis algorithm. Therefore, in order to overcome the inherent defects of incomplete single exposure image information and fully capture all visual information in complex lighting scenes, this application is based on the basic principle of bracketing (Exposure Bracketing) and controls the imaging parameters to obtain image sequences of the same scene at different exposure levels. Specifically, automated cameras deployed in the field (e.g., mounted on fixed monitoring poles, tracked robots, or drones) are set to capture three consecutive photos of the same field of view within extremely short time intervals by rapidly adjusting the shutter speed or sensitivity (ISO): one underexposed image using a shorter exposure time that clearly preserves details in the scene's highlights; one normally exposed image using standard metering values ​​as a baseline; and one overexposed image using a longer exposure time that effectively captures details in the scene's shadows. This method fully records all visual characteristic information of pests and diseases, from the darkest to the brightest areas. This fundamentally addresses information loss caused by lighting issues and lays a solid data foundation for subsequent high-precision pest and disease detection and identification.

[0029] In the above-mentioned agricultural pest and disease monitoring and early warning method based on image processing, in step S2, the underexposed image data, the normally exposed image data and the overexposed image data are aligned and arranged into high dynamic range image data according to the channel dimension. It should be understood that when the camera continuously captures three images with different exposures, even if the interval is extremely short, there may be slight spatial displacement and deformation between the three images due to slight vibrations of the device itself (such as wind blowing the monitoring pole) or movement of the photographed object (such as leaves swaying in the wind). If the three are directly fused, ghosting or blurred artifacts will be generated in the image, interfering with the judgment of the model. Therefore, in order to construct a unified data representation that is precisely aligned in space, this application is based on image registration technology. By matching feature points of image data from multiple exposure sources, the spatial transformation relationship between images is calculated, and all images are aligned so that the three images accurately correspond in the same spatial coordinate system, thereby being combined into high dynamic range image data.

[0030] Figure 3 FIG. 1 is a flow chart of step S2 in the agricultural pest monitoring and early warning method based on image processing according to an embodiment of the present application. Figure 3 As shown, step S2 includes: S21, calculating the affine transformation matrices of the underexposed image data and the overexposed image data relative to the normally exposed image data based on the SIFT algorithm to obtain an underexposed image affine transformation matrix and an overexposed image affine transformation matrix; S22, performing an affine transformation on the underexposed image data and the overexposed image data based on the underexposed image affine transformation matrix and the overexposed image affine transformation matrix to obtain aligned underexposed image data and aligned overexposed image data; and S23, arranging the aligned underexposed image data, the normally exposed image data, and the aligned overexposed image data according to the channel dimension to obtain the high dynamic range image data. Specifically, first, using the normally exposed image as a reference image, the SIFT algorithm is used to extract key points from the three image frames and calculate the affine transformation matrices of the underexposed image and the overexposed image relative to the reference image. Subsequently, the respective affine transformation matrices are applied to the underexposed image and the overexposed image to ensure strict spatial alignment of their pixels with the reference image. After alignment, the three images are stacked channel-wise (e.g., C1 = underexposed, C2 = normal, C3 = overexposed) to create high-dynamic-range image data. This approach not only eliminates registration errors caused by displacement, ensuring the accuracy of subsequent pixel-level analysis, but also, by constructing multi-channel high-dynamic-range data, it losslessly passes the original exposure information to subsequent models. Compared to traditional HDR fusion algorithms, this method preserves more image detail and color information, providing a richer and more accurate data source for subsequent pest and disease identification.

[0031] In the above-mentioned agricultural pest and disease monitoring and early warning method based on image processing, the step S3 performs pest and disease target detection on the high dynamic range image data to obtain a binary mask map of the suspicious lesion area. In a specific example of the present application, the high dynamic range image data is input into a pest and disease target detection model based on the Mask R-CNN network to obtain the binary mask map of the suspicious lesion area, wherein the Mask R-CNN network includes a feature extraction module, a candidate region generation module, a candidate region classification and regression module, and a mask generation module. That is, considering the rectangular frame target detection method commonly used in the prior art, when locating the lesion, a large amount of healthy leaf background or irrelevant interference objects will inevitably be framed together, which seriously interferes with the subsequent learning of the core features of the disease, thereby reducing the precision and accuracy of pest and disease identification. Therefore, in order to achieve high-precision stripping of the lesion area in the image, the present application further realizes pixel-level positioning of the lesion by introducing an advanced Mask R-CNN network. Specifically, the Mask R-CNN model uses carefully manually annotated pixel-level lesion masks for supervised learning during the training phase. During object detection, nine-channel high dynamic range image data is input into the trained Mask R-CNN model. The Mask R-CNN network first extracts image features from the high dynamic range image data and, using its Region Proposal Network (RPN), generates a series of candidate target regions (Region Proposals) based on the multi-exposure high dynamic range image feature map. Subsequently, while performing classification and bounding box regression on each candidate region, the mask prediction branch simultaneously outputs a high-resolution binary mask of the suspected lesion region. In this binary mask, regions with a pixel value of 1 precisely correspond to the lesion, while regions with a pixel value of 0 represent the background. This generates a binary mask that closely matches the lesion outline, accurately indicating the lesion's location and shape, and separating the lesion from healthy tissue at the pixel level, providing clean data input for subsequent targeted lesion feature analysis and type identification.

[0032] In the above-mentioned agricultural pest and disease monitoring and early warning method based on image processing, the step S4 extracts underexposed images of the farmland crop lesion area, normally exposed images of the farmland crop lesion area, and overexposed images of the farmland crop lesion area from the high dynamic range image data based on the binary mask map of the suspected lesion area. It should be understood that although the high dynamic range image data improves the overall visual effect, its multi-exposure tone overlay mapping process will weaken the discriminant features under specific exposure (such as leaf vein texture in the underexposed image). Therefore, in order to retain the lesion-specific information under the original exposure, the present application uses the binary mask map of the suspected lesion area as a spatial filter based on the spatial mask reverse mapping principle, and performs pixel-by-pixel multiplication operations with the aligned underexposed image, normally exposed image, and overexposed image in the high dynamic range image data. Under this operation, for each original exposure image, only the pixels corresponding to the pixel value of 1 (i.e., the lesion area) on the mask image retain their original brightness and color values, while the pixels corresponding to the pixel value of 0 (i.e., the background area) are forced to be black (value 0), thereby effectively eliminating the background information and obtaining three sets of lesion images with geometric alignment but complementary radiation characteristics.

[0033] In the above-mentioned agricultural pest and disease monitoring and early warning method based on image processing, step S5 performs pest and disease type identification based on dynamic compensation aggregation of multi-source information on the underexposed image of the crop lesion area, the normally exposed image of the crop lesion area, and the overexposed image of the crop lesion area to obtain a pest and disease type identification result. That is, considering that the feature importance of the different exposure lesion images changes dynamically with the pest and disease type and the actual environmental conditions (such as downy mildew identification relies on the reflection of frost in the underexposed image, and aphid identification requires the transparent secretion features of the overexposed image, etc.). Therefore, in order to adaptively fuse the discriminative information of multi-source features, the present application further performs dynamic compensation aggregation of features on the underexposed image of the crop lesion area, the normally exposed image of the crop lesion area, and the overexposed image of the crop lesion area, so as to comprehensively utilize the complementary advantages of each exposure image to identify the pest and disease type, thereby improving the robustness and accuracy of pest and disease identification.

[0034] Figure 4 FIG. 5 is a flow chart of step S5 in the agricultural pest monitoring and early warning method based on image processing according to an embodiment of the present application. Figure 4As shown, the step S5 includes: S51, respectively extracting the pest and disease features of the underexposed image of the farmland crop lesion area, the normally exposed image of the farmland crop lesion area and the overexposed image of the farmland crop lesion area to obtain the underexposed image lesion area feature coding vector, the normally exposed image lesion area feature coding vector and the overexposed image lesion area feature coding vector; S52, performing feature dynamic compensation aggregation on the underexposed image lesion area feature coding vector, the normally exposed image lesion area feature coding vector and the overexposed image lesion area feature coding vector to obtain a comprehensive representation vector of farmland crop pest and disease features; S53, inputting the comprehensive representation vector of farmland crop pest and disease features into the trained pest and disease classifier to obtain the recognition result of the pest and disease type.

[0035] Specifically, the step S51 extracts the pest and disease features of the underexposed image of the lesion area of ​​the crop field, the normally exposed image of the lesion area of ​​the crop field, and the overexposed image of the lesion area of ​​the crop field to obtain the feature coding vector of the lesion area of ​​the underexposed image, the feature coding vector of the lesion area of ​​the normally exposed image, and the feature coding vector of the lesion area of ​​the overexposed image. In a specific example of the present application, the underexposed image of the lesion area of ​​the crop field, the normally exposed image of the lesion area of ​​the crop field, and the overexposed image of the lesion area of ​​the crop field are respectively subjected to pest and disease feature extraction based on the MobileNet model to obtain the feature coding vector of the lesion area of ​​the underexposed image, the feature coding vector of the lesion area of ​​the normally exposed image, and the feature coding vector of the lesion area of ​​the overexposed image. It should be understood that the underexposed image retains the lesion texture in the highlight area, the normally exposed image reflects the baseline color and morphology of the disease, and the overexposed image can clearly show the disease edge and infection outline in the dark area. The three have different visual emphases. Therefore, to achieve efficient feature-level representation of multi-source data, this application, based on the lightweight feature extraction principle of depthwise separable convolution, constructs three parallel feature extraction channels. Each channel uses a lightweight, highly efficient MobileNet model as its backbone network. A series of depthwise separable convolution, pointwise convolution, and pooling operations are performed on the underexposed image of the lesion area of ​​the crop field, the normally exposed image of the lesion area of ​​the crop field, and the overexposed image of the lesion area of ​​the crop field, layer by layer, extracting semantic features from the raw pixels, ranging from low-level (such as edges, color, and texture) to high-level (such as lesion shape and combination pattern). Ultimately, each feature extraction channel outputs a fixed-dimensional lesion area feature encoding vector, thereby converting the three lesion area images of different exposure levels into low-dimensional feature encoding representations containing their respective core pathological information: the lesion area feature encoding vector of the underexposed image, the lesion area feature encoding vector of the normally exposed image, and the lesion area feature encoding vector of the overexposed image. This lays a solid foundation for the subsequent effective fusion of multi-source information and pest and disease identification.

[0036] Specifically, step S52 performs dynamic feature compensation aggregation on the feature coding vectors for the lesion regions in the underexposed image, the feature coding vectors for the lesion regions in the normally exposed image, and the feature coding vectors for the lesion regions in the overexposed image to obtain a comprehensive representation vector for crop pests and diseases. It should be understood that simple static fusion strategies, such as feature concatenation or mean fusion, cannot adaptively prioritize the most effective information sources based on the actual conditions of the lesions to be identified, and may even reduce recognition accuracy by introducing invalid or interfering features. Therefore, in order to be able to adaptively select and emphasize the most discriminative features based on the input sample information, the present application proposes a feature dynamic compensation aggregation method, which performs feature baseline learning on the feature coding vector of the lesion area of ​​the underexposed image, the feature coding vector of the lesion area of ​​the normally exposed image, and the feature coding vector of the lesion area of ​​the overexposed image to mine the common feature patterns among the three, and further calculates the feature offsets of the three relative to the common feature patterns as a dynamic measurement indicator of feature importance, and performs attention aggregation on the three to form a compensatory feature representation relative to the common pest and disease feature pattern, thereby enhancing the recognition sensitivity of specific pest and disease types, thereby achieving comprehensive capture of the pest and disease characteristics of farmland crops by integrating the baseline pest and disease characteristics of farmland crops and the compensation features under different exposure conditions, and obtaining a comprehensive representation vector of the pest and disease characteristics of farmland crops.

[0037] Figure 5 FIG. 5 is a flow chart of step S52 in the agricultural pest monitoring and early warning method based on image processing according to an embodiment of the present application. Figure 5 As shown, the step S52 includes: S521, combining the underexposed image lesion area feature coding vector, the normally exposed image lesion area feature coding vector and the overexposed image lesion area feature coding vector into a multi-source feature set of farmland crop disease and insect pest feature coding vectors, and inputting the feature baseline learning network to obtain a farmland crop disease and insect pest feature baseline regression coding vector; S522, based on the feature offset of each farmland crop disease and insect pest feature coding vector in the multi-source feature set of the farmland crop disease and insect pest feature coding vector relative to the farmland crop disease and insect pest feature baseline regression coding vector, dynamically compensating the farmland crop disease and insect pest feature baseline regression coding vector to obtain the farmland crop disease and insect pest feature comprehensive representation vector.

[0038] In a specific example of the present application, step S521 can be expressed as follows:

[0039]

[0040]

[0041] in, 、 and They represent the feature coding vector of the lesion area in the underexposed image, the feature coding vector of the lesion area in the normally exposed image, and the feature coding vector of the lesion area in the overexposed image, respectively. A multi-source feature set representing the feature coding vector of crop diseases and insect pests, The first farmland crop pest and disease feature coding vectors, and =1,2,3; is the characteristic scale value of each farmland crop disease and insect pest characteristic coding vector, Learning a network for feature baselines, is the learnable parameter matrix, is the learnable bias vector, is the Sigmoid activation function, The baseline regression coding vector of the farmland crop disease and insect pest characteristics.

[0042] That is, since the lesion features under different exposure conditions carry complementary information and basic common patterns (such as the basic morphological structure of pests and diseases, etc.), in order to mine the stable essential representation of multi-source features, this application constructs a feature baseline learning network based on a deep neural network structure, combines the underexposed image lesion area feature coding vector, the normally exposed image lesion area feature coding vector, and the overexposed image lesion area feature coding vector into a multi-source feature set, inputs the feature baseline learning network for deep feature learning and regression, and mines the common feature patterns in the multi-source feature set to obtain the baseline regression coding vector of the farmland crop disease and pest feature, thereby establishing a common feature benchmark that is insensitive to light and providing a baseline anchor point for subsequent dynamic compensation.

[0043] In a specific example of the present application, step S522 includes: first, respectively calculating the characteristic dynamic compensation factor of each farmland crop disease and insect pest characteristic coding vector in the multi-source feature set of the farmland crop disease and insect pest characteristic coding vector relative to the farmland crop disease and insect pest characteristic baseline regression coding vector to obtain a set of farmland crop disease and insect pest characteristic dynamic compensation factors, which is expressed as follows:

[0044] in, represents the two-norm of the vector, Indicates point multiplication by position, represents the hyperbolic tangent function, for Dynamic compensation factors for crop disease and insect pest characteristics in farmland.

[0045] It should be understood that relying solely on the baseline regression coding vector of the crop disease and insect pest characteristics cannot fully characterize the specific discriminant information of each exposure source (such as the high-contrast texture at the edge of the lesion in the underexposed image), and directly classifying and identifying it will lose key detail information. Therefore, in order to quantify the discriminative offset of multi-source features relative to the baseline, this application evaluates the importance of the specific information of each crop disease and insect pest characteristic coding vector in the multi-source feature set relative to the baseline feature to calculate its characteristic dynamic compensation factor relative to the common feature baseline, thereby forming a set of dynamic compensation factors for crop disease and insect pest characteristics.

[0046] Next, the set of dynamic compensation factors for the characteristics of farmland crop diseases and insect pests is regularized to obtain a set of dynamic compensation weight factors for the characteristics of farmland crop diseases and insect pests. In a specific example of the present application, the set of dynamic compensation factors for the characteristics of farmland crop diseases and insect pests is regularized, which can be expressed as follows:

[0047]

[0048] in, represents the exponential function with base e, is the temperature coefficient, which is used to control the smoothness of the Softmax function. for Dynamic compensation weight factors for farmland crop disease and insect pest characteristics.

[0049] That is, considering that the original dynamic compensation factors of farmland crop disease and insect pest characteristics have magnitude differences and lack a measure of relative importance, using them directly as weights will lead to aggregation bias. Therefore, in order to achieve adaptive focusing of discriminative features, this application adopts a regularization process based on the Softmax function. By dividing each dynamic compensation factor of farmland crop disease and insect pest characteristics by the temperature coefficient and inputting it into the Softmax function for normalization operation, the original unconstrained compensation factor set is converted into a probability distribution with a sum of 1, and a set of dynamic compensation weight factors of farmland crop disease and insect pest characteristics is obtained. Among them, the feature source with a larger dynamic compensation weight factor of farmland crop disease and insect pest characteristics dominates the aggregation, and vice versa, it is suppressed, so that the network can adaptively focus on the important specific discriminant information.

[0050] In particular, in a preferred example of the present application, the set of dynamic compensation factors of the characteristics of farmland crop diseases and insect pests is regularized, including: first, each dynamic compensation factor of the characteristics of farmland crop diseases and insect pests in the set of dynamic compensation factors of the characteristics of farmland crop diseases and insect pests is respectively optimized based on the stability mapping of the characteristic baseline deviation constraint to obtain a set of optimized dynamic compensation factors of the characteristics of farmland crop diseases and insect pests. Specifically, here, considering that each dynamic compensation factor of the characteristics of farmland crop diseases and insect pests is evaluated, Regression encoding vector relative to the established baseline of crop pest and disease characteristics When the contribution difference is To feature baseline With smooth mapping, the characteristic coding vectors of each farmland crop disease and insect pest are The calculated dynamic compensation weight factors of crop disease and insect pest characteristics are universal.

[0051] Based on this, we first target the characteristic coding vectors of crop diseases and insect pests To the baseline regression coding vector of crop disease and insect pest characteristics The baseline shift change is obtained by mapping the deviation of

[0052]

[0053] in, represents the partial derivative, for The corresponding baseline offset gradient vector of the characteristics of crop diseases and insect pests in farmland;

[0054] Then, the baseline gradient vector is offset by the crop pest and disease characteristic Constructing a correlation change matrix of characteristic metrics of crop diseases and insect pests in farmland , wherein the baseline offset gradient vector of the crop pest and disease characteristic is is a column vector.

[0055] That is, if the above encoding vector based on the characteristics of crop diseases and insect pests is To the baseline regression coding vector of crop disease and insect pest characteristics The distance mapping differential term is used as a parameterized mapping, which will drive the mapping evolution through the baseline shift change. Thus, by solving the baseline shift gradient vector of the crop pest and disease feature , and construct the correlation change matrix of the characteristic metrics of crop diseases and insect pests To maintain structural conservation, the change matrix can be associated with the characteristic metric of crop diseases and insect pests in farmland to dominate the mapping robustness.

[0056] Therefore, the farmland crop pest and disease characteristic metric correlation change matrix is ​​further calculated The L2 norm of , i.e., the compression stability characterization, is used as the stability coefficient in the calculation of the compensation factor to enhance the universality of the dynamic compensation factor of the characteristics of crop diseases and insect pests, namely:

[0057]

[0058] in, express The corresponding optimized dynamic compensation factor of farmland crop disease and insect pest characteristics is realized. Based on this, the farmland crop disease and insect pest characteristic coding vector is realized. To feature baseline Subsequently, the set of optimized dynamic compensation factors for crop pests and diseases characteristics is regularized using a Softmax activation function to obtain a set of dynamic compensation weight factors for crop pests and diseases characteristics, thereby further improving the robustness and universal accuracy of the dynamic compensation weight factors for crop pests and diseases characteristics.

[0059] Then, based on the set of dynamic compensation weight factors of the farmland crop disease and pest characteristics and the multi-source feature set of the farmland crop disease and pest characteristic coding vector, the farmland crop disease and pest characteristic baseline regression coding vector is dynamically compensated to obtain the farmland crop disease and pest characteristic comprehensive representation vector. More specifically, first, based on the set of dynamic compensation weight factors of the farmland crop disease and pest characteristics, the multi-source feature set of the farmland crop disease and pest characteristic coding vector is weightedly aggregated to obtain the farmland crop disease and pest characteristic baseline compensation coding vector; then, the positional sum between the farmland crop disease and pest characteristic baseline compensation coding vector and the farmland crop disease and pest characteristic baseline regression coding vector is calculated to obtain the farmland crop disease and pest characteristic comprehensive representation vector, which is expressed as follows:

[0060]

[0061] in, It is a comprehensive representation vector of the characteristics of crop diseases and pests.

[0062] That is, the robust common baseline characteristics of farmland crop diseases and pests are used as the skeleton, and the dynamic compensation weight factors of each farmland crop disease and pest characteristic are used to dynamically weighted aggregate the disease and pest characteristics in each exposure source, so as to enrich and correct the baseline skeleton using the importance-weighted multi-exposure source specific information, which not only retains the overall stable trend of the data, but also keenly captures and incorporates the unique contribution and relative importance of each independent information source, thereby achieving a comprehensive and accurate description of the characteristics of farmland crop diseases and pests, and obtaining a comprehensive representation vector of the characteristics of farmland crop diseases and pests.

[0063] Specifically, in step S53, the comprehensive characterization vector of the characteristics of the crop diseases and insect pests is input into the trained disease and insect pest classifier to obtain the identification result of the disease and insect pest type. That is, in order to convert the comprehensive characterization vector of the characteristics of the crop diseases and insect pests into a specific disease and insect pest category conclusion, the present application is based on the classification principle in supervised learning and constructs a disease and insect pest classification decision module based on a deep neural network to achieve accurate mapping from abstract feature representation to disease and insect pest category space. Specifically, first, the comprehensive characterization vector of the characteristics of the crop diseases and insect pests is input into the trained disease and insect pest classifier, and the classifier is composed of multiple fully connected layers, and the last layer is a Softmax activation function layer. The fully connected layer is responsible for performing nonlinear transformation and deep feature extraction on the comprehensive characterization vector of the characteristics of the crop diseases and insect pests, while the Softmax layer converts the output of the network into a probability distribution, in which each value represents the predicted probability of the corresponding disease and insect pest category. Furthermore, throughout the model's training phase, the entire process—from pest and disease feature extraction based on the MobileNet model, to dynamic feature compensation aggregation, and finally feature classification—is jointly optimized end-to-end. The goal is to ensure that the probability distribution output by the model for training samples is highly consistent with the true disease labels. In practical applications, the classifier outputs the probabilities of all predefined pest and disease categories, and the category with the highest probability is determined as the final identification result, thereby enabling accurate diagnosis of crop pests and diseases and providing practical early warning information for crop management.

[0064] In summary, according to the embodiment of the present application, the agricultural pest and disease monitoring and early warning method based on image processing is explained, which continuously collects under-exposed, normally exposed and over-exposed images of farmland crops through a camera, and aligns and combines the three into high dynamic range image data that fully retains the highlights and shadow details. Subsequently, target detection is performed on the high dynamic range image data to generate a binary mask map of the suspicious lesion area, and the binary mask map is further used to extract the lesion area images from the three original images with different exposures. The three groups of multi-source suspicious lesion images containing complementary information are subjected to deep pest and disease feature extraction and multi-source feature dynamic compensation aggregation to form a comprehensive and integrated representation of the pest and disease characteristics, thereby achieving fine identification of the pest and disease types on this basis. This method can significantly improve the accuracy and robustness of agricultural pest and disease monitoring under complex natural lighting conditions through the comprehensive utilization of multi-exposure image data.

[0065] Furthermore, the present application also provides an agricultural pest and disease monitoring and early warning system based on image processing.

[0066] Figure 6 FIG is a block diagram of an agricultural pest monitoring and early warning system based on image processing according to an embodiment of the present application. Figure 6As shown, according to the embodiment of the present application, the agricultural pest and disease monitoring and early warning system 100 based on image processing includes: a multi-exposure image acquisition module 110, which is used to use a camera to continuously collect under-exposed image data, normally exposed image data and over-exposed image data of farmland crops; an image alignment and combination module 120, which is used to align the under-exposed image data, the normally exposed image data and the over-exposed image data and arrange them into high dynamic range image data according to channel dimensions; a pest and disease target detection module 130, which is used to perform pest and disease target detection on the high dynamic range image data to obtain A binary mask map of the suspected lesion area; a suspicious lesion area extraction module 140, used to extract under-exposed images of the farmland crop lesion area, normally-exposed images of the farmland crop lesion area, and over-exposed images of the farmland crop lesion area from the high dynamic range image data based on the binary mask map of the suspected lesion area; a pest and disease type identification module 150, used to perform pest and disease type identification on the under-exposed images, normally-exposed images, and over-exposed images of the farmland crop lesion area based on dynamic compensation aggregation of multi-source information to obtain a pest and disease type identification result.

[0067] Here, those skilled in the art will appreciate that the specific operations of each module in the agricultural pest monitoring and early warning system based on image processing have been described in the above. Figures 1 to 5 The description of the agricultural pest and disease monitoring and early warning method based on image processing has been introduced in detail, and therefore, its repeated description will be omitted.

[0068] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not limiting. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for monitoring and early warning of agricultural pests and diseases based on image processing, characterized in that: include: Use a camera to continuously collect underexposed image data, normally exposed image data, and overexposed image data of farmland crops; Performing image alignment on the underexposed image data, the normally exposed image data, and the overexposed image data, and then arranging them according to channel dimensions into high dynamic range image data; Performing pest and disease target detection on the high dynamic range image data to obtain a binary mask image of a suspicious lesion area; Based on the binary mask image of the suspicious lesion area, extracting an underexposed image of the farmland crop lesion area, a normally exposed image of the farmland crop lesion area, and an overexposed image of the farmland crop lesion area from the high dynamic range image data; performing pest and disease type identification based on multi-source information dynamic compensation aggregation on the underexposed image of the farmland crop lesion area, the normally exposed image of the farmland crop lesion area, and the overexposed image of the farmland crop lesion area to obtain a pest and disease type identification result; Performing image alignment on the underexposed image data, the normally exposed image data, and the overexposed image data and arranging them according to channel dimensions into high dynamic range image data includes: Calculating the affine transformation matrices of the underexposed image data and the overexposed image data relative to the normally exposed image data based on the SIFT algorithm to obtain an underexposed image affine transformation matrix and an overexposed image affine transformation matrix; performing affine transformation on the underexposed image data and the overexposed image data based on the underexposed image affine transformation matrix and the overexposed image affine transformation matrix to obtain aligned underexposed image data and aligned overexposed image data; Arranging the aligned underexposed image data, the normally exposed image data, and the aligned overexposed image data according to a channel dimension to obtain the high dynamic range image data; The underexposed image of the farmland crop lesion area, the normally exposed image of the farmland crop lesion area, and the overexposed image of the farmland crop lesion area are subjected to pest and disease type identification based on dynamic compensation aggregation of multi-source information to obtain a pest and disease type identification result, including: Extracting pest and disease features from the underexposed image of the crop lesion area, the normally exposed image of the crop lesion area, and the overexposed image of the crop lesion area respectively to obtain a feature coding vector of the underexposed image lesion area, a feature coding vector of the normally exposed image lesion area, and a feature coding vector of the overexposed image lesion area; Performing feature dynamic compensation aggregation on the feature coding vector of the lesion area of ​​the underexposed image, the feature coding vector of the lesion area of ​​the normally exposed image, and the feature coding vector of the lesion area of ​​the overexposed image to obtain a comprehensive representation vector of the characteristics of crop diseases and insect pests; The comprehensive representation vector of the characteristics of crop diseases and insect pests is input into the trained disease and insect pest classifier to obtain the identification result of the disease and insect pest type.

2. The agricultural pest monitoring and early warning method based on image processing according to claim 1 is characterized in that: Performing pest and disease target detection on the high dynamic range image data to obtain a binary mask image of a suspicious lesion area includes: The high dynamic range image data is input into a pest and disease target detection model based on the Mask R-CNN network to obtain a binary mask map of the suspicious lesion area, wherein the Mask R-CNN network includes a feature extraction module, a candidate region generation module, a candidate region classification and regression module, and a mask generation module.

3. The agricultural pest monitoring and early warning method based on image processing according to claim 1 is characterized in that: Extracting pest and disease features of the underexposed image of the farmland crop lesion area, the normally exposed image of the farmland crop lesion area, and the overexposed image of the farmland crop lesion area respectively to obtain a feature coding vector of the underexposed image lesion area, a feature coding vector of the normally exposed image lesion area, and a feature coding vector of the overexposed image lesion area, including: The underexposed image of the farmland crop lesion area, the normally exposed image of the farmland crop lesion area and the overexposed image of the farmland crop lesion area are respectively subjected to pest and disease feature extraction based on the MobileNet model to obtain the feature coding vector of the underexposed image lesion area, the feature coding vector of the normally exposed image lesion area and the feature coding vector of the overexposed image lesion area.

4. The agricultural pest monitoring and early warning method based on image processing according to claim 3 is characterized in that: Performing feature dynamic compensation aggregation on the feature coding vector of the lesion area of ​​the underexposed image, the feature coding vector of the lesion area of ​​the normally exposed image, and the feature coding vector of the lesion area of ​​the overexposed image to obtain a comprehensive representation vector of the characteristics of crop diseases and insect pests, including: Combining the underexposed image lesion area feature coding vector, the normally exposed image lesion area feature coding vector, and the overexposed image lesion area feature coding vector into a multi-source feature set of farmland crop disease and insect pest feature coding vectors, and inputting the multi-source feature set into a feature baseline learning network to obtain a farmland crop disease and insect pest feature baseline regression coding vector; Based on the feature offset of each farmland crop disease and insect pest feature coding vector in the multi-source feature set of the farmland crop disease and insect pest feature coding vector relative to the farmland crop disease and insect pest feature baseline regression coding vector, the farmland crop disease and insect pest feature baseline regression coding vector is dynamically compensated to obtain the comprehensive representation vector of the farmland crop disease and insect pest feature.

5. The agricultural pest monitoring and early warning method based on image processing according to claim 4 is characterized in that: Based on the feature offset of each farmland crop disease and insect pest feature coding vector in the multi-source feature set of the farmland crop disease and insect pest feature coding vector relative to the farmland crop disease and insect pest feature baseline regression coding vector, dynamically compensating the farmland crop disease and insect pest feature baseline regression coding vector to obtain the farmland crop disease and insect pest feature comprehensive representation vector, including: respectively calculating a characteristic dynamic compensation factor of each farmland crop disease and insect pest characteristic coding vector in the multi-source feature set of the farmland crop disease and insect pest characteristic coding vector relative to the farmland crop disease and insect pest characteristic baseline regression coding vector to obtain a set of farmland crop disease and insect pest characteristic dynamic compensation factors; Regularizing the set of dynamic compensation factors for crop disease and insect pest characteristics to obtain a set of dynamic compensation weight factors for crop disease and insect pest characteristics; Based on the set of dynamic compensation weight factors of the farmland crop disease and pest characteristics and the multi-source feature set of the farmland crop disease and pest characteristic coding vector, the farmland crop disease and pest characteristic baseline regression coding vector is dynamically compensated to obtain the farmland crop disease and pest characteristic comprehensive representation vector.

6. The agricultural pest monitoring and early warning method based on image processing according to claim 5 is characterized in that: Regularization processing is performed on the set of dynamic compensation factors of farmland crop disease and insect pest characteristics to obtain a set of dynamic compensation weight factors of farmland crop disease and insect pest characteristics, including: performing stability mapping optimization based on characteristic baseline deviation constraints on each of the dynamic compensation factors of the farmland crop disease and insect pest characteristics in the set of dynamic compensation factors of the farmland crop disease and insect pest characteristics to obtain a set of optimized dynamic compensation factors of the farmland crop disease and insect pest characteristics; The set of optimized dynamic compensation factors of farmland crop disease and insect pest characteristics is subjected to regularization processing based on a Softmax activation function to obtain a set of dynamic compensation weight factors of farmland crop disease and insect pest characteristics.

7. The agricultural pest monitoring and early warning method based on image processing according to claim 6 is characterized in that: Based on the set of dynamic compensation weight factors of the farmland crop disease and insect pest characteristics and the multi-source feature set of the farmland crop disease and insect pest characteristic coding vector, dynamically compensating the farmland crop disease and insect pest characteristic baseline regression coding vector to obtain the farmland crop disease and insect pest characteristic comprehensive representation vector, including: Based on the set of dynamic compensation weight factors of the farmland crop disease and insect pest characteristics, weighted aggregation is performed on the multi-source feature set of the farmland crop disease and insect pest characteristic coding vector to obtain a farmland crop disease and insect pest characteristic baseline compensation coding vector; The position-based sum between the baseline compensation coding vector of the farmland crop disease and insect pest characteristics and the baseline regression coding vector of the farmland crop disease and insect pest characteristics is calculated to obtain the comprehensive representation vector of the farmland crop disease and insect pest characteristics.

8. An agricultural pest monitoring and early warning system based on image processing, used to execute the method according to any one of claims 1 to 7, characterized in that: include: A multi-exposure image acquisition module is used to continuously acquire under-exposure image data, normal-exposure image data, and over-exposure image data of farmland crops using a camera; an image alignment and combination module, configured to align the underexposed image data, the normally exposed image data, and the overexposed image data, and then arrange them into high dynamic range image data according to channel dimensions; a pest and disease target detection module, configured to perform pest and disease target detection on the high dynamic range image data to obtain a binary mask image of a suspicious lesion area; a suspicious lesion region extraction module, configured to extract an underexposed image of the farmland crop lesion region, a normally exposed image of the farmland crop lesion region, and an overexposed image of the farmland crop lesion region from the high dynamic range image data based on the binary mask map of the suspicious lesion region; The pest and disease type identification module is used to identify the pest and disease type based on dynamic compensation aggregation of multi-source information on the underexposed image of the farmland crop disease area, the normally exposed image of the farmland crop disease area and the overexposed image of the farmland crop disease area to obtain the identification result of the pest and disease type.

Images