A method and system for detecting plant diseases and pests in a greenhouse

Abstract

The application relates to the technical field of intelligent agriculture, and particularly relates to a greenhouse crop automatic detection method, in particular to a greenhouse crop disease and pest detection method and system. According to the detection data of different regions, the embodiment provided by the application judges the growth deviation degree of crops in the region, configures a corresponding sampling strategy based on different deviation degrees, obtains real-time images of the crops in the region based on the corresponding sampling strategy, extracts features of the real-time images through a detection model, and determines whether the crops in the current region are diseased according to the features. Compared with the prior art, the method provided by the embodiment can configure different detection strategies according to the growth states of different regions, so that the detection accuracy is ensured while the detection cost is reduced.

Description

Technical Field

[0001] This application relates to the field of smart agriculture technology, specifically to an automated detection method for crops in greenhouses, and more specifically to a method and system for detecting crop diseases and pests in greenhouses. Background Technology

[0002] In crop production, pests and diseases are significant factors affecting crop growth and yield. However, due to technological limitations, the occurrence and severity of pests and diseases are typically assessed manually. This method is insufficient for comprehensively understanding crop conditions and may lead to missed opportunities for optimal control. Therefore, utilizing computer technology for crop pest and disease management has become a crucial technological direction in crop pest and disease detection.

[0003] Therefore, overcoming the shortcomings of the existing technology is an urgent problem to be solved in this technical field. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a method and system for detecting crop diseases and pests in greenhouses. Based on the growth status of different crops within the detection range, corresponding detection strategies are formulated for different areas. Image information of the crops in that area is acquired according to the corresponding detection strategy. The presence of diseases and pests is then determined based on the image information, thereby achieving automated crop monitoring while reducing monitoring costs. To achieve the above objectives, the technical solution adopted in this invention application is as follows:

[0005] In a first aspect, a method for detecting crop diseases and pests in greenhouses is provided. The method includes: determining the degree of crop growth deviation in each region based on multiple detection data corresponding to crops in multiple regions within a unit time period; using the multiple detection data to characterize the environmental state in each region; determining a sampling strategy for each region based on the overall deviation degree of each region, and determining the image acquisition time point for each region based on the sampling strategy, and acquiring real-time images of crops corresponding to the image acquisition time point for each region; acquiring crop images of relevant crops in the real-time images, determining image features in the crop images, and determining whether the crop has diseases and pests and the classification result of diseases and pests based on the distribution state of the image features.

[0006] In some specific implementations, the method further includes: when the crop individual image in any region has the feature expression of pests and diseases, updating the sampling strategy of each region, acquiring updated real-time images in other regions, and determining whether the crop in the corresponding region has pests and diseases and the pest and disease classification result based on the updated real-time images.

[0007] In some specific implementations, the various detection data include ambient humidity data, ambient temperature data, soil moisture data, light intensity data, and carbon dioxide data.

[0008] In some specific implementations, the degree of crop growth deviation in each region is determined based on multiple detection data within each region, including: acquiring the degree of deviation corresponding to each detection data, and determining the overall degree of deviation in each region as the degree of crop growth deviation based on the degree of deviation of each detection data.

[0009] In some specific implementations, obtaining the degree of deviation corresponding to each detection data includes: comparing each detection data with the corresponding standard data to determine the difference of each data as the degree of deviation; determining the overall degree of deviation of each region as the degree of crop growth deviation based on the degree of deviation of each detection data includes: updating the weight of the difference of each detection data to obtain the updated deviation, and determining the overall degree of deviation as the degree of crop growth deviation based on the updated deviation corresponding to each detection data.

[0010] In some specific implementations, determining the degree of crop growth deviation in each region based on multiple detection data within each region includes: extracting features from the multiple detection data corresponding to each region to obtain the features corresponding to each detection data; splicing the multiple features to obtain spliced ​​features; obtaining the estimated growth status of the crop in each region based on the spliced ​​features; and comparing the estimated growth status with the growth standard value to obtain the degree of growth deviation in each region.

[0011] In some specific implementations, obtaining crop images related to crops from the real-time image includes: obtaining the pixel distribution state on the grayscale image corresponding to the real-time image, and segmenting the crop-related image according to the pixel edge features of the crop to obtain the crop image; determining the image features in the crop image includes: obtaining multiple scale features under different receptive field conditions, and fusing the multiple scale features to obtain fused features as image features.

[0012] In some specific implementations, determining whether a crop has pests and diseases and the classification result of pests and diseases based on the feature distribution state of the image features includes: segmenting the image features into multiple sub-feature maps, obtaining the dynamic weight corresponding to each sub-feature map, updating each sub-feature map according to the dynamic weight, merging the updated multiple sub-feature maps to obtain multiple intermediate feature maps, and obtaining the bounding box coordinate prediction result and the category probability prediction result corresponding to the features in each intermediate feature map.

[0013] In some specific implementations, the dynamic weights corresponding to each sub-feature map are obtained, and each sub-feature map is updated according to the dynamic weights. This includes: performing dimensionality increase and dimensionality decrease processing on each sub-feature map in sequence, obtaining the spatial attention weights of each sub-feature map based on the spatial attention module, and obtaining the updated sub-feature map by weighted product processing based on the spatial attention weights.

[0014] Secondly, a greenhouse crop pest and disease detection system is provided. The system includes a data acquisition unit and a data processing unit. The data acquisition unit includes multiple sets of growth data acquisition devices and multiple image acquisition devices. The multiple sets of growth data acquisition devices include multiple growth data acquisition modules. Each set of growth data acquisition devices is used to acquire multiple types of detection data corresponding to each region. Each image acquisition device is used to acquire a real-time image corresponding to each region. The data processing unit is used to receive the detection data and real-time images and execute the method described in any one of the above, including: a deviation determination module, used to determine the degree of crop generation deviation corresponding to each region based on the acquired multiple detection data of crops in multiple regions within a unit time; a strategy determination module, used to determine the sampling strategy of each region based on the overall deviation degree of each region, and determine the image acquisition time point of each region based on the sampling strategy, and acquire the real-time image of the crop corresponding to the image acquisition time point of each region; and an identification module, used to acquire individual crop images in the real-time images, determine the image features in each individual crop image, and determine whether the crop has pests and diseases and the classification result of the pests and diseases based on the image features.

[0015] The technical solution provided in this application determines the degree of crop growth deviation in a given region based on detection data from different areas. A corresponding sampling strategy is configured based on different deviation levels, and real-time images of the relevant crops in that region are acquired based on the corresponding sampling strategy. A detection model is then used to extract features from the real-time images, and the features are used to identify whether the crops in the current region are diseased. Compared to existing technologies, the method provided in this application can configure different detection strategies based on the growth status of different regions, thereby reducing detection costs while ensuring detection accuracy. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] The methods, systems, and / or procedures shown in the accompanying drawings will be further described with reference to exemplary embodiments. These exemplary embodiments will be described in detail with reference to the drawings. These exemplary embodiments are non-limiting exemplary embodiments, wherein example figures represent similar mechanisms in the various views of the drawings.

[0018] Figure 1 This is a schematic diagram of the structure of the greenhouse crop pest and disease detection system provided in the embodiments of this application.

[0019] Figure 2 This is a schematic diagram of the greenhouse crop pest and disease detection method provided in the embodiments of this application.

[0020] Figure 3 This is a schematic diagram of the detection results provided in the embodiments of this application.

[0021] Figure 4 This is a schematic diagram of the data processing unit structure provided in the embodiments of this application.

[0022] Figure 5 This is a schematic diagram of the terminal device structure provided in the embodiments of this application. Detailed Implementation

[0023] To better understand the above technical solutions, the technical solutions of this application will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of this application and the specific features in the embodiments are detailed descriptions of the technical solutions of this application, rather than limitations on the technical solutions of this application. In the absence of conflict, the embodiments of this application and the technical features in the embodiments can be combined with each other.

[0024] In the detailed description below, numerous specific details are illustrated with examples to provide a comprehensive understanding of the relevant guidance. However, it will be apparent to those skilled in the art that this application can be practiced without these details. In other instances, well-known methods, procedures, systems, components, and / or circuits have been described at a relatively high level without detail to avoid unnecessarily obscuring aspects of this application.

[0025] This application uses flowcharts to illustrate the execution process performed by a system according to embodiments of this application. It should be clearly understood that the execution processes in the flowcharts may not be executed sequentially. Instead, these execution processes may be executed in reverse order or simultaneously. Additionally, at least one other execution process may be added to the flowchart. One or more execution processes may be deleted from the flowchart.

[0026] Before providing a further detailed description of the embodiments of the present invention, the nouns and terms involved in the embodiments of the present invention will be explained, and the nouns and terms involved in the embodiments of the present invention shall be interpreted as follows.

[0027] (1) In response to, used to indicate the conditions or states on which the operation is performed depends, when the conditions or states on which it depends are met, the one or more operations performed may be performed in real time or may have a set delay; unless otherwise specified, there is no restriction on the order of execution of the multiple operations performed.

[0028] (2) Based on, used to indicate the conditions or states on which the operation is performed depends. When the conditions or states on which it depends are met, one or more operations can be performed in real time or with a set delay. Unless otherwise specified, there is no restriction on the order of execution of the multiple operations.

[0029] In traditional agricultural greenhouses, operators typically rely on subjective experience to assess the match between the cultivation environment and crop growth requirements, and then implement manual control measures. This traditional management model has significant drawbacks. First, a large labor force is required per unit area to maintain suitable crop growth conditions, resulting in high operating costs. Second, manual inspection cannot achieve dynamic perception and real-time feedback of environmental parameters, easily leading to lags in environmental control.

[0030] With increasing levels of intelligence and advancements in artificial intelligence, especially image processing technology, coupled with reduced costs of intelligent equipment, more and more agricultural technologies are adopting intelligent devices for comprehensive monitoring and management of agricultural production processes. The development of greenhouses has also spurred technological iteration, with wireless sensor network technology and artificial intelligence control being successively applied to agricultural modernization. Intelligent management of greenhouses can be categorized into growth management methods, irrigation and planting control methods, and pest and disease monitoring methods. The technical solution in this application embodiment is a pest and disease monitoring method used to monitor and identify pests and diseases affecting crops within a greenhouse. Crops are susceptible to varying degrees of disease during cultivation, and severe pest and disease infestations can lead to reduced yields or even total crop failure. In the past, pest and disease detection relied primarily on visual observation and experience, but early-stage diseases are often subtle, and by the time they are detected, they have already spread.

[0031] With the popularization of artificial intelligence and intelligent devices, target detection technology is increasingly used in existing technologies to monitor the health status of crops, especially image processing technology based on deep learning. However, although deep learning algorithms are mature and can be applied to pest and disease detection, several technical problems still exist in specific greenhouse applications. Currently, the mainstream deep learning-based pest and disease detection methods typically involve setting up cameras to collect images of the crops to be inspected, and then processing these images to obtain information on pest and disease lesions and their types. To ensure the completeness of the detection, a high density of cameras is usually set up for continuous image acquisition and processing. This approach ensures timely and accurate monitoring, but the high setup and continuous processing costs are high, far exceeding the savings in labor costs. To address this issue, another solution exists: using drones for inspection, collecting and processing images of the crops during the inspection process. While this method solves the cost problem of high-density camera deployment, drone inspection requires a separate control system, and the drone control also needs to be highly precise, especially for crops with high foliage density. Although this system reduces the cost of high-density cameras, its control cost and control precision are higher than those of fixed-position camera acquisition solutions.

[0032] Therefore, while the two commonly used technical solutions in the prior art can achieve intelligent detection of crop pests and diseases in greenhouses, in practical scenarios, these solutions are based on high deployment and control costs. Therefore, to solve the above technical problems, this application provides a greenhouse crop pest and disease detection system 10 for intelligent control of greenhouses. For more information on this system, please refer to [link to relevant documentation]. Figure 1 It includes a data acquisition unit 11 and a data processing unit 12, and the data acquisition unit and the data processing unit transmit data through a communication unit.

[0033] The data acquisition unit is used to obtain relevant crop data, while the data processing unit is used to process the data to achieve the detection and identification of pests and diseases. Typically, the acquired data is image data, and detection and identification are achieved through image processing. However, as mentioned in the background section above, directly processing the images would require acquiring an image of each individual crop. To address the high cost of this method, this embodiment employs a data acquisition strategy that acquires images based on the crop's growth status and the corresponding likelihood of pests and diseases, and then uses the acquired images for pest and disease detection and identification.

[0034] Specifically, in this embodiment, the data acquisition unit includes multiple sets of growth data acquisition devices 111 and multiple image acquisition devices 112 configured corresponding to the growth data acquisition devices. Each set of growth data acquisition devices is located in a different area within the greenhouse; for example, in one possible implementation, six sets of growth data acquisition devices are provided, located in the four corner areas and the two central areas of the greenhouse, forming six monitoring zones. These zones are non-overlapping. Each set of growth data acquisition devices includes multiple growth data acquisition modules for acquiring crop growth data within that area.

[0035] In this embodiment, the number of regions is set according to the crop growth patterns and the specific space area of ​​the greenhouse. This means that the number of regions varies depending on the crop and the greenhouse, and each region should consist of a set of crops with the same growth process. Specifically, the growth data acquisition module in this embodiment includes an environmental humidity acquisition module, an environmental temperature acquisition module, a soil moisture acquisition module, a light intensity acquisition module, and a carbon dioxide acquisition module. Specifically, because the temperature distribution differs significantly between the upper part of the crop and the lower part of the root region, the environmental humidity and environmental temperature acquisition modules are respectively located above and below the center of each region. That is, each region has two environmental humidity acquisition modules and two environmental temperature acquisition modules, located at the top and bottom of the region, respectively. The soil moisture acquisition module is located at the center of each region to obtain the soil moisture status. The light intensity acquisition module is located in the upper part of each region to monitor changes in light intensity in real time. The carbon dioxide acquisition module is also located in the upper part of each region to obtain carbon dioxide concentration data for each region.

[0036] In this embodiment, the above-described modules can acquire multiple detection data points within each region, collectively referred to as growth data. The image acquisition device is used to acquire real-time images corresponding to each region. This image acquisition device can be a camera positioned in the upper part of each region, or it can be a camera positioned in the side part of each region. The specific location of the image acquisition device will not be elaborated upon in this embodiment, as long as it can acquire images of the relevant crops within that region.

[0037] In this embodiment, the data processing unit communicates with the data acquisition unit through the communication unit, receives multiple growth data, determines the growth deviation corresponding to each region based on the multiple growth data, determines the acquisition strategy for each region based on the degree of growth deviation, acquires real-time images corresponding to each region based on the acquisition strategy, and determines whether the region has pests or diseases based on the real-time images.

[0038] Specifically, regarding the data processing unit's execution of the greenhouse crop pest and disease detection method provided in this application embodiment, the processing procedure for this method can be found in [reference needed]. Figure 2 As shown, the specific steps include:

[0039] Step S21. Based on the multiple detection data corresponding to crops in multiple areas within the greenhouse per unit time, determine the degree of crop growth deviation corresponding to each area.

[0040] In this embodiment, multiple detection data are used to characterize the environmental state of each area, which are data collected by multiple growth data acquisition modules, including environmental humidity data, environmental temperature data, soil moisture data, light intensity data, and carbon dioxide data.

[0041] The degree of growth deviation reflects the growth level of crops within the current area. Deviation refers to the difference between the current growth level and the standard or preset growth level, including growth exceeding or falling short of expectations. Generally, the growth level of crops is determined based on their growth status, such as the growth of branches and leaves, and the ripening of fruits. However, obtaining this data requires manual observation and experience, and the actual growth outcome is determined by the plant variety and environmental factors. While the crop variety is controllable, the growth status depends on environmental factors. This means that changes in the environment or deviations in environmental data will affect the final crop growth. Crops in favorable environmental conditions tend to grow better than those in relatively unfavorable conditions. In this embodiment, pests and diseases, especially diseases, are more likely to occur in crops with poor growth. When crops are disturbed by adverse environmental factors, their metabolic processes are disrupted and affected, resulting in physiological and tissue structural lesions and morphological abnormalities. Furthermore, a lack of essential nutrients can also lead to a decrease in the crop's resistance, such as a lack of necessary light, temperature, and water.

[0042] Therefore, this application provides a pest and disease detection method that reduces sampling costs and achieves high accuracy, based on the patterns of crop growth. Specifically, it acquires the crop growth status within each region and generates a predictive mapping between the growth status and the probability of disease, determining a sampling strategy based on this mapping. This method enables the assessment of crop growth status and the prediction of pest and disease probability by acquiring commonly used monitoring data, and allows for corresponding sampling based on the probability of pest and disease, thereby reducing sampling costs while ensuring detection accuracy.

[0043] In this embodiment, the degree of crop growth deviation is determined by acquiring the degree of deviation corresponding to each detection data, determining the overall degree of deviation for each region based on the degree of deviation of each detection data, and using this overall degree of deviation as the degree of crop growth deviation.

[0044] Specifically, the deviation for each test data point refers to the difference between that test data and the standard data. The test data is time-series data acquired within a time period according to the sampling time, including data from multiple time points. Furthermore, the sampling time for the test data is determined based on the crop's growth condition. This can be understood as setting multiple time sampling points according to different plants and their corresponding growth patterns, acquiring data from different time points within different time periods, and then forming the corresponding time-series data. The standard data is also the standard data at the corresponding time sampling points.

[0045] Then, the deviation between the time-series data and the standard data is calculated to obtain the data deviation corresponding to each time point, thus obtaining the deviation sequence corresponding to this detection data. Different weights are assigned to the degree of data deviation at different time points; for example, the weight of carbon dioxide data at night is lower than that during the day. Therefore, to accurately represent the overall deviation degree corresponding to each detection data point, this embodiment assigns weights to different sampling time points when obtaining the deviation degree of each detection data point. Then, based on the weight corresponding to each sampling time point, the difference between the sampling time points is updated, and a weighted sum is performed to obtain the deviation result of each detection data point within that time period. The sum of the weights in the detection data sequence is 1.

[0046] In this embodiment, the deviation degree of each detection data corresponding to each region can be obtained through the above results. Since the expression of crop growth status is based on multiple detection data, this embodiment also needs to integrate the deviation degrees of the multiple detection data corresponding to each region to obtain the overall deviation degree as the crop growth deviation degree.

[0047] Specifically, the calculation of the degree of deviation in each of the aforementioned detection data is the same. In this embodiment, the determination of the overall deviation degree for each region is achieved by updating the deviation degree of each detection data based on its corresponding weight, obtaining an updated deviation, and then integrating the updated deviations corresponding to each detection data to obtain the overall deviation as the degree of crop growth deviation. In this embodiment, the weights are obtained based on linear fitting; this process employs a fitting approach from existing technologies and will not be elaborated upon further in this embodiment.

[0048] As mentioned above, the degree of crop growth deviation in this embodiment can be either exceeding the standard value or falling below the standard value.

[0049] In this embodiment, the degree of deviation corresponding to each sampling point in each detection data is obtained by assigning weights, and then the degree of deviation corresponding to each region is obtained by assigning weights to each detection data. This method can quickly obtain the crop growth deviation corresponding to each region, providing support for determining the subsequent image sampling strategy. Furthermore, in this embodiment, the growth deviation is obtained by constructing a mapping relationship based on environmental data related to crop growth, and the crop growth deviation is determined based on the environmental data.

[0050] In another possible implementation, a method for determining the degree of deviation is also provided, employing a different technical approach than the process described above. Specifically, this embodiment includes an estimation model that, by acquiring multiple detection data corresponding to each region, can infer an estimated value of the growth state of that region within a given time period. This estimated value of the growth state is then compared with a standard growth value to obtain the degree of growth deviation for each region.

[0051] The estimation model in this process first extracts features from multiple detection data points corresponding to each region, obtaining the features corresponding to each detection data point. Then, these multiple features are concatenated to obtain concatenated features. The concatenated features are then updated using a multi-channel attention mechanism to obtain the updated target features. Finally, the probability of the corresponding estimated value is calculated based on the target features, and the estimated value with the highest probability is output as the target, which is the growth state estimate.

[0052] In this embodiment, feature extraction is implemented through a feature extraction module. Specifically, the feature extraction module includes a first depthwise separable convolution, a second depthwise separable convolution, a third depthwise separable convolution, a max pooling layer, and a dual attention mechanism. The first, second, and third depthwise separable convolutions each have different kernels; in this embodiment, they are 3, 5, and 1 kernels, respectively. The input detection data undergoes feature extraction through these convolutions, and the resulting features are then concatenated to obtain the initial features of the convolution output. This design aims to capture feature information at different scales, thereby improving the perception capability for features at different scales.

[0053] The initial features obtained after feature extraction are processed by a max pooling layer and then updated with feature weights based on a dual attention mechanism. This enhances the focus on key features of each detection data point, thereby improving the ability to capture and utilize important information. In this embodiment, the dual attention mechanism combines spatial attention and channel attention. Specifically, spatial attention scores and channel attention scores corresponding to the initial features are obtained through spatial attention and channel attention mechanisms, respectively.

[0054] Then, the initial features corresponding to the acquired detection data are concatenated to obtain concatenated features. The concatenation process discards features based on the acquired spatial attention score and channel attention score. Specifically, features with spatial attention scores and channel attention scores below a set threshold are discarded. Then, the importance of different channel features in the concatenated features is calculated to determine the weight coefficient of each channel. Weighted processing is then performed based on these weight coefficients to obtain the final target features. The calculation of the importance of different channel features is implemented through a compressed activation network. First, global average pooling is performed on the concatenated features to obtain the corresponding global compressed feature vector. Then, two fully connected layers are used to perform a nonlinear transformation on the compressed features to obtain the weight value of each channel. The weight coefficients are then multiplied one by one with the concatenated features to obtain weighted features. Finally, residual connections are used to combine the weighted features with the concatenated features to obtain the final target features.

[0055] In this embodiment, the calculation of the estimated probability is determined based on a convergent estimation model. The convergence of this estimation model is determined by calculating the regression loss between the predicted and actual values. Specifically, the mean squared error is used as the metric for the regression loss to accurately quantify the deviation between the model's predictions and the actual results.

[0056] Step S22. Determine the sampling strategy for each region based on the overall deviation of each region, determine the image acquisition time point for each region based on the sampling strategy, and obtain the real-time image of the crop corresponding to the image acquisition time point for each region.

[0057] In this embodiment, the sampling strategy refers to the frequency and timing of image acquisition for each region. It can be understood that, within the same background environment, the different growth states of different regions are primarily due to the presence of pests and diseases. Therefore, regions with different growth states should be given different levels of attention. For example, regions with larger growth deviations should have a higher sampling frequency to capture short-term changes in the crop within that region; while regions with smaller growth deviations should have a relatively lower sampling frequency to avoid excessive processing costs caused by excessive sampling.

[0058] Therefore, based on this logic, the sampling strategy for each region in this embodiment is determined based on the overall deviation level of each region. Specifically, this embodiment sets multiple deviation intervals, and each deviation interval is configured with a corresponding sampling strategy. The crop growth deviation level obtained in step S21 is mapped to the deviation intervals to determine the deviation interval and corresponding sampling strategy for each region.

[0059] The sampling strategy includes image acquisition frequency, which determines the corresponding image acquisition time point, and at the corresponding image acquisition time point, image acquisition is performed on the corresponding area to obtain real-time images of crops in that area.

[0060] This can be understood as the processing of steps S21 and S22 being performed synchronously. Specifically, in step S21, the degree of crop growth deviation is determined by acquiring data within the t-1 time period, and then the sampling strategy for each region within the t time period is determined and real-time images corresponding to multiple image acquisition time points within the t time period are acquired. Simultaneously, multiple detection data are acquired within the t time period to determine the crop growth deviation and degree in each region, and then the sampling strategy for each region in the t+1 time period is determined.

[0061] Furthermore, this embodiment also includes a mechanism for updating the sampling strategy based on the detection results. Specifically, when a crop image in any region contains features representing pests and diseases, the sampling strategy corresponding to each region is updated, and updated real-time images of other regions are acquired. Based on the updated real-time images, it is determined whether the crop in the corresponding region has pests and diseases and the classification result of the pests and diseases.

[0062] The mechanism is based on the detection results showing pests and diseases. If a pest or disease result is found in the current area, the sampling time point and sampling interval corresponding to the sampling strategy for that area are shortened to improve the tracking and detection of pest and disease development trends in that area. Similarly, when updating the sampling strategy for areas with pest and disease detection results, the sampling strategies for other areas also need to be updated synchronously, with the update strategy also shortening the previous sampling time point and sampling interval to increase the sampling density and detection frequency for each area. Furthermore, since pests and diseases, especially diseases, are contagious to some extent, when a pest or disease result is found in a certain area, real-time images of crops in other areas need to be collected and pest and disease detection performed on the real-time images of those other areas.

[0063] For details on how to conduct pest and disease detection, please refer to step S23.

[0064] Step S23. Obtain crop images of the relevant crops from the real-time images, determine the image features in the crop images, and determine whether the crop has pests and diseases and the classification results of pests and diseases based on the distribution of the image features.

[0065] In this embodiment, the real-time images acquired are those within the field of view of the image acquisition device. These images contain not only crop information but also background images. Therefore, before image recognition, the background images in the real-time images need to be removed, retaining only the images related to the crops, thereby reducing the processing costs of subsequent feature extraction and recognition.

[0066] This process employs a fusion of mask processing and Canny edge detection. Specifically, Canny edge detection first acquires edge information from the real-time image. Then, morphological processing is applied to the edge detection results to reduce noise interference and edge breaks, ensuring image continuity and providing more complete edge information for subsequent target extraction and segmentation. Next, a binary mask is constructed from the morphologically processed image, where the pixel values ​​of the target region are set to 1, and the background to 0. Closure operations are used to remove small holes to enhance mask quality. A bitwise AND operation is then used to extract the target region, preserving the areas with mask values ​​of 1 and their corresponding regions in the original image. Simultaneously, bitwise operations darken the background, making the target region more prominent, ultimately resulting in a clear image with separate target and background.

[0067] The morphological processing in this embodiment includes dilation, erosion, and closing operations. Furthermore, to make Canny edge detection more adaptable to complex texture features and background interference in images, the existing Canny edge detection method is improved. Specifically, the acquired real-time image is converted from its initial RGB channels to HSV channels; this process can be implemented using existing technology and will not be elaborated upon in this embodiment.

[0068] For the acquired HSV image, because the S channel has a stronger ability to represent the difference in pigment distribution between the leaf and the background, especially in uneven lighting scenarios, it can significantly suppress the interference of background noise such as soil on the gradient amplitude. Therefore, in this embodiment, the S channel image information in the HSV image is extracted separately and used as the bottom layer image for Canny edge detection. Then, based on the Canny edge detection method, the gradient amplitude and direction of each pixel in the S channel image are obtained. Then, non-maximum suppression is used to traverse each pixel in the S channel image information, comparing it with neighboring pixels, retaining only the pixel with the largest local gradient amplitude, and setting other pixels to 0, thereby making the edges more refined and retaining only the extreme points on the edges. Then, based on the set first and second thresholds, the S channel image information is divided into three edge classification results: strong edge, weak edge, and non-edge. Then, an elliptic kernel is used to perform a closing operation on the segmentation results to repair the edge breaks caused by occlusion or shadows, thereby forming a topologically coherent leaf boundary.

[0069] In this embodiment, the first threshold and the second threshold are respectively a high threshold and a low threshold. By setting two threshold ranges, the transition response of weak edges can be avoided while preserving the main contour. In this embodiment, the low threshold is 50 and the high threshold is 150.

[0070] In this embodiment, the above processing can achieve accurate segmentation of the foreground and background in a real-time image, and the segmented foreground image is a crop image that only contains the crop leaves to be detected.

[0071] The identification of pests and diseases is performed on the acquired crop images. The identification is implemented using a model built on YOLO, which includes a backbone network, a neck network, and a head network. The backbone network acts as a feature extractor to obtain multiple scale features under different receptive field conditions from the input crop image. The neck network is used to fuse multiple scale features to obtain fused features. The fused features are used as image features and predicted based on the head network.

[0072] Specifically, the backbone network in this embodiment employs multiple feature extraction modules and performs multi-layer feature extraction based on a downsampling mechanism to obtain features for each layer. The acquired features are then input into the next layer's feature extraction module. A pooling operation is performed after the output of the last feature extraction module to stitch together feature maps of different scales, enhancing multi-scale target recognition capabilities. The neck network aggregates features from different levels obtained in the backbone network, thereby enhancing the perception of small and occluded targets.

[0073] Specifically, the feature extraction module in this embodiment includes two sets of convolutional modules with different convolutional processing. The first set of convolutional modules has two convolutional blocks. The first convolutional block receives the original feature data from the upper layer. It first performs pointwise convolution on the feature data to reduce the number of feature channels and obtain initial features. Then, it performs convolution and linear transformation on the initial features, and merges the two processing results to obtain a merged feature. The second convolutional block performs the same processing on the merged feature to obtain a second merged feature. Then, the second merged feature is superimposed with the original feature data and the initial features. Finally, two convolutional processes are used to filter important features, resulting in the output feature of this feature extraction module.

[0074] In the neck network, an upsampling path and a skip connection mechanism are first set up to restore the resolution of multi-layer features in the backbone network. A downsampling reconstruction path is set up for the restored feature map. The downsampling reconstruction process is carried out through the downsampling reconstruction path to compress the feature map size and expand the receptive field while preserving key feature information.

[0075] Specifically, the neck network incorporates multiple bottleneck feature layers in its upsampling path. The feature output of the last bottleneck feature layer is fed into the input of the downsampling reconstruction path and a detector head in the head network. Furthermore, the output of the bottleneck feature layer in the upsampling path is concatenated with the feature output of the corresponding layer in the downsampling reconstruction path, thereby fusing the output features from the upsampling and downsampling reconstruction paths. This fused feature is then input into the next layer of the downsampling reconstruction path for feature extraction. Additionally, the output of each layer in the downsampling reconstruction path is connected to the corresponding detector head in the head network.

[0076] Furthermore, this embodiment includes multiple detection heads, each used to identify a classification task. The number of detection heads is configured based on the detection task. Since the number of detection heads is related to the number of layers in the downsampling reconstruction path and the number of upsampling layers, in this embodiment, the number of layers in the downsampling reconstruction path and the number of upsampling layers are determined based on the number of categories required for the detection task. For example, for the detection task in this embodiment, which is to determine whether anthrax, powdery mildew, and sharps burns are present, the corresponding head network should have three detection heads, and its output should have three values. Each output should contain the coordinate offset, confidence level, and category probability of each anchor box, ultimately parsed as a detection box + label. The corresponding upsampling path should have at least two layers for feature fusion and input of features required by the detection heads.

[0077] The bottleneck feature layer in the upsampling path includes a concatenation module and a feature processing module. The concatenation module concatenates the feature output of the corresponding feature extraction module in the backbone network with the feature output of the previous bottleneck feature layer in the upsampling path. Specifically, for the processing of the first concatenated feature layer in the upsampling path, the output features of the last feature extraction module in the backbone network are first upsampled, and the upsampled features are concatenated with the feature output of the corresponding feature extraction module in the backbone network before being input into the feature processing module.

[0078] In this embodiment, the feature processing module corresponding to each layer in the upsampling path segments the input image features into multiple sub-feature maps based on the channel dimension. Then, it obtains the dynamic weights corresponding to each sub-feature map based on a channel attention mechanism. Multiple sub-feature maps are sequentially processed through two fully connected layers for dimensionality upscaling and dimensionality reduction, respectively. Attention analysis is performed on each channel to determine their weight relationships: critical sub-feature maps receive larger weights, and non-critical sub-feature maps receive smaller weights. The obtained dynamic attention weights are then used to perform a weighted product operation on the multiple sub-feature maps to obtain the updated feature map. By segmenting the image features to independently process different channel groups, feature homogenization is avoided, thereby improving the small target detection capability and enabling the features corresponding to minor crop lesions to be highlighted.

[0079] The purpose of setting up the downsampling reconstruction path is to reduce the loss of spatial details caused by over-compression of feature maps during downsampling. Each layer in the downsampling reconstruction path includes a feature enhancement module and a feature processing module. For the input features, the feature enhancement module first obtains the global context information and local fine-grained features, then fuses the global context information and local fine-grained features to enhance the input features. The enhanced features are then input into the feature processing module to obtain the output features. The feature processing module in the downsampling reconstruction path has the same structure as the feature processing module in the upsampling path, and will not be described again in this embodiment.

[0080] Specifically, the feature enhancement module has two processing branches. One branch uses standard convolution to extract local fine-grained features from the input features, while the other branch uses dilatant convolution with a dilation rate of 3 to expand the receptive field and obtain long-range contextual information. The obtained local fine-grained features and long-range contextual information are then concatenated to form a multi-scale fused feature map. This multi-scale fused feature map is then processed by global average pooling to finally generate the global features.

[0081] Each detection head in this embodiment is configured with a first branch and a second branch. The first branch is used to predict the bounding box coordinates, and the second branch is used to predict the class probability. As mentioned earlier, each detection head is configured to correspond to a detection task, and the class probability indicates the probability of having the target object corresponding to that detection task. The output results of the detection heads can be found in [reference needed]. Figure 3 As shown, according to Figure 3 The test results are clearly displayed using a test box and labels.

[0082] The method for detecting pests and diseases in greenhouses provided in this application embodiment judges the degree of crop growth deviation in different areas based on detection data, configures corresponding sampling strategies based on different deviation degrees, acquires real-time images of the relevant crops in that area based on the corresponding sampling strategies, and extracts features from the real-time images using a detection model to identify whether the crops in the current area are diseased. Compared with existing technologies, the method provided in this application embodiment can configure different detection strategies according to the growth status of different areas, thereby reducing detection costs while ensuring detection accuracy.

[0083] See Figure 4 The data processing unit 12 in this embodiment specifically includes the following modules:

[0084] The deviation determination module 121 is used to determine the degree of crop generation deviation for each area based on multiple detection data corresponding to crops in multiple areas within a unit of time in the greenhouse.

[0085] The strategy determination module 122 is used to determine the sampling strategy for each region based on the overall deviation of each region, determine the image acquisition time point for each region based on the sampling strategy, and acquire the real-time image of the crop corresponding to the image acquisition time point for each region.

[0086] The identification module 123 is used to acquire individual crop images in the real-time image, determine the image features in each individual crop image, and determine whether the crop has pests and diseases and the classification result of pests and diseases based on the image features.

[0087] See Figure 5 The above methods can also be integrated into the provided terminal device 500. Since the device may vary significantly due to differences in configuration or performance, it may include one or more processors 501 and memories 502. The memories 502 may store one or more application programs or data. The memories 502 can be temporary or persistent storage. The application programs stored in the memories 502 may include one or more modules (not shown in the figure), each module may include a series of computer-executable instructions from the terminal device. Furthermore, the processor 501 may be configured to communicate with the memories 502, and the terminal device may execute the series of computer-executable instructions stored in the memories 502. The terminal device may also include one or more power supplies 503, one or more wired / wireless network interfaces 504, one or more input / output interfaces 505, one or more keyboards 506, etc.

[0088] In one specific embodiment, the terminal device includes a memory and one or more programs, wherein the one or more programs are stored in the memory, and the one or more programs may include one or more modules, and each module may include a series of computer-executable instructions for use in the terminal device, and is configured to be executed by one or more processors. The one or more programs include computer-executable instructions for performing the following:

[0089] Based on the multiple detection data corresponding to crops in multiple areas within the greenhouse per unit time, the degree of crop growth deviation corresponding to each area is determined.

[0090] The sampling strategy for each region is determined based on the overall deviation of each region, and the image acquisition time point for each region is determined based on the sampling strategy. Real-time images of crops corresponding to the image acquisition time points for each region are then obtained.

[0091] Obtain crop images of relevant crops from the real-time images, determine the image features in the crop images, and determine whether the crop has pests and diseases and the classification results of pests and diseases based on the distribution of the image features.

[0092] Optionally, the processor can perform various functions, such as the above-mentioned functions, by running or executing software programs stored in memory and by calling data stored in memory. Figure 2 The method shown.

[0093] In a specific implementation, as one example, the processor may include one or more microprocessors.

[0094] The memory is used to store the software program that executes the solution of this application, and the execution is controlled by the processor. The specific implementation method can be referred to the above method embodiment, which will not be repeated here.

[0095] In this application, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0096] It should be understood that in the various embodiments of this application, the order of the above-mentioned processes does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0097] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0098] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0099] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0100] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for detecting crop diseases and pests in greenhouses, characterized in that, The method includes: Based on multiple detection data points corresponding to crops in multiple areas within a unit of time within the greenhouse, the degree of crop growth deviation in each area is determined. These multiple detection data points characterize the environmental state within each area. Specifically, this includes: extracting features from the multiple detection data points corresponding to each area to obtain features for each data point; concatenating these features to obtain a concatenated feature; updating the concatenated feature using a multi-channel attention mechanism to obtain an updated target feature; calculating the probability of the corresponding estimated value based on the target feature; outputting the estimated value with the highest probability as the target as the growth state estimate; and comparing the growth state estimate with the growth standard value to obtain the degree of growth deviation in each area. The multiple detection data points include environmental humidity data, environmental temperature data, soil moisture data, light intensity data, and carbon dioxide data. The sampling strategy for each region is determined based on the overall deviation level of each region, and the image acquisition time point for each region is determined based on the sampling strategy. Real-time images of the crops corresponding to the image acquisition time points for each region are then acquired. Specifically, this includes: determining the degree of crop growth deviation by acquiring data within a time period of t-1; then determining the sampling strategy for each region within a time period of t and acquiring real-time images corresponding to multiple image acquisition time points within the time period of t; simultaneously acquiring multiple detection data within the time period of t to determine the crop growth deviation and degree in each region; and then determining the sampling strategy for each region within a time period of t+1. The sampling strategy includes the image acquisition frequency. The process involves acquiring crop images related to crops from real-time images, determining image features in the crop images, and determining whether the crop has pests or diseases and the classification results of pests or diseases based on the distribution of the image features. Acquiring crop images related to crops from real-time images includes: acquiring the pixel distribution on the grayscale image corresponding to the real-time image, and segmenting the crop-related image according to the pixel edge features of the crop to obtain the crop image. Specifically, this includes: converting the acquired real-time image from the initial RGB channel to the HSV channel, extracting the S-channel image information from the HSV image separately, using the S-channel image information as the bottom layer image for Canny edge detection, then acquiring the gradient magnitude and direction of each pixel in the S-channel image based on the Canny edge detection method, then traversing each pixel in the S-channel image information using non-maximum suppression, comparing it with adjacent pixels, retaining only the pixel with the largest local gradient magnitude, and then classifying the S-channel image information into three edge classification results: strong edge, weak edge, and non-edge based on a set first threshold and second threshold, and then performing a closing operation on the segmentation results using an elliptic kernel.

2. The method for detecting crop diseases and pests in greenhouses according to claim 1, characterized in that, The method further includes: when a crop individual image in any region contains features of pests and diseases, updating the sampling strategy of each region, acquiring updated real-time images of other regions, and determining whether the crop in the corresponding region has pests and diseases and the pest and disease classification result based on the updated real-time images.

3. The method for detecting crop diseases and pests in greenhouses according to claim 1, characterized in that, Determining the degree of crop growth deviation in each region based on multiple detection data within each region includes: acquiring the degree of deviation corresponding to each detection data, and determining the overall degree of deviation in each region as the degree of crop growth deviation based on the degree of deviation of each detection data.

4. The method for detecting crop diseases and pests in greenhouses according to claim 3, characterized in that, Obtaining the degree of deviation corresponding to each detection data includes: comparing each detection data with the corresponding standard data to determine the difference of each data as the degree of deviation; determining the overall degree of deviation of each region as the degree of crop growth deviation based on the degree of deviation of each detection data includes: updating the weight of the difference of each detection data to obtain the updated deviation, and determining the overall degree of deviation as the degree of crop growth deviation based on the updated deviation corresponding to each detection data.

5. The method for detecting crop diseases and pests in greenhouses according to claim 1, characterized in that, The feature extraction is implemented based on a feature extraction module, which includes a first depthwise separable convolution, a second depthwise separable convolution, a third depthwise separable convolution, a max pooling layer, and a dual attention mechanism. Different convolution kernels are set for the first depthwise separable convolution, the second depthwise separable convolution, and the third depthwise separable convolution. The input detection data is subjected to feature extraction through the above convolutions, and then the obtained features are concatenated to obtain the initial features of the convolution output. The initial features obtained after feature extraction are processed by a max pooling layer and then the feature weights are updated based on a dual attention mechanism, which is a combination of spatial attention and channel attention.

6. The method for detecting crop diseases and pests in greenhouses according to claim 1, characterized in that, Determining image features in the crop image includes: acquiring multiple scale features under different receptive field conditions, and fusing the multiple scale features to obtain fused features as image features.

7. The method for detecting crop diseases and pests in greenhouses according to claim 6, characterized in that, Determining whether a crop has pests and diseases and the classification result of pests and diseases based on the feature distribution state of the image features includes: segmenting the image features into multiple sub-feature maps, obtaining the dynamic weight corresponding to each sub-feature map, updating each sub-feature map according to the dynamic weight, merging the updated multiple sub-feature maps to obtain multiple intermediate feature maps, and obtaining the bounding box coordinate prediction result and the category probability prediction result corresponding to the features in each intermediate feature map.

8. The method for detecting crop diseases and pests in greenhouses according to claim 7, characterized in that, The process involves obtaining the dynamic weights corresponding to each sub-feature map and updating each sub-feature map based on the dynamic weights, including: sequentially performing dimensionality increase and dimensionality decrease processing on each sub-feature map, obtaining the spatial attention weights in each sub-feature map based on the spatial attention module, and obtaining the updated sub-feature map by weighted product processing based on the spatial attention weights.

9. A greenhouse crop pest and disease detection system, characterized in that, The system includes a data acquisition unit and a data processing unit. The data acquisition unit includes multiple sets of growth data acquisition devices and multiple image acquisition devices. Each set of growth data acquisition devices includes multiple growth data acquisition modules. Each set of growth data acquisition devices is used to acquire multiple types of detection data corresponding to each region. Each image acquisition device is used to acquire a real-time image corresponding to each region. The data processing unit is used to receive the detection data and real-time images, and execute the method according to any one of claims 1-8, including: The deviation determination module is used to determine the degree of crop generation deviation for each area based on multiple detection data corresponding to crops in multiple areas within a unit of time. The strategy determination module is used to determine the sampling strategy for each region based on the overall deviation of each region, determine the image acquisition time point for each region based on the sampling strategy, and acquire the real-time image of the crop corresponding to the image acquisition time point for each region. The identification module is used to acquire individual crop images in the real-time image, determine the image features in each individual crop image, and determine whether the crop has pests or diseases and the classification result of the pests or diseases based on the image features.

Images