Crop anomaly detection method and device, electronic equipment and storage medium

Abstract

This disclosure provides a crop anomaly detection method, comprising: acquiring multiple images, wherein the multiple images are acquired by an image acquisition device at multiple image acquisition points within a preset height range based on a preset angle and a preset direction, each image including a target part of the crop to be monitored, the preset angle being the pitch angle when the image acquisition device acquires the image, and the preset direction being perpendicular to the row direction of the crop; and processing the multiple images using a trained semantic segmentation model to determine the area of ​​the abnormal part in the multiple images.

Description

Technical Field

[0001] This disclosure relates to the field of intelligent agricultural detection, and in particular to a method, apparatus, electronic device, computer-readable storage medium, and computer program product for detecting crop anomalies. Background Technology

[0002] Currently, crops such as wheat frequently suffer from diseases during their growth. Therefore, in plant protection, it is necessary to monitor crop health to detect and control diseases early, and to accurately assess the severity of diseases after they occur to inform decisions about crop protection for the next season. Existing technologies typically employ manual on-site surveys, but this method is inefficient and heavily influenced by personnel experience and subjective biases. Furthermore, since observations are often conducted at only a few points, it is difficult to gain an objective understanding of the entire field's condition. Existing methods using drone remote sensing to create complete field maps and then assess pest and disease occurrence are limited by the spatial resolution of current imaging equipment. These methods are suitable for situations where plant growth is significantly altered and large areas of disease have occurred; therefore, they are not accurate enough for a comprehensive assessment of the entire field. Summary of the Invention

[0003] In view of this, the present disclosure provides a method, apparatus, electronic device, computer-readable storage medium, and computer program product for detecting crop anomalies.

[0004] One aspect of this disclosure provides a crop anomaly detection method, comprising: acquiring multiple images, wherein the multiple images are acquired by an image acquisition device at multiple image acquisition points within a preset height range based on a preset angle and a preset direction, each image including a target part of the crop to be monitored, the preset angle being the pitch angle when the image acquisition device acquires the image, and the preset direction being perpendicular to the row direction of the crop; and processing the multiple images using a trained semantic segmentation model to determine the area of ​​the abnormal part in the multiple images.

[0005] According to embodiments of this disclosure, the trained semantic segmentation model is trained using the following operations: obtaining a labeled dataset, which includes multiple sample images and corresponding labeled images, where the corresponding labeled images represent images that annotate abnormal parts in the sample images; dividing the labeled dataset into a training set and a validation set according to a preset ratio, the preset ratio being determined based on the amount of data in the labeled dataset; and training an initial semantic segmentation model using the training set and the validation set to obtain the trained semantic segmentation model.

[0006] According to embodiments of this disclosure, training an initial semantic segmentation model using a training set and a validation set to obtain a trained semantic segmentation model includes: training the initial semantic segmentation model using the training set to obtain a semantic segmentation model to be validated; determining the accuracy of the semantic segmentation model to be validated based on the validation set; if the accuracy is greater than or equal to a first preset threshold, using the semantic segmentation model to be validated as the trained semantic segmentation model; if the accuracy is less than the first preset threshold, repeating the following steps until the accuracy is greater than the first preset threshold: enriching and / or augmenting the training set to obtain an updated training set; training the semantic segmentation model to be validated using the updated training set and validating it based on the validation set; data enrichment includes adding sample images and corresponding labeled images to the training set, and data augmentation includes at least one of the following: adjusting the saturation of the image, adjusting the hue of the image, rotating the image, and scaling the image.

[0007] According to embodiments of this disclosure, the method further includes: determining a preset height range based on the resolution of images acquired by image acquisition devices positioned at different vertical heights.

[0008] According to embodiments of this disclosure, the method further includes: projecting the area of ​​the abnormal part onto a horizontal plane to obtain the area of ​​the abnormal region; determining the field of view area of ​​the image acquisition device based on the height during image acquisition, a preset angle, and the field of view angle of the image acquisition device; and obtaining the percentage of the abnormal region area based on the area of ​​the abnormal region and the field of view area of ​​the image acquisition device.

[0009] According to an embodiment of this disclosure, the method further includes: obtaining an abnormal situation scatter plot based on the proportion of abnormal areas corresponding to each image acquisition point; performing spatial interpolation on the abnormal situation scatter plot to obtain an abnormal area distribution map; and determining the crop in the abnormal area corresponding to the abnormality degree as a seed if the abnormality degree is less than a second preset threshold, based on the abnormality degree of multiple abnormal areas in the abnormal area distribution map.

[0010] Another aspect of this disclosure provides a remote sensing monitoring device, comprising: an acquisition module for acquiring multiple images, wherein the multiple images are acquired by an image acquisition device at multiple image acquisition points within a preset height range based on a preset angle and a preset direction, each image including a target part of the crop to be monitored, the preset angle being the pitch angle when the image acquisition device acquires the image, and the preset direction being perpendicular to the row direction of the crop; and a processing module for processing the multiple images using a trained semantic segmentation model to determine the area of ​​abnormal parts in the multiple images.

[0011] Another aspect of this disclosure provides an electronic device, including: one or more processors; and a storage device for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the above-described crop anomaly detection method.

[0012] Another aspect of this disclosure provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the aforementioned crop anomaly detection method.

[0013] Another aspect of this disclosure provides a computer program product, including a computer program stored on at least one of a readable storage medium and an electronic device, wherein the computer program implements the above-described crop anomaly detection method when executed by a processor.

[0014] According to embodiments of this disclosure, images acquired from preset angles and preset directions can extract subtle features of the target parts of crops, thereby determining whether there are any abnormalities. At the same time, the proportion of abnormal areas can be determined based on the area of ​​abnormal parts in multiple images. Based on the proportion of abnormal areas, the overall situation of the entire area to be detected can be evaluated, thus achieving both microscopic and macroscopic considerations. Attached Figure Description

[0015] The above and other objects, features and advantages of this disclosure will become clearer from the following description of embodiments with reference to the accompanying drawings, in which:

[0016] Figure 1 The illustration shows an application scenario of the crop anomaly detection method, apparatus, device, medium, and program product according to embodiments of the present disclosure;

[0017] Figure 2 A flowchart illustrating a crop anomaly detection method according to an embodiment of the present disclosure is shown schematically.

[0018] Figure 3 The illustration shows a schematic diagram of image acquisition according to an embodiment of the present disclosure;

[0019] Figure 4 A schematic diagram illustrating the acquisition of an image according to another embodiment of the present disclosure is shown;

[0020] Figure 5 A schematic diagram illustrating the acquisition of an image according to another embodiment of the present disclosure is shown;

[0021] Figure 6 A schematic diagram of a dataset according to an embodiment of the present disclosure is shown;

[0022] Figure 7 A schematic diagram illustrating the acquisition of an image according to another embodiment of the present disclosure is shown;

[0023] Figure 8 A schematic diagram illustrating the proportion of abnormal regions according to an embodiment of the present disclosure is shown.

[0024] Figure 9 A schematic diagram illustrating the distribution of abnormal regions according to an embodiment of the present disclosure is shown.

[0025] Figure 10 A schematic diagram illustrating the structure of a crop anomaly detection device according to an embodiment of the present disclosure is shown.

[0026] Figure 11 A block diagram schematically illustrates an electronic device suitable for implementing a crop anomaly detection method according to an embodiment of the present disclosure. Detailed Implementation

[0027] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0028] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0029] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0030] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0031] This disclosure provides a method for detecting crop anomalies. The method includes: acquiring multiple images, which are acquired by an image acquisition device at multiple image acquisition points within a preset height range based on preset angles and preset directions. Each image includes a target part of the crop to be monitored. The preset angle is the pitch angle of the image acquisition device when acquiring the image, and the preset direction is perpendicular to the row direction of the crop. The method also involves processing the multiple images using a trained semantic segmentation model to determine the area of ​​abnormal parts in the multiple images. By employing the technical solution proposed in this disclosure, acquiring images from preset angles and preset directions can extract subtle features of the target parts of the crop, thereby determining whether there are any anomalies. This method can also determine the proportion of abnormal regions based on the area of ​​abnormal parts in multiple images. Based on this proportion of abnormal regions, the overall condition of the entire area to be detected, such as a field, can be evaluated, achieving both micro and macroscopic considerations.

[0032] Figure 1 This is an application scenario diagram of a crop anomaly detection method, apparatus, device, medium, and program product according to an embodiment of this disclosure. It should be noted that... Figure 1 The diagrams shown are merely examples of application scenarios for crop anomaly detection methods, apparatuses, devices, media, and program products that can be applied according to the embodiments of this disclosure, in order to help those skilled in the art understand the technical content of this disclosure. However, they do not mean that the embodiments of this disclosure cannot be used in other devices, systems, environments, or scenarios.

[0033] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, and 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the terminal devices 101, 102, and 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0034] Users can use terminal devices 101, 102, and 103 to interact with server 105 via network 104 to receive or send messages, etc. Terminal devices 101, 102, and 103 can be various electronic devices with displays and web browsing capabilities, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0035] Server 105 can be a server that provides various services, such as a back-end management server that supports images acquired by agricultural technicians using terminal devices 101, 102, and 103 (for example only). The back-end management server can analyze and process the received images and feed back the processing results (such as an abnormal area distribution map obtained by spatial interpolation of an anomaly scatter plot) to the terminal devices.

[0036] It should be noted that the crop anomaly detection method provided in this disclosure embodiment can generally be executed by server 105. Correspondingly, the crop anomaly detection device provided in this disclosure embodiment can generally be located in server 105. The crop anomaly detection method provided in this disclosure embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105. Correspondingly, the crop anomaly detection device provided in this disclosure embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with terminal devices 101, 102, 103 and / or server 105.

[0037] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0038] The following will be based on Figure 1 The described scene, through Figure 2 The crop anomaly detection method of the disclosed embodiments is described in detail.

[0039] Figure 2 A flowchart illustrating a crop anomaly detection method according to an embodiment of this disclosure is shown schematically. Figure 2 As shown, the crop anomaly detection method of this embodiment includes operations S210 to S220.

[0040] In operation S210, multiple images are acquired. These images are acquired by an image acquisition device at multiple image acquisition points within a preset height range based on preset angles and preset directions. Each image includes the target part of the crop to be monitored. The preset angle is the pitch angle when the image acquisition device acquires the image, and the preset direction is perpendicular to the row direction of the crop.

[0041] For example, the image acquisition device can be a drone. More specifically, the image acquisition device can include two parts: a camera and an aircraft. The specific parameters are shown in Table 1, which shows the parameters involved in the camera part and the parameters involved in the aircraft part of the image acquisition device.

[0042] Table 1 Parameter Table of Image Acquisition Device

[0043]

[0044] The acquired images can be images captured by an image acquisition device, such as a drone, through point sampling. The image acquisition points can be pre-set or set to acquire images at intervals. Figure 3 This schematic diagram illustrates an image acquisition process according to an embodiment of the present disclosure, such as... Figure 3As shown, there are multiple image acquisition points 320, and the image acquisition device 310 acquires images at each image acquisition point 320 to obtain multiple images.

[0045] Taking wheat as the crop and the wheat ear as the target part as an example, the preset angle can be an angle that can capture a large area of ​​the wheat ear. Figure 4 This schematic diagram illustrates an image acquisition process according to another embodiment of the present disclosure. Figure 4 The left image is a vertically downward image, meaning it was acquired at a 0-degree pitch angle. Due to the characteristics of the wheat ear, if the image acquisition device shoots vertically downwards, the ear will appear as dots, making it difficult to capture its complete features. For example... Figure 4 As shown in the right figure, when the pitch angle of the image acquisition device reaches 50 degrees, the side features of the crop ear can be displayed in a larger area. Therefore, the preset angle can be set to 50 degrees, and thus, acquiring images from the preset angle can improve the accuracy of detection.

[0046] Besides the angle at which the image acquisition device captures the image, the direction in which the image is acquired also affects the accuracy of the monitoring. Taking wheat as the crop and the wheat ear as the target part as an example... Figure 5 This schematic diagram illustrates an image acquisition process according to another embodiment of the present disclosure. Figure 5 The left image is an image acquired parallel to the direction of row 510. Figure 5 The image on the right is a picture acquired perpendicular to row 520. For example... Figure 5 As shown, images acquired perpendicular to the row direction, or images acquired between rows, can better display the ears of crops than images acquired parallel to the row direction. Therefore, images acquired perpendicular to the row direction show a larger area of ​​the ears, resulting in higher detection accuracy.

[0047] In addition, the sampling interval involved in this disclosure can be 10 meters, that is, images are collected every 10 meters in the area to be detected. The sampling time can be from 10:00 to 14:00, the camera sensitivity is 100-200, the camera shutter speed is 1 / 3200 second, the camera pixel is more than 20 million, and the camera field of view can be 70° to 80°.

[0048] Understandably, images acquired from preset angles and directions can better showcase the characteristics of wheat ears, thereby improving the accuracy of detecting abnormalities and providing a more accurate assessment of the overall condition of the entire field.

[0049] In operation S220, a trained semantic segmentation model is used to process multiple images to determine the area of ​​abnormal regions in the multiple images.

[0050] Semantic segmentation models can employ U-Net models. These models classify and label each pixel based on the semantic information contained in an image, thereby accurately segmenting specific targets. The dataset can be divided into training and validation sets. An initial semantic segmentation model is trained using the training set to obtain a validation model. The validation set is then used to validate the model and obtain its accuracy. Based on this accuracy, it is determined whether the semantic segmentation model can be used as a trained semantic segmentation model to detect anomalies in images.

[0051] By inputting multiple images acquired by an image acquisition device into a trained semantic segmentation model, image segmentation results can be obtained, that is, images of abnormal parts can be segmented, thereby determining the area of ​​abnormal parts in each image.

[0052] Understandably, since multiple images can better display the target part, such as the ear of wheat, the trained semantic segmentation model can be used to process multiple images and determine the area of ​​abnormal parts in multiple images more accurately.

[0053] According to embodiments of this disclosure, the trained semantic segmentation model is trained using the following operations: obtaining a labeled dataset, which includes multiple sample images and corresponding labeled images, where the corresponding labeled images represent images that annotate abnormal parts in the sample images; dividing the labeled dataset into a training set and a validation set according to a preset ratio, the preset ratio being determined based on the amount of data in the labeled dataset; and training an initial semantic segmentation model using the training set and the validation set to obtain the trained semantic segmentation model.

[0054] The labeled dataset can include multiple sample images and corresponding labeled images. The sample images can be images captured by an image acquisition device, while the labeled images can be manually annotated by professional agricultural technicians who circle the objects of interest (ROIs) within the images. ROIs can be abnormal areas. The labeled dataset is divided into training and validation sets according to a preset ratio. To ensure sufficient training samples and a reasonable validation scale, the preset ratio can be determined based on the data volume of the labeled dataset. With sufficient data, a 7:3 ratio is generally recommended for dividing the labeled dataset into training and validation sets.

[0055] Figure 6 A schematic diagram of a dataset according to an embodiment of the present disclosure is shown, such as Figure 6 As shown, the labeled dataset includes a training set and a validation set. The training set includes sample images 610 and corresponding labeled images 620, and the validation set includes sample images 630 and corresponding labeled images 640.

[0056] It is understandable that training an initial semantic segmentation model using the training and validation sets to obtain a trained semantic segmentation model can ensure that the semantic segmentation model has high accuracy.

[0057] According to embodiments of this disclosure, training an initial semantic segmentation model using a training set and a validation set to obtain a trained semantic segmentation model includes: training the initial semantic segmentation model using the training set to obtain a semantic segmentation model to be validated; determining the accuracy of the semantic segmentation model to be validated based on the validation set; if the accuracy is greater than or equal to a first preset threshold, using the semantic segmentation model to be validated as the trained semantic segmentation model; if the accuracy is less than the first preset threshold, repeating the following steps until the accuracy is greater than the first preset threshold: enriching and / or augmenting the training set to obtain an updated training set; training the semantic segmentation model to be validated using the updated training set and validating it based on the validation set; data enrichment includes adding sample images and corresponding labeled images to the training set, and data augmentation includes at least one of the following: adjusting the saturation of the image, adjusting the hue of the image, rotating the image, and scaling the image.

[0058] The initial semantic segmentation model can be an untrained U-Net model. The model can be trained using the training set to obtain a semantic segmentation model to be validated. Then, the semantic segmentation model to be validated can be used with the validation set.

[0059] Specifically, the sample images from the validation set are input into the semantic segmentation model to be validated, resulting in corresponding output images. Based on the output images and their corresponding labeled images, the accuracy of the semantic segmentation model to be validated can be obtained. The accuracy is then used to assess whether the semantic segmentation model needs further training. If the accuracy is greater than a first preset threshold, the semantic segmentation model can be directly used as a trained semantic segmentation model. If the accuracy is less than the first preset threshold, it indicates that the semantic segmentation model still needs further training.

[0060] For example, the first preset threshold could be 80%. If the accuracy is less than 80%, the training set can be enriched and / or augmented to obtain an updated training set. Then, the semantic segmentation model to be validated is trained using the updated training set and validated based on the validation set. This process is repeated until the accuracy is greater than or equal to the first preset threshold. Data enrichment can include adding sample images and corresponding labeled images to the training set—data that has not been used in previous training. Data augmentation includes at least one of the following: adjusting image saturation, adjusting image hue, image rotation, and image scaling.

[0061] It is understandable that the semantic segmentation model obtained through this training method can guarantee that the semantic segmentation model has a high accuracy rate, thereby ensuring that the area of ​​the abnormal part obtained based on the semantic segmentation model is more accurate.

[0062] According to embodiments of this disclosure, the method further includes: determining a preset height range based on the resolution of images acquired by image acquisition devices positioned at different vertical heights.

[0063] To clearly monitor the occurrence of ear diseases, this invention employs low-altitude, close-range observation using unmanned aerial vehicles (UAVs). Lowering the flight altitude allows for more precise monitoring, but it also reduces the efficiency of monitoring large areas. Therefore, images can be acquired at a height of 1.5-2.0 meters above the canopy, while simultaneously sampling large areas of the field to improve data collection efficiency. Taking the parameters involved in this application as an example, images sampled at a distance of 1.5 meters from the crop canopy have a resolution of approximately 0.7 mm, and images sampled at a height of 2.0 meters have a resolution of approximately 1 mm, both of which can accurately monitor ear characteristics. Therefore, the preset height range can be between 1.5 meters and 2 meters.

[0064] It is understandable that capturing images within a preset height range can ensure that the images captured by the image acquisition device have good resolution without affecting the acquisition efficiency.

[0065] According to embodiments of this disclosure, the method further includes: projecting the area of ​​the abnormal part onto a horizontal plane to obtain the area of ​​the abnormal region; determining the field of view area of ​​the image acquisition device based on the height during image acquisition, a preset angle, and the field of view angle of the image acquisition device; and obtaining the percentage of the abnormal region area based on the area of ​​the abnormal region and the field of view area of ​​the image acquisition device.

[0066] To more accurately calculate the proportion of abnormal areas to normal areas, the area of ​​the abnormal area in the image can be projected onto a horizontal plane to obtain the area of ​​the abnormal region. Then, based on this area of ​​the abnormal region and the field of view of the image acquisition device, the proportion of the abnormal region area can be obtained. Taking wheat smut as an example, in practical applications, the proportion of the abnormal region area can be used to represent the proportion of smut in wheat.

[0067] Specifically, the calculation process of projecting the area of ​​the abnormal part in the image onto the horizontal plane to obtain the area of ​​the abnormal region can be shown in formula (1).

[0068] Formula (1)

[0069] in, This represents the area of ​​the abnormal region, specifically the area of ​​the monitored area affected by pests, diseases, and weeds. 'n' is the total number of pixels in the image that detected the abnormal areas, and 'H' is the height of the image acquisition device above the crop canopy. This represents the physical size of a camera pixel, f represents the camera's focal length, and L represents the size of the camera's charge-coupled device (CCD) along the pixel row direction. This represents the component of the distance of a pixel from the top-left corner on the CCD along the pixel row direction. This indicates the pitch angle of the image acquisition device.

[0070] After obtaining the area of ​​the abnormal region, the proportion of the abnormal region area can be obtained based on the area of ​​the abnormal region and the field of view of the image acquisition device. The specific calculation process is shown in formula (2).

[0071] Formula (2)

[0072] Where r represents the percentage of abnormal area, specifically the percentage of area affected by pests, diseases, and weeds. H represents the area of ​​the abnormal region, and H represents the height of the image acquisition device from the crop canopy. Indicates the pitch angle of the image acquisition device. This indicates the field of view of the image acquisition device.

[0073] Figure 7 A schematic diagram illustrating the acquisition of an image according to another embodiment of this disclosure is shown, such as... Figure 7 As shown, the image acquisition device 710 acquires an image of the crop 720, through... Figure 7 This allows for a direct understanding of the altitude, pitch angle, and field of view involved in this disclosure.

[0074] It is understandable that, due to optical distortion of the acquired image pixels, directly using the image to obtain the proportion of abnormal areas cannot accurately reflect the true situation. Therefore, this disclosure takes into account three-dimensional projection relationships, and obtains the area of ​​the abnormal region by projecting the area of ​​the abnormal part onto a horizontal plane. Based on the area of ​​the abnormal region and the field of view of the image acquisition device, the proportion of the abnormal region area is obtained, which can more accurately and realistically reflect the abnormal situation of the corresponding area of ​​the image in the actual application scenario.

[0075] According to an embodiment of this disclosure, the method further includes: obtaining an abnormal situation scatter plot based on the proportion of abnormal areas corresponding to each image acquisition point; performing spatial interpolation on the abnormal situation scatter plot to obtain an abnormal area distribution map; and determining the crop in the abnormal area corresponding to the abnormality degree as a seed if the abnormality degree is less than a second preset threshold, based on the abnormality degree of multiple abnormal areas in the abnormal area distribution map.

[0076] Figure 8 The diagram illustrates the proportion of abnormal regions according to an embodiment of the present disclosure, specifically a scatter plot of abnormal conditions. Since the acquired images are captured at image acquisition points, and the entire area to be detected includes areas where no images were captured, spatial interpolation is required on the scatter plot of abnormal conditions to obtain an abnormal region distribution map.

[0077] Figure 9 A schematic diagram illustrating the distribution of abnormal regions according to embodiments of the present disclosure is shown, such as... Figure 9 As shown, the distribution map of abnormal areas obtained by spatial interpolation of the scatter plot of abnormal conditions can reflect the severity of the abnormality in the entire area to be detected. Specifically, the degree of abnormality of each abnormal area can be determined based on the results of spatial interpolation. Taking wheat as the crop and the wheat ear as the abnormal part as an example, the distribution map of abnormal areas obtained by spatial interpolation of the scatter plot of abnormal conditions can accurately determine how many ears per acre have been affected by disease in different locations in the field, and thus make precise decisions.

[0078] Taking the area to be tested as a production field as an example, the second preset threshold can be one ten-thousandth. If the abnormality of the abnormal area is less than one ten-thousandth, it can be said that the abnormality in the area is not serious, and the crops in the area can be used as seeds.

[0079] Understandably, by performing spatial interpolation on the scatter plot of anomalies, an anomaly distribution map can be obtained, which can then be used to assess the overall condition of the area to be detected.

[0080] Figure 10 A schematic block diagram of a crop anomaly detection device according to an embodiment of the present disclosure is shown.

[0081] like Figure 10 As shown, the crop anomaly detection device 1000 of this embodiment includes...

[0082] The acquisition module 1010 is used to acquire multiple images. These images are acquired by an image acquisition device at multiple image acquisition points within a preset height range based on preset angles and preset directions. Each image includes a target part of the crop to be monitored. The preset angle is the pitch angle of the image acquisition device when acquiring the image, and the preset direction is perpendicular to the row direction of the crop. In one embodiment, the acquisition module 1010 can be used to perform the operation S210 described above, which will not be repeated here.

[0083] The processing module 1020 is used to process multiple images using a trained semantic segmentation model to determine the area of ​​abnormal regions in the multiple images. In one embodiment, the processing module 1020 can be used to perform the operation S220 described above, which will not be repeated here.

[0084] According to embodiments of this disclosure, the trained semantic segmentation model is trained using the following operations: obtaining a labeled dataset, which includes multiple sample images and corresponding labeled images, where the corresponding labeled images represent images that annotate abnormal parts in the sample images; dividing the labeled dataset into a training set and a validation set according to a preset ratio, the preset ratio being determined based on the amount of data in the labeled dataset; and training an initial semantic segmentation model using the training set and the validation set to obtain the trained semantic segmentation model.

[0085] According to embodiments of this disclosure, training an initial semantic segmentation model using a training set and a validation set to obtain a trained semantic segmentation model includes: training the initial semantic segmentation model using the training set to obtain a semantic segmentation model to be validated; determining the accuracy of the semantic segmentation model to be validated based on the validation set; if the accuracy is greater than or equal to a first preset threshold, using the semantic segmentation model to be validated as the trained semantic segmentation model; if the accuracy is less than the first preset threshold, repeating the following steps until the accuracy is greater than the first preset threshold: enriching and / or augmenting the training set to obtain an updated training set; training the semantic segmentation model to be validated using the updated training set and validating it based on the validation set; data enrichment includes adding sample images and corresponding labeled images to the training set, and data augmentation includes at least one of the following: adjusting the saturation of the image, adjusting the hue of the image, rotating the image, and scaling the image.

[0086] According to embodiments of this disclosure, the crop anomaly detection device 1000 further includes a first determining module, used to determine a preset height range based on the resolution of images acquired by image acquisition devices set at different vertical heights.

[0087] According to an embodiment of this disclosure, the crop anomaly detection device 1000 further includes a projection module, a second determination module, and a first processing module. The projection module is used to project the area of ​​the abnormal part onto a horizontal plane to obtain the area of ​​the abnormal region. The second determination module is used to determine the field of view area of ​​the image acquisition device based on the height during image acquisition, a preset angle, and the field of view angle of the image acquisition device. The first processing module is used to obtain the percentage of the abnormal region area based on the area of ​​the abnormal region and the field of view area of ​​the image acquisition device.

[0088] According to embodiments of this disclosure, the crop anomaly detection device 1000 further includes a second processing module, an interpolation module, and a third determination module. The second processing module is used to obtain an anomaly scatter plot based on the proportion of anomaly regions corresponding to each image acquisition point. The interpolation module is used to perform spatial interpolation on the anomaly scatter plot to obtain an anomaly region distribution map. The second processing module is used to determine, based on the anomaly degree of multiple anomaly regions in the anomaly region distribution map, if the anomaly degree is less than a second preset threshold, the crop in the anomaly region corresponding to the anomaly degree is a seed.

[0089] According to embodiments of this disclosure, any plurality of modules in the acquisition module 1010 and the processing module 1020 may be combined into one module, or any one of these modules may be split into multiple modules. Alternatively, at least a portion of the functionality of one or more of these modules may be combined with at least a portion of the functionality of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the acquisition module 1010 and the processing module 1020 may be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the acquisition module 1010 and the processing module 1020 may be at least partially implemented as a computer program module, which, when run, can perform corresponding functions.

[0090] Figure 11 A block diagram schematically illustrates an electronic device suitable for implementing a crop anomaly detection method according to an embodiment of the present disclosure.

[0091] like Figure 11 As shown, an electronic device 1100 according to an embodiment of the present disclosure includes a processor 1101, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1102 or a program loaded from a storage portion 1108 into a random access memory (RAM) 1103. The processor 1101 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 1101 may also include onboard memory for caching purposes. The processor 1101 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0092] RAM 1103 stores various programs and data required for the operation of electronic device 1100. Processor 1101, ROM 1102, and RAM 1103 are interconnected via bus 1104. Processor 1101 performs various operations of the method flow according to embodiments of this disclosure by executing programs in ROM 1102 and / or RAM 1103. It should be noted that the programs may also be stored in one or more memories other than ROM 1102 and RAM 1103. Processor 1101 may also implement the methods provided in embodiments of this disclosure by executing programs stored in said one or more memories.

[0093] According to embodiments of this disclosure, the electronic device 1100 may further include an input / output (I / O) interface 1105, which is also connected to a bus 1104. The electronic device 1100 may also include one or more of the following components connected to the I / O interface 1105: an input section 1106 including a keyboard, mouse, etc.; an output section 1107 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 1108 including a hard disk, etc.; and a communication section 1109 including a network interface card such as a LAN card, modem, etc. The communication section 1109 performs communication processing via a network such as the Internet. A drive 1110 is also connected to the I / O interface 1105 as needed. A removable medium 1111, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 1110 as needed so that computer programs read from it can be installed into the storage section 1108 as needed.

[0094] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0095] According to embodiments of this disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 1102 and / or RAM 1103 and / or one or more memories other than ROM 1102 and RAM 1103 described above.

[0096] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the methods provided in the embodiments of this disclosure.

[0097] When the computer program is executed by the processor 1101, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0098] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 1109, and / or installed from the removable medium 1111. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0099] In such an embodiment, the computer program can be downloaded and installed from a network via communication section 1109, and / or installed from removable medium 1111. When the computer program is executed by processor 1101, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0100] It should be noted that the collection, storage, use, processing, transmission, provision, disclosure, and application of user personal information in this disclosed technical solution comply with relevant laws and regulations, necessary confidentiality measures have been taken, and it does not violate public order and good morals. In this disclosed technical solution, user authorization or consent has been obtained before acquiring or collecting user personal information.

[0101] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on a user's computing device, partially on a user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0102] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0103] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure can be combined and / or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0104] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A method for detecting crop anomalies, characterized in that, The method includes: Multiple images are acquired by an image acquisition device at multiple image acquisition points within a preset height range based on a preset angle and a preset direction. Each image includes a target part of the crop to be monitored. The preset angle is the pitch angle when the image acquisition device acquires the image, and the preset direction is perpendicular to the row direction of the crop. The trained semantic segmentation model is used to process the multiple images to determine the area of ​​abnormal regions in the multiple images; The area of ​​the abnormal part is projected onto a horizontal plane to obtain the area of ​​the abnormal region. The field of view area of ​​the image acquisition device is determined based on the height during image acquisition, the preset angle, and the field of view angle of the image acquisition device. Based on the area of ​​the abnormal region and the field of view of the image acquisition device, the percentage of the abnormal region area is obtained; Based on the area ratio of abnormal regions corresponding to each image acquisition point, a scatter plot of abnormal situations is obtained; Spatial interpolation is performed on the scatter plot of the abnormal situation to obtain the distribution map of the abnormal area; Based on the abnormality degree of multiple abnormal regions in the abnormal region distribution map, if the abnormality degree is less than a second preset threshold, the crop in the abnormal region corresponding to the abnormality degree is determined to be a seed. The area of ​​the abnormal region is determined based on the following formula: in, The area represents the abnormal region, n represents the total number of pixels in the abnormal region detected from the image, and H represents the height of the image acquisition device from the crop canopy. This represents the physical size of a camera pixel, f represents the camera's focal length, and L represents the size of the camera's charge-coupled device (CCD) along the pixel row direction. This represents the component of the distance of a pixel from the top-left corner on the camera's charge-coupled device along the pixel row direction. The pitch angle of the image acquisition device is indicated, which includes the camera.

2. The method according to claim 1, characterized in that, The trained semantic segmentation model is obtained using the following operations: Obtain a labeled dataset, which includes multiple sample images and corresponding labeled images, wherein the corresponding labeled images represent images that annotate abnormal parts in the sample images; The labeled dataset is divided into a training set and a validation set according to a preset ratio, the preset ratio being determined based on the amount of data in the labeled dataset. The initial semantic segmentation model is trained using the training set and the validation set to obtain the trained semantic segmentation model.

3. The method according to claim 2, characterized in that, The step of training an initial semantic segmentation model using the training set and validation set to obtain the trained semantic segmentation model includes: The initial semantic segmentation model is trained using the training set to obtain the semantic segmentation model to be validated; Based on the validation set, the accuracy of the semantic segmentation model to be validated is determined; If the accuracy is greater than or equal to a first preset threshold, the semantic segmentation model to be verified is used as the trained semantic segmentation model. If the accuracy rate is less than a first preset threshold, repeat the following steps until the accuracy rate is greater than the first preset threshold: The training set is updated by enriching and / or augmenting the training set. The semantic segmentation model to be validated is trained using the updated training set and validated based on the validation set; the data enrichment includes adding sample images and corresponding labeled images to the training set, and the data augmentation includes at least one of the following: adjusting the saturation of the image, adjusting the hue of the image, rotating the image, and scaling the image.

4. The method according to claim 1, characterized in that, The method further includes: The preset height range is determined based on the resolution of images acquired by image acquisition devices positioned at different vertical heights.

5. A remote sensing monitoring device, characterized in that, The device includes: The acquisition module is used to acquire multiple images. The multiple images are acquired by an image acquisition device at multiple image acquisition points within a preset height range based on a preset angle and a preset direction. Each image includes a target part of the crop to be monitored. The preset angle is the pitch angle when the image acquisition device acquires the image, and the preset direction is perpendicular to the row direction of the crop. The processing module is used to process the multiple images using a trained semantic segmentation model to determine the area of ​​abnormal regions in the multiple images; The device further includes: A projection module is used to project the area of ​​the abnormal part onto a horizontal plane to obtain the area of ​​the abnormal region; The second determining module is used to determine the field of view area of ​​the image acquisition device based on the height during image acquisition, the preset angle, and the field of view angle of the image acquisition device. The first processing module is used to obtain the percentage of the abnormal area based on the area of ​​the abnormal area and the field of view of the image acquisition device. The second processing module is used to obtain a scatter plot of abnormal situations based on the area ratio of the abnormal region corresponding to each image acquisition point. The interpolation module is used to perform spatial interpolation on the scatter plot of the abnormal situation to obtain an abnormal area distribution map; The third determining module is used to determine, based on the abnormality degree of multiple abnormal regions in the abnormal region distribution map, that crops in the abnormal region corresponding to the abnormality degree are seeds when the abnormality degree is less than a second preset threshold. The area of ​​the abnormal region is determined based on the following formula: in, The area represents the abnormal region, n represents the total number of pixels in the abnormal region detected from the image, and H represents the height of the image acquisition device from the crop canopy. This represents the physical size of a camera pixel, f represents the camera's focal length, and L represents the size of the camera's charge-coupled device (CCD) along the pixel row direction. This represents the component of the distance of a pixel from the top-left corner on the camera's charge-coupled device along the pixel row direction. The pitch angle of the image acquisition device is indicated, which includes the camera.

6. An electronic device, comprising: One or more processors; Storage device for storing one or more programs. Wherein, when the one or more programs are executed by the one or more processors, the one or more processors perform the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 4.

8. A computer program product comprising a computer program stored on at least one of a readable storage medium and an electronic device, the computer program implementing the method according to any one of claims 1 to 4 when executed by a processor.

Images