Crop anomaly detection method and device, electronic equipment and storage medium

Abstract

The disclosure provides a crop anomaly detection method, comprising: acquiring a plurality of images, the plurality of images being acquired by an image acquisition device at a plurality of image acquisition points based on a preset angle and a preset direction within a preset height range, each image comprising a target part of a to-be-monitored crop, the preset angle is a pitch angle when the image acquisition device acquires images, and the preset direction is a row ridge direction perpendicular to crops; and processing the plurality of images by using the trained semantic segmentation model, and determining the areas of the abnormal parts in the plurality of images.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of agricultural intelligent detection, and in particular, to a crop anomaly detection method and device, an electronic device, a computer readable storage medium, and a computer program product. BACKGROUND

[0002] At present, crops such as wheat in agriculture often suffer from diseases during the growth process. Therefore, in the plant protection link, on the one hand, the health status of crops needs to be monitored to discover the occurrence of diseases as early as possible and to prevent and control them, and on the other hand, the severity of the occurrence of diseases needs to be accurately assessed to provide a basis for decision-making for the protection of crops in the next season. The existing technology usually adopts the way of artificial on-site reconnaissance, but this way is low in efficiency and is greatly influenced by the experience and subjective consciousness of personnel. At the same time, since it is often observed at a few points, it is difficult to have an objective understanding of the real situation of the whole field. The existing way of using unmanned aerial vehicles to remotely sense and prepare a complete map of the field, and then assessing the occurrence of diseases and pests is limited by the spatial resolution of the existing imaging equipment, is suitable for the case where the plant body changes obviously and a large area is affected, and therefore, the overall assessment of the whole field is not accurate enough. SUMMARY

[0003] Therefore, the present disclosure provides a crop anomaly detection method and device, an electronic device, a computer readable storage medium, and a computer program product.

[0004] One aspect of the present disclosure provides a crop anomaly detection method, comprising: acquiring a plurality of images, the plurality of images being acquired by an image acquisition device at a plurality of image acquisition points within a preset height range based on a preset angle and a preset direction, each image including a target part of a crop to be monitored, the preset angle being a pitch angle when the image acquisition device acquires the image, and the preset direction being a row direction perpendicular to the crop; processing the plurality of images by using a trained semantic segmentation model to determine the area of an abnormal part in the plurality of images.

[0005] According to an embodiment of the present disclosure, the trained semantic segmentation model is trained by the following operations: acquiring a labeled data set, the labeled data set including a plurality of sample images and corresponding labeled images, the corresponding labeled images representing images in which the abnormal parts in the sample images are labeled; dividing the labeled data set into a training set and a validation set according to a preset proportion, the preset proportion being determined based on the data quantity of the labeled data set; training an initial semantic segmentation model by using the training set and the validation set to obtain the trained semantic segmentation model.

[0006] According to an embodiment of the present disclosure, training the initial semantic segmentation model by using the training set and the validation set comprises: training the initial semantic segmentation model by using the training set to obtain a semantic segmentation model to be validated; determining an accuracy of the semantic segmentation model to be validated based on the validation set; in a case where the accuracy is greater than or equal to a first preset threshold, taking the semantic segmentation model to be validated as the trained semantic segmentation model; in a case where the accuracy is less than the first preset threshold, repeating the following steps until the accuracy is greater than the first preset threshold: performing data enrichment and / or data enhancement on the training set to obtain an updated training set; training the semantic segmentation model to be validated by using the updated training set and performing validation based on the validation set; the data enrichment comprises adding sample images and corresponding labeled images in the training set, and the data enhancement comprises at least one of the following: adjusting saturation of the images, adjusting hue of the images, image rotation, image scaling.

[0007] According to an embodiment of the present disclosure, the method further comprises: determining the preset height range based on resolutions of images collected by the image collection devices arranged at different vertical heights.

[0008] According to an embodiment of the present disclosure, the method further comprises: projecting the area of the abnormal part to a horizontal plane to obtain an abnormal area; determining a field of view area of the image collection device based on a height at which the image is collected, a preset angle, and a field of view angle of the image collection device; and obtaining an abnormal area proportion based on the abnormal area and the field of view area of the image collection device.

[0009] According to an embodiment of the present disclosure, the method further comprises: obtaining an abnormal condition scatter plot based on the abnormal area proportion corresponding to each image collection point; performing spatial interpolation on the abnormal condition scatter plot to obtain an abnormal area distribution map; and determining, according to the abnormal degree of the multiple abnormal areas in the abnormal area distribution map, that the crop in the abnormal area corresponding to the abnormal degree is a seed in a case where the abnormal degree is less than a second preset threshold.

[0010] Another aspect of the present disclosure also provides a remote sensing monitoring device, comprising: an acquisition module configured to acquire a plurality of images, the plurality of images being collected by an image collection device at a plurality of image collection points within a preset height range based on a preset angle and a preset direction, each image comprising a target part of a crop to be monitored, the preset angle being a pitch angle at which the image collection device collects the image, and the preset direction being a row direction perpendicular to the crop; and a processing module configured to process the plurality of images by using a trained semantic segmentation model to determine areas of abnormal parts in the plurality of images.

[0011] Another aspect of the present disclosure also provides an electronic device, comprising: one or more processors; and a storage device configured to store one or more programs, wherein the one or more programs, when executed by the one or more processors, cause the one or more processors to perform the crop abnormality detection method.

[0012] Another aspect of the present disclosure also provides a computer-readable storage medium having stored thereon executable instructions that, when executed by a processor, cause the processor to perform the crop anomaly detection method.

[0013] Another aspect of the present disclosure also provides a computer program product comprising a computer program stored on at least one of a readable storage medium and an electronic device, which, when executed by a processor, implements the crop anomaly detection method.

[0014] According to the embodiments of the present disclosure, the fine features of the target part of the crop can be extracted from the images collected from the preset angle and the preset direction, so as to determine whether the crop is abnormal, and the area ratio of the abnormal area can be determined based on the area of the abnormal part in the multiple images, and the overall situation of the entire detection area can be evaluated based on the area ratio of the abnormal area, so that both micro and macro aspects are considered. BRIEF DESCRIPTION OF DRAWINGS

[0015] The above and other objects, features and advantages of the present disclosure will become more apparent from the following description when taken in conjunction with the accompanying drawings, in which:

[0016] Figure 1 A schematic diagram of an application scenario of the crop anomaly detection method, device, equipment, medium and program product according to the embodiments of the present disclosure is shown;

[0017] Figure 2 A flowchart of the crop anomaly detection method according to the embodiments of the present disclosure is shown;

[0018] Figure 3 A schematic diagram of acquiring images according to the embodiments of the present disclosure is shown;

[0019] Figure 4 A schematic diagram of acquiring images according to another embodiment of the present disclosure is shown;

[0020] Figure 5 A schematic diagram of acquiring images according to another embodiment of the present disclosure is shown;

[0021] Figure 6 A schematic diagram of a data set according to the embodiments of the present disclosure is shown;

[0022] Figure 7 A schematic diagram of acquiring images according to another embodiment of the present disclosure is shown;

[0023] Figure 8 A schematic diagram of the area ratio of the abnormal area according to the embodiments of the present disclosure is shown;

[0024] Figure 9 A schematic diagram illustrating abnormal region distribution according to an embodiment of the present disclosure is shown;

[0025] Figure 10 A structural block diagram of a crop abnormality detection apparatus according to an embodiment of the present disclosure is shown schematically;

[0026] Figure 11 A block diagram of an electronic device suitable for implementing a crop abnormality detection method according to an embodiment of the present disclosure is shown schematically. DETAILED DESCRIPTION

[0027] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. It should be understood, however, that the description which follows is merely illustrative and is not intended to limit the scope of the present disclosure. In the following detailed description of embodiments of the present disclosure, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. However, it will be apparent to one skilled in the art that one or more embodiments of the present disclosure can be practiced without these specific details. In other instances, well-known structures and functions have not been described in detail in order to avoid obscuring aspects of the present disclosure.

[0028] The terms used herein are merely used to describe specific embodiments and are not intended to limit the present disclosure. The terms "include", "comprise" and the like used herein indicate the presence of the described features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0029] All terms used herein, including technical and scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of the present specification, and should not be interpreted in an idealized or overly formal manner.

[0030] In the case of using expressions similar to "at least one of A, B, and C, etc.", it should generally be interpreted to include at least one of A, B, or C, or a combination thereof (e.g., "a system having at least one of A, B, and C" should include but not be limited to a system having A alone, a system having B alone, a system having C alone, a system having A and B together, a system having A and C together, a system having B and C together, and / or a system having A, B, and C together, etc.).

[0031] The crop anomaly detection method provided by the embodiments of the present disclosure comprises: acquiring a plurality of images, the plurality of images being acquired by an image acquisition device at a plurality of image acquisition points within a preset height range based on a preset angle and a preset direction, each image comprising a target part of a crop to be monitored, the preset angle being a pitch angle when the image acquisition device acquires the image, and the preset direction being a row direction perpendicular to the crop; processing the plurality of images by using a trained semantic segmentation model to determine the area of an abnormal part in the plurality of images. By using the technical solution provided by the present disclosure, the fine features of the target part of the crop can be extracted from the images acquired at the preset angle and the preset direction, so that it can be determined whether the target part is abnormal. The method can also determine the abnormal area ratio based on the area of the abnormal part in the plurality of images. Based on the abnormal area ratio, the overall situation of the entire detection area, for example, the entire field, can be evaluated, and both micro and macro aspects are considered.

[0032] Figure 1 FIG. 1 is an application scenario diagram of a crop anomaly detection method, device, equipment, medium and program product according to an embodiment of the present disclosure. It should be noted that Figure 1 the application scenario diagram of the crop anomaly detection method, device, equipment, medium and program product according to the embodiments of the present disclosure is only an example to help those skilled in the art understand the technical content of the present disclosure, but does not mean that the embodiments of the present disclosure cannot be applied to other equipment, systems, environments or scenarios.

[0033] As Figure 1 shown, the system architecture 100 according to the embodiment can comprise terminal equipment 101, 102, 103, a network 104 and a server 105. The network 104 is used as a medium to provide a communication link between the terminal equipment 101, 102, 103 and the server 105. The network 104 can comprise various connection types, such as wired and / or wireless communication links, etc.

[0034] A user can use the terminal equipment 101, 102, 103 to interact with the server 105 through the network 104 to receive or send messages, etc. The terminal equipment 101, 102, 103 can be various electronic devices with a display screen and supporting web browsing, including but not limited to smartphones, tablet computers, laptop computers and desktop computers, etc.

[0035] The server 105 can be a server providing various services, such as a background management server providing support for agricultural technicians to acquire images by using the terminal equipment 101, 102, 103 (only as an example). The background management server can analyze and process the received images, and feed back the processing results (such as an abnormal area distribution map obtained by spatial interpolation on an abnormal situation scatter plot) to the terminal equipment.

[0036] It should be noted that the crop anomaly detection method provided in the embodiments of the present disclosure can generally be executed by the server 105. Accordingly, the crop anomaly detection apparatus provided in the embodiments of the present disclosure can generally be located in the server 105. The crop anomaly detection method provided in the embodiments of the present disclosure can also be executed by a server or server cluster that is different from the server 105 and that is capable of communicating with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the crop anomaly detection apparatus provided in the embodiments of the present disclosure can also be located in a server or server cluster that is different from the server 105 and that is capable of communicating with the terminal devices 101, 102, 103 and / or the server 105.

[0037] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is merely illustrative. Any number of terminal devices, networks and servers may be provided as required.

[0038] The following will be based on Figure 1 The scene described by Figure 2 The crop anomaly detection method of the disclosed embodiment is described in detail.

[0039] Figure 2 The flowchart of the crop abnormality detection method according to the embodiment of the present disclosure is schematically shown. Figure 2 As shown, the crop abnormality detection method of this embodiment includes operations S210 to S220.

[0040] In operation S210, a plurality of images are acquired. The plurality of images are acquired by an image acquisition device at a plurality of image acquisition points based on a preset angle and a preset direction within a preset height range. Each image includes a target portion of the crop to be monitored. The preset angle is the pitch angle when the image acquisition device acquires the image, and the preset direction is a direction perpendicular to the crop row.

[0041] For example, the image acquisition device may be a drone. More specifically, the image acquisition device may include two parts: a camera and an aircraft. The specific parameters may be shown in Table 1, which shows the parameters involved in the camera part and the parameters involved in the aircraft part of the image acquisition device.

[0042] Table 1 Parameters of image acquisition device

[0043]

[0044] The multiple images obtained may be images collected by an image acquisition device such as a drone through point sampling. The image acquisition points may be pre-set or may be set to capture images at each distance. Figure 3 The schematic diagram of obtaining an image according to an embodiment of the present disclosure is schematically shown. Figure 3As shown, there are multiple image collection points 320, and the image collection device 310 collects images at each image collection point 320 to obtain multiple images.

[0045] Taking wheat as the crop and the ear part of the wheat as the target part as an example, the preset angle can be an angle at which a larger area of the ear part of the wheat can be collected, Figure 4 The left part of FIG. 6 schematically shows images obtained according to another embodiment of the present disclosure, Figure 4 The left part of FIG. 6 schematically shows images obtained according to another embodiment of the present disclosure, Figure 4 As shown in the right part of FIG. 6, when the pitch angle of the image collection device reaches 50 degrees, the side features of the ear part of the crop can be displayed in a larger area, and thus the preset angle can be set as 50 degrees, so that the accuracy of detection can be improved by collecting images from the preset angle.

[0046] In addition to the angle at which the image collection device collects images, the direction in which the images are collected also affects the accuracy of monitoring. Taking wheat as the crop and the ear part of the wheat as the target part as an example, Figure 5 The left part of FIG. 6 schematically shows images obtained according to another embodiment of the present disclosure, Figure 5 The left part of FIG. 6 schematically shows images obtained according to another embodiment of the present disclosure, Figure 5 The right part of FIG. 6 schematically shows images obtained according to another embodiment of the present disclosure. As shown, Figure 5 As shown, the images collected perpendicularly to the direction of the row ridge or the images collected between the ridges can better display the ear part of the crop than the images collected parallel to the direction of the row ridge, and thus the area of the ear part presented from the images collected perpendicularly to the direction of the row ridge is larger, and the accuracy of detection is higher.

[0047] In addition, the sampling interval involved in the present disclosure can be 10 meters, that is, images are collected every 10 meters in the area to be detected, the sampling time can be from 10:00 to 14:00, the camera sensitivity can be 100-200, the camera shutter can be 1 / 3200 seconds, the camera pixels can be more than 20 million, and the field angle of view of the camera can be 70°-80°.

[0048] It can be understood that the images collected from the preset angle and the preset direction can better display the features of the ear part of the wheat, so as to improve the accuracy of detecting whether it is abnormal and more accurately evaluate the overall situation of the entire field.

[0049] In operation S220, the plurality of images are processed by using the trained semantic segmentation model to determine the area of the abnormal part in the plurality of images.

[0050] The semantic segmentation model can adopt a U-shaped network (U-Net) model. The semantic segmentation model can classify and label each pixel according to semantic information contained in the image, thereby accurately segmenting a specific target. The dataset can be divided into a training set and a validation set, an initial semantic segmentation model is trained using the dataset to obtain a semantic segmentation model to be verified, the semantic segmentation model to be verified is verified using the validation set to obtain an accuracy rate, and based on the accuracy rate, it is determined whether the semantic segmentation model can be used as a trained semantic segmentation model to detect abnormal conditions in the image.

[0051] The plurality of images collected by the image collection device are input into the trained semantic segmentation model, and an image segmentation result, i.e., an image in which an abnormal part is segmented, can be obtained, so that the area of the abnormal part in each image can be determined.

[0052] It can be understood that, since the plurality of images can better show the target part, e.g., the ear of the wheat, the area of the abnormal part in the plurality of images can also be more accurately determined by processing the plurality of images using the trained semantic segmentation model.

[0053] According to an embodiment of the present disclosure, the trained semantic segmentation model is trained by: obtaining a labeled dataset, the labeled dataset including a plurality of sample images and corresponding labeled images, the corresponding labeled images representing images in which abnormal parts in the sample images are labeled; dividing the labeled dataset into a training set and a validation set according to a preset ratio, the preset ratio being determined based on a data amount of the labeled dataset; training an initial semantic segmentation model using the training set and the validation set to obtain the trained semantic segmentation model.

[0054] The labeled dataset can include a plurality of sample images and corresponding labeled images, wherein the sample images can be images collected by the image collection device, and the corresponding labeled images can be images manually labeled by professional agricultural technicians on the collected images, and the images can be obtained by circling the target of interest in the images, and the target of interest can be an abnormal part. The labeled dataset is divided into a training set and a validation set according to a preset ratio, in order to ensure sufficient training samples and reasonable validation size, the preset ratio can be determined based on the data amount of the labeled dataset, and in the case of sufficient data amount, it is generally recommended to divide the labeled dataset into a training set and a validation set in a ratio of 7:3.

[0055] Figure 6 A schematic diagram of a dataset according to an embodiment of the present disclosure is schematically shown, as shown in Figure 6 The labeled dataset includes a training set and a validation set, the training set includes sample images 610 and corresponding labeled images 620, and the validation set includes sample images 630 and corresponding labeled images 640.

[0056] It can be understood that training the initial semantic segmentation model by using the training set and the validation set to obtain the trained semantic segmentation model can ensure that the semantic segmentation model has high accuracy.

[0057] According to an embodiment of the present disclosure, training the initial semantic segmentation model by using the training set and the validation set to obtain the trained semantic segmentation model comprises: training the initial semantic segmentation model by using the training set to obtain a semantic segmentation model to be verified; determining the accuracy of the semantic segmentation model to be verified based on the validation set; in the case that the accuracy is greater than or equal to a first preset threshold, taking the semantic segmentation model to be verified as the trained semantic segmentation model; in the case that the accuracy is less than the first preset threshold, repeating the following steps until the accuracy is greater than the first preset threshold: enriching data and / or enhancing data of the training set to obtain an updated training set; training the semantic segmentation model to be verified by using the updated training set and verifying based on the validation set; the data enrichment comprises adding sample images and corresponding labeled images in the training set, and the data enhancement comprises at least one of the following: adjusting the saturation of the image, adjusting the hue of the image, image rotation, image scaling.

[0058] The initial semantic segmentation model can be an untrained U-Net model, and training the model by using the training set can obtain the semantic segmentation model to be verified, and then the semantic segmentation model to be verified can be verified by using the validation set.

[0059] Specifically, inputting the sample images in the validation set into the semantic segmentation model to be verified can obtain corresponding output images, and based on the obtained output images and the corresponding labeled images, the accuracy of the semantic segmentation model to be verified can be obtained, and based on the accuracy, whether the semantic segmentation model to be verified needs to be trained again can be evaluated. If the accuracy is greater than the first preset threshold, the semantic segmentation model can be directly taken as the trained semantic segmentation model, and if the accuracy is less than the first preset threshold, it indicates that the semantic segmentation model needs to be trained.

[0060] For example, the first preset threshold can be 80%, and in the case that the accuracy is less than 80%, the training set can be enriched and / or enhanced to obtain an updated training set, and then the semantic segmentation model to be verified is trained by using the updated training set and verified based on the validation set, and the whole process is repeated until the accuracy is greater than or equal to the first preset threshold. The data enrichment can include adding sample images and corresponding labeled images in the training set, i.e. data that has not been used in the previous training process, and the data enhancement comprises at least one of the following: adjusting the saturation of the image, adjusting the hue of the image, image rotation, image scaling.

[0061] It can be understood that the semantic segmentation model obtained by this training method can ensure that the semantic segmentation model has a higher accuracy, thereby ensuring that the area of ​​the abnormal part obtained based on the semantic segmentation model is more accurate.

[0062] According to an embodiment of the present disclosure, the method further includes: determining a preset height range based on the resolution of images captured by image capture devices disposed at different vertical heights.

[0063] In order to be able to clearly monitor the occurrence of ear diseases, the present invention adopts a method of low-altitude close observation by unmanned aerial vehicles. Lowering the flight altitude can enable more accurate monitoring, but it also reduces the efficiency of monitoring large areas. Therefore, image acquisition can be carried out at a height of 1.5-2.0 meters from the canopy, and at the same time, large-scale field plots can be sampled to improve data acquisition efficiency. Taking the parameters involved in this application as an example, the image resolution of the sample taken at a height of 1.5 meters from the canopy of the crop is about 0.7mm, and the image resolution of the sample taken at a height of 2.0 meters is about 1mm, both of which can accurately monitor the characteristics of the ear. Therefore, the preset height range can be 1.5 meters to 2 meters.

[0064] It is understandable that capturing images within a preset height range can ensure that the images captured by the image capture device have a good resolution without affecting the capture efficiency.

[0065] According to an embodiment of the present disclosure, the method further includes: projecting the area of ​​the abnormal part onto a horizontal plane to obtain the area of ​​the abnormal region; determining the field of view area of ​​the image acquisition device based on the height, preset angle and field of view angle of the image acquisition device during image acquisition; and obtaining the area ratio of the abnormal region based on the area of ​​the abnormal region and the field of view area of ​​the image acquisition device.

[0066] To more accurately calculate the ratio of abnormal to normal areas, the area of ​​the abnormal area in the image can be projected onto a horizontal plane to obtain the area of ​​the abnormal region. The area of ​​the abnormal region and the field of view of the image acquisition device can then be used to calculate the abnormal region area ratio. For example, if the abnormal area is wheat black ear, in practical applications, the abnormal region area ratio can be used to represent the proportion of wheat black ear.

[0067] Specifically, the calculation process of projecting the area of ​​the abnormal part in the image onto the horizontal plane to obtain the area of ​​the abnormal region can be shown as formula (1).

[0068] Formula (1)

[0069] in, represents the area of ​​the abnormal region, specifically the area of ​​the monitored pest and disease area, n is the total number of pixels of the abnormal part detected from the image, H is the height of the image acquisition device from the crop canopy, Indicates the physical size of the camera pixel, f represents the focal length of the camera, L represents the physical size of the camera's charge coupled device (CCD) along the pixel row direction, It represents the distance of the pixel from the upper left corner on the CCD in the direction of the pixel row. Indicates the pitch angle of the image acquisition device.

[0070] After obtaining the area of ​​the abnormal region, the area ratio of the abnormal region can be obtained based on the area of ​​the abnormal region and the field of view area of ​​the image acquisition device. The specific calculation process is shown in formula (2).

[0071] Formula (2)

[0072] Among them, r represents the proportion of abnormal area, which can be the proportion of area where pests, diseases and weeds occur. represents the area of ​​abnormal region, H represents the height of image acquisition device from crop canopy, represents the pitch angle of the image acquisition device, Represents the field of view of the image acquisition device.

[0073] Figure 7 A schematic diagram of acquiring an image according to another embodiment of the present disclosure is schematically shown. Figure 7 As shown, the image acquisition device 710 acquires an image of the crop 720. Figure 7 The height, pitch angle, and field of view involved in this disclosure can be intuitively understood.

[0074] It is understandable that due to optical distortion of captured image pixels, directly using the image to determine the abnormal area ratio cannot accurately reflect the actual situation. Therefore, the present disclosure takes into account the three-dimensional projection relationship and calculates the abnormal area area by projecting the area of ​​the abnormal part onto the horizontal plane. Based on the abnormal area area and the field of view of the image acquisition device, the abnormal area ratio is calculated. This can more accurately and realistically reflect the abnormal situation of the corresponding area in the image in actual application scenarios.

[0075] According to an embodiment of the present disclosure, the method further includes: obtaining an abnormal situation scatter plot based on the proportion of abnormal areas corresponding to each image acquisition point; performing spatial interpolation on the abnormal situation scatter plot to obtain an abnormal area distribution map; and according to the abnormality degree of multiple abnormal areas in the abnormal area distribution map, when the abnormality degree is less than a second preset threshold, determining that the crop in the abnormal area corresponding to the abnormality degree is a seed.

[0076] Figure 8 A schematic diagram of the proportion of the abnormal area according to an embodiment of the present disclosure is shown, i.e., a schematic diagram of the abnormal condition scatter plot is shown. Since the collected images are images collected at image collection points, and the entire area to be detected includes areas where images are not collected, it is necessary to perform spatial interpolation on the abnormal condition scatter plot to obtain an abnormal area distribution map.

[0077] Figure 9 A schematic diagram of the abnormal area distribution according to an embodiment of the present disclosure is shown, as shown in Figure 9 The abnormal area distribution map obtained by spatial interpolation on the abnormal condition scatter plot can reflect the severity of the abnormality of the entire area to be detected. Specifically, the severity of each abnormal area can be determined according to the result of spatial interpolation. Taking crops as wheat and the abnormal part as the ear of wheat as an example, spatial interpolation of the abnormal condition scatter plot to obtain an abnormal area distribution map can accurately grasp how many ears per mu of the field have diseases, and make precise decision-making accordingly.

[0078] Taking the area to be detected as a production field as an example, the second preset threshold can be one ten-thousandth. If the severity of the abnormal area is less than one ten-thousandth, it can be concluded that the abnormal condition of the area is not serious, and then the crops in the area can be used as seeds.

[0079] It can be understood that by performing spatial interpolation on the abnormal condition scatter plot to obtain an abnormal area distribution map, the overall situation of the area to be detected can be evaluated based on the abnormal area distribution map.

[0080] Figure 10 A structural block diagram of a crop abnormality detection device according to an embodiment of the present disclosure is shown.

[0081] As shown in Figure 10 The crop abnormality detection device 1000 of this embodiment includes.

[0082] The acquisition module 1010 is configured to acquire a plurality of images, the plurality of images being acquired by an image acquisition device at a plurality of image collection points within a preset height range based on a preset angle and a preset direction. Each image includes a target part of a crop to be monitored. The preset angle is the pitch angle when the image acquisition device collects the image. The preset direction is the row direction perpendicular to the crop. In an embodiment, the acquisition module 1010 can be configured to perform the operation S210 described above, and thus the details are not repeated here.

[0083] The processing module 1020 is configured to process the multiple images using the trained semantic segmentation model to determine the areas of abnormal parts in the multiple images. In one embodiment, the processing module 1020 may be configured to perform the operation S220 described above, which will not be described in detail here.

[0084] According to an embodiment of the present disclosure, the trained semantic segmentation model is trained using the following operations: obtaining a labeled data set, the labeled data set including multiple sample images and corresponding labeled images, the corresponding labeled images representing images with abnormal parts in the sample images labeled; dividing the labeled data set into a training set and a validation set according to a preset ratio, the preset ratio being determined based on the data volume of the labeled data set; using the training set and the validation set to train an initial semantic segmentation model to obtain a trained semantic segmentation model.

[0085] According to an embodiment of the present disclosure, the initial semantic segmentation model is trained using a training set and a validation set to obtain a trained semantic segmentation model, including: using the training set to train the initial semantic segmentation model to obtain a semantic segmentation model to be verified; based on the validation set, determining the accuracy of the semantic segmentation model to be verified; when the accuracy is greater than or equal to a first preset threshold, using the semantic segmentation model to be verified as the trained semantic segmentation model; when the accuracy is less than the first preset threshold, repeating the following steps until the accuracy is greater than the first preset threshold: performing data enrichment and / or data enhancement on the training set to obtain an updated training set; using the updated training set to train the semantic segmentation model to be verified and verifying it based on the validation set; data enrichment includes adding sample images and corresponding annotated images to the training set, and data enhancement includes at least one of the following: adjusting the saturation of the image, adjusting the hue of the image, rotating the image, and scaling the image.

[0086] According to an embodiment of the present disclosure, the crop anomaly detection apparatus 1000 further includes a first determination module for determining a preset height range based on the resolution of images captured by image capture devices disposed at different vertical heights.

[0087] According to an embodiment of the present disclosure, the crop abnormality detection device 1000 further includes a projection module, a second determination module, and a first processing module. The projection module is used to project the area of ​​the abnormal part onto a horizontal plane to obtain the area of ​​the abnormal region; the second determination module is used to determine the field of view area of ​​the image acquisition device based on the height during image acquisition, a preset angle, and the field of view angle of the image acquisition device; and the first processing module is used to obtain the area ratio of the abnormal region based on the area of ​​the abnormal region and the field of view area of ​​the image acquisition device.

[0088] According to an embodiment of the present disclosure, the crop anomaly detection apparatus 1000 further comprises a second processing module, an interpolation module, and a third determining module. The second processing module is configured to obtain an anomaly condition scatter plot based on the anomaly area proportion corresponding to each image collection point. The interpolation module is configured to perform spatial interpolation on the anomaly condition scatter plot to obtain an anomaly area distribution map. The second processing module is configured to determine, according to the anomaly degree of the multiple anomaly areas in the anomaly area distribution map, that the crop in the anomaly area corresponding to the anomaly degree is a seed if the anomaly degree is less than a second preset threshold

[0089] According to an embodiment of the present disclosure, any of the acquisition module 1010 and the processing module 1020 can be combined in one module, or any of the modules can be split into multiple modules. Alternatively, at least part of the function of one or more of the modules can be combined with at least part of the function of another module, and implemented in one module. According to an embodiment of the present disclosure, at least one of the acquisition module 1010 and the processing module 1020 can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on chip, a system on substrate, a system on package, an application specific integrated circuit (ASIC), or any other reasonable manner of integrating or packaging a circuit, etc. hardware or firmware, or implemented in any one of software, hardware and firmware or in a proper combination of any of them. Alternatively, at least one of the acquisition module 1010 and the processing module 1020 can be at least partially implemented as a computer program module which can perform corresponding functions when executed.

[0090] Figure 11 A block diagram of an electronic device suitable for implementing the crop anomaly detection method according to an embodiment of the present disclosure is schematically shown.

[0091] As shown in Figure 11 The electronic device 1100 according to an embodiment of the present disclosure includes a processor 1101 which can perform various appropriate actions and processes according to programs stored in a read only memory (ROM) 1102 or loaded from a storage portion 1108 into a random access memory (RAM) 1103. The processor 1101 can include, for example, a general purpose microprocessor (e.g., a CPU), an instruction set processor, and / or a related chipset, and / or a special purpose microprocessor (e.g., an application specific integrated circuit (ASIC)), etc. The processor 1101 can also include an on-board memory for cache use. The processor 1101 can include a single processing unit or multiple processing units for performing different actions of the method processes according to embodiments of the present disclosure.

[0092] In the RAM 1103, various programs and data required for the operation of the electronic device 1100 are stored. The processor 1101, the ROM 1102, and the RAM 1103 are connected to each other via the bus 1104. The processor 1101 performs various operations of the method flow according to the embodiments of the present disclosure by executing the programs in the ROM 1102 and / or the RAM 1103. It should be noted that the programs can also be stored in one or more memories other than the ROM 1102 and the RAM 1103. The processor 1101 can also implement the method provided by the embodiments of the present disclosure by executing the programs stored in the one or more memories.

[0093] According to the embodiments of the present disclosure, the electronic device 1100 can further include an input / output (I / O) interface 1105, which is also connected to the bus 1104. The electronic device 1100 can further include one or more of the following components connected to the I / O interface 1105: an input part 1106 including a keyboard, a mouse, and the like; an output part 1107 including a cathode ray tube (CRT), a liquid crystal display (LCD), and the like, and a speaker, and the like; a storage part 1108 including a hard disk, and the like; and a communication part 1109 including a network interface card such as a LAN card, a modem, and the like. The communication part 1109 performs communication processing via a network such as the Internet. A drive 1110 is also connected to the I / O interface 1105 as necessary. A removable medium 1111 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like is mounted on the drive 1110 as necessary, so that a computer program read out therefrom is installed in the storage part 1108 as necessary.

[0094] The present disclosure also provides a computer readable storage medium, which can be included in the device / apparatus / system described in the above embodiments; or can exist separately without being assembled into the device / apparatus / system. The above computer readable storage medium carries one or more programs, when the one or more programs are executed, the method according to the embodiments of the present disclosure is implemented.

[0095] According to an embodiment of the present disclosure, the computer readable storage medium can be a nonvolatile computer readable storage medium, for example, can include, but is not limited to, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any appropriate combination thereof. In the present disclosure, the computer readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present disclosure, the computer readable storage medium can include one or more memories such as the ROM 1102 and / or the RAM 1103 described above and / or one or more memory other than the ROM 1102 and the RAM 1103.

[0096] Embodiments of the present disclosure also include a computer program product that includes a computer program containing program codes for executing the methods shown in the flowcharts. When the computer program product is run in a computer system, the program codes are used to make the computer system implement the methods provided by the embodiments of the present disclosure.

[0097] The above-described functions defined in the system / device of the embodiments of the present disclosure are performed when the computer program is executed by the processor 1101. According to an embodiment of the present disclosure, the system, device, module, unit, etc. described above can be implemented by computer program modules.

[0098] In one embodiment, the computer program can rely on a tangible storage medium such as an optical storage device, a magnetic storage device, etc. In another embodiment, the computer program can also be transmitted, distributed, and downloaded in the form of a signal on a network medium and installed and downloaded through the communication part 1109 and / or installed from the detachable medium 1111. The program codes contained in the computer program can be transmitted by any appropriate network medium, including but not limited to wireless, wired, etc., or any appropriate combination thereof.

[0099] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 1109 and / or installed from the detachable medium 1111. When the computer program is executed by the processor 1101, the above-described functions defined in the system of the embodiments of the present disclosure are performed. According to an embodiment of the present disclosure, the system, device, apparatus, module, unit, etc. described above can be implemented by computer program modules.

[0100] It should be noted that in the technical solutions of the present disclosure, the collection, storage, use, processing, transmission, provision, disclosure and application of user personal information comply with relevant laws and regulations, necessary security measures are taken, and the public order and good customs are not violated. In the technical solutions of the present disclosure, the authorization or consent of the user is obtained before the user personal information is acquired or collected.

[0101] According to the embodiments of the present disclosure, the program code for executing the computer programs provided by the embodiments of the present disclosure can be written in any combination of one or more programming languages, and specifically, these computer programs can be implemented by using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. The program code can be executed completely on a user computing device, partially on a user device, partially on a remote computing device, or completely on a remote computing device or server. In the case involving a remote computing device, the remote computing device can be connected to the user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, connected to the Internet through an Internet service provider).

[0102] The flowcharts and block diagrams in the drawings illustrate the possible implementation architectures, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowcharts or block diagrams can represent a module, a program segment, or a part of code containing one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in different orders than those noted in the drawings. For example, two blocks that are represented in succession can actually be executed in parallel, and sometimes they can be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams or flowcharts, and the combination of blocks in the block diagrams or flowcharts, can be implemented by a dedicated hardware-based system that performs the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0103] Those skilled in the art can understand that the features described in various embodiments and / or claims of the present disclosure can be combined and / or integrated in various combinations, even if such combinations are not explicitly described in the present disclosure. In particular, the features described in various embodiments and / or claims of the present disclosure can be combined and / or integrated in various combinations without departing from the spirit and teachings of the present disclosure. All such combinations and / or integrations fall within the scope of the present disclosure.

[0104] The above describes embodiments of the present disclosure. However, these embodiments are merely for illustrative purposes, and are not intended to limit the scope of the present disclosure. Although each embodiment is described above separately, this does not mean that the measures in each embodiment cannot be used advantageously in combination. The scope of the present disclosure is defined by the appended claims and their equivalents. Those skilled in the art can make various substitutions and modifications without departing from the scope of the present disclosure, and these substitutions and modifications should all fall within the scope of the present disclosure.

Claims

1. A crop anomaly detection method, characterized in that: The method comprises: Acquiring a plurality of images, the plurality of images being acquired by an image acquisition device at a plurality of image acquisition points based on a preset angle and a preset direction within a preset height range, wherein each image includes a target portion of the crop to be monitored, the preset angle being a pitch angle when the image acquisition device acquires the image, and the preset direction being a direction perpendicular to the crop row; The plurality of images are processed using the trained semantic segmentation model to determine areas of abnormal parts in the plurality of images.

2. The method according to claim 1, characterized in that The trained semantic segmentation model is trained using the following operations: Acquire a labeled data set, the labeled data set including a plurality of sample images and corresponding labeled images, the corresponding labeled images representing images in which abnormal parts in the sample images are labeled; Dividing the labeled data set into a training set and a validation set according to a preset ratio, wherein the preset ratio is determined based on the data volume of the labeled data set; An initial semantic segmentation model is trained using the training set and the validation set to obtain the trained semantic segmentation model.

3. The method according to claim 2, characterized in that The step of training the initial semantic segmentation model using the training set and the validation set to obtain the trained semantic segmentation model comprises: Using the training set to train an initial semantic segmentation model to obtain a semantic segmentation model to be verified; Determining the accuracy of the semantic segmentation model to be verified based on the verification set; When the accuracy is greater than or equal to a first preset threshold, the semantic segmentation model to be verified is used as a trained semantic segmentation model; If the accuracy rate is less than the first preset threshold, repeat the following steps until the accuracy rate is greater than the first preset threshold: Perform data enrichment and / or data augmentation on the training set to obtain an updated training set; The semantic segmentation model to be verified is trained using the updated training set and verified based on the verification set; the data enrichment includes adding sample images and corresponding annotated images to the training set, and the data enhancement includes at least one of the following: adjusting the saturation of the image, adjusting the hue of the image, rotating the image, and scaling the image.

4. The method according to claim 1, wherein The method further comprises: The preset height range is determined based on the resolution of images captured by image capture devices arranged at different vertical heights.

5. The method according to claim 1, wherein The method further comprises: Projecting the area of ​​the abnormal part onto a horizontal plane to obtain the area of ​​the abnormal region; Determining the field of view area of ​​the image acquisition device based on the height at the time of image acquisition, the preset angle, and the field of view angle of the image acquisition device; Based on the area of ​​the abnormal region and the field of view area of ​​the image acquisition device, the area ratio of the abnormal region is obtained.

6. The method according to claim 5, characterized in that The method further comprises: Obtaining an abnormal situation scatter plot based on the proportion of abnormal areas corresponding to each image acquisition point; Performing spatial interpolation on the abnormal situation scatter plot to obtain an abnormal area distribution map; According to the abnormality degrees of the plurality of abnormal areas in the abnormal area distribution map, when the abnormality degrees are less than a second preset threshold, it is determined that the crops in the abnormal areas corresponding to the abnormality degrees are seeds.

7. A remote sensing monitoring device, characterized in that: The device comprises: an acquisition module, configured to acquire a plurality of images, the plurality of images being acquired by an image acquisition device at a plurality of image acquisition points based on a preset angle and a preset direction within a preset height range, each of the images including a target portion of the crop to be monitored, the preset angle being a pitch angle of the image acquisition device when acquiring the image, and the preset direction being a direction perpendicular to the crop row; A processing module is used to process the multiple images using the trained semantic segmentation model to determine the areas of abnormal parts in the multiple images.

8. An electronic device comprising: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors are enabled to perform the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to perform the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, wherein the computer program is stored on at least one of a readable storage medium and an electronic device, and when the computer program is executed by a processor, implements the method according to any one of claims 1 to 6.

Images