Anomaly detection apparatus, learning apparatus, anomaly detection method, and program

Abstract

To improve the accuracy of anomaly detection using training images prepared appropriately.SOLUTION: An anomaly detection apparatus includes: an acquisition unit which acquires a farmland image; and a detection unit which detects an anomaly of crops in a farmland by comparing pixel values between a reference image created by a trained generative model and a farmland image in an anomaly detection stage. The trained generative model is trained by leaning of a generative adversarial network so that a classification model discriminates a positive-sample image from a negative-sample image, the positive-sample image being generated so as to include a plurality of growth patterns using a crop growth model configured to estimate a future farmland image on receipt of the farmland image, the negative-sample image being obtained by inputting a latent variable to a generative model, and outputs a reference image by inputting a trained latent variable to the trained generative model. The trained latent variable has been trained so as to minimize an error index between the reference image and the farmland image in a learning stage.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an anomaly detection device, a learning device, an anomaly detection method, and a program. [Background technology]

[0002] Previously, attempts have been made to detect abnormalities in crops based on images of farmland captured from the air by drones or satellites. For example, a system is known that converts images of farmland captured by drones into vegetation indices or heat maps, compares them with past image data, and predicts the number of germinated seeds, the number of non-germinated seeds, and the germination and non-germination rates (Patent Document 1). Furthermore, when applying machine learning to this type of technology, there is the problem of difficulty in preparing a sufficient number of training images. To address this issue, a technology is known that adds synthetic images generated by AR-GAN to a dataset (Non-Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-153109 [Patent Document 2] Patent No. 7316004 [Non-patent literature]

[0004] [Non-Patent Document 1] “Unsupervised image translation using adversarial networks for improved plant disease recognition”, Haseeb Nazki et al., Computers and Electronics in Agriculture Volume 168, January 2020, 105117 [Non-patent document 2] “ORYZA2000: modeling lowland rice”, BAM Bouman et al., (2001), International Rice Research Institute, Wageningen University and Research Centre, Los Banos, Philippines, Wageningen, Netherlands [Non-patent document 3] “The rice simulation model SIMRIW and its testing.”, T. HORIE et al., (1995), Modeling the impact of climate change on rice production in Asia. CABI, UK, IRRI, Philippines, pp. 95-139 [Non-patent document 4] ”Simulation of the effects of genotype and N availability on rice growth and yield response to an atmospheric elevated CO2concentration”, Hiroe Yoshida et al., (2011), Field Crops Research 124(3) 433-440. Summary of the Invention [Problem to be solved by the invention]

[0005] With conventional techniques, it is difficult to prepare an appropriate and sufficient number of learning images, and there are cases where the accuracy of anomaly detection is not sufficient.

[0006] The present invention has been made in consideration of these circumstances, and one of its objects is to provide an anomaly detection device, a learning device, an anomaly detection method, and a program that are capable of improving the accuracy of anomaly detection by using a sufficient number of appropriately prepared learning images. [Means for solving the problem]

[0007] An anomaly detection device according to the present invention comprises an acquisition unit that acquires images of farmland taken from above, and a detection unit that detects abnormalities in crops in the farmland by comparing pixel values between a reference image created by a trained generative model and the farmland image in the anomaly detection stage, or derives the probability of an abnormality occurring. The trained generative model is trained by learning a generative adversarial network so that a classification model can distinguish between positive example images generated to include multiple growth patterns using a crop growth model that estimates future farmland images when farmland images are input, and negative example images obtained by inputting latent variables into the generative model. The reference image is output by inputting trained latent variables into the trained generative model, and the trained latent variables have been trained so that the error index between the reference image and the farmland image in the learning stage is minimized. [Effects of the Invention]

[0008] According to each of the above aspects, it is possible to improve the accuracy of anomaly detection by using a sufficient number of appropriately prepared learning images. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram illustrating an example of a usage environment and configuration of an anomaly detection device 100. FIG. [Figure 2] FIG. 2 is a diagram showing an overview of processing at an anomaly detection stage in the anomaly detection device 100. [Figure 3] FIG. 1 is a diagram illustrating an overview of the process of training a trained generative model and a classification model. [Figure 4] FIG. 10 is a diagram illustrating an outline of a process for generating a positive example image. [Figure 5] FIG. 1 is a diagram illustrating an outline of a learning process of latent variables. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of an anomaly detection device, a learning device, an anomaly detection method, and a program according to the present invention will be described with reference to the drawings.

[0011] FIG. 1 illustrates an example of the usage environment and configuration of the anomaly detection device 100. For example, the anomaly detection device 100 acquires images captured by one or more satellites 10 via a satellite image provider server 20 and a network NW, or acquires images captured by a drone 30 via a relay device 40 and a network NW (either one is acceptable, or images may be acquired by another method). These images are images of farmland captured from above. Note that the images captured by the drone 30 may be orthoimages created by stitching images together. The anomaly detection device 100 provides detection results to a terminal device 50 via the network NW. The network NW may be any network, such as a LAN, a WAN, or the Internet, and may be wired or wireless. Note that the image acquisition method is not limited to this. Images may also be acquired by inserting images stored on a storage medium into a drive device of the anomaly detection device 100. Furthermore, the images may be preprocessed before being passed to the anomaly detection device 100.

[0012] The anomaly detection device 100 includes, for example, an acquisition unit 110, a detection unit 120, a learning unit 130, and a storage unit 150. The learning unit 130 may be configured as a separate device (learning device) from the anomaly detection device 100, but is considered to be included in the anomaly detection device 100 here. The components other than the storage unit 150 are implemented by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be implemented by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), a GPU (Graphics Processing Unit), or an SOC (System On Chip), or may be implemented by a combination of software and hardware. The program may be stored in advance in a storage device (a storage device with a non-transitory storage medium) such as an HDD (Hard Disk Drive) or flash memory, or may be stored in a removable storage medium (a non-transitory storage medium) such as a DVD or CD-ROM and installed by inserting the storage medium into a drive device. The anomaly detection device 100 may be configured as a dedicated device, or may be a general-purpose device such as a smartphone or tablet terminal with the program installed.

[0013] The storage unit 150 is a RAM, HDD, flash memory, etc. The storage unit 150 stores information such as a trained generative model 152, trained latent variables 154, and a crop growth model 156.

[0014] As described above, the acquisition unit 110 acquires farmland images captured from above. The detection unit 120 detects abnormalities in crops in the farmland by comparing pixel values between a reference image created by the trained generative model and the farmland image in the anomaly detection stage. The learning unit 130 generates a trained generative model 152 and trained latent variables 154. Details of each will be explained in order.

[0015] [Anomaly detection stage] FIG. 2 is a diagram illustrating an overview of the anomaly detection stage processing in the anomaly detection device 100. The detection unit 120 inputs the learned latent variables generated by learning from the farmland image (T) to be detected as an anomaly into the corresponding trained generative model, and derives a judgment result based on a comparison image obtained by calculating the pixel-by-pixel difference between the generated reference image and the farmland image (T). Here, the numbers in parentheses (i = 1 to n) are growth pattern identifiers for the combination of the trained latent variables and trained generative model set to include multiple growth patterns (described below). Furthermore, T is the date on which the farmland image was captured. The farmland image (T) does not need to be acquired every day and can be acquired on any day. For example, the detection unit 120 sets the judgment result R(i) to 1 if the comparison image (i) contains a pixel value equal to or greater than a threshold, and sets the judgment result R(i) to zero if the comparison image (i) does not contain a pixel value equal to or greater than a threshold.

[0016] Then, the detection unit 120 derives the value p obtained by equation (1) as the abnormality occurrence probability.

[0017]

number

[0018] In equation (1), the value p is the simple average of the determination results R(i). However, as shown in equation (2), the detection unit 120 may obtain a weighted average of the determination results R(i) as the value p. In the equation, v(i) is a weight assigned to each of the aforementioned "plurality of growth patterns." By doing so, for example, by assigning a large weight to a determination result derived using a highly reliable crop growth model 156, meteorological data, or internal parameters, the reliability of the detection results of the anomaly detection device 100 can be improved.

[0019]

number

[0020] When the value p is equal to or greater than a threshold, the detection unit 120 may output information indicating that an abnormality has occurred as a detection result, or may output the value p as is.

[0021] [Learning stage] 3 to 5 are diagrams showing an overview of the learning stage processing in the anomaly detection device 100. The learning unit 130 first learns a trained generative model and a classification model, and then learns the corresponding trained latent variables. FIG. 3 is a diagram showing an overview of the processing for training a trained generative model and a classification model. The learning unit 130 trains the classification model and the generative model so that when a positive example image (explained in the next paragraph) is input to the classification model, information indicating a positive result is output, and when a negative example image is input to the classification model, information indicating an error is output. Negative example images are generated by inputting latent variables in the form of images of random noise into the generative model. The classification model is a multi-layer model, and information indicating whether it is true or false is output from the output layer.

[0022] FIG. 4 is a diagram illustrating an outline of the process for generating positive example images. The learning unit 130 uses a farmland image (T0) at an arbitrary time point as an input image and generates positive example images that include multiple growth patterns using a crop growth model 156 that estimates future farmland images. The positive example images are assigned a crop growth model 156 identifier i and a growth day identifier t. The learning unit 130 may use different crop growth models 156, or may generate multiple growth patterns by providing different parameters (time-series weather data, internal parameters, etc.) to the same crop growth model 156. The crop growth model 156 may be, for example, some or all of OYAZA2000 (Non-Patent Document 2), SIMLIW (Non-Patent Document 3), GEMRICE (Non-Patent Document 4), etc., or other crop growth models 156 may be used. These crop growth models 156 output pixel values (images) in a time series by providing initial values for the image, etc., and external parameters, such as weather data. Patent Document 2 describes applying a leaf area index calculated using SIMLIW to a crop canopy radiative transfer simulation model such as PROSAIL to calculate the reflectance (≒ pixel value) of an arbitrary band (color band). For example, the learning unit 130 generates positive example images using this method.

[0023] FIG. 5 is a diagram illustrating an outline of the latent variable learning process. The learning unit 130 learns the learned latent variables so as to minimize the error index L between the reference image (which at this stage is merely an image obtained by inputting the latent variables to be learned into the learned generative model, but is referred to as such because it will become an image equivalent to the reference image after learning is complete) and the farmland image (T). The error index L is a weighted sum of the first error E1 between the reference image and the farmland image (T) and the second error E2 between the feature images obtained by inputting the reference image and the farmland image (T) into the classification model. The feature image is an input image to a layer one layer before the output layer of the classification model. The first error E1 and the second error E2 are each an L1 error. The learning unit 130 performs this process for each learned generative model (i.e., for each crop growth pattern) and each time a farmland image (T) is captured.

[0024] According to the embodiment described above, anomalies are detected by comparing captured farmland images with reference images generated using deep learning and adversarial networks, enabling highly accurate anomaly detection. Furthermore, there is no need to capture and prepare in advance the large number of training images required for deep learning. Once an image of farmland (agricultural field) is captured, a classifier for detecting anomalies in that farmland is generated, enabling repeated and easy anomaly detection. In other words, the accuracy of anomaly detection can be improved by using a sufficient number of appropriately prepared training images.

[0025] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0026] 100 Anomaly detection device 110 Acquisition Department 120 Detection unit 130 Learning unit (learning device) 150 Storage section 152 Trained generative models 154 Trained Latent Variables 156 Crop Growth Model

Claims

1. an acquisition unit that acquires farmland images captured from above; A detection unit that detects abnormalities in crops in the farmland by comparing pixel values of a reference image created by a trained generative model with the farmland image in the anomaly detection stage, or derives the probability of an abnormality occurring; Equipped with The trained generative model is trained by learning a generative adversarial network so that a classification model can distinguish between positive example images generated to include multiple growth patterns using a crop growth model that estimates future farmland images when farmland images are input, and negative example images obtained by inputting latent variables into the generative model, and outputs the reference image by inputting the trained latent variables into the trained generative model; The learned latent variables are learned so as to minimize an error index between the reference image and the farmland image in the learning stage. Anomaly detection device.

2. the error index is a weighted sum of a first error between the reference image and the farmland image in the learning stage and a second error between feature images obtained by inputting the reference image and the farmland image in the learning stage into the classification model, The anomaly detection device according to claim 1.

3. the classification model is a multi-layer model; The feature image is an input image to a layer one input layer later than an output layer in the classification model. The anomaly detection device according to claim 2.

4. Each of the first error and the second error is an L1 error. The abnormality detection device according to claim 2.

5. The system further comprises a learning unit that generates the trained generative model by training a generative adversarial network so that a classification model can distinguish between positive example images generated to include multiple growth patterns using a crop growth model that estimates a future farmland image when a farmland image is input, and negative example images obtained by inputting latent variables into the generative model, and that trains the trained latent variables so that an error index between the reference image and the farmland image in the training stage is minimized. The anomaly detection device according to claim 1.

6. A trained generative model is generated by training a generative adversarial network so that a classification model can distinguish between positive example images generated to include multiple growth patterns using a crop growth model that estimates future farmland images when farmland images are input, and negative example images obtained by inputting latent variables into the generative model; A learning device that learns a learned latent variable so as to minimize an error index between a reference image and the farmland image, The reference image is obtained by inputting the learned latent variables into the learned generative model, and is used to detect abnormalities in crops in the farmland by comparing pixel values with the farmland image in the anomaly detection stage. Learning device.

7. The anomaly detection device Acquiring farmland images captured from above; Detecting abnormalities in crops in the farmland by comparing pixel values between a reference image created by the trained generative model and the farmland image in the anomaly detection stage, or deriving the probability of an abnormality occurring; Run The trained generative model is trained by learning a generative adversarial network so that a classification model can distinguish between positive example images generated to include multiple growth patterns using a crop growth model that estimates future farmland images when farmland images are input, and negative example images obtained by inputting latent variables into the generative model, and outputs the reference image by inputting the trained latent variables into the trained generative model; The learned latent variables are learned so as to minimize an error index between the reference image and the farmland image in the learning stage. Anomaly detection methods.

8. The processor of the anomaly detection device Acquiring farmland images captured from above; Detecting abnormalities in crops in the farmland by comparing pixel values between a reference image created by the trained generative model and the farmland image in the anomaly detection stage, or deriving the probability of an abnormality occurring; A program for executing The trained generative model is trained by learning a generative adversarial network so that a classification model can distinguish between positive example images generated to include multiple growth patterns using a crop growth model that estimates future farmland images when farmland images are input, and negative example images obtained by inputting latent variables into the generative model, and outputs the reference image by inputting the trained latent variables into the trained generative model; The learned latent variables are learned so as to minimize an error index between the reference image and the farmland image in the learning stage. program.

Images