Detection device, detection system, detection method, and model generation device

Abstract

To provide a detection device, detection system, and detection method, which enable accurate detection of a detection target existing in a monitoring target area even in the presence of obstacles or objects with low transmittance between the monitoring target area and an image capturing device.SOLUTION: A detection device of the present invention is for determining whether a determination target image is a detection target image including a detection target object or not, and comprises: an image restoration and generation unit configured to generate a non-detection target image corresponding to the determination target image by inputting data of the determination target image to a learning model configured to use processed image data as input data and non-detection target image data, including no detection target object, corresponding to the processed image as output data; a difference computation unit for generating a differential image of the determination target image and the non-detection target image generated by the image restoration and generation unit; and a detection unit configured to determine whether the determination target image is a detection target image or not according to the differential image generated by the difference computation unit.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a detection device, a detection system, a detection method, and a model generation device that determine whether a determination target image is a detection target image including a detection target object.

Background Art

[0002] Patent Document 1 describes a device that extracts a change region from an image sequentially captured by an imaging device in a monitoring target region, and compares a feature amount of the monitoring target region based on the extracted change region with a set value of a predetermined feature amount to determine the presence or absence of an event.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the device described in Patent Document 1, when there are obstacles such as a safety fence, a fence, or a cover for dust and drip prevention between the monitoring target region and the imaging device, there is a possibility that a detection target object existing in the monitoring target region cannot be detected. For this reason, even when there are obstacles or objects with low transmittance between the monitoring target region and the imaging device, it has been expected to provide a technology capable of accurately detecting a detection target object existing in the monitoring target region.

[0005] The present invention has been made in view of the above problems, and an object thereof is to accurately detect a detection target object existing in a monitoring target area even when there are obstacles or objects with low transmittance between the monitoring target area and the imaging device. A detection device, a detection system, and a detection method are provided. Another object of the present invention is to provide a model generation device capable of generating a learning model for accurately detecting a detection target object existing in a monitoring target area even when there are obstacles or objects with low transmittance between the monitoring target area and the imaging device.

Means for Solving the Problems

[0006] The detection device according to the present invention is a detection device that determines whether a determination target image is a detection target image including a detection target object, and inputs data of the determination target image to a learning model that uses data of a processed image as input data and data of a non-detection target image that does not include the detection target object corresponding to the processed image as output data, thereby generating a non-detection target image corresponding to the determination target image; a difference calculation unit that generates a difference image between the determination target image and the non-detection target image generated by the image restoration generation unit; and a detection unit that determines whether the determination target image is the detection target image based on the difference image generated by the difference calculation unit.

[0007] An image processing unit that executes image processing for emphasizing the features of the detection target object on the difference image generated by the difference calculation unit is provided, and the detection unit may determine whether the determination target image is the detection target image based on the difference image on which the image processing is executed by the image processing unit.

[0008] The image processing unit may perform a masking process on an image area other than a predetermined image area where the detection target object may exist.

[0009] It is preferable to provide a model generation unit that generates the learning model by learning to reconstruct a non-detection target image that is identical to the learning target data or a non-detection target image that approximates the learning target data, using the data of the non-detection target image as learning source data.

[0010] The model generator may be an autoencoder.

[0011] The model generation unit may add data of a determination target image that the detection unit has determined to be a non-detection target image to the learning source data, and generate the learning model again.

[0012] The detection system of the present invention comprises the detection device of the present invention and an imaging unit that captures an image of the monitored area and outputs data of the captured image of the monitored area to the detection device as data of the image to be determined.

[0013] The detection method of the present invention is a detection method for determining whether an image to be determined is a detection target image containing a detection target object, and includes an image restoration generation step for generating a non-detection target image corresponding to the image to be determined by inputting data of the image to be determined into a learning model which has data of a processed image as input data and data of a non-detection target image corresponding to the processed image as output data, a difference calculation step for generating a difference image between the image to be determined and the non-detection target image generated in the image restoration generation step, and a detection step for determining whether the image to be determined is the detection target image based on the difference image generated in the difference calculation step.

[0014] The model generation device of the present invention is a model generation device that generates a learning model for determining whether an image to be determined is a detection target image that contains a detection target, and is equipped with a model generation unit that generates a learning model using data of a processed image as input data and data of a non-detection target image that does not contain the detection target corresponding to the processed image as output data, and the model generation unit generates the learning model by learning to reconstruct a non-detection target image that is identical to the learning target data or a non-detection target image that approximates the learning target data, using the data of the non-detection target image as learning source data. [Effects of the Invention]

[0015] The detection device, detection system, and detection method according to the present invention can accurately detect a detection target present in a monitored area even if an obstacle or an object with low transmittance exists between the monitored area and the imaging device.Furthermore, the model generation device according to the present invention can generate a learning model that accurately detects a detection target present in a monitored area even if an obstacle or an object with low transmittance exists between the monitored area and the imaging device. [Brief explanation of the drawings]

[0016]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

[0017] Hereinafter, with reference to the drawings, the configuration and operation of a detection system according to an embodiment of the present invention will be described.

[0018] 〔Configuration〕 FIG. 1 is a block diagram showing the configuration of a detection system according to an embodiment of the present invention. As shown in FIG. 1, a detection system 1 according to an embodiment of the present invention is a system that determines whether a determination target image is a detection target image including a detection target object, and includes an imaging unit 2, a data acquisition unit 3, a model generation unit 4, a learning database 5, and an arithmetic unit 6. The data acquisition unit 3, the model generation unit 4, and the arithmetic unit 6 function as a detection device according to the present invention.

[0019] The imaging unit 2 is composed of an imaging device such as a camera or an industrial monitoring camera (ITV). The imaging unit 2 captures an image of a monitoring target area and outputs the captured image data of the monitoring target area to the data acquisition unit 3. Specifically, the imaging unit 2 captures images of the monitoring target area before and after the switching of an event or before and after the movement of a movable object, and outputs the captured image data of the monitoring target area to the data acquisition unit 3.

[0020] The data acquisition unit 3 and the model generation unit 4 are functional blocks realized by an arithmetic processing device such as a CPU inside an information processing device executing a computer program. The functions of the data acquisition unit 3 and the model generation unit 4 will be described later.

[0021] The learning database 5 is configured with a non-volatile storage device. The learning database 5 stores learning source data 5a and a learning model 5b. As will be described in detail later, the learning source data 5a is image data of the monitored area at a time when no detection target object is present in the monitored area. The learning model 5b is a machine learning model that takes image data of the monitored area to be judged (data of the image to be judged) as input data and image data of the monitored area at a time when no detection target object is present in the monitored area corresponding to the input data as output data. The model generation unit 4 and the learning database 5 may be configured as a system or device other than the detection system 1.

[0022] The calculation unit 6 is configured by an arithmetic processing device such as a CPU inside the information processing device. The calculation unit 6 functions as an image restoration generation unit 6a, a difference calculation unit 6b, an image processing unit 6c, a detection unit 6d, and an output unit 6e when the arithmetic processing device executes a computer program. The functions of each of these units will be described later. The information processing device that realizes each of these units and the information processing device that realizes the data acquisition unit 3 and the model generation unit 4 may be the same device or different devices. If they are different devices, the information processing devices are connected to each other via a telecommunications line.

[0023] The detection system 1 having such a configuration executes the detection process and learning model generation process described below, thereby enabling accurate detection of a detection target object present in a monitored area even if an obstacle or an object with low transmittance exists between the monitored area and the imaging unit 2. Below, the operation of the detection system 1 when executing the detection process and learning model generation process will be described with reference to the flowchart shown in Fig. 2.

[0024] [Detection process] First, with reference to FIG. 2, the operation of the detection system 1 when performing the detection process will be described.

[0025] 2 is a flowchart showing the flow of detection processing according to one embodiment of the present invention. The flowchart shown in FIG. 2 starts when an execution command for the detection processing is input to the calculation unit 6, and the detection processing proceeds to step S1.

[0026] In the process of step S1, the data acquisition unit 3 inputs the image data of the area to be monitored output from the imaging unit 2 as data of the image to be determined to the calculation unit 6. This completes the process of step S1, and the detection process proceeds to the process of step S2.

[0027] In the process of step S2, the image restoration generation unit 6a inputs the data of the determination target image input by the data acquisition unit 3 as input data to the learning model 5b, thereby generating image data (output image) of the monitoring target area at a time when the detection target object does not exist, reflecting the features of the determination target image. This completes the process of step S2, and the detection process proceeds to the process of step S3.

[0028] In the process of step S3, the difference calculation unit 6b generates a difference image between the determination target image input by the data acquisition unit 3 and the image of the monitoring target area generated by the image restoration generation unit 6a. An image of the difference image generation process is shown in Figure 3. Details of Figure 3 will be described later. This completes the process of step S3, and the detection process proceeds to the process of step S4.

[0029] In the process of step S4, the image processing unit 6c performs image processing such as binarization, masking, edge detection, smoothing, filtering, correction, expansion, and contraction on the generated difference image in order to enhance the features of the detection object, such as its shape, to make it easier to detect the detection object. This completes the process of step S4, and the detection process proceeds to the process of step S5.

[0030] In the process of step S5, the detection unit 6d determines whether a detection target object exists in the differential image by determining whether features such as the shape (target shape) of the object to be dealt with are present in the differential image after image processing. When the target shape is a circular shape, the detection unit 6d may detect the target shape using methods such as Hough transform or minimum circumscribed circle method. If the detection target object is reflected in the determination target image, as shown in FIG. 3, the differential image is in a state where features of the detection target object such as the target shape remain only at the location where the detection target object exists. On the other hand, when the detection target image is not reflected in the determination target image, the differential image shows nothing. Thereby, the process of step S5 is completed, and the detection process proceeds to the process of step S6.

[0031] In the process of step S6, the output unit 6e outputs to the output device whether a detection target object exists in the determination target image according to the determination result of the detection unit 6d. Examples of the output device include a printing device, a display device, an audio output device, etc. Thereby, the process of step S6 is completed, and a series of detection processes are terminated.

[0032] 〔Learning model generation process〕 Next, with reference to FIG. 4, the operation of the detection system 1 when executing the learning model generation process will be described.

[0033] FIG. 4 is a flowchart showing the flow of the learning model generation process according to an embodiment of the present invention. The flowchart shown in FIG. 4 starts at the timing when the image data of the monitoring target area determined not to have a detection target object is acquired, and the learning model generation process proceeds to the process of step S11. The image data of the monitoring target area determined not to have a detection target object may be the one obtained in the above-described detection process, or may be separately collected and determined.

[0034] In the process of step S11, the model generation unit 4 adds the image data of the monitored area determined to have no detection target object to the learning source data 5a stored in the learning database 5. Thereby, the process of step S11 is completed, and the learning model generation process proceeds to the process of step S12.

[0035] In the process of step S12, the model generation unit 4 generates a learning model 5b using the learning source data 5a. As the learning model 5b, a learning model capable of reconstructing an image identical or approximate to the learning source data 5a is used. An example of such a learning model is an autoencoder. A configuration example of the autoencoder is shown in FIG. 5. As shown in FIG. 5, the autoencoder is a learning model that learns the features from the learning image, extracts the features by compressing the input image to reduce the dimension, and reconstructs and outputs an image in a state that matches or approximates the learned content.

[0036] Therefore, when the data of the determination target image is input, the learning model 5b learned using the learning source data 5a outputs, as output data, the image data of the monitored area at the time when there is no detection target object in the monitored area corresponding to the determination target image. As described above, the autoencoder has the advantage that, due to the nature of learning the parameters to restore the learned image, less adjustment is required to cope with unknown objects and disturbances, and even a small amount of learning data is sufficient. For this reason, even if the determination target image changes according to the change in the surrounding environment of the imaging unit 2, the learning model 5b generates an output image corresponding to the determination target image, so there is no need to make adjustment corresponding to various conditions. Thereby, the process of step S12 is completed, and the series of learning model generation processes ends.

[0037] As is clear from the above description, the detection system 1 according to one embodiment of the present invention includes an image restoration generation unit 6a that generates a non-detection target image corresponding to the determination target image by inputting data of the determination target image to a learning model in which data of a processed image is input data and data of a non-detection target image that does not contain a detection target object corresponding to the processed image is output data; a difference calculation unit 6b that generates a difference image between the determination target image and the non-detection target image generated by the image restoration generation unit 6a; and a detection unit 6d that determines whether the determination target image is a detection target image based on the difference image generated by the difference calculation unit 6b. This allows for accurate detection of a detection target object present in the monitored area even if an obstacle or an object with low transmittance is present between the monitored area and the image capture unit 2. Furthermore, the model generation unit 4 and the learning database 5 allow for generation of a learning model that accurately detects a detection target object present in the monitored area even if an obstacle or an object with low transmittance is present between the monitored area and the image capture unit 2. [Example]

[0038] In this example, the present invention was applied to a process for determining whether a metal tube (whopper) supporting a metal product to prevent its inner diameter from being crushed when it is wound during the manufacturing process of the metal product was used as the detection object. Furthermore, a wire mesh fence or a dust-proof / water-proof cover was assumed as an obstacle between the detection object and the imaging unit 2. In the learning model generation process, images of the monitored area when the whopper was not inserted in its designated position were captured, and the 25 captured images were used for learning by an autoencoder to generate learning model 5b. The autoencoder had the following layer structure, and structural similarity indexes were evaluated based on changes in pixel values, contrast, and structure. This resulted in a learning model that could generate images of a state when the whopper was not inserted in its designated position.

[0039] Encoder: Compresses a 300x300 pixel image to 100 dimensions Decoder: Reconstructs 100 dimensions into a 300x300 pixel image · Number of learning epochs: 500 epochs · Loss function: SSIM (structural similarity)

[0040] When an image (regardless of the presence or absence of the object to be detected) is input into this learning model, an image in a state where the wrapper is not loaded at a predetermined position is output. Then, the difference between the input image and the output image is calculated. The difference image between the input image with the wrapper loaded at the predetermined position and the output image with the wrapper not loaded at the predetermined position is an image of the wrapper loaded at the predetermined position. Next, after emphasizing the features of the wrapper by performing image processing such as binarization processing, edge detection processing, and smoothing processing, the image portion other than the predetermined position is masked in order to further improve the detection accuracy. Then, circles and arcs were detected using the Hough transform. The radius and center position of the circle, which are parameters of the Hough transform, were those previously considered from the captured image. As a result, although a circle may be detected at a location other than the predetermined position, when the camera is fixed, the radius and center position of the circle when the wrapper is at the predetermined position are known, and the circle can be detected at a reasonable position. Thus, as shown in FIG. 6, even when there is an obstacle or an object with a low transmittance between the monitoring target area and the imaging unit, it was possible to accurately determine whether or not the object to be detected appears in the determination target image.

[0041] As described above, the embodiments to which the invention made by the present inventors is applied have been described, but the present invention is not limited by the description and drawings that form a part of the disclosure of the present invention according to this embodiment. That is, all other embodiments, examples, and operation techniques made by those skilled in the art based on this embodiment are included in the scope of the present invention.

Explanation of reference numerals

[0042] 1 Detection system 2 Imaging unit 3 Data acquisition unit 4 Model generation unit 5 Learning database 5a Learning source data 5b Learning Model 6 Arithmetic Unit 6a Image Restoration and Generation Unit 6b Difference Calculation Unit 6c Image Processing Unit 6d Detection Unit 6e Output Unit

Claims

1. A detection device for determining whether a determination target image is a detection target image including a detection target object, an image restoration generation unit that inputs data of the determination target image to a learning model that uses data of a processed image as input data and data of a non-detection target image that does not include the detection target object corresponding to the processed image as output data, thereby generating a non-detection target image corresponding to the determination target image; a difference calculation unit that generates a difference image between the determination target image and the non-detection target image generated by the image restoration generation unit; a detection unit that determines whether the determination target image is the detection target image based on the difference image generated by the difference calculation unit; A detection device comprising:

2. The detection device according to claim 1, further comprising an image processing unit that performs image processing for emphasizing features of the detection target object on the difference image generated by the difference calculation unit, wherein the detection unit determines whether the determination target image is the detection target image based on the difference image on which the image processing has been performed by the image processing unit.

3. The detection device according to claim 2, wherein the image processing unit performs a masking process on an image area other than a predetermined image area where the detection target object may exist.

4. The detection device according to claim 1, further comprising a model generation unit that generates the learning model by learning to reconstruct the same non-detection target image as the learning source data or a non-detection target image approximating the learning source data using the data of the non-detection target image as the learning source data.

5. The detection device according to claim 4, wherein the model generation unit is an autoencoder.

6. The detection device according to claim 4, wherein the model generation unit adds data of a determination target image determined by the detection unit to be a non-detection target image to the learning source data and regenerates the learning model.

7. A detection system comprising: the detection device according to any one of claims 1 to 6; and an imaging unit that captures an image of a monitoring target area and outputs data of the captured image of the monitoring target area to the detection device as data of the determination target image.

8. A detection method for determining whether a determination target image is a detection target image including a detection target object, An image restoration generation step of generating a non-detection target image corresponding to the determination target image by inputting the data of the determination target image to a learning model that uses the data of the processed image as input data and the data of the non-detection target image that does not include the detection target object corresponding to the processed image as output data; A difference calculation step of generating a difference image between the determination target image and the non-detection target image generated in the image restoration generation step; A detection step of determining whether or not the determination target image is the detection target image based on the difference image generated in the difference calculation step; A detection method including the above.

9. A model generation device for generating a learning model for determining whether or not a determination target image is a detection target image including a detection target object, Comprising a model generation unit that generates a learning model that uses the data of the processed image as input data and the data of the non-detection target image that does not include the detection target object corresponding to the processed image as output data, The model generation unit generates the learning model by learning to reconstruct the same non-detection target image as the learning source data or a non-detection target image approximating the learning source data using the data of the non-detection target image as the learning source data.

Images