Detection device, detection system, detection method, and model generation device

Abstract

A detection device according to the present invention determines whether an image subject to determination is an image subject to detection that includes an object to be detected. The detection device comprising: an image restoration generation unit that generates an image not subject to detection and corresponding to the image subject to determination by inputting data of the image subject to determination into a learning model that uses data of a processed image as input data, and creates, as output data, data of an image not subject to detection and not including the object to be detected corresponding to the processed image; a difference calculation unit that generates a difference image representing the difference between the image subject to determination and the image not subject to detection generated by the image restoration generation unit; and a detection unit that determines whether the image subject to determination is an image subject to detection on the basis of the difference image generated by the difference calculation unit.

Description

Detection device, detection system, detection method, and model generation device

[0001] The present invention relates to a detection device, a detection system, a detection method, and a model generation device that determine whether or not a determination target image is a detection target image that includes a detection target object.

[0002] Patent document 1 describes a device that extracts change areas from images of a monitored area captured sequentially by an imaging device, and compares the feature values ​​of the monitored area based on the extracted change areas with predetermined set values ​​of the feature values ​​to determine whether an event has occurred.

[0003] Patent No. 5388827

[0004] The device described in Patent Document 1 may not be able to detect a detection target that exists within a monitored area if there is an obstacle such as a safety fence, a fence, a dust-proof or drip-proof cover, etc. between the monitored area and the imaging device. For this reason, there has been a demand for a technology that can accurately detect a detection target that exists within a monitored area even if there is an obstacle or an object with low transmittance between the monitored area and the imaging device.

[0005] The present invention has been made in consideration of the above-mentioned problems, and its object is to provide a detection device, a detection system, and a detection method that can accurately detect a detection object present in a monitored area even if an obstacle or an object with low transmittance exists between the monitored area and the imaging device. Another object of the present invention is to provide a model generation device that can generate a learning model that accurately detects a detection object present in a monitored area even if an obstacle or an object with low transmittance exists between the monitored area and the imaging device.

[0006] The detection device of the present invention is a detection device that determines whether an image to be determined is a detection target image that contains a detection target object, and is equipped with an image restoration generation unit that generates a non-detection target image corresponding to the image to be determined by inputting data of the image to be determined into a learning model that has data of a processed image as input data and data of a non-detection target image that does not contain the detection target object corresponding to the processed image as output data, a difference calculation unit that generates a difference image between the image to be determined and the non-detection target image generated by the image restoration generation unit, and a detection unit that determines whether the image to be determined is the detection target image based on the difference image generated by the difference calculation unit.

[0007] The image processing unit may be provided with an image processing unit that performs image processing on the difference image generated by the difference calculation unit to emphasize the features of the object to be detected, and the detection unit may determine whether the image to be determined is the image to be detected based on the difference image on which image processing has been performed by the image processing unit.

[0008] The image processing unit may perform a mask process on an image area other than a predetermined image area in which the object to be detected may exist.

[0009] It is preferable to provide a model generation unit that generates the learning model by learning to reconstruct a non-detection target image that is identical to the learning target data or a non-detection target image that approximates the learning target data, using the data of the non-detection target image as learning source data.

[0010] The model generator may be an autoencoder.

[0011] The model generation unit may add data of a determination target image that the detection unit has determined to be a non-detection target image to the learning source data, and generate the learning model again.

[0012] The detection system of the present invention comprises the detection device of the present invention and an imaging unit that captures an image of the monitored area and outputs data of the captured image of the monitored area to the detection device as data of the image to be determined.

[0013] The detection method of the present invention is a detection method for determining whether an image to be determined is a detection target image containing a detection target object, and includes an image restoration generation step for generating a non-detection target image corresponding to the image to be determined by inputting data of the image to be determined into a learning model which has data of a processed image as input data and data of a non-detection target image corresponding to the processed image as output data, a difference calculation step for generating a difference image between the image to be determined and the non-detection target image generated in the image restoration generation step, and a detection step for determining whether the image to be determined is the detection target image based on the difference image generated in the difference calculation step.

[0014] The model generation device of the present invention is a model generation device that generates a learning model for determining whether an image to be determined is a detection target image that contains a detection target, and is equipped with a model generation unit that generates a learning model using data of a processed image as input data and data of a non-detection target image that does not contain the detection target corresponding to the processed image as output data, and the model generation unit generates the learning model by learning to reconstruct a non-detection target image that is identical to the learning target data or a non-detection target image that approximates the learning target data, using the data of the non-detection target image as learning source data.

[0015] The detection device, detection system, and detection method according to the present invention can accurately detect a detection target present in a monitored area even if an obstacle or an object with low transmittance exists between the monitored area and the imaging device.Furthermore, the model generation device according to the present invention can generate a learning model that accurately detects a detection target present in a monitored area even if an obstacle or an object with low transmittance exists between the monitored area and the imaging device.

[0016] FIG. 1 is a block diagram showing the configuration of a detection system according to one embodiment of the present invention. FIG. 2 is a flowchart showing the flow of detection processing according to one embodiment of the present invention. FIG. 3 is a diagram for explaining differential image generation processing. FIG. 4 is a flowchart showing the flow of learning model generation processing according to one embodiment of the present invention. FIG. 5 is a diagram showing an example configuration of an autoencoder. FIG. 6 is a diagram for explaining detection processing according to an embodiment.

[0017] Hereinafter, the configuration and operation of a detection system according to one embodiment of the present invention will be described with reference to the drawings.

[0018] [Configuration] Fig. 1 is a block diagram showing the configuration of a detection system according to one embodiment of the present invention. As shown in Fig. 1, the detection system 1 according to one embodiment of the present invention is a system that determines whether a determination target image is a detection target image that includes a detection target object, and includes an imaging unit 2, a data acquisition unit 3, a model generation unit 4, a learning database 5, and a calculation unit 6. The data acquisition unit 3, the model generation unit 4, and the calculation unit 6 function as a detection device according to the present invention.

[0019] The imaging unit 2 is composed of an imaging device such as a camera or an industrial surveillance camera (ITV). The imaging unit 2 captures images of the monitored area and outputs the captured image data of the monitored area to the data acquisition unit 3. Specifically, the imaging unit 2 captures images of the monitored area before and after an event changes or before and after a movable object moves, and outputs the captured image data of the monitored area to the data acquisition unit 3.

[0020] The data acquisition unit 3 and the model generation unit 4 are functional blocks that are realized by a processing unit such as a CPU in the information processing device executing a computer program. The functions of the data acquisition unit 3 and the model generation unit 4 will be described later.

[0021] The learning database 5 is configured with a non-volatile storage device. The learning database 5 stores learning source data 5a and a learning model 5b. As will be described in detail later, the learning source data 5a is image data of the monitored area at a time when no detection target object is present in the monitored area. The learning model 5b is a machine learning model that takes image data of the monitored area to be judged (data of the image to be judged) as input data and image data of the monitored area at a time when no detection target object is present in the monitored area corresponding to the input data as output data. The model generation unit 4 and the learning database 5 may be configured as a system or device other than the detection system 1.

[0022] The calculation unit 6 is configured by an arithmetic processing device such as a CPU inside the information processing device. The calculation unit 6 functions as an image restoration generation unit 6a, a difference calculation unit 6b, an image processing unit 6c, a detection unit 6d, and an output unit 6e when the arithmetic processing device executes a computer program. The functions of each of these units will be described later. The information processing device that realizes each of these units and the information processing device that realizes the data acquisition unit 3 and the model generation unit 4 may be the same device or different devices. If they are different devices, the information processing devices are connected to each other via a telecommunications line.

[0023] The detection system 1 having such a configuration executes the detection process and learning model generation process described below, thereby enabling accurate detection of a detection target object present in a monitored area, even if an obstacle or an object with low transmittance exists between the monitored area and the imaging unit 2. Below, the operation of the detection system 1 when executing the detection process and learning model generation process will be described with reference to the flowchart shown in FIG.

[0024] [Detection Process] First, with reference to FIG. 2, the operation of the detection system 1 when performing the detection process will be described.

[0025] 2 is a flowchart showing the flow of the detection process according to one embodiment of the present invention. The flowchart shown in FIG. 2 starts when an execution command for the detection process is input to the calculation unit 6, and the detection process proceeds to step S1.

[0026] In the process of step S1, the data acquisition unit 3 inputs the image data of the area to be monitored output from the imaging unit 2 as data of the image to be determined to the calculation unit 6. This completes the process of step S1, and the detection process proceeds to the process of step S2.

[0027] In the process of step S2, the image restoration generation unit 6a inputs the data of the determination target image input by the data acquisition unit 3 as input data to the learning model 5b, thereby generating image data (output image) of the monitoring target area at a time when the detection target object does not exist, reflecting the features of the determination target image. This completes the process of step S2, and the detection process proceeds to the process of step S3.

[0028] In the process of step S3, the difference calculation unit 6b generates a difference image between the determination target image input by the data acquisition unit 3 and the image of the monitoring target area generated by the image restoration generation unit 6a. An image of the difference image generation process is shown in Figure 3. Details of Figure 3 will be described later. This completes the process of step S3, and the detection process proceeds to the process of step S4.

[0029] In the process of step S4, the image processing unit 6c performs image processing such as binarization, masking, edge detection, smoothing, filtering, correction, expansion, and contraction on the generated difference image in order to enhance the features of the detection object, such as its shape, to make it easier to detect the detection object. This completes the process of step S4, and the detection process proceeds to the process of step S5.

[0030] In the process of step S5, the detection unit 6d determines whether or not a feature such as the shape of the object to be detected (target shape) is present in the difference image after image processing, thereby determining whether or not the detection target is present in the difference image. If the target shape is circular, the detection unit 6d may detect the target shape using a method such as the Hough transform or the minimum circumscribed circle method. If the detection target is present in the image to be determined, as shown in FIG. 3, the difference image will have the features of the detection target, such as the target shape, remaining only in the area where the detection target is present. On the other hand, if the detection target is not present in the image to be determined, the difference image will have nothing present. This completes the process of step S5, and the detection process proceeds to the process of step S6.

[0031] In the process of step S6, the output unit 6e outputs whether or not the detection target object exists in the determination target image according to the determination result of the detection unit 6d to an output device. Examples of the output device include a printing device, a display device, an audio output device, etc. This completes the process of step S6, and the series of detection processes ends.

[0032] [Learning Model Generation Process] Next, with reference to FIG. 4, the operation of the detection system 1 when executing the learning model generation process will be described.

[0033] 4 is a flowchart showing the flow of a learning model generation process according to one embodiment of the present invention. The flowchart shown in FIG. 4 starts when image data of a monitored area in which it has been determined that no detection target object is present is acquired, and the learning model generation process proceeds to step S11. The image data of the monitored area in which it has been determined that no detection target object is present may be obtained in the detection process described above, or may be separately collected and determined.

[0034] In the process of step S11, the model generation unit 4 adds the image data of the monitored area in which it is determined that no detection target object exists to the learning source data 5a stored in the learning database 5. This completes the process of step S11, and the learning model generation process proceeds to the process of step S12.

[0035] In the process of step S12, the model generation unit 4 generates a learning model 5b using the learning source data 5a. As the learning model 5b, a learning model capable of reconstructing an image identical to or similar to the learning source data 5a is used. An example of such a learning model is an autoencoder. An example configuration of an autoencoder is shown in FIG. 5. As shown in FIG. 5, an autoencoder is a learning model that learns features from training images, extracts features by compressing input images to reduce their dimensions, and reconstructs and outputs an image that matches or approximates the training content.

[0036] Therefore, when data of a target image is input, the learning model 5b, which has been trained using the learning source data 5a, outputs as output data image data of the target area at a time when no target object is present in the target area corresponding to the target image. As described above, the autoencoder has the advantage of requiring fewer adjustments to accommodate unknown objects or disturbances and requiring a small amount of training data, due to its nature of learning parameters to restore the trained image. Therefore, even if the target image changes in response to changes in the surrounding environment of the imaging unit 2, the learning model 5b generates an output image corresponding to the target image, eliminating the need for adjustments according to various conditions. This completes the processing of step S12, and the series of learning model generation processes ends.

[0037] As is clear from the above description, the detection system 1 according to one embodiment of the present invention includes an image restoration generation unit 6a that generates a non-detection target image corresponding to the determination target image by inputting data of a judgment target image into a learning model in which data of a processed image is input data and data of a non-detection target image that does not contain a detection target object corresponding to the processed image is output data; a difference calculation unit 6b that generates a difference image between the judgment target image and the non-detection target image generated by the image restoration generation unit 6a; and a detection unit 6d that determines whether the judgment target image is a detection target image based on the difference image generated by the difference calculation unit 6b. This allows for accurate detection of a detection target object present in the monitored area even if an obstacle or an object with low transmittance is present between the monitored area and the image capture unit 2. Furthermore, the model generation unit 4 and the learning database 5 allow for generation of a learning model that accurately detects a detection target object present in the monitored area even if an obstacle or an object with low transmittance is present between the monitored area and the image capture unit 2.

[0038] In this example, the present invention was applied to a process for determining whether a metal tube (whopper) supporting a metal product to prevent the inner diameter of the metal product from being crushed when it is wound during the manufacturing process of the metal product was used as the detection object. Furthermore, a wire mesh fence, dustproof / waterproof cover, etc. were assumed as obstacles between the detection object and the imaging unit 2. In the learning model generation process, images of the monitored area when the whopper was not installed in its designated position were captured, and the 25 captured images were used for learning by an autoencoder to generate learning model 5b. The autoencoder had the following layer structure, and structural similarity indexes were used to evaluate changes in pixel values, contrast, and structure. This resulted in a learning model that generates images of a state when the whopper was not installed in its designated position.

[0039] ・Encoder: Compresses a 300x300 pixel image to 100 dimensions ・Decoder: Restores the 100 dimensions to a 300x300 pixel image ・Training times: 500 epochs ・Loss function: SSIM (structural similarity)

[0040] When an image (regardless of whether or not the detection target is present) is input into this learning model, an image of the whopper not in its designated position is output. The difference between the input image and the output image is then calculated. The difference image between the input image with the whopper in its designated position and the output image without the whopper in its designated position is an image of the whopper in its designated position. Next, image processing such as binarization, edge detection, and smoothing was performed to emphasize the whopper's features. To further improve detection accuracy, the image portions other than the designated position were masked. Circles and arcs were then detected using the Hough transform. The parameters for the Hough transform, such as the radius and center position of the circle, were determined in advance from the captured image. While this may result in the detection of a circle in a location other than the designated position, if the camera is fixed, the radius and center position of the circle when the whopper is in its designated position are known, allowing the circle to be detected in a convincing position. As a result, as shown in Figure 6, it was possible to accurately determine whether the detection target is captured in the image to be determined, even when an obstacle or an object with low transmittance is present between the monitored area and the imaging unit.

[0041] Although the present invention has been described above as an embodiment, the present invention is not limited to the descriptions and drawings that form part of the disclosure of the present invention. In other words, other embodiments, examples, and operational techniques that can be made by those skilled in the art based on the present invention are all included in the scope of the present invention.

[0042] According to the present invention, it is possible to provide a detection device, a detection system, and a detection method that can accurately detect a detection object that exists in a monitored area even if an obstacle or an object with low transmittance exists between the monitored area and the imaging device. Also, according to the present invention, it is possible to provide a model generation device that can generate a learning model that accurately detects a detection object that exists in a monitored area even if an obstacle or an object with low transmittance exists between the monitored area and the imaging device.

[0043] REFERENCE SIGNS LIST 1 Detection system 2 Imaging unit 3 Data acquisition unit 4 Model generation unit 5 Learning database 5a Learning source data 5b Learning model 6 Calculation unit 6a Image restoration generation unit 6b Difference calculation unit 6c Image processing unit 6d Detection unit 6e Output unit

Claims

1. A detection device for determining whether a determination target image is a detection target image including a detection target object, comprising: an image restoration generation unit that inputs data of the determination target image to a learning model that uses data of a processed image as input data and data of a non-detection target image that does not include the detection target object corresponding to the processed image as output data, thereby generating a non-detection target image corresponding to the determination target image; a difference calculation unit that generates a difference image between the determination target image and the non-detection target image generated by the image restoration generation unit; and a detection unit that determines whether the determination target image is the detection target image based on the difference image generated by the difference calculation unit.

2. The detection device according to claim 1, further comprising an image processing unit that performs image processing to emphasize features of the detection target object on the difference image generated by the difference calculation unit, and the detection unit determines whether the determination target image is the detection target image based on the difference image on which the image processing is performed by the image processing unit.

3. The detection device according to claim 2, wherein the image processing unit performs a masking process on an image area other than a predetermined image area where the detection target object may exist.

4. The detection device according to any one of claims 1 to 3, further comprising a model generation unit that generates the learning model by learning to reconstruct the same non-detection target image as the learning source data or a non-detection target image approximating the learning source data using the data of the non-detection target image as the learning source data.

5. The detection device according to claim 4, wherein the model generation unit is an autoencoder.

6. The detection device according to claim 4 or 5, wherein the model generation unit adds data of the determination target image determined by the detection unit to be a non-detection target image to the learning source data and regenerates the learning model.

7. A detection system, comprising: the detection device according to any one of claims 1 to 6; and an imaging unit that captures an image of a monitoring target area and outputs data of the captured image of the monitoring target area to the detection device as data of the determination target image.

8. A detection method for determining whether a determination target image is a detection target image including a detection target object, the method comprising: an image restoration generation step of inputting data of the determination target image to a learning model that uses data of a processing image as input data and data of a non-detection target image not including the detection target object corresponding to the processing image as output data, thereby generating a non-detection target image corresponding to the determination target image; a difference calculation step of generating a difference image between the determination target image and the non-detection target image generated in the image restoration generation step; and a detection step of determining whether the determination target image is the detection target image based on the difference image generated in the difference calculation step.

9. A model generation device for generating a learning model for determining whether a determination target image is a detection target image including a detection target object, the device comprising: a model generation unit that generates a learning model that uses data of a processing image as input data and data of a non-detection target image not including the detection target object corresponding to the processing image as output data, wherein the model generation unit generates the learning model by learning to reconstruct the same non-detection target image as the learning source data or a non-detection target image approximating the learning source data, using the data of the non-detection target image as the learning source data.

Images