Detection device, detection system, detection method, and model generation device

Abstract

A detection device according to the present invention is a detection device for determining whether a judgment target image is a detection target image that includes a detection target object, and comprises: an image restoration generation unit that generates a non-detection target image corresponding to a judgment target image by inputting data of a judgment target image to a learning model having data of a processed image as input data and data of a non-detection target image that does not include a detection target object according to the processed image as output data; a difference calculation unit that generates a difference image of the judgment target image and the non-detection target image generated by the image restoration generation unit; and a detection unit that determines whether the judgment target image is a detection target image based on the difference image generated by the difference calculation unit.

Description

Technology Field

[0001] The present invention relates to a detection device, a detection system, a detection method, and a model generation device for determining whether a judgment target image is a detection target image containing a detection target object. Background Technology

[0002] Patent Document 1 describes a device that extracts a change area from images of a monitored area sequentially captured by an imaging device, and determines whether an event has occurred by comparing a feature amount of a monitored area based on the extracted change area with a set value of a predetermined feature amount. Prior art literature

[0003] Japanese Patent Publication No. 5388827 The problem to be solved

[0004] In the device described in Patent Document 1, if there are obstacles such as safety barriers, fences, or dustproof / drip-proof covers between the monitoring area and the imaging device, there is a possibility that the detection target present within the monitoring area cannot be detected. For this reason, there was an expectation for a technology that could detect the detection target present within the monitoring area with high precision, even if there are obstacles or objects with low transmittance between the monitoring area and the imaging device.

[0005] The present invention has been made in consideration of the above-mentioned problem, and its purpose is to provide a detection device, a detection system, and a detection method capable of detecting a target object present within a monitored area with high precision, even when there are obstacles or objects with low transmittance between the monitored area and the imaging device. In addition, another objective of the present invention is to provide a model generation device capable of generating a learning model capable of detecting a target object present within a monitored area with high precision, even when there are obstacles or objects with low transmittance between the monitored area and the imaging device. means of solving the problem

[0006] A detection device according to the present invention is a detection device for determining whether a judgment target image is a detection target image that includes a detection target object, and comprises: an image restoration generation unit that generates a non-detection target image corresponding to a judgment target image by inputting data of the judgment target image to a learning model having data of a processing image as input data and data of a non-detection target image that does not include the detection target object according to the processing image as output data; a difference calculation unit that generates a difference image of the judgment target image and the non-detection target image generated by the image restoration generation unit; and a detection unit that determines whether the judgment target image is a detection target image based on the difference image generated by the difference calculation unit.

[0007] The present invention may include an image processing unit that performs image processing to emphasize the features of the detection target object on the difference image generated by the difference calculation unit, and the detection unit may determine whether the judgment target image is the detection target image or not based on the difference image on which image processing is performed by the image processing unit.

[0008] The image processing unit may perform mask processing on an image area other than a predetermined image area where the detection target object may exist.

[0009] It is preferable to provide a model generation unit that generates the learning model by using the data of the above-mentioned non-detection target image as learning source data and learning to reconstruct a non-detection target image identical to the said learning source data or a non-detection target image that approximates the said learning source data.

[0010] The above model generation unit may be an autoencoder.

[0011] The above model generation unit may generate the learning model again by adding the data of the judgment target image, which the detection unit determined to be a non-detection target image, to the learning source data.

[0012] A detection system according to the present invention comprises a detection device according to the present invention and an imaging unit that captures an image of a surveillance target area and outputs the data of the captured image of the surveillance target area as the data of the image of the judgment target to the detection device.

[0013] A detection method according to the present invention is a detection method for determining whether a judgment target image is a detection target image that includes a detection target object, and comprises: an image restoration generation step for generating a non-detection target image corresponding to a judgment target image by inputting data of the judgment target image to a learning model having data of a processed image as input data and data of a non-detection target image that does not include the detection target object according to the processed image as output data; a difference calculation step for generating a difference image of the judgment target image and the non-detection target image generated in the image restoration generation step; and a detection step for determining whether the judgment target image is a detection target image based on the difference image generated in the difference calculation step.

[0014] A model generation device according to the present invention is a model generation device for generating a learning model for determining whether a judgment target image is a detection target image containing a detection target object, and comprises a model generation unit for generating a learning model having data of a processed image as input data and data of a non-detection target image that does not contain the detection target object according to the processed image as output data, wherein the model generation unit generates the learning model by using the data of the non-detection target image as source data for learning and by learning to reconstruct a non-detection target image identical to the source data or a non-detection target image that approximates the source data for learning. Effects of the invention

[0015] According to the detection device, detection system, and detection method of the present invention, even if there are obstacles or objects with low transmittance between the monitoring target area and the imaging device, detection targets existing within the monitoring target area can be detected with high precision. In addition, according to the model generation device of the present invention, even if there are obstacles or objects with low transmittance between the monitoring target area and the imaging device, a learning model can be generated that detects detection targets existing within the monitoring target area with high precision. Brief explanation of the drawing

[0016] FIG. 1 is a block diagram showing the configuration of a detection system, which is an embodiment of the present invention. FIG. 2 is a flowchart showing the flow of a detection process, which is an embodiment of the present invention. Figure 3 is a diagram illustrating the generation process of a difference image. FIG. 4 is a flowchart showing the flow of a learning model generation process, which is an embodiment of the present invention. Figure 5 is a diagram showing an example of the configuration of an autoencoder. Figure 6 is a diagram illustrating the detection process of an embodiment. Specific details for implementing the invention

[0017] (Form for carrying out the invention)

[0018] Hereinafter, the configuration and operation of a detection system, which is an embodiment of the present invention, will be described with reference to the drawings.

[0019] 〔composition〕

[0020] FIG. 1 is a block diagram showing the configuration of a detection system according to one embodiment of the present invention. As shown in FIG. 1, the detection system (1) according to one embodiment of the present invention is a system for determining whether a judgment target image is a detection target image that includes a detection target object, and is equipped with an imaging unit (2), a data acquisition unit (3), a model generation unit (4), a learning database (5), and a calculation unit (6). The data acquisition unit (3), the model generation unit (4), and the calculation unit (6) function as a detection device according to the present invention.

[0021] The imaging unit (2) is configured by an imaging device such as a camera or an industrial surveillance camera (ITV). The imaging unit (2) captures an image of the surveillance target area and outputs the captured image data of the surveillance target area to the data acquisition unit (3). Specifically, the imaging unit (2) captures an image of the surveillance target area before and after an event transition or before and after a movable object moves, and outputs the captured image data of the surveillance target area to the data acquisition unit (3).

[0022] The data acquisition unit (3) and the model generation unit (4) are functional blocks realized by an arithmetic processing unit, such as a CPU, inside an information processing device executing a computer program. The functions of the data acquisition unit (3) and the model generation unit (4) will be described later.

[0023] The learning database (5) is configured by a non-volatile memory device. The learning database (5) stores source data (5a) and a learning model (5b). Although details will be described later, source data (5a) is image data of the monitoring area at a time when no object to be detected exists within the monitoring area. The learning model (5b) is a machine learning model that takes image data of the monitoring area of ​​the judgment target (data of the judgment target image) as input data and image data of the monitoring area at a time when no object to be detected exists within the monitoring area corresponding to the input data as output data. The model generation unit (4) and the learning database (5) may be configured as a system or device different from the detection system (1).

[0024] The operation unit (6) is configured by an operation processing unit, such as a CPU, inside the information processing device. The operation unit (6) functions as an image restoration generation unit (6a), a difference calculation unit (6b), an image processing unit (6c), a detection unit (6d), and an output unit (6e) by executing a computer program by the operation processing unit. The functions of each of these units will be described later. The information processing device realizing each of these units and the information processing device realizing the data acquisition unit (3) and the model generation unit (4) may be the same device or different devices. In the case of different devices, the information processing devices are connected to each other via an electrical communication line.

[0025] A detection system (1) having such a configuration enables high-precision detection of objects within a monitored area even when there are obstacles or objects with low transmittance between the monitored area and the imaging unit (2) by executing the detection processing and learning model generation processing described below. Below, with reference to the flowchart shown in FIG. 2, the operation of the detection system (1) when executing the detection processing and learning model generation processing will be explained.

[0026] [Detection Processing]

[0027] First, with reference to FIG. 2, the operation of the detection system (1) when performing detection processing will be explained.

[0028] FIG. 2 is a flowchart showing the flow of a detection process, which is an embodiment of the present invention. The flowchart shown in FIG. 2 is initiated at the timing when an execution command for the detection process is input to the operation unit (6), and the detection process proceeds to the processing of step S1.

[0029] In the processing of step S1, the data acquisition unit (3) inputs the image data of the monitored target area output from the imaging unit (2) into the calculation unit (6) as the data of the image to be determined. Accordingly, the processing of step S1 is completed, and the detection processing proceeds to the processing of step S2.

[0030] In the processing of Step S2, the image restoration generation unit (6a) inputs the data of the image to be determined, which is input by the data acquisition unit (3), into the learning model (5b) as input data, thereby generating image data (output image) of the monitoring target area at a time when no detection target exists, which reflects the characteristics of the image to be determined. Accordingly, the processing of Step S2 is completed, and the detection processing proceeds to the processing of Step S3.

[0031] In the processing of step S3, the difference calculation unit (6b) generates a difference image between the image of the target area to be judged input by the data acquisition unit (3) and the image of the target area to be monitored generated by the image restoration generation unit (6a). An image of the processing of generating the difference image is shown in FIG. 3. Details of FIG. 3 will be described later. Accordingly, the processing of step S3 is completed, and the detection processing proceeds to the processing of step S4.

[0032] In the processing of step S4, the image processing unit (6c) performs image processing such as binarization, masking, edge detection, smoothing, filtering, correction, expansion, and shrinking on the generated difference image to emphasize features such as the shape of the object to be detected and to make it easier to detect the object to be detected. Accordingly, the processing of step S4 is completed, and the detection processing proceeds to the processing of step S5.

[0033] In the processing of step S5, the detection unit (6d) determines whether or not a detection target exists within the difference image by determining whether or not features such as the shape of the detection target (target shape) exist within the difference image after image processing. If the target shape is a circular shape, the detection unit (6d) may detect the target shape using methods such as the Hough transform or the least circumscribed circle method. For example, if the detection target is reflected in the image to be judged, as shown in FIG. 3, the difference image is in a state where features of the detection target, such as the target shape, remain only at the locations where the detection target exists. On the other hand, if the detection target image is not reflected in the image to be judged, the difference image is in a state where nothing is reflected. Accordingly, the processing of step S5 is completed, and the detection processing proceeds to the processing of step S6.

[0034] In the processing of step S6, the output unit (6e) outputs to the output device whether or not a detection target exists within the image to be determined, based on the determination result of the detection unit (6d). Examples of output devices include a printing device, a display device, and a voice output device. Accordingly, the processing of step S6 is completed, and the series of detection processes is terminated.

[0035] [Training Model Generation Processing]

[0036] Next, with reference to FIG. 4, the operation of the detection system (1) when executing the learning model generation process will be described.

[0037] FIG. 4 is a flowchart illustrating the flow of a learning model generation process, which is an embodiment of the present invention. The flowchart shown in FIG. 4 is initiated at the timing when image data of a monitored area determined not to have a detection target is acquired, and the learning model generation process proceeds to the processing of step S11. The image data of the monitored area determined not to have a detection target may be obtained in the aforementioned detection process, or it may be collected and determined separately.

[0038] In the processing of step S11, the model generation unit (4) adds image data of the monitored area, where it is determined that no detection target exists, to the learning source data (5a) stored in the learning database (5). Accordingly, the processing of step S11 is completed, and the learning model generation process proceeds to the processing of step S12.

[0039] In the processing of step S12, the model generation unit (4) generates a learning model (5b) using the source data (5a). As the learning model (5b), a learning model capable of reconstructing an image identical or similar to the source data (5a) is used. An autoencoder (self-encoder) can be exemplified as such a learning model. An example of the configuration of an autoencoder is shown in FIG. 5. As shown in FIG. 5, the autoencoder is a learning model that learns features from a learning image, extracts features by compressing the input image to reduce dimensions, and reconstructs and outputs an image that matches or approximates the learning content.

[0040] Accordingly, the learning model (5b) trained using the source data (5a) outputs the image data of the monitoring area at the time when no object to be detected exists within the monitoring area corresponding to the image to be determined, when data of the image to be determined is input. As described above, the autoencoder has the advantage that, due to the nature of learning parameters to restore the learned image, adjustment response to respond to unknown objects or disturbances is minimal, and the amount of training data is small. For this reason, even if the image to be determined changes due to changes in the surrounding environment of the imaging unit (2), the learning model (5b) generates an output image according to the image to be determined, so there is no need to perform adjustment response according to various conditions. Accordingly, the processing of step S12 is completed, and the series of learning model generation processing is terminated.

[0041] As is evident from the above description, a detection system (1) which is an embodiment of the present invention comprises an image restoration generation unit (6a) that generates a non-detection target image corresponding to a judgment target image by inputting data of a judgment target image to a learning model that has data of a processing image as input data and data of a non-detection target image that does not include a detection target object according to the processing image as output data, a difference calculation unit (6b) that generates a difference image of the judgment target image and the non-detection target image generated by the image restoration generation unit (6a), and a detection unit (6d) that determines whether the judgment target image is a detection target image or not based on the difference image generated by the difference calculation unit (6b). Accordingly, even if there are obstacles or objects with low transmittance between the monitoring target area and the imaging unit (2), detection target objects existing within the monitoring target area can be detected with high precision. In addition, according to the model generation unit (4) and the learning database (5), even if there are obstacles or objects with low transmittance between the monitoring target area and the imaging unit (2), a learning model can be generated that detects objects within the monitoring target area with high precision.

[0042] Examples

[0043] In this embodiment, the present invention was applied to a process for determining whether a metal container (wafer) that supports a metal product so that the inner diameter of the metal product is not crushed when the metal product is wound during the manufacturing process of the metal product is used as the detection target, and to determine whether the wafer is loaded at a predetermined position. Additionally, a wire mesh fence or a dustproof / spray-proof cover was assumed as an obstacle between the detection target and the imaging unit (2). In the learning model generation process, an image of the monitoring target area in a state where the wafer is not loaded at a predetermined position was captured, and a learning model (5b) was generated by training an autoencoder using 25 captured images. The layer structure of the autoencoder was set as follows, using an index of structural similarity that evaluates changes in pixel values, changes in contrast, and changes in structure. Accordingly, a learning model was obtained that generates an image in a state where the wafer is not loaded at a predetermined position.

[0044] · Encoder: Compresses a 300×300 pixel image into 100 dimensions

[0045] · Decoder: Restores a 100-dimensional image to a 300×300 pixel image

[0046] · Training count: 500 epochs

[0047] · Loss function: SSIM(structural similarity)

[0048] When an image (regardless of the presence or absence of a detection target) is input into this learning model, an image is output in which the wafer is not loaded at a predetermined position. Subsequently, a difference calculation is performed between the input image and the output image. The difference image between the input image in which the wafer is loaded at the predetermined position and the output image in which the wafer is not loaded at the predetermined position is the image of the wafer loaded at the predetermined position. Next, image processing such as binarization, edge detection, and smoothing is performed to emphasize the features of the wafer, and then, to further increase detection accuracy, parts of the image other than the predetermined position are masked. After that, circles and arcs are detected using the Hough transform. The radius and center position of the circle, which are parameters of the Hough transform, were used based on values ​​previously determined from the captured image. Accordingly, there are cases where a circle is detected in a location other than the predetermined position; however, when the camera is fixed, the radius and center position of the circle when the wafer is at the predetermined position are known, and the circle can be detected at a reasonable location. Accordingly, as shown in FIG. 6, even if there are obstacles or objects with low transmittance between the monitoring area and the imaging unit, it was possible to determine with high precision whether or not the object to be detected is reflected in the image to be determined.

[0049] Although embodiments applying the invention made by the inventors have been described above, the present invention is not to be limited by the description and drawings that constitute part of the disclosure of the present invention according to the present embodiments. That is, other embodiments, examples, and operational techniques made by those skilled in the art based on the present embodiments are all included within the scope of the present invention.

[0050] Industrial applicability

[0051] According to the present invention, even if there are obstacles or objects with low transmittance between the monitored area and the imaging device, a detection device, a detection system, and a detection method capable of detecting a target object within the monitored area with high precision can be provided. Furthermore, according to the present invention, even if there are obstacles or objects with low transmittance between the monitored area and the imaging device, a model generation device capable of generating a learning model capable of detecting a target object within the monitored area with high precision can be provided. Explanation of the symbols

[0052] 1 : Detection System 2 : Imaging unit 3 : Data Acquisition Unit 4: Model Creation Section 5: Training Database 5a : Learning Center data 5b: Learning model 6 : Operation unit 6a : Image restoration generation unit 6b : Difference calculation section 6c : Image processing unit 6d: Detector 6e : Output section

Claims

Claim 1 A detection device for determining whether a judgment target image is a detection target image containing a detection target object, comprising: an image restoration generation unit that generates a non-detection target image corresponding to a judgment target image by inputting data of the judgment target image to a learning model having data of a processed image as input data and data of a non-detection target image that does not contain the detection target object according to the processed image as output data; a difference calculation unit that generates a difference image of the judgment target image and the non-detection target image generated by the image restoration generation unit; and a detection unit that determines whether the judgment target image is a detection target image based on the difference image generated by the difference calculation unit. Claim 2 A detection device according to claim 1, comprising an image processing unit that performs image processing to emphasize the features of the object to be detected on a difference image generated by the difference calculation unit, and the detection unit that determines whether the image to be determined is the image to be detected based on the difference image on which image processing is performed by the image processing unit. Claim 3 In paragraph 2, the image processing unit performs mask processing on an image area other than a predetermined image area where the detection target may exist, a detection device. Claim 4 A detection device comprising a model generation unit that generates a learning model by using the data of the non-detectable target image as learning source data in any one of claims 1 to 3, and learning to reconstruct a non-detectable target image identical to the learning source data or a non-detectable target image approximated by the learning source data. Claim 5 In paragraph 4, the above model generation unit is a detection device that is an autoencoder. Claim 6 A detection device according to claim 4 or 5, wherein the model generation unit generates the learning model again by adding the data of the judgment target image, which the detection unit has determined to be a non-detection target image, to the learning source data. Claim 7 A detection system comprising a detection device described in any one of claims 1 to 6, and an imaging unit that captures an image of a surveillance target area and outputs the data of the captured image of the surveillance target area as the data of the image to be determined to the detection device. Claim 8 A detection method for determining whether a judgment target image is a detection target image containing a detection target object, comprising: a learning model having data of a processed image as input data and data of a non-detection target image not containing the detection target object according to the processed image as output data, wherein the data of the judgment target image is input to generate a non-detection target image corresponding to the judgment target image; a difference calculation step for generating a difference image of the judgment target image and the non-detection target image generated in the image restoration generation step; and a detection step for determining whether the judgment target image is a detection target image based on the difference image generated in the difference calculation step. Claim 9 A model generating device for generating a learning model for determining whether a judgment target image is a detection target image containing a detection target object, comprising a model generating unit for generating a learning model having data of a processed image as input data and data of a non-detection target image that does not contain the detection target object according to the processed image as output data, wherein the model generating unit generates the learning model by using the data of the non-detection target image as source data for learning and learning to reconstruct a non-detection target image identical to the source data or a non-detection target image that approximates the source data for learning.

Images