Detection device, detection system, detection method, and model generation device
The detection device and model generation system address the challenge of detecting objects through image restoration and difference processing, ensuring accurate detection even with obstacles or low transmittance, using a learning model to reconstruct images and generate difference images.
Patent Information
- Application Number
- JP2024006100
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2024-01-18
- Publication Date
- 2025-07-31
AI Technical Summary
Existing detection devices struggle to accurately detect objects in a monitored area when obstacles or objects with low transmittance exist between the monitoring target area and the imaging device.
A detection device equipped with an image restoration generation unit, difference calculation unit, and detection unit, utilizing a learning model to generate and process difference images, and a model generation unit to create a learning model that reconstructs detection target images, enabling accurate detection even with obstacles or low transmittance objects.
The device can accurately detect objects in the monitored area despite obstacles or low transmittance objects, and the model generation device can create a learning model for precise detection.
Smart Images
Figure 2025112048000001_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a detection device, a detection system, a detection method, and a model generation device for determining whether a determination target image is a detection target image including a detection target object.
Background Art
[0002] Patent Document 1 describes a device that extracts a change region from an image sequentially captured by an imaging device in a monitoring target area, and compares a feature amount of the monitoring target area based on the extracted change region with a set value of a predetermined feature amount to determine the presence or absence of an event.
Prior Art Documents
Patent Documents
[0003]
Patent Document 1
Summary of the Invention
Problems to be Solved by the Invention
[0004] In the device described in Patent Document 1, when there are obstacles such as a safety fence, a fence, or a cover for dust and drip prevention between the monitoring target area and the imaging device, there is a possibility that a detection target object existing in the monitoring target area cannot be detected. For this reason, even when there are obstacles or objects with low transmittance between the monitoring target area and the imaging device, it has been expected to provide a technology capable of accurately detecting a detection target object existing in the monitoring target area.
[0005] The present invention has been made in consideration of the above-mentioned problems, and its object is to provide a detection device, a detection system, and a detection method that can accurately detect a detection object present in a monitored area even if an obstacle or an object with low transmittance exists between the monitored area and the imaging device. Another object of the present invention is to provide a model generation device that can generate a learning model that accurately detects a detection object present in a monitored area even if an obstacle or an object with low transmittance exists between the monitored area and the imaging device. [Means for solving the problem]
[0006] The detection device of the present invention is a detection device that determines whether an image to be determined is a detection target image that includes a detection target object, and is equipped with an image restoration generation unit that generates a detection target image corresponding to the image to be determined by inputting data of the image to be determined into a learning model that has data of a processed image as input data and data of a detection target image that includes the detection target object corresponding to the processed image as output data, a difference calculation unit that generates a difference image between the image to be determined and the detection target image generated by the image restoration generation unit, and a detection unit that determines whether the image to be determined is the detection target image based on the difference image generated by the difference calculation unit.
[0007] The image processing unit may be provided with an image processing unit that performs image processing on the difference image generated by the difference calculation unit to emphasize the features of the object to be detected, and the detection unit may determine whether the image to be determined is the image to be detected based on the difference image on which image processing has been performed by the image processing unit.
[0008] The detection section may determine whether the determination target image is the detection target image based on the number of pixels in the difference image whose luminance value is equal to or greater than a predetermined threshold value.
[0009] A model generation unit may be provided that generates the learning model by learning to reconstruct a detection target image that is the same as the learning source data or a detection target image that approximates the learning source data, using the data of the detection target image as the learning source data.
[0010] The model generation unit may be an autoencoder.
[0011] The model generation unit may add the data of the determination target image determined by the detection unit to be a detection target image to the learning source data and regenerate the learning model.
[0012] The detection system according to the present invention includes the detection device according to the present invention and an imaging unit that captures an image of a monitoring target area and outputs the data of the captured image of the monitoring target area to the detection device as the data of the determination target image.
[0013] The detection method according to the present invention is a detection method for determining whether a determination target image is a detection target image including a detection target object, and includes an image restoration generation step of generating a detection target image corresponding to the determination target image by inputting the data of the determination target image to a learning model that uses the data of the processed image as input data and the data of the detection target image including the detection target object corresponding to the processed image as output data, a difference calculation step of generating a difference image between the determination target image and the detection target image generated in the image restoration generation step, and a detection step of determining whether the determination target image is the detection target image based on the difference image generated in the difference calculation step.
[0014] The model generation device of the present invention is a model generation device that generates a learning model for determining whether an image to be determined is a detection target image that includes a detection target object, and is equipped with a model generation unit that generates a learning model using data of a processed image as input data and data of a detection target image that includes the detection target object corresponding to the processed image as output data, and the model generation unit generates the learning model by learning to reconstruct a detection target image that is identical to the learning target data or a detection target image that approximates the learning target data, using the data of the detection target image as learning source data. [Effects of the Invention]
[0015] The detection device, detection system, and detection method according to the present invention can accurately detect a detection target that exists in a monitored area even if an obstacle or an object with low transmittance exists between the monitored area and the imaging device.Furthermore, the model generation device according to the present invention can generate a learning model that accurately detects a detection target that exists in a monitored area even if an obstacle or an object with low transmittance exists between the monitored area and the imaging device. [Brief explanation of the drawings]
[0016]
Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
[0017] Hereinafter, with reference to the drawings, the configuration and operation of a detection system according to an embodiment of the present invention will be described.
[0018] 〔Configuration〕 FIG. 1 is a block diagram showing the configuration of a detection system according to an embodiment of the present invention. As shown in FIG. 1, a detection system 1 according to an embodiment of the present invention is a system that determines whether a determination target image is a detection target image including a detection target object, and includes an imaging unit 2, a data acquisition unit 3, a model generation unit 4, a learning database 5, and a calculation unit 6. The data acquisition unit 3, the model generation unit 4, and the calculation unit 6 function as a detection device according to the present invention.
[0019] The imaging unit 2 is composed of an imaging device such as a camera or an industrial monitoring camera (ITV). The imaging unit 2 captures an image of a monitoring target area and outputs the captured image data of the monitoring target area to the data acquisition unit 3. Specifically, the imaging unit 2 captures images of the monitoring target area before and after the switching of an event or before and after the movement of a movable object, and outputs the captured image data of the monitoring target area to the data acquisition unit 3.
[0020] The data acquisition unit 3 and the model generation unit 4 are functional blocks realized by an arithmetic processing device such as a CPU inside an information processing device executing a computer program. The functions of the data acquisition unit 3 and the model generation unit 4 will be described later.
[0021] The learning database 5 is composed of a non-volatile memory device. The learning database 5 stores learning source data 5a and a learning model 5b. Although details will be described later, the learning source data 5a is image data of the monitoring target area at the time when a detection target object exists in the monitoring target area. The learning model 5b is a machine learning model that takes the image data (data of the determination target image) of the monitoring target area to be determined as input data, and the image data of the monitoring target area at the time when a detection target object exists in the monitoring target area corresponding to the input data as output data. The model generation unit 4 and the learning database 5 may be configured as other systems or devices different from the detection system 1.
[0022] The arithmetic unit 6 is composed of an arithmetic processing device such as a CPU inside the information processing device. The arithmetic unit 6 functions as an image restoration generation unit 6a, a difference calculation unit 6b, an image processing unit 6c, a detection unit 6d, and an output unit 6e when the arithmetic processing device executes a computer program. The functions of these units will be described later. The information processing device that realizes these units and the information processing device that realizes the data acquisition unit 3 and the model generation unit 4 may be the same device or different devices. In the case of different devices, the information processing devices are connected via a telecommunication line.
[0023] By executing the following detection process and learning process, the detection system 1 having such a configuration can accurately detect a detection target object existing in the monitoring target area even when there is an obstacle or an object with a low transmittance between the monitoring target area and the imaging unit 2. Hereinafter, with reference to the flowchart shown in FIG. 2, the operation of the detection system 1 when executing the detection process and the learning model generation process will be described.
[0024] 〔Detection process〕 First, with reference to FIG. 2, the operation of the detection system 1 when executing the detection process will be described.
[0025] 2 is a flowchart showing the flow of detection processing according to one embodiment of the present invention. The flowchart shown in FIG. 2 starts when an execution command for the detection processing is input to the calculation unit 6, and the detection processing proceeds to step S1.
[0026] In the process of step S1, the data acquisition unit 3 inputs the image data of the area to be monitored output from the imaging unit 2 as data of the image to be determined to the calculation unit 6. This completes the process of step S1, and the detection process proceeds to the process of step S2.
[0027] In the process of step S2, the image restoration generation unit 6a inputs the data of the image to be determined input by the data acquisition unit 3 as input data to the learning model 5b, thereby generating image data (output image) of the area to be monitored at the time when the detection object exists in the area to be monitored, reflecting the features of the image to be determined. This completes the process of step S2, and the detection process proceeds to the process of step S3.
[0028] In the process of step S3, the difference calculation unit 6b generates a difference image between the determination target image input by the data acquisition unit 3 and the image of the monitoring target area generated by the image restoration generation unit 6a. An image of the difference image generation process is shown in Figure 3. Details of Figure 3 will be described later. This completes the process of step S3, and the detection process proceeds to the process of step S4.
[0029] In the process of step S4, the image processing unit 6c performs image processing such as binarization, masking, edge detection, smoothing, filtering, correction, expansion, and contraction on the generated difference image to emphasize features such as the shape of the detection object to make it easier to detect the detection object. In the binarization process, the image processing unit 6c converts the difference image into an image containing only two values: white and black. This completes the process of step S4, and the detection process proceeds to the process of step S5.
[0030] In the process of step S5, the detection unit 6d determines whether or not the detection target object exists in the difference image by determining whether or not features such as the shape of the object to be detected (target shape) exist in the difference image after image processing. Specifically, if the detection target object is reflected in the image to be determined, the difference image will be completely black, with nothing reflected, as shown in Fig. 3. On the other hand, if the detection target image is not reflected in the image to be determined, the difference image will show features of the detection target object, such as the target shape, reflected near the detection target object.
[0031] Therefore, when the binarization process is performed in the process of step S4, the detection unit 6d counts the number of white pixels in the difference image and determines whether the count value is equal to or less than a threshold value preset according to the image size and the purpose of detection. If the count value is equal to or less than the threshold value, the detection unit 6d determines that the detection target object is included in the determination target image. The number of threshold values is not limited to one, and multiple threshold values may be used to gradually determine the change in events or scenes included in the difference image. This completes the process of step S5, and the detection process proceeds to the process of step S6.
[0032] In the process of step S6, the output unit 6e outputs to an output device whether or not the detection target object exists in the image to be determined, based on the determination result of the detection unit 6d. Examples of the output device include a printing device, a display device, and an audio output device. This completes the process of step S6, and the series of detection processes ends.
[0033] [Learning model generation process] Next, with reference to FIG. 4, the operation of the detection system 1 when executing the learning model generation process will be described.
[0034] Fig. 4 is a flowchart showing the flow of a learning model generation process according to one embodiment of the present invention. The flowchart shown in Fig. 4 starts when image data of a monitored area in which it has been determined that a detection target object exists is acquired, and the learning model generation process proceeds to step S11. The image data of the monitored area in which it has been determined that a detection target object exists may be acquired in the detection process described above, or may be separately collected and determined.
[0035] In the process of step S11, the model generation unit 4 adds the image data of the monitoring area in which it is determined that the detection target object exists to the learning source data 5a stored in the learning database 5. This completes the process of step S11, and the learning model generation process proceeds to the process of step S12.
[0036] In the process of step S12, the model generation unit 4 generates a learning model 5b using the learning source data 5a. As the learning model 5b, a learning model capable of reconstructing an image identical or similar to the learning source data is used. An example of such a learning model is an autoencoder. An example of the configuration of an autoencoder is shown in FIG. 5. As shown in FIG. 5, an autoencoder is a learning model that learns features from training images, extracts features by compressing the input image and reducing the dimensions, and reconstructs and outputs an image that matches or approximates the training content.
[0037] Therefore, when data of an image to be determined is input to the learning model 5b learned using the learning source data 5a, the learning model 5b outputs, as output data, image data of the monitoring target area at the time when a detection target object exists in the monitoring target area corresponding to the image to be determined. As described above, since the autoencoder has the property of learning parameters so as to restore the learned image, there is an advantage that less adjustment is required to cope with unknown objects and disturbances, and less learning data is sufficient. For this reason, even if the image to be determined has changed according to the change in the surrounding environment of the imaging unit 2, the learning model 5b generates an output image corresponding to the image to be determined, so it is not necessary to perform adjustment corresponding to various conditions. As a result, the process of step S12 is completed, and the series of learning processes is terminated.
[0038] As is clear from the above description, the detection system 1 according to an embodiment of the present invention includes an image restoration generation unit 6a that generates a detection target image corresponding to an image to be determined by inputting data of the image to be determined to a learning model that uses data of a processed image as input data and data of a detection target image including a detection target object corresponding to the processed image as output data, a difference calculation unit 6b that generates a difference image between the image to be determined and the detection target image generated by the image restoration generation unit 6a, and a detection unit 6d that determines whether or not the image to be determined is a detection target image based on the difference image generated by the difference calculation unit 6b. Thereby, even when an obstacle or an object with a low transmittance exists between the monitoring target area and the imaging unit 2, the detection target object existing in the monitoring target area can be accurately detected. Further, according to the model generation unit 4 and the learning database 5, even when an obstacle or an object with a low transmittance exists between the monitoring target area and the imaging unit 2, a learning model that can accurately detect the detection target object existing in the monitoring target area can be generated.
Example
[0039] In this example, the present invention was applied to a process for determining whether a metal tube (whopper) supporting a metal product to prevent its inner diameter from being crushed when it is wound during the manufacturing process of the metal product was used as the detection object. Furthermore, obstacles such as a wire mesh fence or a dust-proof / water-proof cover were assumed to exist between the detection object and the imaging unit 2. In the learning model generation process, images of the monitored area with the whopper inserted in its designated position were captured, and 25 of the captured images were trained using an autoencoder to generate learning model 5b. The autoencoder had the following layer structure, and structural similarity indexes were used to evaluate changes in pixel values, contrast, and structure. This resulted in a learning model that generates images of the whopper inserted in its designated position.
[0040] Encoder: Compresses a 150x150 pixel image to 100 dimensions Decoder: Reconstructs 100 dimensions into a 150x150 pixel image Training times: 500 epochs Loss function: SSIM (structural similarity)
[0041] When an image (regardless of the presence or absence of the object to be detected) is input into this learning model, an image in a state where the wrapper is loaded at a predetermined position is output. Then, the difference between the input image and the output image is calculated. The difference image between the input image with the wrapper loaded at the predetermined position and the output image with the wrapper loaded at the predetermined position is a black image. Next, after performing binarization processing on the difference image, the portions other than the predetermined position in the difference image are masked, and the number of pixels with high luminance (white portions) is counted. Then, by determining whether the counted value is less than or equal to a predetermined threshold, it is determined whether the object to be detected is reflected in the input image. As the thresholds, a first threshold (=30) and a second threshold (=100) that were previously considered based on the image size of 150×150 pixels and the size of the region where the object to be detected exists were used. When the counted value is less than or equal to the first threshold, it is determined that the scene is one where the wrapper is loaded at the predetermined position, and when the counted value is greater than or equal to the first threshold and less than or equal to the second threshold, it is determined that the wrapper is not loaded at the predetermined position. Also, when the counted value is greater than the second threshold, it is determined that an obstacle is moving or that it is an unknown scene. As a result, as shown in FIG. 6, even when there is an obstacle or an object with a low transmittance between the monitoring target area and the imaging unit, it was possible to accurately determine whether the object to be detected is reflected in the image to be determined.
[0042] As described above, the embodiments to which the invention made by the present inventors is applied have been described, but the present invention is not limited by the description and drawings that form a part of the disclosure of the present invention according to this embodiment. That is, all other embodiments, examples, and operation techniques made by those skilled in the art based on this embodiment are included in the scope of the present invention.
Explanation of Reference Numerals
[0043] 1 Detection system 2 Imaging unit 3 Data acquisition unit 4 Model generation unit 5 Learning database 5a Learning source data 5b Learning model 6 Arithmetic unit 6a Image restoration and generation unit 6b Difference calculation unit 6c Image processing unit 6d Detection unit 6e Output unit
Claims
1. A detection device for determining whether a determination target image is a detection target image including a detection target object, an image restoration generation unit that inputs data of the determination target image to a learning model that uses data of a processed image as input data and data of a detection target image including the detection target object corresponding to the processed image as output data, thereby generating a detection target image corresponding to the determination target image; a difference calculation unit that generates a difference image between the determination target image and the detection target image generated by the image restoration generation unit; a detection unit that determines whether the determination target image is the detection target image based on the difference image generated by the difference calculation unit; A detection device comprising:
2. The detection device according to claim 1, further comprising an image processing unit that performs image processing for emphasizing features of the detection target object on the difference image generated by the difference calculation unit, and the detection unit determines whether the determination target image is the detection target image based on the difference image on which the image processing has been performed by the image processing unit.
3. The detection device according to claim 1, wherein the detection unit determines whether the determination target image is the detection target image based on the number of pixels in the difference image whose luminance value is equal to or greater than a predetermined threshold value.
4. The detection device according to claim 1, further comprising a model generation unit that generates the learning model by learning to reconstruct the same detection target image as the learning source data or a detection target image approximating the learning source data using the data of the detection target image as the learning source data.
5. The detection device according to claim 4, wherein the model generation unit is an autoencoder.
6. The detection device according to claim 4, wherein the model generation unit adds data of the determination target image determined by the detection unit to be the detection target image to the learning source data and regenerates the learning model.
7. A detection system comprising: the detection device according to any one of claims 1 to 6; and an imaging unit that captures an image of a monitoring target area and outputs data of the captured image of the monitoring target area to the detection device as data of the determination target image.
8. A detection method for determining whether a determination target image is a detection target image including a detection target object, An image restoration generation step of generating a detection target image corresponding to the determination target image by inputting data of the determination target image to a learning model that uses data of a processed image as input data and data of a detection target image including the detection target object corresponding to the processed image as output data; A difference calculation step of generating a difference image between the determination target image and the detection target image generated in the image restoration generation step; A detection step of determining whether the determination target image is the detection target image based on the difference image generated in the difference calculation step; A detection method comprising the above.
9. A model generation device for generating a learning model for determining whether a determination target image is a detection target image including a detection target object, Comprising a model generation unit that generates a learning model that uses data of a processed image as input data and data of a detection target image including the detection target object corresponding to the processed image as output data; The model generation unit generates the learning model by learning to reconstruct the same detection target image as the learning source data or a detection target image approximating the learning source data using the data of the detection target image as the learning source data.
Citation Information
Patent Citations
Inspection device and inspection method
WO2018105028A1
Coating composition for flexible foam material
JP1978088827A