Image processing device
An image processing device installed in existing analog systems enhances surveillance capabilities by reducing costs and construction work, enabling efficient anomaly detection and recording.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- MAXELL FRONTIER CO LTD
- Filing Date
- 2022-04-13
- Publication Date
- 2026-05-29
Smart Images
Figure 0007867365000001 
Figure 0007867365000002 
Figure 0007867365000003
Abstract
Description
Technical Field
[0001] The present invention relates to an image processing apparatus used in a video information processing system that performs monitoring and inspection using video information.
Background Art
[0002] As one of video information processing systems, there is a monitoring system that performs monitoring using video information. As background art of the monitoring system, for example, Patent Documents 1 and 2 exist. Patent Document 1 discloses a camera monitoring system that records video information captured by an analog camera with a recording device and transmits the video information to a monitor device. Further, Patent Document 2 discloses a camera monitoring system that captures a monitoring target using an IP camera and collects the video data via an IP network such as the Internet or a LAN.
Prior Art Documents
Patent Documents
[0003]
Patent Document 1
Patent Document 2
Summary of the Invention
Problems to be Solved by the Invention
[0004] In a monitoring system using an analog camera as in Patent Document 1, a monitor is required to check for abnormalities, and further, it is difficult to identify abnormal scenes from a large amount of recorded video. Also, there was a problem that the camera had low image quality and was unclear. Further, in order to solve this, in a monitoring system using an IP camera as in Patent Document 2, it is possible to automatically discriminate whether there is an abnormality by image processing or AI processing of a personal computer, and there is an advantage that a clear video can be obtained due to the high pixel count of the camera.
[0005] However, a problem with migrating from an analog camera-based surveillance system to an IP camera-based surveillance system was the need to completely replace existing analog equipment and the large-scale construction work required, such as wiring in the ceiling, resulting in high implementation costs. This is a common challenge not only for surveillance systems but also for any video information processing system using cameras.
[0006] In view of the above background technology and problems, the object of the present invention is to provide an image processing device that enables the construction of an intelligent image information processing system using a camera simply by installing it on existing equipment, and that reduces implementation costs by reusing existing equipment. [Means for solving the problem]
[0007] One example of the present invention is an image processing device used in a video information processing system having a camera, a recording device for recording video information captured by the camera, and a monitor device for displaying the information. The device comprises a video input unit for capturing video information captured by the camera as input video, a processing unit for processing the input video, and a video output unit for outputting the video processed by the processing unit as output video. The processing unit processes the video so that the input video and the output video have the same specifications, and is configured to be installed between the camera, the recording device, and the monitor device via a cable. [Effects of the Invention]
[0008] According to the present invention, an intelligent monitoring system can be constructed in a video information processing system using a camera simply by installing it on existing equipment, and an image processing device can be provided that reduces implementation costs by reusing existing equipment. [Brief explanation of the drawing]
[0009] [Figure 1] This is a schematic diagram of a conventional surveillance system using analog cameras. [Figure 2] This is a schematic diagram of a conventional surveillance system using IP cameras. [Figure 3] This is a schematic diagram of the monitoring system in Example 1. [Figure 4] This is a functional configuration diagram of the image processing device used in the monitoring system in Example 1. [Figure 5] This is a block diagram illustrating the functional configuration of the image processing apparatus in Example 1. [Figure 6] This is an explanatory diagram of output mode 1 of the output video of the image processing device in Embodiment 1. [Figure 7] This is an explanatory diagram of output mode 2 of the output video of the image processing device in Embodiment 1. [Figure 8] This is an explanatory diagram of output mode 3 of the output video of the image processing device in Embodiment 1. [Figure 9] This diagram illustrates the necessity of image-audio merging processing in Example 1. [Figure 10] This is an explanatory diagram of the image-audio merging process of the image processing apparatus in Embodiment 1. [Figure 11] This is an image diagram of the merged image data generated by the image and audio merging process in Example 1. [Figure 12] This is an explanatory diagram illustrating the anomaly detection method using merged image data from the image processing device in Example 1. [Figure 13] This is an explanatory diagram of an anomaly detection method using AI with merged image data from the image processing device in Example 1. [Figure 14] This is an image diagram of the merged image data in Example 2. [Figure 15] This is a functional configuration diagram of the image processing device used in the monitoring system in Example 2. [Figure 16] This is a specific image diagram of the merged image data in Example 2. [Figure 17] This is a schematic diagram of a conventional industrial image inspection system. [Figure 18] This is a schematic diagram of the industrial image inspection system using the image processing device in Example 3. [Modes for carrying out the invention]
[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings.
Embodiment
[0011] In this embodiment, a monitoring system that performs monitoring using video information as a video information processing system using a camera will be described.
[0012] FIG. 1 is a schematic configuration diagram of a monitoring system using an analog camera that is a premise of this embodiment. As shown in FIG. 1, in a monitoring system using an analog camera, the analog camera 10, a recording device 20 for recording, and a monitor device 30 are connected by a cable 13 laid on the ceiling space 12 or the like. The video information captured by the analog camera 10 is recorded by the recording device 20 and transmitted to the monitor device 30. The monitor 40 monitors the monitor device 30 and checks for abnormalities. Generally, it is difficult for the monitor to identify abnormal scenes from a large amount of recorded video, and there is also a problem that the analog camera has low image quality and is unclear.
[0013] On the other hand, FIG. 2 is a schematic configuration diagram of a monitoring system using an IP camera that is a premise of this embodiment. In a monitoring system using an IP camera, the monitoring target is photographed using the IP camera, and the video data is transmitted and processed to a recording device or an information processing device via an IP network such as the Internet or a LAN. FIG. 2 shows the case of LAN connection. The IP camera 11 is connected to the recording device
[0014] However, a problem with migrating from an analog camera-based surveillance system to an IP camera-based surveillance system was that it required the complete replacement of existing analog equipment and involved large-scale construction work, such as wiring in the ceiling, resulting in high implementation costs.
[0015] Therefore, in this embodiment, an image processing device that enables the construction of an intelligent monitoring system using an analog camera simply by installing it on existing equipment, and that reduces implementation costs by reusing existing equipment, will be described below.
[0016] Figure 3 is a schematic diagram of the monitoring system in this embodiment. In Figure 3, components identical to those in Figure 1 are denoted by the same reference numerals, and their descriptions are omitted. The difference between Figure 3 and Figure 1 is that an image processing device 60 is provided between the analog camera 10, the recording device 20, and the monitoring device 30 via a cable 13.
[0017] Figure 4 is a functional configuration diagram of the image processing device used in the surveillance system in this embodiment. In Figure 4, the image processing device 60 receives video and audio information captured by the analog camera 10 as input video and input audio, converts it into digital data by the video / audio decoder 61, and takes it into the video / audio input unit 62. The data from the video / audio input unit 62 is then processed as images and audio by the processor (processing arithmetic unit) 70. The processor 70 is composed of a CPU and an image processing processor as hardware, and has intelligent functions such as image processing and AI (Artificial Intelligence) processing to increase image resolution 71, image and audio merging processing 72, and anomaly detection 73.
[0018] The processor 70 implements the above functions through software processing, which involves reading and executing an operation program stored in memory (not shown) that is controlled by the CPU, etc. The image memory 63 is used as a buffer to implement the above functions and to store the processed image data.
[0019] The video processed by the processor 70 is output to the video / audio output unit 64, where it is converted to a predetermined format by the video / audio encoder 65 and then output as output video and output audio to the external recording device 20 and monitoring device 30. The processor 70 also detects abnormalities in the monitored object through the abnormality determination process 73. When the processor 70 detects an abnormality, it outputs an alert and controls the video / audio output unit 64 to output the image in which the abnormality was detected. It also performs processes such as sending an email 31 to an external PC or smartphone or activating a warning light 32 via the alert interface (hereinafter referred to as I / F) 66.
[0020] It should be noted that the image processing apparatus in this embodiment is not limited to having all the configurations shown in Figure 4; for example, it may have a configuration specifically for image processing.
[0021] Figure 5 is a block diagram illustrating the functional configuration of the image processing device in this embodiment. In Figure 5, the explanation focuses specifically on image processing. In Figure 5, the same functions as in Figure 4 are denoted by the same reference numerals, and their explanations are omitted. In Figure 5, selector 74 selects the image to be used in the abnormality detection 73, with (1) selecting the camera image and (2) selecting a high-resolution image created by increasing the resolution of the camera image. Selector 75 selects the image to be output to the video / audio output unit 64, with (3) selecting only the image determined to be abnormal by the abnormality detection 73, and (4) selecting all images used in the abnormality detection 73. The selections of these selectors 74 and 75 may be set by the user each time, or they may be pre-set. Furthermore, since the image processing device 60 is intended to be connectable to existing recording devices 20 and monitoring devices 30, the input video and output video must have the same specifications.
[0022] Next, the output modes of the output video for the input video of the image processing device in this embodiment will be described. Figure 6 is an explanatory diagram of output mode 1 of the output video of the image processing device in this embodiment. In Figure 6, output mode 1 is a mode in which all input images are output. As shown in Figure 6, for an input video consisting of input images arranged in chronological order, which are video information captured by the analog camera 10, the output video outputs all input images. In this case, if an anomaly is detected in scenes 4, 5, and 7, for example, as shown in Figure 6, by the anomaly detection process 73, an alert is output at the timing of scenes 4, 5, and 7.
[0023] Figure 7 is an explanatory diagram of output mode 2 of the output video of the image processing device in this embodiment. In Figure 7, output mode 2 is a mode that outputs only abnormal images, and shows the same case as in Figure 6. As shown in Figure 7, the output video outputs only scenes 4, 5, and 7, which are abnormal images. This makes it possible to record only abnormal scenes, and improves the efficiency of the recording capacity of the recording device.
[0024] Figure 8 is an explanatory diagram of output mode 3 of the output video of the image processing device in this embodiment. In Figure 8, output mode 3 is a mode that outputs all images + the number of abnormal scenes, and shows the same case as in Figure 6. As shown in Figure 8, the output video outputs all input images with the cumulative number of abnormal scenes added. That is, the output image of scene 4 is output with the cumulative number of abnormal scenes of 1 added, the output image of scene 5 is output with the cumulative number of abnormal scenes of 2 added, and the output image of scene 7 is output with the cumulative number of abnormal scenes of 3 added. This improves the efficiency of identifying abnormal scenes.
[0025] Next, the image-audio merging process in this embodiment will be described. Figure 9 illustrates the necessity of the image-audio merging process in this embodiment. In Figure 9, for example, when monitoring an intruder entering through a window, there is no sound in normal circumstances, as shown in the upper figure. However, in abnormal circumstances, an abnormal sound is generated, for example, when opening a window to enter, as shown in the middle figure. Then, an abnormal image appears, as shown in the lower figure, in which the intruder appears. Thus, abnormal sounds may accompany the abnormal image. Therefore, by merging image data and audio data for analysis, it is possible to determine signs of an anomaly from the audio data.
[0026] Figure 10 is an explanatory diagram of the image-audio merging process of the image processing device in this embodiment. In Figure 10, input video consisting of time-series input images, which are video information captured by the analog camera 10, and input audio corresponding to the input video are taken into the video / audio input unit 62 of the image processing device 60 via the video / audio decoder 61. Then, in the image-audio merging process 72 of the processor 70 within the image processing device 60, a merged image is generated by merging the input video and input audio. Here, as shown in Figure 10, if there are abnormalities in scenes 4, 5, and 8 of the input video, and abnormalities in scenes 4, 5, 7, and 8 of the input audio, by performing the abnormality determination process 73 using the merged image, it becomes possible to detect abnormalities based on abnormal audio in scene 7, prior to abnormality detection based on abnormal image in scene 8, and thus it becomes possible to detect early signs of abnormality.
[0027] Figure 11 is an image diagram of the merged image data generated by the image-audio merging process in this embodiment. As shown in Figure 11, the merged image data has an image data area and an audio data area, and the audio data area is an area where one-dimensional audio data is stored as two-dimensional data.
[0028] Figure 12 is an explanatory diagram of the anomaly detection method using merged image data in the image processing device of this embodiment. Figure 12 shows the anomaly detection process using difference comparison by image processing. As shown in Figure 12, a difference comparison is performed by image processing between a template merged image consisting of normal image data and silent audio data, and the target merged image data. As a result, if there is a difference in the audio data portion as comparison result 1, or if there is a difference in both the image data and audio data as comparison result 2, or if there is a difference only in the image data portion (not shown), each is judged as an anomaly. In this way, by using merged image data, it becomes possible to perform anomaly detection including audio data using a general image processing anomaly detection algorithm.
[0029] Figure 13 is an explanatory diagram of the AI-based anomaly detection using merged image data in this embodiment. As shown in Figure 13, the AI performs a comparative processing using training data consisting of normal image data and silent audio data, and the input image, and makes an anomaly determination based on the processing result. In this way, by using merged image data, it becomes possible to perform anomaly detection including audio data using a general AI-based anomaly detection algorithm.
[0030] As described above, by installing the image processing device in this embodiment between the analog camera 10, the recording device 20, and the monitor device 30 via a cable 13, an intelligent surveillance system can be easily constructed. Furthermore, the existing analog camera 10, recording device 20, monitor device 30, and cable 13 can all be reused, and large-scale construction work such as wiring in the ceiling space is unnecessary, thus reducing installation costs. In addition, the image processing device can monitor and notify whether or not there is an abnormality, enabling labor savings. Moreover, the image processing device can record only abnormal scenes and improve the image quality of camera footage.
[0031] As described above, according to this embodiment, an intelligent monitoring system can be constructed in a monitoring system using analog cameras simply by installing it on existing equipment, and an image processing device can be provided that reduces implementation costs by reusing existing equipment. [Examples]
[0032] In Example 1, we described how the image-audio merging process generates a merged image by merging the input video and input audio. In this example, we will describe a process for merging data that is not limited to input video and audio data.
[0033] Figure 14 is an illustrative diagram of merged image data in this embodiment. As shown in Figure 14, in this embodiment, the merged image data has an image data area and a data area that is not limited to audio data. The data area is an area where data such as light, temperature, humidity, pressure, distance, and current is stored.
[0034] Figure 15 is a functional configuration diagram of the image processing device used in the monitoring system in this embodiment. In Figure 15, the same components as in Figure 4 are denoted by the same reference numerals, and their descriptions are omitted. The difference between Figure 15 and Figure 4 is that Figure 15 includes a sensor 17, a sensor decoder 67, and a sensor input unit 68.
[0035] In Figure 15, data such as light, temperature, humidity, pressure, distance, and current from the sensor 17, corresponding to the input video captured by the analog camera 10, are acquired by the sensor input unit 68 of the image processing device 60 via the sensor decoder 67. Then, in the image and audio merging process 72 of the processor 70 within the image processing device 60, a merged image is generated by merging the input video and data, as shown in Figure 14.
[0036] Figure 16 is a concrete image diagram of the merged image data in this embodiment. In Figure 16, it is assumed that plant growth is monitored using image data, temperature, humidity, and light sensor data. The image data area stores plant image data, and the data area stores temperature, humidity, and light data. Using this merged image data, the processor 70 of the image processing device 60 described in Embodiment 1 performs image processing and AI-based monitoring. This makes it possible to perform detailed control by using various data in addition to image data, based on their monitoring.
[0037] As described above, according to this embodiment, in addition to the same effects as in Embodiment 1, a monitoring system that enables fine-grained control can be constructed by using input video and various corresponding data. [Examples]
[0038] In this embodiment, an industrial image inspection system that uses video information for inspection is described as a video information processing system using a camera.
[0039] Figure 17 is a schematic diagram of a conventional industrial image inspection system that serves as the premise for this embodiment. As shown in Figure 17, in industrial image inspection, image data of the object to be imaged 90, captured by an industrial camera 16, is transmitted via cable 13 to an image processing device such as a personal computer. In Figure 17, if the processing device is, for example, a personal computer 50, the personal computer 50 has a video capture board (frame grabber) 51, an image processing arithmetic unit 52, and memory 53. For example, the video capture board 51 and memory 53 are connected by PCI Express, which is a serial transmission I / F, and the image processing arithmetic unit 52 performs image processing calculations by exchanging image data with memory 53.
[0040] In industrial image inspection, it is necessary to process image data captured by a camera at high speed without losing frames. Therefore, the image processing unit must be equipped with a high-performance image processing arithmetic unit and large-capacity memory. Furthermore, a relay device called a repeater may be installed between the camera and the image processing unit for long-distance transmission of captured image data.
[0041] Figure 18 is a schematic diagram of an industrial image inspection system using the image processing device in this embodiment. In Figure 18, the same functions as in Figure 17 are denoted by the same reference numerals, and their explanations are omitted. In Figure 18, the difference from Figure 17 is that the image processing device 60 is provided between the industrial camera 16 and the personal computer 50 via a cable 13. The image processing device 60 has the intelligent functions described in Embodiment 1, and for example, by using output mode 2 of the output video of the image processing device described in Figure 7, only the data of abnormal scenes can be transferred to the personal computer 50, thereby reducing the processing specifications and memory capacity of the personal computer 50, which is the image processing device. In addition, because the image processing device 60 rebuffers the input signal before outputting it, it can have a repeater function and can support long-distance transmission.
[0042] As described above, according to this embodiment, it is possible to provide an image processing device that can reduce the processing specifications and memory capacity of the image processing device and also incorporate repeater functionality simply by installing it in a conventional industrial image inspection system.
[0043] Although embodiments have been described above, the present invention is not limited to the embodiments described above and includes various modifications. For example, the embodiments described above are described in detail to make the present invention easy to understand and are not necessarily limited to those having all the configurations described. Furthermore, it is possible to replace parts of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add configurations from other embodiments to the configuration of one embodiment. In addition, it is possible to add, delete, or replace parts of the configuration of each embodiment with other configurations. [Explanation of symbols]
[0044] 10: Analog camera, 11: IP camera, 12: Attic, 13: Cable, 14: LAN cable, 15: HUB, 16: Industrial camera, 17: Sensor, 20: Recording device, 30: Monitoring device, 31: Email transmission, 32: Warning light, 40: Surveillance personnel, 50: PC, 51: Video capture board (frame grabber), 52: Image processing unit, 53: Memory, 60: Image processing unit, 61: Video / audio decoder, 62: Video / audio input unit, 63: Image memory, 64: Video / audio output unit, 65: Video / audio encoder, 66: Alert I / F, 67: Sensor decoder, 68: Sensor input unit, 70: Processor, 71: High resolution enhancement, 72: Image / audio merging processing, 73: Anomaly detection, 74, 75: Selector, 90: Target to be imaged
Claims
1. An image processing device used in a video information processing system having a camera, a recording device for recording video information captured by the camera, and a monitor device for displaying the information, A video input unit that captures video information captured by the aforementioned camera as input video, A processing unit for processing the input video, It has a video output unit that outputs the video processed by the processing unit as output video, The processing unit has a function to decode the input video and convert it into digital data, and a function to encode the video into the same format as the input video. A cable is provided between the camera, the recording device, and the monitoring device. The image processing device is characterized in that it has a function for increasing the resolution of the input video and a function for determining abnormalities, and has a first selector function for selecting whether to use the input video as the image used for determining abnormalities or to use the input video as the high-resolution image.
2. In the image processing apparatus according to claim 1, The image processing apparatus is characterized in that the processing unit has a second selector function that selects whether the output video consists only of images that have been determined to be abnormal in the abnormality determination, or all images used in the abnormality determination.
3. In the image processing apparatus according to claim 2, The image processing device is characterized in that, when the processing arithmetic unit selects the output video to be all the images used for the anomaly determination using a second selector function, it adds the cumulative number of abnormal scenes determined to be abnormal in the anomaly determination to each output image that makes up the output video and outputs it.
4. In the image processing apparatus according to claim 1, The image processing device is characterized in that the processing arithmetic unit performs abnormality detection on the input video and outputs an alert if an abnormality is detected.