Image processing device, surveillance system, image processing method, and program

Abstract

To provide an image processing device which has few blind spots caused by obstacles and can easily detect a person.SOLUTION: An image processing device includes an overhead image acquisition unit that acquires an overhead image, a detection image acquisition unit that acquires a detection image, a person detection unit that detects a person from the detection image, a coordinate conversion unit that converts the floor position of a person detected by the person detection unit into a floor position in the overhead image, and an image synthesis unit that superimposes an image indicating the floor position converted by the coordinate conversion unit onto the overhead image.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a video processing apparatus, a monitoring system, a video processing method, and a program.

Background Art

[0002] Conventionally, there is a monitoring device that transforms a camera image of a monitored area photographed by a camera to generate an aerial view image, and highlights a portion where a moving object such as a person in the camera image exists in the aerial view image (for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the above-described monitoring device, when the camera is installed with a deep depression angle so as to reduce the blind spots due to obstacles or the like, it becomes difficult to detect a person, and when the camera is installed with a shallow depression angle so as to facilitate the detection of a person, there may be a problem that the blind spots due to obstacles or the like increase.

[0005] The present invention has been made in view of such circumstances, and provides a video processing apparatus, a monitoring system, a video processing method, and a program that have few blind spots due to obstacles or the like and can easily detect a person.

Means for Solving the Problems

[0006] This invention was made to solve the above-mentioned problems, and one aspect of the present invention is an image processing device comprising: an overhead image acquisition unit for acquiring an overhead image; a detection image acquisition unit for acquiring a detection image; a person detection unit for detecting a person from the detection image; a coordinate transformation unit for converting the floor position of the person detected by the person detection unit to the floor position in the overhead image; and an image synthesis unit for superimposing an image showing the floor position transformed by the coordinate transformation unit onto the overhead image.

[0007] Another aspect of the present invention is the image processing apparatus described above, wherein the person detection unit comprises a person region detection unit that detects a region containing the person in the detection image, and a person region correction unit that detects the skeleton of the person and corrects the region containing the person by supplementing the missing parts of the skeleton.

[0008] Another aspect of the present invention is the image processing apparatus described above, wherein the person detection unit comprises a person region detection unit that detects a region containing the person in the detection image, and a person region correction unit that corrects the region containing the person based on the size of a region containing the person that was previously detected.

[0009] Another aspect of the present invention is the image processing apparatus described above, wherein the person detection unit comprises a person region detection unit that detects a region containing the person in the detection image, and a person region correction unit that corrects the region containing the person based on the distance to the person in the detection image.

[0010] Another aspect of the present invention is the image processing apparatus described above, wherein the person detection unit comprises: a person region detection unit that detects a region containing the person in the detection image; a first person region correction unit that detects the skeleton of the person and corrects the region containing the person by supplementing the missing parts of the skeleton; and a second person region correction unit that corrects the region containing the person corrected by the first person region correction unit based on the size of the region containing the person detected in the past.

[0011] Another aspect of the present invention is the image processing apparatus described above, wherein the person detection unit detects the position of the person on the floor by detecting a mark attached to the person's footwear.

[0012] Another aspect of the present invention is a surveillance system comprising an overhead view camera, a detection camera, an image processing device, and an image display device, wherein the image processing device includes an overhead view image acquisition unit that acquires an overhead view image from the overhead view camera, a detection image acquisition unit that acquires a detection image from the detection camera, a person detection unit that detects a person from the detection image, a coordinate transformation unit that converts the floor position of the person detected by the person detection unit to the floor position in the overhead view image, and an image synthesis unit that superimposes an image showing the floor position transformed by the coordinate transformation unit onto the overhead view image and displays it on the image display device.

[0013] Another aspect of the present invention is an image processing method comprising: a first step of acquiring an overhead view image; a second step of acquiring a detection image; a third step of detecting a person from the detection image; a fourth step of converting the floor position of the person detected in the third step into a floor position in the overhead view image; and a fifth step of superimposing an image showing the floor position converted in the fourth step onto the overhead view image.

[0014] Another aspect of the present invention is a program that causes a computer to function as an overhead image acquisition unit for acquiring overhead images, a detection image acquisition unit for acquiring detection images, a person detection unit for detecting people from the detection images, a coordinate transformation unit for converting the floor position of the person detected by the person detection unit into a floor position in the overhead image, and an image synthesis unit for superimposing an image showing the floor position transformed by the coordinate transformation unit onto the overhead image. [Effects of the Invention]

[0015] This invention provides an image processing device for a surveillance system that minimizes blind spots caused by obstacles and facilitates the detection of people. [Brief explanation of the drawing]

[0016] [Figure 1] It is a schematic block diagram showing the configuration of the monitoring system according to the first embodiment of this invention. [Figure 2] It is a side view showing an installation example of the overhead view camera 100 and the detection camera 200 in the same embodiment. [Figure 3] It is a schematic diagram for explaining the operation of the coordinate conversion unit 304 in the same embodiment. [Figure 4] It is an image for explaining the operation of the person detection unit 303 in the same embodiment. [Figure 5] It is a diagram showing an example of the video synthesized by the video synthesis unit 305 in the same embodiment. [Figure 6] It is a flowchart for explaining the operation of the video processing apparatus 300 in the same embodiment. [Figure 7] It is a schematic block diagram showing the configuration of the person detection unit 303 according to the second embodiment of this invention. [Figure 8] It is a schematic block diagram showing the configuration of the person detection unit 303 according to the third embodiment of this invention. [Figure 9] It is a schematic diagram for explaining the operation of the second person area correction unit 333 according to the same embodiment. [Figure 10] It is a schematic block diagram showing the configuration of the person detection unit 303 according to the fourth embodiment of this invention. [Figure 11] It is an explanatory diagram for explaining the hardware configuration of each device according to each embodiment.

Mode for Carrying Out the Invention

[0017] <First Embodiment> Hereinafter, the first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a schematic block diagram showing the configuration of the monitoring system according to the first embodiment of this invention. The monitoring system in this embodiment includes an overhead view camera 100, a detection camera 200, a video processing apparatus 300, and a video display apparatus 400.

[0018] The overhead view camera 100 is a video camera that captures overhead views. The overhead view camera 100 may capture overhead views with a single video camera, or it may generate overhead views by combining images captured by multiple video cameras. Alternatively, the video processing device 300 may generate overhead views by combining images captured by multiple overhead view cameras 100. The overhead view camera 100 may attach a timestamp synchronized with the detection camera 200 to the overhead views.

[0019] The detection camera 200 is a video camera that captures images for person detection (detection images). There may be multiple detection cameras 200. Furthermore, the detection camera 200 may be positioned so that the downward angle of its optical axis is shallower than the downward angle of the optical axis of the overhead image camera 100, in order to capture images suitable for person detection. Additionally, the detection camera 200 may add a timestamp synchronized with the overhead image camera 100 to the overhead image.

[0020] The video processing device 300 includes an overhead video receiving unit 301 (overhead video acquisition unit), a detection video receiving unit 302 (detection video acquisition unit), a person detection unit 303, a coordinate transformation unit 304, and a video synthesis unit 305. The overhead video receiving unit 301 receives overhead video images captured by the overhead video camera 100 from the overhead video camera 100. If the overhead video is encoded by the overhead video camera 100, the overhead video receiving unit 301 also decodes the encoded overhead video.

[0021] The detection video receiving unit 302 receives the detection video captured by the detection camera 200 from the detection camera 200. If the detection video is encoded by the detection camera 200, the detection video receiving unit 302 also decodes the encoded detection video.

[0022] The person detection unit 303 detects a person from the detection video acquired by the detection video receiving unit 302. The person detection unit 303 may detect a person from the frame of the detection video by image recognition, or it may detect a person by detecting a predetermined mark attached to the person's clothing or other items. The person detection unit 303 may also use the lower end of the detected person as the person's floor position, or it may detect the detected person's footwear or a predetermined mark attached to the footwear and use that detection position as the person's floor position.

[0023] The coordinate transformation unit 304 transforms the floor position of the person detected by the person detection unit 303 into the floor position in the overhead view image. The coordinate transformation unit 304 uses a transformation formula (e.g., a homography matrix) or transformation table that assumes the floor position of the person detected by the person detection unit 303 is at the floor height in real space to transform it into the floor position in the overhead view image. The coordinate transformation unit 304 may generate the transformation formula or transformation table from the focal length, installation position, and optical axis orientation of the overhead view camera 100 and the detection camera 200.

[0024] The image synthesis unit 305 superimposes the image showing the floor position, which has been transformed by the coordinate transformation unit 304, onto the overhead view image. The image synthesis unit 305 may also synchronize the superimposition of the image showing the floor position and the overhead view image based on timestamps attached by the overhead view camera 100 and the detection camera 200.

[0025] The video display device 400 is equipped with video display means such as a liquid crystal display or an organic EL (Electro Luminescence) display, and displays an overhead view image in which an image indicating the floor position is superimposed by the video synthesis unit 305.

[0026] Figure 2 is a side view showing an example of the installation of the overhead view camera 100 and the detection camera 200 in this embodiment. The overhead view camera 100 may be installed such that the downward angle β1 of its optical axis is less than 90 degrees, as shown in camera 100a in Figure 2, or it may be installed such that the downward angle β2 of its optical axis is 90 degrees, as shown in camera 100b in Figure 2. The detection camera 200 may also be installed such that the downward angle α of its optical axis is less than the downward angle β1 of camera 100a, which is the overhead view camera 100, or the downward angle β2 of camera 100b.

[0027] Figure 3 is a schematic diagram illustrating the operation of the coordinate transformation unit 304 in this embodiment. The coordinate transformation unit 304 transforms the two-dimensional coordinates of the floor position P1 of the person detected in the detection video G11 into the two-dimensional coordinates of the floor position P2 in the overhead view video G12. This transformation is performed using a transformation formula or transformation table that the coordinate transformation unit 304 stores in advance. The coordinate transformation unit 304 stores this transformation formula or transformation table for each detection camera 200.

[0028] Figure 4 is an image illustrating the operation of the person detection unit 303 in this embodiment. The person detection unit 303 detects people by performing image recognition processing on frames of the detection video, as shown in the image in Figure 4. The person detection unit 303 may perform person detection on all frames of the detection video, or it may perform person detection on only some frames, such as one frame every few frames. In the example shown in Figure 4, a rectangle indicating the area where a person was detected and a dotted line indicating the trajectory of the floor position of the person detected in previous frames are also shown. The video synthesis unit 305 may generate an image by superimposing the above-mentioned rectangle and dotted line onto the detection video and display it on the video display device 400.

[0029] Figure 5 shows an example of an image synthesized by the image synthesis unit 305 in this embodiment. As shown in Figure 5, the image synthesis unit 305 superimposes an image M1 (a triangle in Figure 5) indicating the person detection position onto the floor position transformed by the coordinate transformation unit 304 within each frame of the overhead view image. Furthermore, when the person detection unit 303 is tracking and detecting a person, the image synthesis unit 305 may superimpose a string of characters identifying the tracking ("Tracking 1" in Figure 5) along with the image M1 indicating the person detection position. In this way, by superimposing an image indicating the person detection position onto the overhead view image, the image processing device 300 can generate an image in which the person detection position can be easily grasped intuitively.

[0030] Figure 6 is a flowchart illustrating the operation of the video processing device 300 in this embodiment. The overhead video receiving unit 301 of the video processing device 300 acquires an overhead video from the overhead video camera 100 (step Sa1). Next, the detection video receiving unit 302 of the video processing device 300 acquires a video for person detection from the detection camera 200 (step Sa2). Steps Sa1 and Sa2 may be performed in reverse order or in parallel.

[0031] Next, the person detection unit 303 of the video processing device 300 detects a person from the detection video acquired in step Sa2 (step Sa3). Next, the coordinate transformation unit 304 of the video processing device 300 transforms the floor position of the person detected in step Sa3 to the floor position in the overhead view video (step Sa4). Next, the video synthesis unit 305 of the video processing device 300 superimposes the image showing the floor position transformed in step Sa4 onto the overhead view video acquired in step Sa1 (step Sa5). Next, the video synthesis unit 305 of the video processing device 300 displays the overhead view video with the floor position image superimposed in step Sa5 on the video display device 400 (step Sa6). The video processing device 300 returns to step Sa1 and repeats these processes until it finishes processing (step Sa7).

[0032] <Second Embodiment> A second embodiment of the present invention will be described below with reference to the drawings. The monitoring system according to the second embodiment estimates the floor position of a person even when a part of the person, such as their legs, is hidden by furniture or the like in the detection video. The configuration of the monitoring system in this embodiment is the same as that in Figure 1.

[0033] Figure 7 is a schematic block diagram showing the configuration of a person detection unit 303 according to a second embodiment of the present invention. As shown in Figure 7, the person detection unit 303 in this embodiment comprises a person region detection unit 331 and a first person region correction unit 332. The person region detection unit 331 detects a person from the detection video by image recognition and determines a rectangular region containing the person.

[0034] The first person area correction unit 332 corrects the rectangular area determined by the person area detection unit 331, and sets the lower end (center of the lower edge) of the corrected rectangular area as the floor position of the person. The first person area correction unit 332 estimates the skeleton of the person included in the rectangular area using, for example, a technique called bone print (registered trademark) (https: / / www.mitsubishielectric.co.jp / news / 2019 / 1009.html). If there are any missing parts in the estimated skeleton, the first person area correction unit 332 fills in the missing parts and corrects the rectangular area to include the entire skeleton with the missing parts filled in. The first person area correction unit 332 may determine the size when filling in the missing parts based on the size of any part of the estimated skeleton. In addition, the first person area correction unit 332 may perform the process of filling in the missing parts only when the feet are missing.

[0035] <Third Embodiment> A third embodiment of the present invention will be described below with reference to the drawings. Similar to the second embodiment, the surveillance system according to the third embodiment estimates the floor position of a person even when part of the person is hidden in the detection image. However, in the third embodiment, this estimation is performed using past information. The configuration of the surveillance system in this embodiment is the same as that in Figure 1.

[0036] Figure 8 is a schematic block diagram showing the configuration of a person detection unit 303 according to a third embodiment of the present invention. In Figure 8, the same reference numerals are used for parts that are the same as in Figure 7, and their descriptions are omitted. The person detection unit 303 according to this embodiment includes a person region detection unit 331, a second person region correction unit 333, and a past region storage unit 334. The past region storage unit 334 stores rectangular regions detected in past frames of the detection video. The second person region correction unit 333 corrects the rectangular region detected by the person region detection unit 331 by referring to the past region storage unit 334.

[0037] The second person area correction unit 333 compares the size of the rectangular area detected by the person area detection unit 331 with the size of past rectangular areas stored in the past area storage unit 334. If the size is smaller than a predetermined ratio, it corrects the rectangular area by extending it downwards based on the size of past rectangular areas. The second person area correction unit 333 may use the average or median size of rectangular areas over a predetermined number of frames as the size of past rectangular areas. Alternatively, the second person area correction unit 333 may treat the size of past rectangular areas as time-series data and use the size obtained by extrapolating it in the time direction as the comparison target.

[0038] Furthermore, the second person area correction unit 333 may correct the size of the rectangular area based on the distance from the detection camera 200 to the person. The second person area correction unit 333 may also correct the size of the rectangular area using the relationship between the size of past rectangular areas and the distance to the person stored in the past area storage unit 334. The relationship between the size of past rectangular areas and the distance to the person used in this case may be one that is estimated to be the same person. The second person area correction unit 333 may also estimate the distance from the detection camera 200 to the person from the detection images captured by multiple detection cameras 200. Alternatively, the detection camera 200 may be equipped with a distance measuring sensor such as a depth camera, and the second person area correction unit 333 may use the measurement results from the distance measuring sensor.

[0039] Figure 9 is a schematic diagram illustrating the operation of the second person area correction unit 333 according to this embodiment. In Figure 9, rectangles F1 to F6 are rectangular areas stored by the past area storage unit 334. Rectangle F7 is a rectangular area corrected by the second person area correction unit 333. The person in rectangle F7 is hidden from the waist down. Therefore, the area detected by the person area detection unit 331 may be a rectangular area that does not include the lower half of the person, and its lower end is not at the floor level of the person. Therefore, the second person area correction unit 333 extends the rectangular area detected by the person area detection unit 331 downwards to obtain rectangle F7 so that it is the same as the average size of rectangles F1 to F6 detected in past frames and stored by the past area storage unit 334, and sets its lower end at the floor level.

[0040] The past area storage unit 334 may also store the size (especially the height) of the rectangular area detected in past frames.

[0041] <Fourth Embodiment> A fourth embodiment of the present invention will now be described with reference to the drawings. The monitoring system according to the fourth embodiment performs correction of the person detection area in the second embodiment and correction of the person detection area in the third embodiment. The configuration of the monitoring system in this embodiment is the same as that in Figure 1.

[0042] Figure 10 is a schematic block diagram showing the configuration of a person detection unit 303 according to a fourth embodiment of the present invention. The person detection unit 303 in this embodiment comprises a person area detection unit 331, a first person area correction unit 332, a second person area correction unit 333, and a past area storage unit 334. The person area detection unit 331 and the first person area correction unit 332 are the same as the person area detection unit 331 and the first person area correction unit 332 in Figure 7. The second person area correction unit 333 and the past area storage unit 334 are the same as the second person area correction unit 333 and the past area storage unit 334 in Figure 8.

[0043] Note that the processing order of the first person area correction unit 332 and the second person area correction unit 333 may be reversed.

[0044] Figure 11 is an explanatory diagram illustrating the hardware configuration of each device according to the above-described embodiment. Each device is an image processing device 300. Each device comprises an input / output module I, a memory module M, and a control module P. The input / output module I is implemented by including some or all of a communication module H11, a connection module H12, a pointing device H21, a keyboard H22, a display H23, buttons H3, a microphone H41, a speaker H42, a camera H51, or a sensor H52. The memory module M is implemented by including a drive H7. The memory module M may further comprise some or all of a memory H8. The control module P is implemented by including a memory H8 and a processor H9. These hardware components are connected to each other via a bus so as to be able to communicate with each other, and are powered by a power supply H6.

[0045] The connection module H12 is a digital input / output port such as USB (Universal Serial Bus). In the case of a portable device, the pointing device H21, keyboard H22, and display H23 are touch panels. Sensors H52 include accelerometers, gyroscopes, GPS receiver modules, proximity sensors, etc. Power supply H6 is a power supply unit that supplies the electricity necessary to operate each device. In the case of a portable device, power supply H6 is a battery. Drive H7 is an auxiliary storage medium such as a hard disk drive or solid-state drive. Drive H7 may be non-volatile memory such as EEPROM or flash memory, or a magneto-optical disk drive or flexible disk drive. Also, drive H7 is not limited to those built into each device, for example, but may be an external storage device connected to the connector of the connection module H12. Memory H8 is a main memory medium such as random access memory. Memory H8 may also be cache memory. Memory H8 stores instructions when they are executed by one or more processors H9. Processor H9 is the CPU (Central Processing Unit). The processor H9 may be an MPU (microprocessing unit) or a GPU (graphics processing unit). The processor H9 reads programs and various data from drive H7 via memory H8 and performs calculations to execute instructions stored in one or more memory H8s.

[0046] The input / output module I is used for the overhead image receiving unit 301, the detection image receiving unit 302, the image synthesis unit 305, etc. The memory module M implements the past memory area storage unit 334. The control module P is used for the implementation of each part of the image processing device 300. In this specification, the descriptions of the image processing device 300, the overhead image receiving unit 301 (overhead image acquisition unit), the detection image receiving unit 302 (detection image acquisition unit), the person detection unit 303, the coordinate transformation unit 304, the image synthesis unit 305, the person area detection unit 331, the first person area correction unit 332, the second person area correction unit 333, and the past memory area storage unit 334 may be replaced with the description of the control module P.

[0047] The following embodiments may also be used. (1) One embodiment is an image processing device comprising: an overhead image acquisition unit for acquiring an overhead image; a detection image acquisition unit for acquiring a detection image; a person detection unit for detecting a person from the detection image; a coordinate transformation unit for converting the floor position of the person detected by the person detection unit into a floor position in the overhead image; and an image synthesis unit for superimposing an image showing the floor position transformed by the coordinate transformation unit onto the overhead image. This allows the video processing device to detect people easily with fewer blind spots caused by obstacles.

[0048] (2) Another embodiment is the image processing apparatus described in (1) above, wherein the person detection unit comprises a person region detection unit that detects a region containing the person in the detection image, and a person region correction unit that detects the skeleton of the person and corrects the region containing the person by supplementing the missing parts of the skeleton. As a result, the video processing device can estimate the floor position of a person even if part of the person is not captured in the detection video.

[0049] (3) Another embodiment is the image processing apparatus described in (1) above, wherein the person detection unit comprises a person region detection unit that detects a region containing the person in the detection image, and a person region correction unit that corrects the region containing the person based on the size of a region containing the person that was previously detected. This allows the image processing device to correct the area containing the detected person to an area that takes into account its past size.

[0050] (4) Another embodiment is the image processing apparatus described in (1) above, wherein the person detection unit comprises a person region detection unit that detects a region containing the person in the detection image, and a person region correction unit that corrects the region containing the person based on the distance to the person in the detection image. This allows the image processing device to correct the area containing the detected person to an area that takes into account the distance to the person.

[0051] (5) Another embodiment is the image processing apparatus described in (1) above, wherein the person detection unit comprises: a person region detection unit that detects a region containing the person in the detection image; a first person region correction unit that detects the skeleton of the person and corrects the region containing the person by supplementing the missing parts of the skeleton; and a second person region correction unit that corrects the region containing the person corrected by the first person region correction unit based on the size of the region containing the person that was previously detected. This allows the video processing device to correct the area containing the detected person, taking into account past size, even if part of the person is not captured in the detection video.

[0052] (6) Another embodiment is the image processing apparatus described in (1) above, wherein the person detection unit detects the position of the person on the floor by detecting a mark attached to the person's footwear. This allows the video processing device to detect the floor position of a person wearing marked footwear.

[0053] (7) Another embodiment is a surveillance system comprising an overhead view camera, a detection camera, an image processing device, and an image display device, wherein the image processing device comprises an overhead view image acquisition unit that acquires an overhead view image from the overhead view camera, a detection image acquisition unit that acquires a detection image from the detection camera, a person detection unit that detects a person from the detection image, a coordinate transformation unit that converts the floor position of the person detected by the person detection unit to the floor position in the overhead view image, and an image synthesis unit that superimposes an image showing the floor position transformed by the coordinate transformation unit onto the overhead view image and displays it on the image display device. This allows the surveillance system to have fewer blind spots caused by obstacles and to easily detect people.

[0054] (8) Another embodiment is an image processing method comprising: a first step of acquiring an overhead image; a second step of acquiring a detection image; a third step of detecting a person from the detection image; a fourth step of converting the floor position of the person detected in the third step into a floor position in the overhead image; and a fifth step of superimposing an image showing the floor position converted in the fourth step onto the overhead image. This allows for video processing methods that minimize blind spots caused by obstacles and facilitate the detection of people.

[0055] (9) Another embodiment is a program that causes a computer to function as an overhead image acquisition unit for acquiring overhead images, a detection image acquisition unit for acquiring detection images, a person detection unit for detecting people from the detection images, a coordinate transformation unit for converting the floor position of the person detected by the person detection unit into the floor position in the overhead image, and an image synthesis unit for superimposing an image showing the floor position converted by the coordinate transformation unit onto the overhead image. This allows the program to have fewer blind spots caused by obstacles and to easily detect people.

[0056] Alternatively, the functions of each part of the video processing device 300 shown in Figure 1 may be realized by recording a program for realizing each part of the video processing device 300 on a computer-readable recording medium, and then loading and executing the program recorded on this recording medium into a computer system. The term "computer system" here includes hardware such as the operating system and peripheral devices.

[0057] Furthermore, "computer system" shall also include the homepage provisioning environment (or display environment) if a WWW system is being used. Furthermore, "computer-readable recording media" refers to portable media such as flexible disks, magneto-optical disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into computer systems. Moreover, "computer-readable recording media" also includes those that dynamically hold programs for a short period of time, such as communication lines used when transmitting programs over networks such as the Internet or communication lines such as telephone lines, and those that hold programs for a fixed period of time, such as volatile memory inside a computer system acting as a server or client in such cases. In addition, the above-mentioned program may be intended to implement only a part of the functions described above, and may also be a program that can implement the aforementioned functions in combination with a program already recorded in the computer system.

[0058] While embodiments of this invention have been described in detail above with reference to the drawings, the specific configuration is not limited to these embodiments and may include design modifications and the like that do not depart from the spirit of this invention. [Explanation of symbols]

[0059] 100 Cameras for overhead shots 200 detection cameras 300 Video Processing Devices 301 Overhead View Receiving Unit 302 Detection video receiver 303 Person Detection Department 304 Coordinate Transformation Unit 305 Video Compositing Section 331 Human Area Detection Unit 332 1st human area correction section 333 2nd human area correction section 334 Past area storage unit

Claims

1. An overhead image acquisition unit that acquires an overhead image with a time stamp, A detection video acquisition unit acquires a detection video with a timestamp synchronized with the aforementioned overhead view video, A person detection unit that detects a person from the aforementioned detection video, A coordinate transformation unit converts the floor position of the person detected by the person detection unit into the floor position in the overhead view image, The image synthesis unit superimposes the image showing the floor position transformed by the coordinate transformation unit onto the overhead view image in synchronization with the timestamps attached to the overhead view image and the detection image. A video processing device equipped with the following features.

2. The aforementioned person detection unit, The aforementioned detection video includes a person region detection unit that detects the region containing the person, A person region correction unit detects the skeleton of the person and corrects the region containing the person by supplementing the missing parts of the skeleton. The image processing apparatus according to claim 1, comprising:

3. The aforementioned person detection unit, The aforementioned detection video includes a person region detection unit that detects the region containing the person, A person region correction unit corrects the region containing the person based on the size of the region containing the person that was previously detected. The image processing apparatus according to claim 1, comprising:

4. The aforementioned person detection unit, The aforementioned detection video includes a person region detection unit that detects the region containing the person, A person area correction unit corrects the area containing the person based on the distance to the person in the detection video. The image processing apparatus according to claim 1, comprising:

5. The aforementioned person detection unit, The aforementioned detection video includes a person region detection unit that detects the region containing the person, A first person region correction unit detects the skeleton of the person and corrects the region containing the person by supplementing the missing parts of the skeleton, A second person area correction unit corrects the area containing the person corrected by the first person area correction unit based on the size of the area containing the person previously detected. The image processing apparatus according to claim 1, comprising:

6. The image processing apparatus according to claim 1, wherein the person detection unit detects the position of the person on the floor by detecting a mark attached to the person's footwear.

7. A surveillance system comprising an overhead view camera, a detection camera, an image processing device, and an image display device, The aforementioned video processing device is An overhead image acquisition unit that acquires overhead images with timestamps from the overhead image camera, A detection video acquisition unit acquires detection video from the aforementioned detection camera, which has a timestamp synchronized with the overhead view video. A person detection unit that detects a person from the aforementioned detection video, A coordinate transformation unit converts the floor position of the person detected by the person detection unit into the floor position in the overhead view image, An image synthesis unit superimposes an image showing the floor position transformed by the coordinate transformation unit onto the overhead view image in synchronization with the timestamps attached to the overhead view image and the detection image, and displays it on the image display device. A monitoring system equipped with the following features.

8. A first step of acquiring an overhead image with a timestamp, A second step is to acquire a detection video with a timestamp synchronized with the aforementioned overhead view video, A third step involves detecting a person from the aforementioned detection video, A fourth step involves converting the floor position of the person detected in the third step to the floor position in the overhead view image, A fifth step involves superimposing the image showing the floor position converted in the fourth step onto the overhead view image in synchronization with the timestamps attached to the overhead view image and the detection image. A video processing method having the following characteristics.

9. Computers, An overhead image acquisition unit that acquires overhead images with timestamps. A detection video acquisition unit acquires a detection video with a timestamp synchronized with the aforementioned overhead view video. A person detection unit that detects a person from the aforementioned detection video, A coordinate transformation unit converts the floor position of the person detected by the person detection unit into the floor position in the overhead view image. An image synthesis unit superimposes the image showing the floor position transformed by the coordinate transformation unit onto the overhead view image, synchronized with the timestamps attached to the overhead view image and the detection image. A program designed to function as such.

Images