Image rotation processing device, image rotation processing method and program

Abstract

A video rotation processing device that can perform appropriate rotation processing on a video signal is provided. [Solution] A video rotation processing device 16 is provided with an estimated tilt angle detection unit 20 that detects an estimated tilt angle θ, which is the angle that an estimated vertical direction, which is an estimated vertical direction, forms with a predetermined direction of the video, for each of multiple video signals V, and a rotation processing unit 30 that rotates the video signals V as necessary so that the estimated vertical direction in each video signal V coincides with the predetermined direction. This makes it possible to align the estimated vertical direction in each video signal V so that it coincides with the predetermined direction.

Description

[Technical Field]

[0001] The disclosed technology relates to an image rotation processing device, an image rotation processing method, and a program. [Background technology]

[0002] As background art in this technical field, paragraph 0064 of the specification of Patent Document 1 listed below states, "As shown in FIG. 5, the imaging device 200 is installed so that it can capture an image of the floor surface FL within a predetermined area and the person 500 moving along a movement path PA on the floor surface FL. More specifically, the installation position of the imaging device 200, the orientation of the camera 230 (direction of the optical axis of the lens), lens distortion, etc. are adjusted so that the floor surface FL of the predetermined area and the standing person 500 fit within the field of view of the camera 230. For example, the imaging device 200 is installed a predetermined distance from the movement path PA of the person 500. Furthermore, the imaging device 200 is adjusted so that the orientation of the camera 230 is slightly downward relative to the horizontal so that the entire floor surface FL within the predetermined area fits within the field of view of the camera 230."

[0003] Furthermore, paragraph 0035 of the specification of Patent Document 2 below states, "In this embodiment, the skeletal information is defined by a total of 25 joints, joint numbers 0 to 24. Hereinafter, the joint with joint number 0 will be abbreviated as J0, etc. A joint is a part that functions as a joint in the skeletal information, and the posture can be changed by rotating the line segment connecting these joints around the joint..." The descriptions in the above documents are included as part of the specification of this application. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2024 / 232157 [Patent Document 2] Japanese Patent Publication No. 2022-86610 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the above-mentioned Patent Documents 1 and 2 do not particularly mention the application of appropriate rotation processing to video signals of multiple systems. The disclosed technology has been made in consideration of the above-mentioned circumstances, and aims to provide a video rotation processing device, a video rotation processing method, and a program that perform appropriate rotation processing on video signals of multiple systems. [Means for solving the problem]

[0006] In order to solve the above problem, a video rotation processing device according to one aspect of the disclosed technology includes an estimated tilt angle detection unit that detects an estimated tilt angle, which is an angle between an estimated vertical direction, which is an estimated vertical direction, and a predetermined direction of the video, for each of a plurality of video signals, and a rotation processing unit that rotates the video signals as necessary so that the estimated vertical direction in each of the video signals coincides with the predetermined direction. The estimated tilt angle detection unit includes a person image detection unit that detects a person image from each of the video signals, a skeleton information extraction unit that extracts skeleton information from each of the person images, a standing person detection unit that detects people in a standing posture by excluding people who have fallen down based on the skeleton information, and an averaging processing unit that calculates a person estimated vertical direction that is a vertical direction estimated for each person in a standing posture, and calculates the estimated tilt angle by taking an average value of the person estimated vertical directions in each of the video signals as the estimated vertical direction for that video signal. It is characterized by: [Effects of the Invention]

[0007] According to the disclosed technology, it is possible to perform appropriate rotation processing on video signals of multiple systems. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a block diagram of a video monitoring system according to a first embodiment of the disclosed technique. [Figure 2] FIG. 10 is a diagram showing a display example based on a video signal. [Figure 3] FIG. 1 is a block diagram of a computer. DETAILED DESCRIPTION OF THE INVENTION

[0009] [First embodiment] <Configuration of the first embodiment> FIG. 1 is a block diagram of a video monitoring system 10 according to a first embodiment of the disclosed technique. The video surveillance system 10 comprises n cameras 12-1 to 12-n, a display 14, and a video rotation processing device 16 (computer). In the following description, multiple components, physical quantities, information, etc. having the same or similar functions or significance may be referred to by the same reference numeral with a "-" and an alphanumeric character added, such as "cameras 12-1, 12-2." However, when it is not necessary to distinguish between these multiple components, etc., they may be referred to by omitting the "-" and the alphanumeric character, such as "camera 12." The cameras 12-1 to 12-n are installed throughout a facility such as a train station, and output the results of capturing images of the locations where they are installed as video signals V-1 to Vn.

[0010] FIG. 2 is a diagram showing a display example based on the video signal V. Image G1 shown in FIG. 2 is an example of an image obtained when camera 12 is installed in a normal installation state. Generally, image G1 based on video signal V has an aspect ratio such as "4:3" or "16:9," resulting in an image in which the horizontal direction is longer than the vertical direction. In image G1, the vertical direction is the vertical direction. However, in cases such as when a user wants to obtain more depth information at the installation location of camera 12, camera 12 may be installed by rotating it 90° along the optical axis. An image such as that shown in image G2 is obtained from camera 12 installed with the camera rotated 90°. In image G2, the horizontal direction is the vertical direction. Note that the vertical direction is the same as the direction of gravity.

[0011] If multiple images with different vertical directions were displayed together on the display 14, the images would be difficult to see, so it is preferable to display them on the display 14 with the same vertical direction. In the example of FIG. 2, for example, rotating image G2 by 90 degrees would align the vertical directions of images G1 and G2. However, due to restrictions on the installation locations of the cameras 12, the rotation angle of the images may not be exactly 90 degrees. When multiple cameras 12 are installed in a facility, it is extremely cumbersome for a user to individually set the rotation angle for each camera 12. Therefore, the image rotation processing device 16 of this embodiment automatically performs rotation processing as needed for each video signal V from each camera 12.

[0012] 1, the image rotation processing device 16 includes an estimated tilt angle detection unit 20 (estimated tilt angle detection means, estimated tilt angle detection process), a rotation processing unit 30 (rotation processing means, rotation processing process), and an image synthesis unit 32. The estimated tilt angle detection unit 20 includes a person image detection unit 22, a skeletal information extraction unit 24, a standing posture person detection unit 26, and an averaging processing unit 28.

[0013] The person image detection unit 22 detects one or more person images, which are image areas in which people are captured, from each video signal V. The skeletal information extraction unit 24 extracts skeletal information of the corresponding person from each person image. That is, the skeletal information extraction unit 24 detects feature points (joint points) from the person image and extracts skeletal information using multiple lines connecting the feature points. The standing posture person detection unit 26 detects people who are standing based on each skeletal information. This is to exclude skeletal information of people who are lying down or crouching, for example, from the estimation of the vertical direction.

[0014] The averaging processor 28 calculates an estimated human vertical direction, which is the vertical direction estimated for each piece of skeletal information, based on skeletal information of a person in a standing posture belonging to multiple frames of the video signal Vk (where 1≦k≦n). That is, skeletal information whose vertical direction is known is acquired in advance as reference skeletal information, and the estimated human vertical direction can be calculated by comparing the skeletal information acquired from the video signal Vk with the reference skeletal information. Furthermore, the averaging processor 28 sets the average value of the estimated human vertical directions estimated for each piece of skeletal information included in the video signal Vk as the estimated vertical direction of the video signal Vk. Then, the averaging processor 28 outputs the angle formed by the estimated vertical direction with respect to a predetermined direction of the video signal Vk (e.g., the vertical direction) as the estimated tilt angle θ-k of the video signal Vk.

[0015] As described above, estimated tilt angle detection unit 20 outputs estimated tilt angles θ-1 to θ-n of video signals V-1 to Vn. Rotation processing unit 30 rotates video signals V-1 to Vn by estimated tilt angles θ-1 to θ-n, respectively, except when the estimated tilt angle θ is 0°, and outputs the results as processed video signals VR-1 to VR-n. Video synthesis unit 32 synthesizes processed video signals VR-1 to VR-n and displays the synthesized video signals on display 14.

[0016] As a result, images GR-1 to GR-n based on the processed video signals VR-1 to VR-n are displayed on the display 14. Here, the images GR-1 to GR-n may be displayed with the same display area, or only some of the images GR designated by the user may be enlarged and displayed. In this embodiment, the rotation processing unit 30 performs a rotation process on each video signal V as needed. That is, the rotation processing unit 30 rotates the video signal as needed so that the estimated vertical direction coincides with a predetermined direction (for example, the vertical direction), and therefore the images GR-1 to GR-n with the vertical direction aligned (for example, the vertical direction) can be displayed on the display 14.

[0017] 3 is a block diagram of the computer 980. The image rotation processing device 16 shown in FIG. 1 includes one or more computers 980 shown in FIG. 3, a computer 980 includes a CPU 981, a storage unit 982, a communication port 983, an input / output port 984, and a media port 985. The storage unit 982 includes a RAM 982a, a ROM 982b, and an SSD (Solid State Drive) 982c. The communication port 983 is connected to a communication circuit 986. The input / output port 984 is connected to an input / output device 987. The media port 985 reads and writes data from a recording medium 988.

[0018] The ROM 982b stores an IPL (Initial Program Loader) executed by the CPU, etc. The SSD 982c stores application programs, various data, etc. The CPU 981 executes application programs, etc. loaded from the SSD 982c to the RAM 982a, thereby realizing various functions. The interior of the video rotation processing device 16 shown in FIG. 1 is primarily shown as a block diagram of functions realized by application programs, etc.

[0019] [Variations] The disclosed technology is not limited to the above-described embodiments, and various modifications are possible. The above-described embodiments are provided as examples to facilitate understanding of the disclosed technology, and are not necessarily limited to those including all of the described configurations. Furthermore, other configurations may be added to the configurations of the above-described embodiments, and some of the configurations may be replaced with other configurations. Furthermore, the control lines and information lines shown in the figures are those considered necessary for explanation, and do not necessarily represent all control lines and information lines necessary in the product. In reality, it can be assumed that almost all configurations are interconnected. Possible modifications of the above-described embodiments include, for example, the following:

[0020] (1) In the above embodiment, the estimated inclination angle detection unit 20 calculates the estimated inclination angle θ based on the image of a person included in the video signal V. However, the estimated inclination angle detection unit 20 may calculate the estimated inclination angle θ based on the inclination of an image of a pillar, floor, ceiling, wall, etc. included in the video signal V.

[0021] (2) Since the hardware of the image rotation processing device 16 in the above embodiment can be realized by a general computer, the above-mentioned block diagrams and other programs for executing the above-mentioned various processes may be stored on a storage medium (a computer-readable storage medium on which a program is recorded) or distributed via a transmission path.

[0022] (3) In the above embodiment, the above-mentioned block diagrams and various other processes are described as software processes using programs, but some or all of them may be replaced with hardware processes using ASICs (Application Specific Integrated Circuits) or FPGAs (Field Programmable Gate Arrays), etc.

[0023] (4) The various processes executed in the above embodiment may be executed by a server computer via a network (not shown), and the various data stored in the above embodiment may also be stored in the server computer.

[0024] [Effects of the embodiment] As described above, according to the embodiment, the video rotation processor 16 includes an estimated tilt angle detector 20 that detects, for each of the video signals V of multiple systems, an estimated tilt angle θ that is the angle that the estimated vertical direction forms with a predetermined direction of the video, and a rotation processor 30 that rotates the video signals V as necessary so that the estimated vertical direction in each video signal V coincides with the predetermined direction. This makes it possible to align the estimated vertical direction in each video signal V so that it coincides with the predetermined direction, and to perform appropriate rotation processing on the video signals of multiple systems.

[0025] Furthermore, it is more preferable that the estimated tilt angle detection unit 20 includes a person image detection unit 22 that detects a person image from each video signal V, a skeletal information extraction unit 24 that extracts skeletal information from each person image, a standing posture person detection unit 26 that detects a person in a standing posture based on the skeletal information, and an averaging processing unit 28 that calculates a person estimated vertical direction, which is the vertical direction estimated for each person in each standing posture, and calculates an estimated tilt angle θ by taking the average value of the person estimated vertical directions in each video signal V as the estimated vertical direction for that video signal V. This makes it possible to calculate the estimated tilt angle θ for each video signal V based on multiple person images, thereby enabling more appropriate rotation processing for multiple systems of video signals. [Explanation of symbols]

[0026] 16 Image rotation processing device (computer) 20 Estimated inclination angle detection unit (estimated inclination angle detection means, estimated inclination angle detection process) 22 Person image detection unit 24 Skeleton information extraction unit 26 Standing posture person detection unit 28 Averaging processing section 30 Rotation processing unit (rotation processing means, rotation processing process) V Video signal θ Estimated tilt angle

Claims

1. an estimated tilt angle detection unit that detects an estimated tilt angle, which is an angle that an estimated vertical direction, which is an estimated vertical direction, forms with respect to a predetermined direction of the video, for each of the video signals of the plurality of systems; a rotation processing unit that rotates the video signal as necessary so that the estimated vertical direction in each of the video signals coincides with the predetermined direction; The estimated tilt angle detection unit a person image detection unit that detects a person image from each of the video signals; a skeleton information extraction unit that extracts skeleton information from each of the person images; a standing person detection unit that detects a person in a standing position by excluding a person who is lying down based on the skeletal information; an averaging processing unit that calculates an estimated person vertical direction, which is a vertical direction estimated for each person in each standing posture, and calculates the estimated tilt angle by taking an average value of the estimated person vertical directions in each of the video signals as the estimated vertical direction in that video signal.

1. A video rotation processing device comprising:

2. an estimated tilt angle detection step of detecting an estimated tilt angle, which is an angle formed by an estimated vertical direction, which is an estimated vertical direction, with respect to a predetermined direction of the video, for each of the video signals of the plurality of systems; a rotation process for rotating the image signals as necessary so that the estimated vertical direction in each of the image signals coincides with the predetermined direction; The estimated tilt angle detection process includes: a person image detecting step of detecting a person image from each of the video signals; a skeleton information extraction step of extracting skeleton information from each of the person images; a standing person detection step of detecting a person in a standing position by excluding a person who is lying down based on the skeletal information; an averaging processing step of calculating an estimated person vertical direction, which is a vertical direction estimated for each person in each standing posture, and calculating the estimated tilt angle by taking an average value of the estimated person vertical directions in each of the video signals as the estimated vertical direction in that video signal.

10. A video rotation processing method comprising:

3. Computer, an estimated tilt angle detection means for detecting an estimated tilt angle, which is an angle formed by an estimated vertical direction, which is an estimated vertical direction, with respect to a predetermined direction of the video, for each of the video signals of the plurality of systems; a rotation processing means for rotating the video signal as necessary so that the estimated vertical direction in each of the video signals coincides with the predetermined direction; A program for causing the device to function as a The estimated tilt angle detection means a person image detecting means for detecting a person image from each of the video signals; a skeleton information extraction means for extracting skeleton information from each of the human images; a standing person detection unit that detects a person in a standing position by excluding a person who is lying down based on the skeletal information; an averaging processing means for calculating an estimated person vertical direction, which is a vertical direction estimated for each person in each standing posture, and calculating the estimated tilt angle by taking an average value of the estimated person vertical directions in each of the video signals as the estimated vertical direction in that video signal. A program characterized by:

Images