Tracking monitoring device, tracking monitoring system, tracking monitoring method, and tracking monitoring program

Abstract

To provide a tracking monitoring technique capable of monitoring an object to be tracked for a longer time.SOLUTION: A tracking monitoring device includes: a video receiving unit configured to receive video data captured by a first camera; a target detecting unit configured to detect a target from received video data and measure a moving speed of a detected target; a subsequent-stage camera comparing unit configured to calculate a predicted arrival time at which the target reaches an imaging range of a second camera based on the measured moving speed; and a control unit configured to transmit a calculated predicted arrival time to a second camera.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to tracking and monitoring technology. [Background technology]

[0002] Patent Document 1 discloses a technology relating to a control device that controls a camera configured to be capable of pan-tilt-zoom (PTZ) control, characterized in that the control device has an acquisition means for acquiring video captured by the camera, a tracking means for PTZ-controlling the camera to detect and track a subject included in the captured video, a determination means for determining a delay time from when a PTZ control instruction is sent to the camera until when an image reflecting the instruction is input from the camera, and a view angle setting means for setting the camera's view angle based on the delay time. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2023-154971 Summary of the Invention [Problem to be solved by the invention]

[0004] For example, when constructing a wide-area surveillance system equipped with the control device described in Patent Document 1 and multiple swivel cameras capable of pan-tilt-zoom (PTZ) control, a fixed distance is set between the swivel cameras to reduce costs. This causes a problem in that the swivel cameras cannot coordinate to track and monitor, and the monitoring time for a particular swivel camera to track a target becomes shorter.

[0005] The present disclosure has been made to solve such problems, and aims to provide a tracking and monitoring technology that can monitor a tracked target for a longer period of time. [Means for solving the problem]

[0006] One aspect of a tracking monitoring device according to an embodiment of the present disclosure includes a video receiving unit that receives video data captured by a first camera, a target detection unit that detects a target from the received video data and measures the movement speed of the detected target, a subsequent camera comparison unit that calculates a predicted arrival time for the target to arrive within the shooting range of a second camera based on the measured movement speed, and a control unit that transmits the calculated predicted arrival time to the second camera. [Effects of the Invention]

[0007] According to the tracking monitoring device of the embodiment of the present disclosure, the control unit transmits the predicted arrival time of the target to the second camera, so that the second camera can prepare to capture the target at the predicted arrival time, thereby enabling the target to be monitored for a longer period of time. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a configuration diagram of a tracking monitoring system according to a first or second embodiment. [Figure 2] 1 is a block diagram of a tracking monitoring system according to a first embodiment. [Figure 3A] 1 is a diagram illustrating an example of a hardware configuration of a tracking monitoring device according to a first or second embodiment. [Figure 3B] 1 is a diagram illustrating an example of a hardware configuration of a tracking monitoring device according to a first or second embodiment. [Figure 4] 1 is a schematic diagram illustrating the operation of the tracking monitoring system according to the first embodiment. [Figure 5] 4 is a flowchart showing the operation of the tracking monitoring device according to the first or second embodiment. [Figure 6] FIG. 10 is a block diagram of a tracking monitoring system according to a second embodiment. [Figure 7] FIG. 10 is a schematic diagram illustrating the operation of the tracking and monitoring system according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Various embodiments of the present disclosure will be described in detail below with reference to the drawings. In the drawings, identical or similar parts are designated by identical or similar reference numerals, and redundant explanations of such parts will be omitted. In addition, in this disclosure, the term "or" is used to mean an inclusive logical OR unless otherwise specified.

[0010] Embodiment 1 The tracking and monitoring device for a rotating camera according to the first embodiment receives information from a rotating camera that has detected a target to be tracked, extracts the speed and characteristics of the target to be tracked, and predicts the time it will take for the target to reach the range of another rotating camera positioned in the direction of travel of the target to be tracked. The tracking and monitoring device controls the pan, tilt, zoom, or focus of the other rotating cameras so that the target to be tracked can be monitored at an optimal angle of view. Hereinafter, the first embodiment will be described in detail with reference to the drawings.

[0011] <Configuration> A tracking monitoring system according to a first embodiment of the present disclosure will be described with reference to Fig. 1. Fig. 1 is a configuration diagram of a tracking monitoring system including a tracking monitoring device according to the first embodiment. As shown in Fig. 1, the tracking monitoring system includes, as an example, a swiveling camera 101-1 (first camera), a swiveling camera 101-2 (second camera), an SW-HUB 102-1 connected to the swiveling camera 101-1, an SW-HUB 102-2 connected to the swiveling camera 101-2, a management PC 103, a tracking monitoring device 104, a decoder 105, and a video display device 106 connected to the decoder 105. The SW-HUB 102-1, the SW-HUB 102-2, the management PC 103, the tracking monitoring device 104, and the decoder 105 are connected via a backbone network NW. In the following description, when there is no need to distinguish between the swivel camera 101-1 and the swivel camera 101-2, the swivel camera 101-1 and the swivel camera 101-2 will be collectively referred to simply as the swivel camera 101. Similarly, when there is no need to distinguish between the SW-HUB 102-1 and the SW-HUB 102-2, the term SW-HUB 102 will be used.

[0012] The video captured by the swivel camera 101 is sent to the management PC 103 and the tracking and monitoring device 104 via the corresponding SW-HUB 102 and the backbone network.

[0013] In the management PC 103, a user of the management PC 103 manually controls the swivel camera 101-1 or 101-2 or checks the log.

[0014] In one aspect, tracking monitoring device 104 detects a target to be tracked, infers characteristics of the target to be tracked, and transmits a control signal to pivoting camera 101-1 based on the image from pivoting camera 101-1. In one aspect, tracking monitoring device 104 detects a target to be tracked, infers characteristics of the target to be tracked, and transmits a control signal to pivoting camera 101-2 based on the image from pivoting camera 101-1.

[0015] In one aspect, the tracking monitoring device 104 detects a target to be tracked, estimates the characteristics of the target to be tracked, and transmits a control signal to the rotating camera 101-2 based on the image from the rotating camera 101-2. In one aspect, tracking monitoring device 104 may detect a target to be tracked, estimate characteristics of the target to be tracked, and transmit a control signal to pivoting camera 101-1 based on the video from pivoting camera 101-2.

[0016] The video display device 106 displays the video from the swivel camera 101 through the decoder 105 .

[0017] The swivel camera 101 is installed in an environment where the subject's travel route is somewhat limited, such as a highway or a river, and the position information of each of the multiple swivel cameras 101 installed is held by a tracking monitoring device 104.

[0018] 2 is a block diagram of the swiveling camera 101, the tracking monitoring device 104, and the video display device 106 in the tracking monitoring system of embodiment 1. The tracking monitoring device 104 includes a video receiving unit 201-1, a video receiving unit 201-2, a target detection unit 202, a rear camera comparison unit 203, and a control unit 204.

[0019] (Video receiving section) Video receiving unit 201-1 receives video data captured by pivoting camera 101-1 and supplies the received video data to object detection unit 202. Video receiving unit 201-2 receives video data captured by pivoting camera 101-2 and supplies the received video data to object detection unit 202.

[0020] (Target detection section) The object detection unit 202 includes a detection unit 301, an extraction unit 302, and a speed measurement unit 303. The object detection unit 202 transmits detected information to the subsequent camera comparison unit 203 and the control unit 204.

[0021] (Detection unit) The detection unit 301 determines whether a tracking target has been detected within the monitoring range of the rotating camera 101, based on the data transmitted from the rotating camera 101 to the tracking monitoring device 104. The detection of the tracking target is performed based on changes in pixel values.

[0022] (Extraction part) The extraction unit 302 uses known image processing techniques to extract subject features such as color or license plate from the video data.

[0023] (Speed ​​measurement section) The speed measurement unit 303 measures the moving speed of the tracking target detected by the extraction unit 302 .

[0024] (Rear camera comparison section) The rear-stage camera comparison unit 203 includes a storage unit 305 and a comparison unit 304 .

[0025] (Storage part) The storage unit 305 stores information acquired by the swiveling camera 101-1.

[0026] (Comparison section) The comparison unit 304 compares the information stored in the storage unit 305 with the information acquired by the swiveling camera 101-2.

[0027] (Control unit) Based on the information obtained from the target detection unit 202 and the information obtained from the downstream camera comparison unit 203, the control unit 204 transmits a control signal to the rotating camera 101 to adjust the pan, tilt, zoom, or focus of the rotating camera 101.

[0028] Assume that pivoting camera 101-1 and pivoting camera 101-2 are installed on the same route, such as a highway or a river, and the tracking target passes through the capture range of pivoting camera 101-1 and then passes through the capture range of pivoting camera 101-2. In such a case, pivoting camera 101-1 adjusts the angle of view to match the movement of the tracking target in accordance with a control signal from control unit 204. In accordance with the control signal from control unit 204, pivoting camera 101-2 starts adjusting the angle of view before the tracking target enters the monitoring range, based on information stored in memory unit 305 from pivoting camera 101-1.

[0029] Next, an example of the hardware configuration of the tracking monitoring device 104 will be described with reference to Figures 3A and 3B. Each function of the tracking monitoring device 104 is realized by a processing circuitry. The processing circuitry may be a dedicated processing circuit 400a as shown in Figure 3A, or a processor 400b that executes a program stored in a memory 400c as shown in Figure 3B.

[0030] When the processing circuitry is a dedicated processing circuit 400a, the dedicated processing circuit 400a may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (application specific integrated circuit), an FPGA (field-programmable gate array), or a combination thereof. The functions of the tracking monitoring device 104 may be realized by a plurality of separate processing circuits, or the functions of the tracking monitoring device 104 may be realized together by a single processing circuit.

[0031] When the processing circuitry is a processor 400b serving as a computer, the functions of the tracking monitoring device 104 are realized by software, firmware, or a combination of software and firmware. The software and firmware are written as programs and stored in the memory 400c. The processor 400b realizes the functions of the tracking monitoring device 104 by reading and executing the programs stored in the memory 400c. Here, examples of the memory 400c include non-volatile or volatile semiconductor memories such as RAM (random access memory), ROM (read-only memory), flash memory, EPROM (erasable programmable read-only memory), and EEPROM (electrically erasable programmable read-only memory), as well as magnetic disks, flexible disks, optical disks, compact disks, minidisks, and DVDs.

[0032] It is also possible to realize some of the functions of the tracking monitoring device 104 using dedicated hardware, and other functions using software or firmware. In this way, the processing circuit can realize the functions of the tracking monitoring device 104 using hardware, software, firmware, or a combination of these.

[0033] <Operation> Next, the operation of the rotating camera 101 and the tracking monitoring device 104 according to embodiment 1 will be described with reference to Fig. 4 and Fig. 5. Fig. 4 is a schematic diagram of the operation when the tracking target is a wrong-way vehicle on a highway and the tracking target is detected by the rotating camera 101-1, and Fig. 5 is a flowchart showing the operation of the tracking monitoring device 104.

[0034] (Step ST101) In step ST101, the tracking monitoring device 104 receives video data from each of the swivel cameras 101. The function of the tracking monitoring device 104 is divided into two: a control function for the swivel camera 101-1 that detects the tracking target, and a control function for the swivel camera 101-2 that is installed in the direction of travel of the detected tracking target.

[0035] (Steps ST102 to ST106: Rotating camera 101-1) The control details regarding the rotating camera 101-1 will be described. In step ST102, the detection unit 301 determines whether a tracking target has been detected within the monitoring range from the data that the rotating camera 101-1 is transmitting to the tracking monitoring device 104. In step ST103, the extraction unit 302 extracts the characteristics of the tracking target from the data that the rotating camera 101-1 is transmitting to the tracking monitoring device 104, and the speed measurement unit 303 extracts the speed information (v [m / s]) of the tracking target.

[0036] In step ST104, based on the extracted information, control unit 204 transmits tracking control information to rotating camera 101-1. Specifically, control unit 204 transmits a field angle control signal to rotating camera 101-1. As a result, rotating camera 101-1 automatically performs pan, tilt, zoom, or focus operations to track the tracking target so that it is always within the field angle.

[0037] In step ST105, control unit 204 determines whether the tracking target has gone outside the monitoring range of rotating camera 101-1. If the tracking target has not gone outside the monitoring range, the process returns to step ST104.

[0038] If the tracking target goes out of the monitoring range, in step ST106, the control unit 204 transmits a normal monitoring control signal to the swiveling camera 101-1 so as to return to normal monitoring control.

[0039] (Steps ST107 to ST114: Rotating camera 101-2) Next, the control details regarding the rotating camera 101-2 will be described. In step ST107, it is determined whether or not tracking target information has been extracted by the swiveling camera 101-1.

[0040] If extracted, in step ST108, comparison unit 304 calculates the time (t [s]) for the tracking target to arrive at the monitoring range of rotating camera 101-2 as a predicted arrival time, based on the speed information of the tracking target extracted by speed measurement unit 303 and the distance (L [km]) from the monitoring range of rotating camera 101-1 to the monitoring range of rotating camera 101-2. The distance (L [km]) between the monitoring ranges of rotating cameras 101-1 and 101-2 is stored in storage unit 305.

[0041] In step ST109, control unit 204 transmits standby control information to rotating camera 101-2. Specifically, control unit 204 transmits the predicted arrival time to rotating camera 101-2. Control unit 204 may transmit the predicted arrival time to video display device 106, which may then display the predicted arrival time. Also, in step ST109, before the tracking target enters the monitoring range of rotating camera 101-2, rotating camera 101-2 adjusts the pan, tilt, zoom, or focus from the predicted arrival time to an angle of view optimal for monitoring the tracking target.

[0042] In step ST110, the comparison unit 304 determines whether the tracking target has been detected based on the features and speed information extracted from the image of the rotating camera 101-2 and the features and speed information extracted from the image of the rotating camera 101-1 stored in the memory unit 305.

[0043] In step ST112, based on the result of the determination in step ST110, control unit 204 transmits tracking control information to rotating camera 101-2. Specifically, control unit 204 transmits a field of view control signal to rotating camera 101-2. Based on the transmitted field of view control signal, rotating camera 101-2 automatically performs pan, tilt, zoom, or focus operations to track the tracking target so that it is always within the field of view.

[0044] In step ST113, control unit 204 determines whether the tracking target has gone outside the monitoring range of pivoting camera 101-2. If the tracking target has not gone outside the monitoring range, the process returns to step ST112.

[0045] If the tracking target goes out of the monitoring range, in step ST114, the control unit 204 transmits a normal monitoring control signal to the swiveling camera 101-2 so as to return to normal monitoring control.

[0046] <Effects> As explained above, in a tracking monitoring system equipped with multiple rotating cameras for monitoring a wide area, the time during which monitoring is unavailable can be shortened by operating each rotating camera in conjunction with the other cameras when a tracking target is detected. The multiple rotating cameras may be installed at intervals that are greater than or equal to a distance such that the monitoring ranges of the respective rotating cameras do not overlap.

[0047] In addition, by adjusting the angle of view of the rear-stage rotating camera in advance, it is possible to extract many features from the target to be tracked, which is expected to improve the detection accuracy of the target to be tracked.

[0048] <Modification> Although the above description has been given in connection with the case where a vehicle traveling in the wrong direction is detected on an expressway, the tracking and monitoring system according to the first embodiment can also be applied to a system for detecting drifting objects in a river.

[0049] Furthermore, fixed cameras may be used instead of rotating cameras as cameras that make up the tracking monitoring system. A system in which multiple fixed cameras are connected can achieve the same functions as the tracking monitoring system described above.

[0050] Furthermore, extraction of the characteristics or speed of the target to be tracked or instructions to rotate the swivel camera can be performed by the swivel camera itself, rather than by the tracking monitoring device.

[0051] In addition, an alarm may be set to be issued early if the tracking target does not arrive within the monitoring range of the rear-stage rotating camera at the predicted arrival time. For example, if the tracking target does not arrive within the predicted arrival time, an alarm may be set to be issued because it is expected that an accident will occur on a highway or that the target will get caught on drifting objects in a river.

[0052] Embodiment 2 In the first embodiment, a tracking monitoring system that uses two swivel cameras to monitor a tracking target is described, but in the second embodiment, a tracking monitoring system that includes three or more swivel cameras will be described.

[0053] 6 is a configuration diagram of a tracking monitoring system using N swivel cameras 101-1 to 101-N, where N is an arbitrary integer equal to or greater than 3. Since the tracking monitoring system is equipped with N swivel cameras 101-1 to 101-N, tracking monitoring device 104 is equipped with N video receiving units 201-1 to 201-N.

[0054] Next, the operation of the swiveling camera 101 and tracking monitoring device 104 according to the second embodiment will be described with reference to Fig. 7. Fig. 7 is a schematic diagram of the operation when N=4.

[0055] The swiveling camera 101-1 is controlled in the same manner as in the first embodiment.

[0056] Next, an explanation will be given of rotating cameras 101-2 and 101-3. As for rotating camera 101-2, as explained with reference to Fig. 4, tracking monitoring device 104 calculates the time (t [s]) for the tracking target to arrive at the monitoring range of rotating camera 101-2 as the predicted arrival time, based on the speed information received from rotating camera 101-1 and the distance (L [km]) from the monitoring range of rotating camera 101-1 to the monitoring range of rotating camera 101-2.

[0057] For the rotating camera 101-3, the tracking monitoring device 104 calculates the time (t3 [s]) for the tracking target to arrive at the monitoring range of the rotating camera 101-3 as the predicted arrival time based on the speed information received from the rotating camera 101-1 and the distance (L3 [km]) from the monitoring range of the rotating camera 101-1 to the monitoring range of the rotating camera 101-3.

[0058] The distance (L [km]) between the monitoring ranges of the pivoting cameras 101-1 and 101-2 and the distance (L3 [km]) between the monitoring ranges of the pivoting cameras 101-1 and 101-3 are both stored in the storage unit 305.

[0059] Control unit 204 transmits standby control information to rotating cameras 101-2 and 101-3. Specifically, control unit 204 transmits predicted arrival time t [s] to rotating camera 101-2 and transmits predicted arrival time t3 [s] to rotating camera 101-3.

[0060] Control unit 204 may transmit these predicted arrival times to video display device 106, which may then display the predicted arrival times. As a display related to rotating camera 101-3, instead of predicted arrival time t3 [s] for rotating camera 101-3, the time obtained by subtracting predicted arrival time t [s] from predicted arrival time t3 [s] may be displayed.

[0061] Thereafter, as explained in accordance with the first embodiment, control is performed to adjust the angle of view of the swivel camera 101-2 to the optimum angle of view for the tracking target. However, there may be cases where the swivel camera 101-2 is unable to detect the tracking target due to some reason, such as bad weather or the presence of an obstacle. Therefore, after t3 seconds, the tracking monitoring device 104 transmits a control signal to the swivel camera 101-3 to adjust the angle of view of the swivel camera 101-3 to the optimum angle of view. If another swivel camera 101 is arranged downstream of the swivel camera 101-3, similar control may be performed on the yet another swivel camera 101 arranged downstream.

[0062] Furthermore, if there is a branch in the travel route of the tracking target, the rotating camera 101-4 at the branch point can also be controlled in the same way.

[0063] As explained above, by linking three or more rotating cameras to monitor the target, even if the second rotating camera misses the target due to bad weather or the presence of an obstacle, it is possible to continue tracking the target using the third rotating camera.

[0064] It is possible to combine the embodiments, and to modify or omit each embodiment as appropriate. [Industrial Applicability]

[0065] The tracking and monitoring device of the present disclosure can be used as a tracking and monitoring system in which the monitoring cameras are arranged far apart so that their imaging ranges do not overlap. [Explanation of symbols]

[0066] 101 (101-1, 101-2, 101-3, 101-4, ..., 101-N) rotating camera, 103 management PC, 104 tracking monitoring device, 105 decoder, 106 video display device, 201-1 video receiving unit, 201-2 video receiving unit, 202 object detection unit, 203 rear camera comparison unit, 204 control unit, 301 detection unit, 302 extraction unit, 303 speed measurement unit, 304 comparison unit, 305 memory unit, 400a processing circuit, 400b processor, 400c memory, NW backbone network.

Claims

1. a video receiving unit that receives video data captured by the first camera; an object detection unit that detects an object from the received video data and measures the moving speed of the detected object; a rear camera comparison unit that calculates a predicted arrival time for the object to arrive within the photographing range of the second camera based on the measured moving speed; a control unit that transmits the calculated predicted arrival time to the second camera; A tracking and monitoring device comprising:

2. the rear stage camera comparison unit calculates the predicted arrival time from the distance from the imaging range of the first camera to the imaging range of the second camera and the measured moving speed; 2. A tracking monitoring device according to claim 1.

3. the control unit transmits a control signal to the second camera for adjusting pan, tilt, zoom, or focus of the second camera; 2. A tracking monitoring device according to claim 1.

4. A tracking monitoring device according to any one of claims 1 to 3; the first camera; the second camera; A tracking and monitoring system comprising:

5. A tracking monitoring method performed by a tracking monitoring device including a video receiving unit, a target detecting unit, a rear camera comparing unit, and a control unit, a step in which the video receiving unit receives video data captured by a first camera; a step in which the object detection unit detects an object from the received video data and measures a moving speed of the detected object; a step in which the rear-stage camera comparison unit calculates a predicted arrival time for the object to arrive within a shooting range of a second camera based on the measured moving speed; a step of the control unit transmitting the calculated predicted arrival time to the second camera; A tracking and monitoring method comprising:

6. a function of receiving video data captured by the first camera; A function to detect an object from the received video data and measure the moving speed of the detected object; a function of calculating a predicted arrival time for the object to arrive within the photographing range of the second camera based on the measured moving speed; a function of transmitting the calculated predicted arrival time to the second camera; A tracking and monitoring program that causes a computer to execute the following.

Images