Monitoring system

Abstract

To make it possible to stably perform the takeover of tracking of a moving object during relay control to switch the video of a camera used for tracking the moving object.SOLUTION: A monitoring system according to one embodiment of the present invention includes a first camera 210A, a second camera 210B installed at a location separate from the first camera 210A, and a server 220 communicatively connected to the first camera 210A and the second camera 210B. In response to the fact that it is determined that a moving object 200 being tracked based on a video from the first camera 210A has reached a relay point, the server 220 performs relay control to adjust the second camera 210B to a zoom magnification corresponding to the type of moving object 200, and to continue tracking the moving object 200 based on a video from the second camera 210B after adjustment.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to a monitoring system that tracks a moving object in a monitoring area based on an image of a camera that photographs the monitoring area.

Background Art

[0002] Conventionally, a monitoring system has been in operation that images a monitoring area with a security monitoring camera or the like and detects that a person, vehicle, or the like has entered the monitoring area. There is also a monitoring system that not only detects an intruder but also controls the pan-tilt head and lens of the monitoring camera to track the movement of the intruder.

[0003] FIG. 1 shows an overview of a monitoring system that detects and tracks an intruder based on a camera image. As shown in the upper part of FIG. 1, the monitoring system in FIG. 1 uses a monitoring camera 110 mounted on a pan-tilt head to detect and track an intruder 100 who has entered the monitoring area. As shown in the lower part of the figure, images 121, 122, and 123 taken by the monitoring camera 110 are displayed on a monitor in the monitoring system. Image 121 is an image at the time when the intruder 100 enters the monitoring area. Image 122 is an image at the time when the intruder 100 is detected, and a broken-line rectangular frame 131 is superimposed so as to surround the intruder 100. Image 123 is an image during the tracking of the detected intruder 100, and a broken-line arrow 132 indicating the movement path of the intruder 100 is further superimposed.

[0004] Examples of the prior art in the technical field related to the present invention are as follows. For example, Patent Document 1 discloses a technique for zooming a imaging lens by a required amount so as to obtain an image of an intruding object having a required size according to the size of the intruding object. Patent Document 2 also discloses a technique for transitioning a first television camera from a normal mode to a tracking mode and transitioning a second camera to a wide-angle mode when an object is detected in an image obtained by the first television camera in the normal mode.

Prior Art Documents

Patent Documents

[0005] [Patent Document 1] Japanese Unexamined Patent Application Publication No. 11-69342 [Patent Document 2] Japanese Patent No. 4499514 [Summary of the Invention] [Problems to be Solved by the Invention]

[0006] In a surveillance system as shown in FIG. 1, during the pursuit of an intruder 100 by a surveillance camera 110, there is a limit point where the field of view of the surveillance camera 110 is blocked by obstacles such as vegetation, structures, and terrain, making it impossible to pursue. As a countermeasure, a relay control function has been developed in which multiple surveillance cameras are installed at different locations, and when the surveillance camera in pursuit is controlled to near the limit point, the pursuit is transferred to another surveillance camera that captures the limit point or its vicinity.

[0007] FIG. 2 shows an overview of a surveillance system having a relay control function. The surveillance system of FIG. 2 includes two surveillance cameras, a first camera 110A and a second camera 110B, and a server 150 communicably connected to these surveillance cameras via a network. The first camera 110A is a surveillance camera that images the surveillance area from the first direction, and there is a limit point where it becomes impossible to pursue due to an obstacle 140. The second camera 110B is a surveillance camera that images the surveillance area from a second direction different from the first direction, and is arranged at a position different from the first camera 110A so that it can capture the limit point or its vicinity of the imaging by the first camera 110A.

[0008] The relay control function of the surveillance system in FIG. 2 is realized by the following procedure. Hereinafter, the "relay point" is a point where relay (handover) of the surveillance camera is required, that is, the above-mentioned limit point or the point immediately before it (the point where pursuit will soon become impossible or difficult). (1) The first camera 110A detects the intruder 100 and starts pursuing the intruder 100. (2) The intruder 100 moves to the relay point. (3) The server 150 detects the movement of the intruder 100 to the relay point. (4) Under the control of the server 150, after directing the second camera 110B to the relay point, the second camera 110B takes over the pursuit of the intruder 100.

[0009] Conventionally, the detection and pursuit of a moving object were carried out by comparing camera images frame by frame, calculating the changed part as a vector, and controlling the pan-tilt unit of the surveillance camera to point in that direction. Even after the relay control was executed, the pursuit of the moving object was carried out in the same manner. However, since the above method is based on the determination by image comparison, in addition to moving objects such as people and vehicles, the movement of plants and trees or the reflection of water within the field of view may be misdetected as a moving object.

[0010] Therefore, in recent years, a function for detecting moving objects such as people and vehicles using AI (Artificial Intelligence) - based image recognition has been implemented. In AI image recognition, by detecting a moving object based on pre - registered images of people, vehicles, etc., misdetection of noise such as plants and water can be suppressed, and the detection accuracy can be improved.

[0011] However, in the case of AI image recognition, since the detection of the moving object is performed based on pre-registered images, when the monitoring camera (the second camera 110B in the above description) after switching by relay control points to the relay point, depending on the size of the moving object to be detected on the video, it may not be detected and lost from sight, resulting in a failure to continue the tracking. For example, assume that the second camera 110B has a zoom ratio suitable for detecting a person, as shown in the video 161 of FIG. 3A. In this situation, if the moving object detected by the first camera 110A is a vehicle and the vehicle reaches the relay point and the second camera 110B takes over the tracking, as shown in the video 162 of FIG. 3A, the vehicle in the video of the second camera 110B may be too large to be detected. Also, for example, assume that the second camera 110B has a zoom ratio suitable for detecting a vehicle, as shown in the video 163 of FIG. 3B. In this situation, if the moving object detected by the first camera 110A is a person and the person reaches the relay point and the second camera 110B takes over the tracking, as shown in the video 164 of FIG. 3B, the person in the video of the second camera 110B may be too small to be detected.

[0012] The present invention has been made in view of the above-described conventional circumstances, and an object thereof is to stably realize the takeover of tracking of a moving object during relay control for switching the video of a camera used for tracking the moving object.

Means for Solving the Problems

[0013] In order to achieve the above object, a monitoring system according to an aspect of the present invention has the following technical features. That is, in a monitoring system that tracks a moving object within a monitoring area based on the video of a camera that shoots the monitoring area, a first camera, a second camera installed at a position different from the first camera, and in response to determining that a moving object being tracked based on the video of the first camera has reached a relay point, a server that performs relay control to adjust the second camera to a zoom ratio corresponding to the type or size of the moving object and continue tracking the moving object based on the video of the adjusted second camera.

[0014] Here, in the above monitoring system, the server stores the zoom ratio to be applied during relay control for each type or size of the moving object to be detected. In response to determining that the moving object being tracked has reached the relay point, the server reads out the zoom ratio corresponding to the type or size of the moving object and transmits it to the second camera, and may operate to adjust the zoom ratio of the second camera.

[0015] Also, in the above monitoring system, the server tracks a plurality of moving objects with different types or sizes based on the video of the first camera. When it is determined that two or more of these moving objects have reached the relay point simultaneously or continuously, one moving object is selected according to the preset priority order for the moving objects to be detected, and the server may operate to adjust the second camera to the zoom ratio corresponding to the type or size of the moving object.

[0016] Also, in the above monitoring system, the server may correct the zoom ratio corresponding to the type or size of the moving object being tracked according to the moving speed of the moving object, and operate to adjust the second camera to the corrected zoom ratio.

[0017] Also, in the above monitoring system, after the server adjusts the second camera to the zoom ratio corresponding to the type or size of the moving object being tracked, the server may further operate to perform zoom-in or zoom-out according to the moving direction of the moving object based on the video of the second camera until the moving object is detected.

[0018] Also, in the above monitoring system, when the moving object being tracked is moving toward the relay point side in the video of the first camera, the server may operate to determine that the moving object has reached the relay point in response to losing sight of the moving object.

[0019] Also, in the above monitoring system, when the moving object being tracked moves toward the relay point in the video of the first camera, the server may be configured to determine that the moving object has reached the relay point in response to the size of the moving object in the video becoming equal to or less than a predetermined value.

Advantages of the Invention

[0020] According to the present invention, when performing relay control for switching the video of the camera used for tracking a moving object, it becomes possible to stably realize the handover of tracking of the moving object.

Brief Description of the Drawings

[0021]

Figure 1

Figure 2

Figure 3A

Figure 3B

Figure 4

Figure 5

Figure 6

Embodiments for Carrying Out the Invention

[0022] An embodiment of the present invention will be described with reference to the drawings. Fig. 4 shows an overview of a surveillance system according to one embodiment of the present invention. The surveillance system shown in Fig. 4 includes two surveillance cameras, a first camera 210A and a second camera 210B, and a server 220 communicatively connected to these surveillance cameras via a network. While Fig. 4 shows two surveillance cameras, this is merely an example, and a greater number of surveillance cameras may be included.

[0023] First camera 210A is a surveillance camera that captures an image of the surveillance area from a first direction, and there is a limit at which tracking becomes impossible due to an obstruction. Second camera 210B is a surveillance camera that captures an image of the surveillance area from a second direction different from the first direction, and is located in a different position from first camera 210A. The following describes an example in which, when moving object 200 being tracked using the image from first camera 210A reaches a relay point, the image is switched to that from second camera 210B and tracking continues, but the roles of first camera 210A and second camera 210B may be reversed.

[0024] Here, a "relay point" refers to the point at which a surveillance camera must take over (relay), i.e., the limit point mentioned above or the point immediately preceding it (the point at which tracking will soon become impossible or difficult). Relay points are automatically determined based on conditions such as the installation location of the surveillance camera and the surrounding environment, so they can be set in advance before the system begins operation. Alternatively, information such as the installation locations of multiple surveillance cameras can be registered in advance, and the relay point can be automatically calculated and set so that the next surveillance camera can begin filming the same moving object at the point before the tracking surveillance camera loses sight of the moving object.

[0025] The surveillance cameras (210A, 210B) of this system are mounted on a pan-tilt unit. By changing the pan angle and tilt angle through the control of the pan-tilt unit, the viewing angle can be moved to track the moving object. Also, the surveillance cameras of this system are equipped with zoom lenses. By changing the zoom ratio through the control of the zoom lenses, the viewing angle can be adjusted to a size suitable for detecting the moving object 200 being tracked. The surveillance cameras of this system can further detect moving objects such as people and vehicles in the surveillance area through AI image recognition and track their movements. Note that a device different from the surveillance camera (for example, the server 220) may receive the video from the surveillance camera and perform the detection and tracking of the moving object through the AI image recognition of the video.

[0026] The server 220 can be realized by a computer equipped with hardware resources such as a processor and a memory, for example, and can be configured to read out a program related to each function according to the present invention from the memory and execute it by the processor. The server 220 may be realized by one computer or may be realized by a plurality of computers connected to be communicable with each other.

[0027] When the server 220 is tracking the moving object 200 based on the video of the first camera 210A, in response to the determination that the moving object 200 being tracked has reached the relay point, the server 220 performs a relay control process of switching the video used for tracking the moving object 200 from the video of the first camera 210A to the video of the second camera 210B. During this relay control process, the server 220 adjusts the second camera 210B to the zoom ratio corresponding to the type of the moving object 200 being tracked, and continues to track the moving object 200 based on the adjusted video of the second camera 210B.

[0028] FIG. 5 shows an example of a zoom setting table used for zoom adjustment during relay control processing. The zoom setting table may be stored in the memory of the server 220 or may be stored in an external device accessible by the server 220. The illustrated zoom setting table sets the zoom magnification to be applied at the time of video switching for each type of moving object to be detected. In the same figure, when the type of the moving object is "large vehicle", it is set to "ultra-wide angle", when it is "ordinary vehicle", it is set to "wide angle", when it is "person", it is set to "small zoom amount", and when it is "small animal", it is set to "large zoom amount". Note that in FIG. 5, from the viewpoint of easy understanding, the zoom magnification is expressed in characters such as "ultra-wide angle", "wide angle", "small zoom amount", and "large zoom amount", but it may be a setting of a numerical value indicating a specific zoom magnification.

[0029] Details of the relay control processing in the monitoring system of this example will be described with reference to FIG. 6. FIG. 6 shows an example of a processing sequence according to the monitoring system of this example. The first camera 210A detects a moving object 200 such as a person or a vehicle in the monitoring area by AI image analysis and starts tracking it (step S101). At that time, the first camera 210A also identifies the type of the moving object 200 by AI image analysis. Then, when the moving object 200 moves and reaches the relay point (step S102), the movement of the moving object 200 to the relay point is detected by the first camera 210A, and a relay point arrival signal is transmitted from the first camera 210A to the server 220 (step S103). At this time, the first camera 210A also transmits moving object additional information indicating the type of the moving object 200 along with the relay point arrival signal.

[0030] Upon receiving the relay point arrival signal and the moving object addition information from the first camera 210A, the server 220 executes relay control processing. In the relay control processing, the server 220 first refers to the zoom setting table using the moving object addition information received from the first camera 210A as a key, and determines the zoom magnification to be applied to the second camera 210B (step S103). Thereafter, the server 220 transmits the zoom setting information indicating the determined zoom magnification to the second camera 210B together with the relay point directed instruction.

[0031] When the second camera 210B receives the relay point directed instruction and the zoom setting information from the server 220, it changes the pan angle and tilt angle so that the second camera 210B points to the relay point (step S104). Next, the second camera 210B changes the zoom magnification so that the angle of view has a size suitable for detecting the moving object 200 according to the zoom setting information (step S105). Thereafter, the second camera 210B detects the moving object 200 existing near the relay point and starts (takes over) tracking it (step S106).

[0032] As described above, the monitoring system of this example includes the first camera 210A, the second camera 210B installed at a position different from that of the first camera 210A, and the server 220 communicably connected to the first camera 210A and the second camera 210B. When the server 220 determines that the moving object 200 being tracked has reached the relay point based on the video of the first camera 210A, the server 220 adjusts the second camera 210B to the zoom magnification corresponding to the type of the moving object 200, and performs relay control to continue tracking the moving object 200 based on the video of the adjusted second camera 210B.

[0033] Therefore, according to the monitoring system of this example, when performing relay control to switch the video of the camera used for tracking the moving object 200 from the video of the first camera 210A to the video of the second camera 210B, the moving object 200 that has become untrackable or difficult to track in the video of the first camera 210A can be quickly detected from the video of the second camera 210B, so that it is possible to stably realize the takeover of tracking of the moving object 200.

[0034] Note that the above description determines the zoom ratio based on the type of the moving object during tracking. However, the zoom ratio may be determined based on the size of the moving object instead of its type. In this case, the size of the moving object may be the length in the vertical direction of the detection frame (e.g., the dashed rectangular frame 131 in FIG. 1) surrounding the moving object in the detection result by AI image recognition, or the length in the horizontal direction of the detection frame, or the area of the detection frame, or other sizes calculable by another method.

[0035] Also, when there are a plurality of moving objects with different types and sizes in the monitoring area, and two or more of these moving objects reach the relay point simultaneously or continuously, it is not possible to uniquely determine the zoom ratio corresponding to the type or size of the moving object. As a countermeasure, a priority order regarding the moving objects is set in advance, and the server 220 selects one moving object from among the two or more moving objects that have reached the relay point according to the preset priority order, and adjusts the second camera 210B to the zoom ratio corresponding to the type or size of that moving object. The method of setting the priority order is arbitrary. For example, it may be set such that a moving object with a higher risk, such as a large vehicle, has a higher priority. Thereby, it becomes possible to deal more appropriately with the situation where two or more moving objects reach the relay point simultaneously or continuously.

[0036] Also, even if the second camera 210B is adjusted to the zoom ratio corresponding to the type or size of the moving object, detection by the second camera 210B may become difficult depending on the moving speed of the moving object. Therefore, the server 220 acquires the moving speed of the moving object before it reaches the relay point, corrects the zoom ratio corresponding to the type or size of the moving object according to the moving speed, and adjusts the second camera 210B to the corrected zoom ratio. As the correction of the zoom ratio, for example, correction may be performed such that the wider the moving speed of the moving object, the wider the angle of view. Thereby, it becomes possible to more reliably detect the moving object after performing the relay control.

[0037] Even if the zoom magnification of the second camera 210B is adjusted according to the type or size of the moving object, it may be difficult to detect the moving object with the second camera 210B depending on the moving direction of the moving object. Therefore, the server 220 acquires the moving direction of the moving object before it reaches the relay point, adjusts the zoom magnification of the second camera 210B according to the type and size of the moving object, and then further performs zoom-in or zoom-out according to the moving direction of the moving object based on the second camera 210B until the moving object is detected. For example, when the moving object in the video of the second camera 210B is moving in a direction where its size becomes larger, it may be controlled in the zoom-out direction, and when the moving object in the video of the second camera 210B is moving in a direction where its size becomes smaller, it may be controlled in the zoom-in direction. As a result, the detection of the moving object after relay control can be more reliably performed.

[0038] Also, it is sufficient to correctly detect that the moving object being tracked has reached the relay point based on the video of the first camera 210A, but there may be cases where this is difficult. Therefore, as an example, when the moving object being tracked is moving toward the relay point side in the video of the first camera 210A, the server 220 may determine that the moving object has reached the relay point in response to losing sight of the moving object. As another example, when the moving object being tracked is moving toward the relay point side in the video of the first camera 210A, the server 220 may determine that the moving object has reached the relay point in response to the size of the moving object on the video becoming equal to or less than a predetermined value. As a result, it is possible to suppress the omission of relay control in a situation where relay control should be performed.

[0039] The embodiments of the present invention have been described above. However, these embodiments are merely examples and do not limit the technical scope of the present invention. The present invention can take various other embodiments and can undergo various modifications such as omission and substitution without departing from the gist of the present invention. These embodiments and their modifications are included in the scope and gist of the invention described in this specification and the like, and are also included in the invention described in the claims and its equivalent scope.

[0040] In addition, the present invention can be provided not only as the devices as described in the above description and the systems composed of these devices, but also as the methods executed by these devices, the programs for realizing the functions of these devices by a processor, the storage media for storing such programs in a computer-readable manner, and the like.

Industrial Applicability

[0041] The present invention can be used in a monitoring system that tracks a moving object in a monitoring area based on the video of a camera that captures the monitoring area.

Explanation of Reference Numerals

[0042] 100: Intruder, 110: Monitoring camera, 110A: Monitoring camera (first camera), 110B: Monitoring camera (second camera), 140: Shield, 150: Server, 200: Moving object, 210A: Monitoring camera (first camera), 210B: Monitoring camera (second camera), 220: Server

Claims

1. In a monitoring system that tracks a moving object within a monitoring area based on an image from a camera that captures the monitoring area, a first camera, a second camera installed at a position different from that of the first camera, and a server that performs relay control to adjust the second camera to a zoom magnification corresponding to the type or size of the moving object when it is determined that the moving object being tracked has reached a relay point based on the image from the first camera, and continues to track the moving object based on the image from the second camera after the adjustment. A monitoring system characterized by comprising.

2. In the monitoring system according to claim 1, the server stores a zoom magnification to be applied during relay control for each type or size of the moving object to be detected, and when it is determined that the moving object being tracked has reached the relay point, reads out the zoom magnification corresponding to the type or size of the moving object and transmits it to the second camera to adjust the zoom magnification of the second camera. A monitoring system characterized by.

3. In the monitoring system according to claim 1, the server is tracking a plurality of moving objects having different types or sizes based on the image from the first camera, and when it is determined that two or more of them have reached the relay point simultaneously or continuously, one moving object is selected according to a preset priority order for the moving objects to be detected, and the second camera is adjusted to the zoom magnification corresponding to the type or size of the moving object. A monitoring system characterized by.

4. In the monitoring system according to claim 1, the server corrects the zoom magnification corresponding to the type or size of the moving object being tracked according to the moving speed of the moving object, and adjusts the second camera to the corrected zoom magnification. A monitoring system characterized by.

5. In the monitoring system according to claim 1, after the server adjusts the second camera to the zoom magnification corresponding to the type or size of the moving object being tracked, further performs zoom-in or zoom-out according to the moving direction of the moving object based on the image from the second camera until the moving object is detected. A monitoring system characterized by.

6. In the monitoring system according to claim 1, The monitoring system is characterized in that when the moving object being tracked is moving toward the relay point in the video of the first camera, the server determines that the moving object has reached the relay point in response to losing sight of the moving object.

7. In the monitoring system according to claim 1, the monitoring system is characterized in that when the moving object being tracked is moving toward the relay point in the video of the first camera, the server determines that the moving object has reached the relay point in response to the size of the moving object on the video becoming equal to or less than a predetermined value.

Images