Surveillance system

Abstract

To provide a surveillance system capable of more clearly grasping a shape of an object detected by a radar device.SOLUTION: A surveillance system includes: a radar device 200 installed toward a predetermined detection range R; a plurality of cameras 220 that image the detection range R from each different position; and a surveillance control device 240 that, when an object T present in the detection range R is detected by the radar device 200, selects two or more cameras 220 at a position close to the object T based on position information of the object T and position information of each of the plurality of cameras 220 and performs control such that the object T is imaged by the two or more cameras 220.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a surveillance system that uses a radar device to detect an object present in a surveillance area. [Background technology]

[0002] Conventionally, radar devices have been used to detect foreign objects on airport runways and quickly remove them, detecting objects within a specified range based on the reception of reflected waves in response to transmitted waves. Radar devices using microwaves or millimeter waves include, for example, FMCW (Frequency Modulated Continuous-Wave) radar devices with a structure as shown in Figure 1.

[0003] 1 amplifies a frequency-modulated radar signal from an FMCW transmission source 101 using a transmission power amplifier 103 and transmits it from a transmission antenna 104. If an object T (a reflecting object) is present within the detection range of the radar device 100, the transmitted wave from the radar device is reflected by the object T. The reflected wave from the object T is received by a receiving antenna 105 of the radar device 100, amplified by a receiving power amplifier 106, and then mixed with a transmitted radar signal component from a power divider 102 by a mixer 107 and converted into an IF signal. The IF signal output from the mixer 107 is A / D converted and processed by a signal processing unit 108. As a result, radar detection results such as the reflected received power from the object T (reflected wave power), the distance to the object T, and the speed of the object T if it is moving (its speed relative to the radar device 100) can be obtained.

[0004] Prior art in the technical field of the present invention includes the following: For example, Patent Document 1 discloses an invention in which a millimeter wave radar is installed on a moving body, and the distance to a target position is measured based on the distance between a first reflector and a second reflector installed near the target position, and the reception results of the reflected waves from these reflectors. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] International Publication No. 2017 / 018021 Summary of the Invention [Problem to be solved by the invention]

[0006] One use of radar devices is to detect objects on roads, runways, and other surfaces. Roads and runways usually do not have any reflective objects, such as fallen or abandoned objects. Therefore, radar devices continuously send radar transmission waves to a monitoring area where there are no reflective objects, and only when a reflective object appears in the monitoring area do they receive received waves (reflected waves), detecting that object.

[0007] Images captured by a camera are effective for confirming the shape of an object detected by a radar device. That is, by identifying the location of the object using a radar device, capturing an image of that location with a camera and displaying it on a display device, a surveillance officer can grasp the shape of the object. In order to grasp the shape of an object more accurately, it is necessary to improve the resolution of the image captured by the camera. However, the resolution of the image captured by the camera varies depending on various shooting conditions, such as the distance between the camera and the object, the weather (rain, snow, fog, etc.), and the time of day or night.

[0008] Fig. 2 shows an overview of a monitoring system that links a radar device and a camera. The monitoring system in Fig. 2 comprises a radar device 100 and a camera 120 that are installed facing a predetermined detection range R, and a monitoring control device 140 and a display device 160 that are installed in a control room or a monitoring room. In the example in Fig. 2, the radar device 100 and the camera 120 are installed in the same location.

[0009] The radar device 100 receives reflected waves of radar transmission waves transmitted within the detection range R, and outputs radar detection results obtained by signal processing to the monitoring control device 140. The radar detection results output from the radar device 100 include information such as the distance from the radar device 100 to the object T and the angle (azimuth) of the object T relative to the radar device 100, and are provided to the camera 120 via the monitoring control device 140. The camera 120 adjusts the angle of view based on the information on the distance and angle from the radar device 100 to the object T, and captures an image of the object T. The image of the object T captured by the camera 120 is displayed on the display device 160. A monitor can confirm the shape of the object T detected by the radar device 100 by looking at the image displayed on the display device 160.

[0010] However, if the distance between object T and camera 120 is great, object T may not fit in a sufficient size within the captured image (i.e., the size of object T in the captured image is small), and the shape of object T may not be clearly visible when viewing the captured image on display device 160. Furthermore, the shape of object T may not be clearly visible if there is rain, snow, fog, or the like between object T and camera 120 due to weather conditions, or if there is insufficient light, such as at night.

[0011] 2, another problem is that the monitor can only see an image of the object T viewed from one direction. In other words, the monitor can only recognize the shape of the object T viewed from the same direction as the radar device 100 and the camera 120, which may cause a discrepancy between the shape of the object T seen at the site where the monitor went to retrieve or remove it and the actual shape, which may lead to confusion.

[0012] The present invention has been made in consideration of the above-described conventional circumstances, and has an object to provide a surveillance system that makes it possible to more clearly grasp the shape of an object detected by a radar device. [Means for solving the problem]

[0013] In order to achieve the above object, a monitoring system according to one aspect of the present invention has the following technical features: That is, the monitoring system according to one aspect of the present invention includes a radar device installed facing a monitoring area, a plurality of cameras that capture images of the monitoring area from different positions, and a control device that, when an object present in the monitoring area is detected by the radar device, selects two or more cameras that are positioned close to the object based on position information of the object and position information of each of the plurality of cameras, and controls the two or more cameras to capture images of the object.

[0014] Here, the above-mentioned surveillance system further comprises a display device that displays two or more images taken by the two or more cameras, and the display device is operable to display, together with the two or more images, information indicating the direction in which each of the two or more images was taken.

[0015] In the above surveillance system, the display device can operate to display the two or more images in a manner that the size of the object in each image is uniform.

[0016] In addition, in the above-mentioned surveillance system, the monitored area may include a road surface on which a moving object travels, and the multiple cameras may be low-profile cameras that can be embedded in the road surface and installed inside the monitored area.

[0017] In addition, in the above-mentioned surveillance system, the multiple cameras are installed facing the direction in which a moving object enters the monitored area, and can operate to photograph the moving object that enters the monitored area until the camera is selected by the control device to photograph the object.

[0018] In addition, in the above surveillance system, the display device can be operated to display, together with the two or more images, an image of the moving body entering the monitored area immediately before the object is detected. [Effects of the Invention]

[0019] According to the present invention, it is possible to provide a surveillance system that makes it possible to more clearly grasp the shape of an object detected by a radar device. [Brief explanation of the drawings]

[0020] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a radar device. [Figure 2] FIG. 1 is a diagram showing an overview of a monitoring system in which a radar device and a camera are linked together. [Figure 3] 1 is a diagram showing an overview of a monitoring system according to a first embodiment of the present invention. [Figure 4] FIG. 4 is a diagram showing an example of the arrangement of radar devices and cameras in the monitoring system of FIG. 3. [Figure 5] 4 is a diagram showing an example of selecting a camera to be operated in the monitoring system of FIG. 3. FIG. [Figure 6] FIG. 10 is a diagram showing an overview of a monitoring system according to a second embodiment of the present invention. [Figure 7] 7 is a diagram showing an example of selection of cameras to be operated in the monitoring system of FIG. 6. FIG. [Figure 8] FIG. 10 is a diagram showing an example of camera placement in a monitoring system according to a third embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0021] Some embodiments of the present invention will be described with reference to the drawings. An overview of a monitoring system according to a first embodiment of the present invention is shown in Fig. 3. The monitoring system according to the first embodiment of Fig. 3 includes a radar device 200 and a camera 220 installed facing a predetermined detection range R, and a monitoring control device 240 and a display device 260 installed in a control room, a monitoring room, or the like.

[0022] 3 shows only one radar device 200 for the sake of simplicity, multiple radar devices 200 may be arranged, each monitoring a different detection range R. For example, the detection range R of each radar device 200 is assigned to a predetermined section obtained by dividing the road surface, such as a road or runway, which is the area to be monitored, in the length direction.

[0023] A plurality of cameras 220 are provided for the detection range R of one radar device 200, and are installed so as to capture the detection range R of the radar device 200 from different positions. In this way, in the present invention, a greater number (higher density) of cameras 220 than the radar device 200 is installed. In the example of Fig. 3, a rectangular detection range R is assigned to one radar device 200, and six cameras 220 are arranged to sandwich it from both sides (long sides) in the longitudinal direction.

[0024] These cameras 220 are communicably connected to the monitoring control device 240 and the display device 260, and are able to share object detection information from the radar device 200. While an example of wired wiring is shown in Fig. 3, a wireless circuit configuration is also possible. Furthermore, power supply to each camera 220 may be via wired wiring using a power cable, or may be independent power supply using solar power generation.

[0025] When the radar device 200 detects an object T present within the detection range R, it also determines the distance from the radar device 200 to the object T and the angle (azimuth) of the object T relative to the radar device 200. The distance from the radar device 200 to the object T can be calculated by signal processing of the waves reflected from the object T. The angle (azimuth) of the object T relative to the radar device 200 can be determined based on antenna rotation angle information if the radar device 200 is configured to mechanically rotate the antenna, or can be determined based on beam scanning angle information if the radar device 200 is configured to electronically scan the beam angle of the radar transmission wave. These methods are merely examples, and the methods for calculating the distance and angle relative to the object T are not particularly limited.

[0026] The object detection information including the distance and angle to the object T obtained by the radar device 200 is transmitted to the monitoring control device 240. Based on the object detection information received from the radar device 200, the monitoring control device 240 calculates the coordinates of the object T as seen from each camera 220 and the distance to the object T for the object T detected by the radar device 200. The calculation method will be described with reference to FIG. 4.

[0027] 4 shows an example of the arrangement of the radar device 200 and the cameras 220. In the illustrated example, four cameras 220A, 220B, 220C, and 220D are installed along one long side of a rectangular detection range R, and four cameras 220E, 220F, 220G, and 220H are installed along the other long side. Furthermore, the monitoring control device 240 is assumed to have stored in advance coordinate information indicating the installation locations of the radar device 200 and the cameras 220. In this example, the installation location of the radar device 200 is set as the origin (0,0), and coordinate information indicating the installation locations of each camera 220 is stored in the monitoring control device 240. However, this is merely an example, and coordinate information with another point as the origin may also be used.

[0028] The monitoring control device 240 calculates the coordinates of the detected position of the object T based on the object detection information including the distance and angle of the object T relative to the radar device 200. As a result, it is assumed that the coordinates of the detected position of the object T, with the installation location of the radar device 200 as the origin, are (a1, b1). Here, for example, if the coordinates of the installation location of the camera 220F are (a2, b2), the coordinates of the object T as seen from the camera 220F are (a2-a1, b2-b1). Furthermore, the distance d between the camera 220F and the object T can be calculated by the following (Equation 1). d=sqrt((a2-a1) 2 +(b2-b1) 2 )...(Formula 1) Here, "sqrt" is a function that calculates the square root.

[0029] The monitoring control device 240 performs similar calculations for the other cameras 220A, 220B, 220C, 220D, 220E, 220F, 220G, and 220H. Note that although the above description is of calculating the coordinates and distance of the object T using a general vector calculation, other calculation methods may be used, such as calculating the coordinates and distance of the object T using latitude and longitude. By such calculations, when the radar device 200 detects an object T present within the detection range R, it is possible to calculate the coordinates of the object T as seen from each of the multiple cameras 220 arranged to surround the detection range R, and the distance to the object T.

[0030] Here, when multiple cameras 220 are arranged to surround the detection range R of the radar device 200, the shape of the object T can be confirmed more clearly if the camera 220 positioned closest to the object T takes an image first. Also, the shape of the object T can be grasped more accurately if the object T is photographed from multiple directions. However, it is inefficient to display images photographed by all of the cameras 220 on the display device 260. Also, since the monitor only needs to look at a few images to grasp the shape of the object, other images are unnecessary.

[0031] Therefore, the monitoring control device 240 uses the distance to the object T as the priority of each of the multiple cameras 220, selects two or more cameras 220 in order of closest distance, and controls them to capture an image of the object T. For example, the monitoring control device 240 selects only cameras 220 whose distance d to the object T is shorter than a reference value as operation targets, and prioritizes capturing an image of the object T. The reference value can be set in advance based on the performance of the camera 220, etc. The reference value may also be changed taking into account the weather at the time of capture, etc.

[0032] Depending on how the reference value is set (for example, in the case of bad weather), there may be no cameras 220 whose distance d to the object T is shorter than the reference value. In that case, a predetermined number (at least two) of cameras 220 may be selected in ascending order of the distance d to the object T. Furthermore, even if a camera 220 has a distance d to the object T shorter than the reference value, it may not be able to properly photograph the object T due to factors such as backlighting. Therefore, cameras 220 that are unsuitable for photographing the object T may be identified and excluded from selection candidates according to photographing conditions such as the time of photographing, weather, and positional relationship with the object T.

[0033] 5, of the six cameras 220A to 220H arranged to surround the detection range R of the radar device 200, only four cameras 220B, 220C, 220F, and 220G whose distances d to the object T are smaller than a reference value are selected as operation targets. As a result, the cameras 220B, 220C, 220F, and 220G are pan-tilted and tilted to face the direction of the object T, and then capture images of the object T. As a result, images of the object T captured from four directions can be obtained.

[0034] The images of object T photographed from four directions by these four cameras 220B, 220C, 220F, and 220G are displayed on display device 260. For example, display device 260 displays the images of object T photographed from the four directions in each of four areas divided into four sections of the screen. Therefore, an observer who views the images displayed on display device 260 can more accurately grasp the shape of object T present within detection range R of radar device 200.

[0035] At this time, the display on the display device 260 may be devised so that it is possible to recognize from which direction each image of the object T was captured. For example, the display device 260 may display orientation information or a graphic (e.g., an arrow) indicating the capture direction superimposed on each image. Alternatively, the display device 260 may display a plan view as shown in FIG. 4, and may also arrange the captured images near each camera selected as the operation target.

[0036] Furthermore, the display device 260 may generate and display a three-dimensional image representing the three-dimensional shape of the object T based on images of the object T photographed from multiple directions, and may further be configured to rotate the three-dimensional image of the object T in response to an instruction from a monitor. A rotatable three-dimensional image can be generated, for example, by inputting images of the object T photographed from multiple directions into a 3D model generation AI.

[0037] Here, the distance from the cameras 220B, 220C, 220F, and 220G selected as operation targets to the object T differs for each camera. Therefore, simply controlling the pan and tilt of each of the cameras 220B, 220C, 220F, and 220G so that they face the direction of the object T will result in the size of the object T being different in the images of each camera. Therefore, the zoom amount may be controlled according to the distance to the object T so that the size of the object T is uniform in the images of each camera (so that the object T is displayed at the same size). Alternatively, the image of each camera may be enlarged or reduced according to the distance to the object T so that images of the object T with the same size are displayed.

[0038] Furthermore, the display device 260 may also display a reduced scale (scale display) so that the monitor can easily grasp the size of the object T, or may additionally display size information of the object T (for example, one or both of the vertical length and the horizontal length). The size of the object T may be calculated by signal processing in the radar device 200, or may be calculated using the distance between the camera 220 and the object T or the zoom control amount of the camera 220, and the calculation method is not particularly limited.

[0039] As described above, the surveillance system according to the first embodiment includes a radar device 200 installed facing a predetermined detection range R, a plurality of cameras 220 that capture images of the detection range R from different positions, and a surveillance control device 240 that, when an object T present within the detection range R is detected by the radar device 200, selects two or more cameras 220 that are close to the object T based on the position information of the object T and the position information of each of the plurality of cameras 220, and controls the two or more cameras 220 to capture images of the object T. With this configuration, the object T detected by the radar device 200 can be captured by two or more cameras 220 that are close to that position, allowing a surveillance officer who views the images to more clearly grasp the shape of the object T.

[0040] In the above description, the monitoring control device 240 has the function of the control device according to the present invention, but another device may have the function of the control device according to the present invention. For example, each of the multiple cameras 220 may have the function of the control device according to the present invention. In this case, each of the multiple cameras 220 stores the position information of each device in advance, performs the same calculation as described above to determine whether or not it should photograph the object T, and operates to photograph the object T as necessary.

[0041] Fig. 6 shows an overview of a monitoring system according to a second embodiment of the present invention. In the monitoring system (first embodiment) of Fig. 3, multiple cameras 220 are placed outside the detection range R, but in the monitoring system (second embodiment) of Fig. 6, multiple cameras 225 are placed inside the detection range R. It is desirable to use low-profile cameras that can be embedded in the road surface so as not to interfere with the movement of moving objects (vehicles and airplanes) within the detection range R (surface such as a road or runway).

[0042] Fig. 7 shows an example of selecting cameras to be operated in the monitoring system of Fig. 6. In the example shown in Fig. 7, of the six cameras 225A to 225H arranged inside the detection range R of the radar device 200, four cameras 225B, 225C, 225F, and 225G are selected as the cameras to be operated. The distance from these four cameras 225B, 225C, 225F, and 225G to the object T is shorter than the distance from the cameras 220B, 220C, 220F, and 220G to the object T in Fig. 5 (first embodiment). Therefore, according to the monitoring system (second embodiment) of Fig. 6, it is possible to capture an image of the object T present in the detection range R of the radar device 200 more clearly than the monitoring system (first embodiment) of Fig. 3.

[0043] As described above, the monitoring system according to the second embodiment includes a radar device 200 installed facing a predetermined detection range R, multiple cameras 225 that capture images of the detection range R from different positions, and a monitoring control device 240 that, when an object T present within the detection range R is detected by the radar device 200, selects two or more cameras 225 that are located close to the object T based on the position information of the object T and the position information of each of the multiple cameras 225, and controls these two or more cameras 225 to capture images of the object T, the multiple cameras 225 being low-profile cameras that can be embedded in the road surface that is the detection range R, and are installed inside the detection range R (i.e., on the road surface on which the mobile object is traveling). With this configuration, the object T detected by the radar device 200 can be captured from a position nearby, making it possible to grasp the shape of the object T more clearly.

[0044] Fig. 8 shows an example of camera arrangement in a monitoring system according to a third embodiment of the present invention. In the monitoring system according to the third embodiment of Fig. 8, multiple cameras 230A-230H are set for when no object is detected and are directed in the direction in which a moving object (a vehicle or an airplane) is approaching detection range R (a road surface such as a road or a runway). In other words, until selected by monitoring control device 240 to photograph object T, cameras 230A-230H operate to photograph a moving object that has entered detection range R. The moving object may be photographed while cameras 230A-230H are fixed in the direction in which the moving object is approaching, or while cameras 230A-230H are changed (rotated) in orientation so as to track the moving object.

[0045] Images of moving objects captured by the cameras 230A to 230H are stored in a storage device (not shown) so that they can be displayed on the display device 260 as needed. For example, when the radar device 200 detects an object T within the detection range R, the display device 260 displays two or more images of the object T captured from different directions, as well as an image of the moving object entering the detection range R immediately before the object T was detected. This makes it easier to identify the moving object that caused the object T to appear within the detection range R. Furthermore, the camera can be used for purposes other than capturing the object T, thereby enabling effective use of the camera. This is particularly effective when a high-resolution camera is installed to capture fallen objects, abandoned objects, etc.

[0046] When the radar device 200 detects an object T in the detection range R, an image of the object T and an image of the moving object immediately before the object T may be stored in a storage device in association with each other. This can be useful for investigating and analyzing the relationship between the object T and the moving object after the fact. In this case, if the image of the moving object is stored as a video, it may be possible to confirm the state when the object T falls from the moving object.

[0047] Here, it is not necessary to use all of the multiple cameras 230A-230H to photograph the moving object, and two or more cameras suitable for photographing the moving object may be selectively used. For example, based on the distribution of points where object T has fallen from the moving object in the past, a point where object T is likely to fall may be specified in advance, and two cameras (for example, camera 230C and camera 230G) on both sides near that point may be used to photograph the moving object.

[0048] Furthermore, cameras 230A-230H may be set to pause in a standby state so that they can immediately start capturing images, except when a capturing operation such as capturing an image of object T or a moving object is required. The timing for capturing an image of a moving object can be determined, for example, based on information from a control system that controls aircraft. That is, monitoring control device 240 recognizes the entry of a moving object into detection range R based on information from the control system, and controls cameras 230A-230H to start capturing an image of the moving object. Note that in an environment where the direction of entry of a moving object into detection range R may change, monitoring control device 240 may identify the direction of entry of the moving object into detection range R based on information from the control system, and control cameras 230A-230H to start capturing an image after pointing them in that direction.

[0049] Although the embodiments of the present invention have been described above, these embodiments are merely illustrative and do not limit the technical scope of the present invention. The present invention can take on various other embodiments, and various modifications such as omissions and substitutions can be made without departing from the spirit of the present invention. These embodiments and modifications thereof are included in the scope and spirit of the invention described in this specification, etc., and are included in the invention described in the claims and their equivalents.

[0050] Furthermore, the present invention can be provided not only as devices such as those described above or as systems composed of these devices, but also as methods executed by these devices, programs for realizing the functions of these devices using a processor, and storage media for storing such programs in a computer-readable manner. [Industrial Applicability]

[0051] The present invention can be used in a surveillance system that uses a radar device to detect objects that exist within a surveillance area. [Explanation of symbols]

[0052] 100: Radar device, 101: FMCW transmission source, 102: Power divider, 103: Transmission power amplifier, 104: Transmission antenna, 105: Receiving antenna, 106: Receiving power amplifier, 107: Mixer, 108: Signal processing unit, 120: Camera, 140: Monitoring and control device, 160: Display device, 200: Radar device, 220 (220A to 220H), 225 (225A to 225H), 230 (230A to 230H): Camera, 240: Monitoring and control device, 260: Display device

Claims

1. a radar device installed facing the area to be monitored; a plurality of cameras each capturing an image of the area to be monitored from a different position; A surveillance system characterized by comprising a control device that, when an object is detected within the monitored area by the radar device, selects two or more cameras located close to the object based on the position information of the object and the position information of each of the multiple cameras, and controls the two or more cameras to photograph the object.

2. 2. The monitoring system according to claim 1, a display device that displays two or more images captured by the two or more cameras; The monitoring system is characterized in that the display device displays, together with the two or more images, information indicating the photographing direction of each of the two or more images.

3. 2. The monitoring system according to claim 1, a display device that displays two or more images captured by the two or more cameras; The monitoring system is characterized in that the display device displays the two or more images in a manner in which the size of the object in each image is uniform.

4. 2. The monitoring system according to claim 1, the monitored area includes a road surface on which a moving object travels, A surveillance system characterized in that the plurality of cameras are low-profile cameras that can be embedded in the road surface and are installed inside the area to be monitored.

5. 2. The monitoring system according to claim 1, A surveillance system characterized in that the multiple cameras are installed facing the direction in which a moving object is entering the monitored area, and photograph the moving object as it enters the monitored area until the control device selects the object for photographing.

6. 6. The monitoring system according to claim 5, a display device that displays two or more images captured by the two or more cameras; The monitoring system is characterized in that the display device displays, together with the two or more images, an image of the moving object entering the monitored area immediately before the object is detected.

Images