Flying object detection device, flying object detection system, flying object detection method, and program storage medium

Abstract

This flying object detection device is provided with the following functions, in order to detect flying objects that are targeted for detection and achieve prevention of missed detection, while suppressing the number of installations of imaging devices, even in a wide monitoring area. The flying object detection device senses a flying object in a detection area by performing signal processing on a radio wave sensing signal outputted from a radio wave sensing device. When a flying object has been sensed in the detection area, the flying object detection device treats the sensed flying object as a flying object targeted for investigation, and variably controls the imaging direction of an imaging device so as to cause the imaging device to image the flying object targeted for investigation. The flying object detection device detects a flying object targeted for detection from the imaged image that was imaged by the imaging device. The flying object detection device outputs flying object detection information indicating that the flying object targeted for detection has been detected.

Description

Flying object detection device, flying object detection system, flying object detection method, and program storage medium

[0001] The present disclosure relates to an airborne object detection device, an airborne object detection system, an airborne object detection method, and a program storage medium for detecting an airborne object to be detected.

[0002] No-fly zones (for example, the airspace above airports and important facilities and their surrounding areas) are designated for unmanned aerial vehicles (small unmanned aircraft), also known as drones or UAVs (Unmanned Aerial Vehicles), and flight permission is required for unmanned aircraft to fly in these no-fly zones. However, with the increasing use of unmanned aircraft, there are concerns that an increasing number of unmanned aircraft will enter these no-fly zones without permission.

[0003] Therefore, technologies for detecting the flight of unmanned aerial vehicles in monitored areas such as no-fly zones have been studied. For example, Patent Document 1 (WO 2023 / 286295) discloses a technology for detecting the intrusion of a suspicious unmanned aerial vehicle into a monitored area using the results of multiple types of detection sensors that detect unmanned aerial vehicles. Patent Document 1 lists a photographing device as one of the detection sensors that detects unmanned aerial vehicles.

[0004] WO 2023 / 286295

[0005] When attempting to monitor a surveillance area that includes a no-fly zone for unmanned aerial vehicles as described above, it is possible to use a camera that captures the surveillance area, as shown in Patent Document 1. In this case, if the surveillance area is large, such as the airspace above an airport or its surrounding area, a large number of camera devices are required to monitor the entire surveillance area. However, due to reasons such as the installation location and cost of the camera devices, it may be desirable to detect unmanned aerial vehicles in the surveillance area with a limited number of camera devices, even if the surveillance area is large.

[0006] The present disclosure has been devised to solve the above-mentioned problems. That is, the main purpose of the present disclosure is to provide a technology for detecting target flying objects while preventing detection omissions, even in a wide surveillance area, while limiting the number of installations of imaging devices.

[0007] In order to achieve the above-mentioned object, one aspect of the flying object detection device of the present disclosure comprises: a detection unit that detects flying objects in a detection area by signal processing a radio detection signal output from a radio detection device that detects flying objects in the detection area using radio waves; an imaging direction control unit that, when an flying object in the detection area is detected, sets the detected flying object as an flying object to be investigated and variably controls the imaging direction of an imaging device that photographs the detection area so that the flying object to be investigated is photographed by the imaging device; a detection unit that detects the flying object to be detected from the photographed image taken by the imaging device; and an output unit that outputs flying object detection information indicating that the flying object to be detected has been detected.

[0008] In addition, one aspect of the flying object detection system according to the present disclosure includes a radio wave detection device that uses radio waves to detect flying objects in a detection area, an imaging device that images the detection area, and an flying object detection device as described above that uses the radio wave detection device and the imaging device.

[0009] Furthermore, in one aspect, the flying object detection method of the present disclosure includes: detecting flying objects in a detection area by a computer by signal processing a radio detection signal output from a radio detection device that uses radio waves to detect flying objects in the detection area; when an flying object in the detection area is detected, the detected flying object is designated as an flying object to be investigated; variably controlling the shooting direction of an imaging device that photographs the detection area so as to photograph the flying object to be investigated; detecting the flying object to be detected from the image photographed by the imaging device; and outputting flying object detection information indicating that the flying object to be detected has been detected.

[0010] Furthermore, in one aspect, the program storage medium according to the present disclosure stores a computer program that causes a computer to execute the following processes: a process of detecting an aircraft in a detection area by signal processing a radio detection signal output from a radio detection device that uses radio waves to detect aircraft in the detection area; a process of designating the detected aircraft in the detection area as an aircraft to be investigated, and variably controlling the shooting direction of an imaging device that photographs the detection area so that the aircraft to be investigated is photographed by the imaging device; a process of detecting the aircraft to be detected from the image photographed by the imaging device; and a process of outputting aircraft detection information indicating that the aircraft to be detected has been detected.

[0011] According to the present disclosure, even in a large monitoring area, it is possible to detect target flying objects while minimizing the number of installed imaging devices and preventing missed detections.

[0012] FIG. 1 is a diagram illustrating an example of the configuration of a flying object detection system. FIG. 2 is a diagram illustrating an example of the configuration of a flying object detection device. FIG. 3 is a diagram illustrating an example of a captured image output by an output unit. FIG. 4 is a diagram illustrating another example of a captured image output by an output unit. FIG. 5 is a diagram illustrating an example of flying object detection information output by an output unit. FIG. 6 is a flowchart illustrating an example of the operation related to the detection of a flying object that is a detection target in a flying object detection device. FIG. 7 is a diagram illustrating another example of the configuration of a flying object detection device. FIG. 8 is a diagram illustrating a modified example of the display on a display device. FIG. 9 is a diagram illustrating an example of an image that reflects the detection result. FIG. 10 is a diagram illustrating another example of the configuration of a flying object detection device. FIG. 11 is a flowchart illustrating another example of the operation related to the detection of a flying object that is a detection target in a flying object detection device.

[0013] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0014] First Embodiment An airborne object detection system according to a first embodiment of the present disclosure is a system that detects an airborne object as a detection target in a detection area, and includes a radio wave detection device 2, an imaging device 3, and an airborne object detection device (hereinafter also simply referred to as a detection device) 5, as shown in FIG. 1 . In the first embodiment, the airborne object detection system 1 is applied to a surveillance system. The surveillance system to which the airborne object detection system 1 is applied is, for example, a system that monitors the skies above and surrounding areas of important facilities such as airports and nuclear power plants, and the surveillance area includes no-fly zones where the flight of unmanned aerial vehicles is prohibited by law or regulation. Because the airborne object detection system 1 is applied to such a surveillance system, the detection area of ​​the airborne object detection system 1 is the surveillance area of ​​the surveillance system to which it is applied.

[0015] The target air vehicles to be detected by the air vehicle detection system 1 are determined in advance by a system designer or the like, taking into consideration, for example, the type of facility monitored by the applicable monitoring system and its surrounding environment. Examples of air vehicles set as the target of detection include unmanned aerial vehicles (unmanned aerial vehicles) that can be flown by remote control or automatic pilot, such as unmanned airplanes, unmanned rotorcraft, and unmanned airships, as well as those flown by humans using specific aviation equipment, such as balloons, hang gliders, and paragliders. In this way, various air vehicles can be set as the target of detection, but in the first embodiment, a so-called drone (unmanned aerial vehicle) is set as the target of detection.

[0016] The radio wave detection device 2 is a device that uses radio waves to detect flying objects in a detection area. Examples of the radio wave detection device 2 include radar and passive radar. Radar transmits radio waves and receives the waves reflected by an object. By using the time from transmitting the radio waves to receiving the reflected waves and the direction in which the reflected waves are received, it is possible to calculate the presence or absence of an object, as well as the distance and direction to the object. Passive radar, also known as a radio wave detection sensor, is able to determine the location of an aircraft by detecting radio waves used for communication between an aircraft, such as an unmanned aerial vehicle, and a ground station. There are multiple types of radar and passive radar. When a radar or passive radar is used as the radio wave detection device 2 constituting the flying object detection system 1, the type of radar or passive radar is determined in advance by a system designer or the like, taking into consideration various factors such as the size of the detection area, the type of flying object to be detected, the type of facility to be monitored by the monitoring system to which the flying object detection system 1 is applied, and the surrounding environment. In the first embodiment, the type of radar or passive radar employed as the radio wave detection device 2 is not limited, and therefore a description thereof will be omitted.

[0017] The imaging device 3 is a device that captures an image of a detection area. In the first embodiment, the imaging device 3 is connected to a drive device 4. The drive device 4 is a device that changes the orientation of the imaging device 3 in order to vary the imaging direction (the direction in which the optical axis is facing) of the imaging device 3. The drive control of the drive device 4 is performed by the flying object detection device 5.

[0018] The installation location of the imaging device 3 may be any suitable location that allows the imaging device 3 to capture an image of the detection area, taking into consideration the presence or absence of obstacles that may hinder imaging, ease of installation, etc. Here, there are no restrictions on the installation location of the imaging device 3 that are related to the installation location of the radio wave detection device 2, such as requiring the imaging device 3 to be installed near the radio wave detection device 2.

[0019] The number of installed image capturing devices 3 may be one or more, and is not limited. However, in the first embodiment, as described above, the image capturing direction of the image capturing device 3 can be changed by the drive device 4, and therefore the image capturing range that can be captured by one image capturing device 3 can be expanded compared to when the image capturing direction of the image capturing device 3 is fixed. Taking this into consideration, an attempt is made to keep the number of installed image capturing devices 3 to a minimum.

[0020] For example, if it is possible to use a single imaging device that can capture an image of the entire detection area, one such imaging device is installed as the imaging device 3 that constitutes the flying object detection system 1.

[0021] Furthermore, for example, if the detection area is as large as the airspace above an airport or its surrounding area, a camera equipped with a telephoto lens can be used as the camera 3 of the flying object detection system 1 to capture flying objects in distant airspace, such as approximately one kilometer away. A camera equipped with a telephoto lens (hereinafter also referred to as a telephoto-type camera) can capture distant objects in a magnified view. On the other hand, a telephoto-type camera cannot clearly capture close objects. In other words, when a telephoto-type camera is used, portions of the detection area will be left uncaptured by the camera. Taking this into consideration, a camera equipped with a wide-angle lens (hereinafter also referred to as a wide-angle-type camera) can also be used as the camera 3 of the flying object detection system 1. The wide-angle-type camera can have a capture range that compensates for portions of the detection area not captured by the telephoto-type camera. Thus, in the first embodiment, a combination of a telephoto-type camera and a wide-angle-type camera can be used as the camera 3 of the flying object detection system 1. When a telephoto type imaging device and a wide-angle type imaging device are used, the telephoto type imaging device and the wide-angle type imaging device are installed in a common installation location where they can capture images of the detection area. In addition, a drive unit 4 corresponding to each of the telephoto type imaging device and the wide-angle type imaging device is provided, so that, for example, the imaging direction of the telephoto type imaging device and the wide-angle type imaging device can be changed independently of each other.

[0022] Lenses equipped in imaging devices can be classified into three types depending on the angle of view: "standard lenses," "wide-angle lenses," and "telephoto lenses." A "standard lens" is a lens with an angle of view of about 45 to 50 degrees, which is said to be close to the human field of view. A "wide-angle lens" is a lens with a wider angle of view than a "standard lens," for example, 60 degrees or more. A "telephoto lens" is a lens with a narrower angle of view than a "standard lens," for example, 30 degrees or less.

[0023] The flying object detection device (hereinafter also referred to as the detection device) 5 is a computer device and is communicatively connected to the radio detection device 2, the imaging device 3, and the drive device 4. The detection device 5 executes a detection process to detect flying objects in a detection area using the radio detection device 2, the imaging device 3, and the drive device 4. That is, as shown in FIG. 2 , the detection device 5 includes a calculation device 50 and a storage device 40. The storage device 40 includes a storage medium for storing data and a computer program (hereinafter also referred to as a program) 41. There are multiple types of storage devices, such as magnetic disk drives and semiconductor memory devices. Furthermore, there are multiple types of semiconductor memory devices, such as RAM (Random Access Memory) and ROM (Read Only Memory). A computer device may be equipped with multiple types of storage devices depending on their intended use, but these storage devices will be collectively referred to as the storage device 40 without distinction. Furthermore, the types and number of storage devices 40 included in the detection device 5 are not limited, and a description thereof will be omitted. The detection device 5 may also be connected to a database 6, which is a storage device. In this case, the detection device 5 may write information to the database 6 or read information from the database 6. However, to avoid complicating the explanation, a description of such a case will be omitted here.

[0024] The arithmetic device 50 is configured with a processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The arithmetic device 50 can have functions based on a program 41 by reading and executing the program 41 stored in the storage device 40. Here, the arithmetic device 50 has a receiving unit 51, a detection unit 52, an imaging direction control unit 53, a detection unit 54, and an output unit 55 as functional units related to detecting an airborne object.

[0025] The receiving unit 51 receives a radio detection signal output from the radio detection device 2. That is, the radio detection signal is a signal output from a radar or a passive radar. The receiving unit 51 also receives information (image data) of an image captured by the imaging device 3. The receiving unit 51 then stores the received radio detection signal and the captured image information in the storage device 40. The radio detection signal and the captured image information stored in the storage device 40 are each associated with information on the time of detection by the radio detection device 2 and information on the time of imaging.

[0026] In the example of FIG. 2, one image capture device 3 and one drive device 4 are shown, but as described above, a plurality of image capture devices 3 and a plurality of drive devices 4 may be used.

[0027] The detection unit 52 detects flying objects in the detection area and calculates the position of the detected flying objects by processing the radio detection signal output from the radio detection device 2. The method of detecting flying objects in the detection area using the radio detection signal depends on the type of radar or passive radar, and is not limited here, so a description thereof will be omitted.

[0028] The imaging direction control unit 53 controls the drive unit 4 to variably control the imaging direction of the imaging device 3. Here, the imaging direction control unit 53 controls the imaging direction variably by alternatively switching between the following first and second modes. That is, the first mode is a mode in which, when an airborne object in the detection area is detected by the detection unit 52, the drive unit 4 is controlled to enable the imaging device 3 to photograph the detected airborne object. In this first mode, the imaging direction control unit 53 controls the drive unit 4 using position information representing the position of the airborne object calculated by the detection unit 52. That is, in the first embodiment, the radio detection device 2 and the detection unit 52 monitor the entire detection area to detect airborne objects in the detection area. In this way, monitoring the entire detection area by the radio detection device 2 and the detection unit 52 makes it possible to detect airborne objects in the detection area while preventing detection misses. However, even if an aircraft can be detected, it is not easy to identify the type of the detected aircraft from among the many types of aircraft using only the radio detection signal output from the radio detection device 2. Therefore, in the first embodiment, the image captured by the imaging device 3 is used to identify the type of aircraft detected by the radio detection device 2 and the detection unit 52. In other words, the detected aircraft is set as the aircraft under investigation, and the imaging direction control unit 53 controls the drive of the drive unit 4 using position information representing the position of the aircraft under investigation (detected aircraft) calculated by the detection unit 52 so that the imaging device 3 can photograph the aircraft under investigation. "So that the imaging device 3 can photograph the aircraft under investigation" preferably means that the optical axis of the imaging device 3 passes through the position of the aircraft under investigation calculated by the detection unit 52, and the aircraft under investigation is captured in the center of the captured image.

[0029] Incidentally, there are cases in which multiple flying objects are detected in the detection area by the radio detection device 2 and the detection unit 52. Anticipating such cases, a priority rule is predefined to determine which of the multiple detected flying objects is to be the highest priority flying object to be investigated. For example, suppose an enhanced surveillance area is defined in the detection area (monitoring area) in which surveillance is strengthened. In this case, a rule is predefined to determine the priority order of the multiple detected flying objects using the distance between the enhanced surveillance area and the location of the detected flying object. The priority rule is defined such that the flying object with the highest priority among the priorities determined according to the rule is to be the highest priority flying object to be investigated.

[0030] The first mode of the imaging direction control unit 53 is a mode in which the control described above is performed. Note that, when multiple imaging devices are installed as the imaging device 3 as described above, each of the imaging devices 3 has a designated imaging area in the detection area in which it is responsible for imaging. In the first mode, the imaging direction control unit 53 uses the position information of the aircraft under investigation obtained by the detection unit 52 to select an imaging device 3 from the multiple imaging devices 3 that will image the aircraft under investigation. Then, the imaging direction control unit 53 performs drive control of the drive unit 4 associated with the selected imaging device 3, thereby variably controlling the imaging direction of the imaging device 3 to image the aircraft under investigation.

[0031] The second mode in the imaging direction control unit 53 is a mode in which the imaging direction of the imaging device 3 is controlled by driving and controlling the drive units 4 in accordance with a predetermined imaging direction control rule, regardless of the results of processing by the radio detection device 2 and the detection unit 52. In the first embodiment, the imaging direction control rule in the second mode is not limited, but examples include a rule that scans the imaging direction over the entire detection area, or a rule that fixes the imaging device 3 in a state facing a predetermined imaging direction and waits in preparation for the first mode. Note that when multiple imaging devices are installed as the imaging device 3 as described above, the imaging direction control unit 53 controls each of the drive units 4 corresponding to each of the multiple imaging devices 3 in the second mode in accordance with the imaging direction control rule.

[0032] In the first embodiment, the imaging direction control unit 53 switches from the second mode to the first mode when the detection unit 52 detects an aircraft to be investigated. Furthermore, when the imaging direction control unit 53 is controlling the imaging direction in the first mode, it switches from the first mode to the second mode at a predetermined timing. Examples of the timing for switching from the first mode to the second mode include the timing when a predetermined time has elapsed since switching to the first mode, or the timing when the aircraft to be investigated is photographed by the imaging device 3 and processing of the photographed image by the detection unit 54 is completed. Note that even when the imaging direction control unit 53 ends its operation in the first mode in response to the detection of an aircraft (here, referred to as aircraft A) by the radio detection device 2 and the detection unit 52 and switches to the second mode, it is assumed that aircraft A is still flying within the detection area. A control method for the imaging direction control unit 53 to switch modes may be defined to account for such cases.

[0033] The detection unit 54 detects the target flying object from the captured image by analyzing the captured image taken by the imaging device 3. Various methods for detecting the target flying object from the captured image have been proposed, and although not limited thereto, one example is a detection method using AI (Artificial Intelligence) technology. In this case, a detection model generated by AI technology is used. This detection model is generated by learning using images of the target flying object as training data. Input information to the detection model is the image captured by the imaging device 3, and output information from the detection model includes information indicating the presence or absence of the target flying object in the input captured image and, if the target flying object is detected, information indicating the position of the detected flying object in the captured image (hereinafter also referred to as detection position information). Note that the imaging device 3 captures video. In this case, for example, frame images to be processed are selected for each predetermined number of frames from multiple frame images constituting the captured image (video). The detection unit 54 performs the detection process described above for the selected frame images to detect the target flying object from the captured image.

[0034] Information indicating the detection result by the detection unit 54 is stored in the storage device 40 in association with, for example, information indicating the frame number of the captured image subjected to the detection process and information on the time of capture.

[0035] As described above, in the first embodiment, there is a case where a plurality of image capturing devices 3 are used. In this case, the detection unit 54 executes a detection process for detecting the target flying object from the captured image for each image captured by the image capturing device 3.

[0036] Furthermore, in the first embodiment, the shooting direction of the imaging device 3 may be suddenly changed due to the aforementioned switching of the imaging direction control unit 53 from the first mode to the second mode. It is expected that the captured image captured by the imaging device 3 during this period of changing the shooting direction may be an image inappropriate for detection processing by the detection unit 54 due to poor shooting conditions, such as blurring of the subject. Taking this into consideration, in the first embodiment, during the driving period of the drive unit 4 caused by the switching of the imaging direction control unit 53 from the first mode to the second mode, the detection unit 54 suspends detection processing for the captured image output from the imaging device 3 whose shooting direction is being variably controlled.

[0037] When the detection unit 54 detects a target flying object, the output unit 55 outputs flying object detection information indicating that the target flying object has been detected. An example of an output destination is the display device 7 shown in FIG. 1. The display device 7 is a device that notifies (provides) information by displaying information on a screen using text or images. Here, the display device 7 receives the flying object detection information from the output unit 55 and provides the flying object detection information to, for example, a user of the flying object detection system (monitoring system). Note that the display device 7 may be, for example, a display device of a terminal device carried by a user of the flying object detection system (monitoring system).

[0038] There are various methods for providing flying object detection information to the user by the display device 7, and here, the method of providing this information is not limited as long as it can inform the user that the target flying object has been detected, but the following methods of providing this information are examples.

[0039] For example, one method of providing the information is to display the flying object detection information on the display device 7 using text. Another method of providing the information is to display a detection result reflection image, such as that shown in FIG. 3, on the display device 7 as the flying object detection information. The detection result reflection image is an image in which a graphic (hereinafter also referred to as the flying object detection graphic) 25 representing the position of the detected flying object 20 is superimposed on a captured image in which the target flying object is photographed. The type of flying object detection graphic is not limited and may be, for example, a circle, triangle, or square, or a symbol or mark resembling an flying object, and is appropriately set by a system designer, etc. Furthermore, when multiple types of flying objects are detected, the same flying object detection graphic 25 may be superimposed on the captured image in the detection result reflection image regardless of the type of flying object detected. However, as shown in FIG. 4, different flying object detection graphics 25 corresponding to the type of flying object may be superimposed on the captured image.

[0040] Furthermore, as another provision method, as shown in Fig. 5, there is a method in which the above-mentioned result reflection image using the image captured by the imaging device 3 and a detection result image using the radio detection signal output from the radio detection device 2 are displayed side by side on the display device 7 as flying object detection information. In this case, for example, information indicating the position of the detected flying object is superimposed on the detection result image, as shown in Fig. 5. Furthermore, it is assumed that the result reflection image and the detection result image displayed side by side are synchronized images (images created in time).

[0041] The detection device 5 of the first embodiment and the flying object detection system 1 including the detection device 5 are configured as described above. Next, an example of the operation of the detection device 5 to detect an flying object will be described with reference to Fig. 6. Fig. 6 is a flowchart illustrating an example of the operation of the detection device 5 to detect an flying object. In other words, Fig. 6 shows an example of a flying object detection method.

[0042] For example, assume that the radio detection device 2 and the image capture device 3 are both operating, and the image capture direction control unit 53 is in the second operating mode. In this case, the receiver 51 receives the captured image output from the image capture device 3 and the radio detection signal output from the radio detection device 2 (step 101 in FIG. 6 ). The detection unit 52 then performs signal processing on the received radio detection signal (step 102). The image capture direction control unit 53, for example, determines whether an aircraft has been detected in the detection area through this signal processing (step 103). If an aircraft has not been detected, the detection device 5 repeats the operations from step 101 onward. If an aircraft has been detected, the image capture direction control unit 53 switches to the first operating mode and controls the drive unit 4 to orient the detected aircraft as the aircraft under investigation so that the image capture direction of the image capture device 3 faces the aircraft under investigation (step 104).

[0043] Thereafter, the detection unit 54 performs a detection process on the captured image captured by the image capture device 3 whose image capture direction has been variably controlled (step 105). Then, for example, the output unit 55 determines whether or not the target flying object has been detected from the captured image (step 106). If the target flying object has not been detected, the detection device 5 repeats the operations from step 101 onward. Furthermore, if the target flying object has been detected, the output unit 55 outputs flying object detection information to, for example, the display device 7 (step 107).

[0044] The detection device 5 of the first embodiment and the flying object detection system 1 including the detection device 5 are configured as described above, and therefore can achieve the following effects: That is, the flying object detection system 1 can detect flying objects in the detection area while preventing any missed detections by monitoring the entire detection area using the radio detection device 2 and the detection unit 52 of the detection device 5.

[0045] Furthermore, it is not easy to identify the type of detected flying object using only the radio detection signal output from the radio detection device 2. In this regard, the flying object detection system 1 is configured to treat the detected flying object as an investigation target flying object, photograph the investigation target flying object, and detect the target flying object from the photographed image. Using the photographed image facilitates identifying the type of flying object and improves the accuracy of detection. Furthermore, the flying object detection system 1 is configured so that the photographing direction of the image capture device 3 can be changed by the drive device 4. This allows for a wider photographing range per image capture device 3 than when the photographing direction of the image capture device 3 is fixed, thereby reducing the number of image capture devices 3 installed. In other words, the detection device 5 of the first embodiment and the flying object detection system 1 equipped with the detection device 5 can detect target flying objects while preventing missed detections, even in a wide monitoring area, while reducing the number of image capture devices installed.

[0046] Second Embodiment A second embodiment according to the present disclosure will be described below. In the description of the second embodiment, components having the same names as those in the description of the first embodiment will be denoted by the same reference numerals, and a duplicate description of the common parts will be omitted.

[0047] The detection device 5 constituting the flying object detection system 1 in the second embodiment includes a tracking unit 56 as shown in Fig. 7 in addition to the configuration of the first embodiment. The tracking unit 56 works in conjunction with the detection unit 54 to track the flying object to be detected (the flying object to be investigated) using captured images. There are various tracking methods for tracking a subject using captured images, and the tracking method is not limited here, so a description thereof will be omitted.

[0048] In addition, in the second embodiment, the second mode in the shooting direction control unit 53 includes a control operation of controlling the drive unit 4 so that the tracking flying object appears in the center of the captured image when the tracking unit 56 is performing a process of tracking the flying object using the captured image.

[0049] If the detection device 5 is equipped with the tracking unit 56 described above, the detection unit 54 may perform a detection process that further uses the results of tracking of the flying object by the tracking unit 56. For example, when the detection unit 54 first detects a target flying object from a captured image, it recognizes the detected flying object as a detection candidate. If the detection unit 54 and the tracking unit 56 then work together to track the detection candidate and thereby obtain information on the flight trajectory of the detection candidate, the detection unit 54 uses the flight trajectory information to determine whether the detection candidate is a target flying object. In such a case, for example, information on the flight trajectory of the flying object is stored in advance in the storage device 40 as information representing the target flying object, and is referenced in the process of determining whether the target flying object is a target flying object.

[0050] In the second embodiment, the receiving unit 51 receives captured images captured while tracking the target flying object through the tracking process of the tracking unit 56 and the drive control of the drive device 4 by the imaging direction control unit 53. The output unit 55 may output the captured images captured while tracking the target flying object to the display device 7.

[0051] The detection device 5 and the flying object detection system 1 including the detection device 5 in the second embodiment have the same configuration as those in the first embodiment, and can therefore achieve the same effects as those in the first embodiment. Furthermore, in the second embodiment, the detection device 5 further includes a tracking unit 56, which can also acquire information about the flight trajectory of the flying object from the captured image. This allows the detection device 5 to improve the detection accuracy of the target flying object and increase the amount of information that can be provided to the user when the target flying object is detected.

[0052] <Other Embodiments> The present disclosure is not limited to the first and second embodiments and may be implemented in various ways. For example, the detection unit 54 may further include a function for determining whether the detected flying object is a suspicious flying object that is not permitted to fly in the detection area, and may issue an alert if it is determined to be a suspicious flying object. This alert is output by the output unit 55 to a predetermined notification destination.

[0053] Furthermore, for example, when a user inputs a request (enlarged display request) to the detection device 5 to enlarge and display an image of the flying object on the display device 7, the output unit 55 may perform display control to enlarge and display the flying object 20 in response to the request, as shown in FIG. 8 . The method by which the user inputs the enlarged display request is not limited here, but an example is a method in which the enlarged display request is input to the detection device 5 by clicking the flying object detection graphic 25 with a cursor 71 as shown in FIG. 8 . Furthermore, the manner in which the flying object is enlarged is not limited here and may be an appropriately set display manner. For example, as shown in FIG. 8 , a window image in which the flying object is enlarged may be superimposed on the wide-angle image, or the wide-angle image and the enlarged display image of the flying object may be displayed side by side.

[0054] Furthermore, although the first and second embodiments illustrate examples in which the flying object detection system is applied to a surveillance system, the application of the flying object detection system of the present disclosure is not limited to surveillance systems. For example, in a traffic control system for unmanned aircraft such as drones in an area surrounding a logistics hub, the flying object detection system of the present disclosure may be applied to detect unmanned aircraft that are subject to management as flying objects to be detected.

[0055] Furthermore, in the first and second embodiments, an air vehicle that is a man-made object has been described as an example of an air vehicle to be detected. However, for example, a bird may be set as an air vehicle to be detected. That is, if a bird collides with an aircraft, a serious accident such as the aircraft crashing may occur. If a bird flies near an aircraft, such a risk may arise, and there is a concern that the operation of the unmanned aircraft may be hindered. For this reason, for example, when the air vehicle detection system of the present disclosure is applied to an unmanned aircraft traffic management system, birds may also be detected as an air vehicle to be detected. Then, when a bird (air vehicle to be detected) is detected around a managed aircraft that is the air vehicle to be detected, the output unit 55 may output a detection result reflection image, such as that shown in FIG. 9 , by superimposing information indicating that a bird is flying nearby and that there is a risk posed by the bird on the captured image.

[0056] Furthermore, in the first and second embodiments, the output unit 55 outputs the flying object detection information to the display device 7. In addition, when the detection unit 54 detects a target flying object, the output unit 55 may output the flying object detection information indicating that the target flying object has been detected to a predetermined notification destination other than the display device. One example of the notification destination is a computer device of a monitoring system to which the flying object detection system is applied.

[0057] Furthermore, the flying object detection device may also have a configuration as shown in Fig. 10. The flying object detection device 60 shown in Fig. 10 is, for example, a computer device, and includes a detection unit 61, an imaging direction control unit 62, a detection unit 63, and an output unit 64 as functional units realized by executing a computer program. The flying object detection device 60 is connected to a radio wave detection device 70, an imaging device 80, and a drive device 90. The radio wave detection device 70 is a device that detects flying objects in a detection area using radio waves and outputs a radio wave detection signal. The imaging device 80 is a device that images the detection area.

[0058] The detection unit 61 detects flying objects in the detection area by processing the radio detection signal output from the radio detection device 70. When an flying object is detected in the detection area, the imaging direction control unit 62 designates the detected flying object as an investigation target flying object and variably controls the imaging direction of the imaging device 80 so that the imaging device 80 photographs the investigation target flying object. The detection unit 63 detects the detection target flying object from the captured image taken by the imaging device 80. The output unit 64 outputs flying object detection information indicating that the detection target flying object has been detected. Note that the detection unit 52, imaging direction control unit 53, detection unit 54, and output unit 55 of the detection device 5 in the first and second embodiments described above are examples of the detection unit 61, imaging direction control unit 62, detection unit 63, and output unit 64, respectively.

[0059] The flying object detection device 60 has the above-described configuration. The flying object detection device 60, together with a radio wave detection device 70 and an imaging device 80 as shown by dotted lines in Fig. 10, constitutes a flying object detection system.

[0060] Next, an example of the operation of the flying object detection device 60 will be described with reference to Fig. 11. Fig. 11 is a flowchart explaining an example of the operation of the flying object detection device 60. Fig. 11 can also be said to be a diagram explaining an example of a flying object detection method.

[0061] For example, when the detection unit 61 of the flying object detection device 60 detects an flying object in the detection area by signal processing the radio detection signal output from the radio detection device 70 (step 204), the imaging direction control unit 62 variably controls the imaging direction of the imaging device 80 (step 202). That is, when an flying object is detected in the detection area, the imaging direction control unit 62 designates the detected flying object as the flying object to be investigated, and variably controls the imaging direction of the imaging device 80 so that the imaging device 80 photographs the flying object to be investigated. Thereafter, the detection unit 63 executes a process to detect the target flying object from the image captured by the imaging device 80 (step 203). As a result, when the target flying object is detected, the output unit 64 outputs flying object detection information indicating that the target flying object has been detected (step 204).

[0062] The flying object detection device 60 detects flying objects in the detection area by signal processing the radio detection signal from the radio detection device 70, allowing for complete detection of flying objects throughout the entire detection area. Furthermore, the flying object detection device 60 uses the detection results to photograph the flying object with the imaging device 80 and detects the target flying object from the photographed image, thereby improving the detection accuracy of the target flying object. Furthermore, the flying object detection device 60 is configured to be able to variably control the imaging direction of the imaging device 80, thereby providing a wider imaging range than when the imaging direction of the imaging device 80 is fixed, and reducing the number of imaging devices required to detect the target flying object. In other words, the flying object detection device 60 can detect target flying objects without missing any, even in a large monitoring area, while minimizing the number of imaging devices installed.

[0063] Some or all of the above embodiments can be described as, but are not limited to, the following supplementary notes. (Supplementary Note 1) An air vehicle detection device comprising: a detection unit that detects air vehicles in a detection area by signal processing a radio detection signal output from a radio detection device that detects air vehicles in the detection area using radio waves; an imaging direction control unit that, when an air vehicle in the detection area is detected, designates the detected air vehicle as an air vehicle to be investigated and variably controls the imaging direction of an imaging device that photographs the detection area so that the air vehicle to be investigated is photographed by the imaging device; a detection unit that detects the air vehicle to be detected from images photographed by the imaging device; and an output unit that outputs air vehicle detection information indicating that the air vehicle to be detected has been detected. (Supplementary Note 2) The air vehicle detection device according to Supplementary Note 1, further comprising a tracking unit that tracks the air vehicle to be detected detected by the detection unit using images photographed by the imaging device. (Supplementary Note 3) The flying object detection device described in Supplementary Note 1, wherein the detection unit suspends the detection process of detecting the target flying object from the captured image when the capturing direction control unit variably controls the capturing direction of the imaging device due to the detection of a flying object using a radio detection signal. (Supplementary Note 4) The flying object detection device described in Supplementary Note 1, wherein the output unit outputs the flying object detection information in the form of a detection result reflection image superimposed on the captured image captured by the imaging device. (Supplementary Note 5) The flying object detection device described in Supplementary Note 1, wherein the output unit further outputs a detection result image displaying information obtained by signal processing of the radio detection signal output from the radio detection device and the captured image captured by the imaging device side by side for display on a display device. (Supplementary Note 6) The flying object detection device described in Supplementary Note 2, wherein the capturing direction control unit further variably controls the capturing direction of the imaging device so as to capture an image of the target flying object being tracked by the tracking unit, and the output unit outputs the captured image while the target flying object is being tracked. (Supplementary Note 7) The flying object detection device described in Supplementary Note 2, wherein the detection unit detects a target flying object from the captured image as a detection candidate, and further determines whether the detection candidate is the target flying object using flight trajectory information from the tracking unit.(Supplementary Note 8) An airborne object detection system comprising: a radio detection device that detects airborne objects in a detection area using radio waves, an imaging device that photographs the detection area, and an airborne object detection device that uses the radio detection device and the imaging device as described in Supplementary Note 1. (Supplementary Note 9) A method of detecting airborne objects, comprising: using a computer to detect airborne objects in the detection area by signal processing a radio detection signal output from the radio detection device that detects airborne objects in the detection area using radio waves; when an airborne object in the detection area is detected, setting the detected airborne object as an airborne object to be investigated; variably controlling the imaging direction of the imaging device that photographs the detection area so that the imaging device photographs the airborne object to be investigated; detecting the airborne object to be detected from the photographed image taken by the imaging device; and outputting airborne object detection information indicating that the airborne object to be detected has been detected. (Supplementary Note 10) A program storage medium that stores a computer program that causes a computer to execute the following processes: a process of detecting an aircraft in a detection area by signal processing a radio detection signal output from a radio detection device that uses radio waves to detect aircraft in the detection area; a process of designating the detected aircraft in the detection area as an aircraft to be investigated when the aircraft is detected in the detection area, and variably controlling the shooting direction of an imaging device that photographs the detection area so that the aircraft to be investigated is photographed by the imaging device; a process of detecting the aircraft to be detected from the photographed image taken by the imaging device; and a process of outputting aircraft detection information indicating that the aircraft to be detected has been detected.

[0064] Note that part or all of the configurations described in Supplementary Notes 2 to 7 that are dependent on Supplementary Note 1 above may also be dependent on Supplementary Notes 8, 9, and 10 in the same dependent relationship as Supplementary Notes 2 to 7. Furthermore, not limited to Supplementary Notes 1, 8, 9, and 10, part or all of the configurations described as Supplements may be made dependent on various hardware, software, various recording means for recording software, or systems, within the scope of each of the above-mentioned embodiments.

[0065] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0066] This application claims priority based on Japanese Patent Application No. 2024-042087, filed March 18, 2024, the disclosure of which is incorporated herein in its entirety by reference.

[0067] REFERENCE SIGNS LIST 1 Flying object detection system 2, 70 Radio wave detection device 3, 80 Photographing device 4 Drive device 5 Detecting device 7 Display device 52, 61 Detecting unit 53, 62 Photographing direction control unit 54, 63 Detecting unit 55, 64 Output unit 56 Tracking unit

Claims

1. A flying object detection device comprising: a detection means for detecting flying objects in a detection area by signal processing a radio detection signal output from a radio detection device that detects flying objects in the detection area using radio waves; a photographing direction control means for, when a flying object in the detection area is detected, designating the detected flying object as a flying object to be investigated and variably controlling the photographing direction of a photographing device that photographs the detection area so that the flying object to be investigated is photographed by the photographing device; a detection means for detecting the flying object to be detected from the photographed image taken by the photographing device; and an output means for outputting flying object detection information indicating that the flying object to be detected has been detected.

2. The flying object detection device according to claim 1, further comprising a tracking means for tracking the target flying object detected by the detection means using images captured by the imaging device.

3. The flying object detection device according to claim 1 or claim 2, wherein the detection means suspends the detection process for detecting the target flying object from the captured image when the shooting direction of the imaging device is variably controlled by the shooting direction control means due to the detection of the flying object using a radio detection signal.

4. A flying object detection device as described in any one of claims 1 to 3, wherein the output means outputs flying object detection information in the form of an image reflecting the detection result, superimposed on the image captured by the imaging device.

5. An air vehicle detection device as claimed in any one of claims 1 to 4, wherein the output means outputs a detection result image displaying information obtained by signal processing of the radio detection signal output from the radio detection device and an image captured by the imaging device so that the detection result image and the captured image are displayed side by side on a display device.

6. The flying object detection device according to claim 2, wherein the photographing direction control means further variably controls the photographing direction of the photographing device so as to photograph the target flying object being tracked by the tracking means, and the output means outputs the photographed image while the target flying object is being tracked.

7. The flying object detection device according to claim 2, wherein the detection means detects the target flying object from the captured image as a detection candidate, and further determines whether the detection candidate is the target flying object using flight trajectory information from the tracking means.

8. An aircraft detection system comprising: a radio wave detection device that uses radio waves to detect aircraft in a detection area; an imaging device that images the detection area; and an aircraft detection device described in any one of claims 1 to 7 that uses the radio wave detection device and the imaging device.

9. A method of detecting flying objects in a detection area, comprising: detecting flying objects in the detection area by signal processing a radio detection signal output from a radio detection device that uses radio waves to detect flying objects in the detection area; when an flying object in the detection area is detected, designating the detected flying object as an flying object to be investigated; variably controlling the photographing direction of a photographing device that photographs the detection area so that the photographing device photographs the flying object to be investigated; detecting the flying object to be detected from the photographed image taken by the photographing device; and outputting flying object detection information indicating that the flying object to be detected has been detected.

10. A program storage medium that stores a computer program that causes a computer to execute the following processes: a process of detecting an aerial vehicle in a detection area by signal processing a radio detection signal output from a radio detection device that uses radio waves to detect an aerial vehicle in the detection area; a process of designating the detected aerial vehicle as an aerial vehicle to be investigated when an aerial vehicle in the detection area is detected, and variably controlling the shooting direction of a photographing device that photographs the detection area so that the photographing device photographs the aerial vehicle to be investigated; a process of detecting the aerial vehicle to be detected from the photographed image taken by the photographing device; and a process of outputting aerial vehicle detection information indicating that the aerial vehicle to be detected has been detected.

11. The flying object detection method according to claim 9, wherein the computer further tracks the detected target flying object using images captured by the imaging device.

12. A method for detecting flying objects as described in claim 9 or claim 11, wherein the computer suspends the detection process for detecting the target flying object from the captured image when the computer is variably controlling the shooting direction of the imaging device due to the detection of the flying object using a radio detection signal.

13. A method for detecting flying objects as described in claim 9, claim 11 or claim 12, wherein the computer outputs the flying object detection information in the form of an image reflecting the detection result superimposed on the image captured by the imaging device.

14. A method for detecting flying objects as claimed in any one of claims 9 or 11 to 13, wherein the computer outputs a detection result image and a captured image so that the detection result image, which displays information obtained by signal processing of the radio detection signal output from the radio detection device, and the captured image by the photographing device are displayed side by side on a display device.

15. The flying object detection method according to claim 11, wherein the computer further variably controls the photographing direction of the photographing device so as to photograph the target flying object being tracked, and outputs the photographed image while tracking the target flying object.

16. A program storage medium according to claim 10, further storing a computer program that causes a computer to execute a process of tracking a detected target flying object using images captured by the imaging device.

17. A program storage medium as claimed in claim 10 or claim 16, further storing a computer program that causes a computer to execute a process to interrupt the detection process of detecting the target flying object from the captured image when the shooting direction of the shooting device is variably controlled due to the detection of the flying object using a radio detection signal.

18. A program storage medium as claimed in claim 10, claim 16 or claim 17, further storing a computer program that causes a computer to execute a process of outputting flying object detection information in the form of an image reflecting the detection results superimposed on an image captured by the imaging device.

19. A program storage medium as claimed in claim 10 or any one of claims 16 to 18, further storing a computer program that causes a computer to execute a process for outputting a detection result image and a captured image so that the detection result image, which displays information obtained by signal processing of the radio detection signal output from the radio detection device, and the captured image by the capturing device are displayed side by side on a display device.

20. A program storage medium as claimed in claim 16, further storing a computer program that causes a computer to execute the following processes: a process of variably controlling the photographing direction of the photographing device so as to photograph the target flying object being tracked; and a process of outputting the photographed image while the target flying object is being tracked.

Images