Flying body detecting device, flying body detecting system, flying body detecting method, and program storage medium

Abstract

In order to prevent the detection accuracy of a flying body in a detection region from decreasing due to the distance of said flying body from an imaging device that captures an image of the detection region, this flying body detecting device comprises the following functions. The flying body detecting device acquires a telephoto image, which is a captured image captured by, from among a plurality of types of imaging devices having different angles of view installed at a common installation location for capturing images of a detection region, a telephoto-type imaging device for capturing an image of a distant portion distant from the installation location in the detection region. The flying body detecting device acquires a wide-angle image, which is a captured image captured by a wide-angle-type imaging device from among the plurality of types of imaging devices. The flying body detecting device detects, from the telephoto image, a detection target flying body in a distant portion in the detection region, and detects, from the wide-angle image, a detection target flying body in a detection region portion other than the distant portion. The flying body detecting device generates a detection result reflecting image by superimposing, on the wide-angle image, information indicating the position of the detected flying body in the image.

Description

Flying object detection device, flying object detection system, flying object detection method, and program storage medium

[0001] The present disclosure relates to an airborne object detection device, an airborne object detection system, an airborne object detection method, and a program storage medium for detecting an airborne object.

[0002] No-fly zones (for example, the airspace above airports and important facilities and their surrounding areas) are designated for unmanned aerial vehicles (small unmanned aircraft), also known as drones or UAVs (Unmanned Aerial Vehicles), and flight permission is required for unmanned aircraft to fly in these no-fly zones. However, with the increasing use of unmanned aircraft, there are concerns that an increasing number of unmanned aircraft will enter these no-fly zones without permission.

[0003] Patent Document 1 (JP 2007-116666 A) discloses a technology for efficiently monitoring a monitoring area spanning several kilometers or more and accurately capturing and photographing moving objects within the monitoring area. That is, in the technology disclosed in Patent Document 1, the entire monitoring area is photographed with a wide-angle camera. When a moving object is detected in an image captured by the wide-angle camera, the attitude of the telephoto camera is controlled to point the optical axis of the telephoto camera in the direction of the moving object. That is, the telephoto camera photographs the moving object while tracking it.

[0004] Japanese Patent Application Laid-Open No. 2007-116666

[0005] In the technology disclosed in Patent Document 1, the presence or absence of moving objects in the entire monitoring area is first detected from images captured by a wide-angle camera. Therefore, even if a moving object moves to a part of the monitoring area far from the wide-angle camera, the moving object may not be detected properly because the distant moving object appears small in the image captured by the wide-angle camera. In other words, the accuracy of detecting moving objects decreases in parts of the monitoring area far from the camera.

[0006] The present disclosure has been devised to solve such problems. That is, a main purpose of the present disclosure is to provide a technology that prevents the detection accuracy of an airborne object in a detection area from decreasing due to the distance from an imaging device that images the detection area.

[0007] In order to achieve the above-mentioned object, one aspect of the flying object detection device of the present disclosure comprises: an acquisition unit that acquires telephoto images, which are images taken by a telephoto type imaging device that captures a distant portion of the detection area that is far from the installation location, among multiple types of imaging devices with different angles of view that are installed in a common installation location for photographing the detection area, and wide-angle images, which are images taken by a wide-angle type imaging device among the multiple types of imaging devices; a detection unit that detects flying objects to be detected in the distant portion of the detection area from the telephoto image, and detects flying objects to be detected in portions of the detection area other than the distant portion from the wide-angle image; a generation unit that generates an image reflecting the detection result by superimposing information representing the position of the detected flying object in the image on the wide-angle image; and an output unit that outputs the image reflecting the detection result.

[0008] Furthermore, in one aspect, the flying object detection system of the present disclosure comprises: a telephoto type imaging device, among multiple types of imaging devices with different angles of view installed at a common installation location for photographing a detection area, that photographs a distant portion of the detection area that is far from the installation location; a wide-angle type imaging device, among the multiple types of imaging devices, that photographs an area including a portion of the detection area that is not photographed by the telephoto type imaging device; and the above-mentioned flying object detection device that uses images photographed by the telephoto type imaging device and the wide-angle type imaging device, respectively.

[0009] Furthermore, in one aspect, the flying object detection method of the present disclosure includes, by a computer, acquiring a telephoto image, which is an image taken by a telephoto type imaging device that captures a distant portion of the detection area that is far from the installation location, among multiple types of imaging devices with different angles of view that are installed in a common installation location for photographing the detection area, and a wide-angle image, which is an image taken by a wide-angle type imaging device among the multiple types of imaging devices, detecting a flying object to be detected in the distant portion of the detection area from the telephoto image, detecting a flying object to be detected in a portion of the detection area other than the distant portion from the wide-angle image, superimposing information representing the position of the detected flying object in the image on the wide-angle image, and generating an image reflecting the detection result, and outputting the image reflecting the detection result.

[0010] Furthermore, in one aspect, the program storage medium of the present disclosure stores a computer program that causes a computer to execute the following processes: a process of acquiring a telephoto image, which is an image taken by a telephoto type imaging device that captures a distant portion of the detection area that is far from the installation location, among multiple types of imaging devices with different angles of view that are installed in a common installation location for photographing the detection area, and a wide-angle image, which is an image taken by a wide-angle type imaging device among the multiple types of imaging devices; a process of detecting a target flying object in the distant portion of the detection area from the telephoto image, and detecting a target flying object in a portion of the detection area other than the distant portion from the wide-angle image; a process of generating an image reflecting the detection result by superimposing information representing the position of the detected flying object in the image on the wide-angle image; and a process of outputting the image reflecting the detection result.

[0011] According to the present disclosure, it is possible to prevent the detection accuracy of an airborne object in a detection area where the airborne object is detected from decreasing due to the distance from an imaging device that images the detection area.

[0012] FIG. 2 is a diagram illustrating the configuration of an embodiment of the flying object detection device in the present disclosure. FIG. 3 is a diagram illustrating an imaging device constituting the flying object detection system in the present disclosure. FIG. 4 is a diagram illustrating the imaging device together with FIG. 2. FIG. 5 is a diagram illustrating an example of an image reflecting a detection result. FIG. 6 is a diagram illustrating another example of an image reflecting a detection result. FIG. 7 is a diagram illustrating yet another example of an image reflecting a detection result. FIG. 8 is a flowchart illustrating an example of operation related to flying object detection in the flying object detection device. FIG. 9 is a diagram illustrating a modified example of the generation unit in the flying object detection device. FIG. 10 is a diagram illustrating other embodiments. FIG. 11 is a diagram illustrating another other embodiment. FIG. 12 is a diagram illustrating yet another other embodiment of the flying object detection device. FIG. 13 is a flowchart illustrating another example of operation related to flying object detection in the flying object detection device.

[0013] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0014] 1 , an airborne object detection system 1 according to a first embodiment of the present disclosure is a system that detects an airborne object as a detection target in a detection area using images captured by an image capture device, and includes multiple types of image capture devices 2 and 3 and an airborne object detection device (hereinafter also simply referred to as a detection device) 5. The detection device 5 is a computer device that detects airborne objects in the detection area by performing image analysis on the images captured by the image capture devices 2 and 3.

[0015] In the first embodiment, the flying object detection system 1 is applied to a surveillance system. The surveillance system to which the flying object detection system 1 is applied is, for example, a system that monitors the airspace above and surrounding areas of important facilities such as airports and nuclear power plants, and the surveillance area includes no-fly zones where the flight of unmanned aerial vehicles is prohibited by law, etc. Since the flying object detection system 1 is applied to such a surveillance system, the detection area of ​​the flying object detection system 1 is the surveillance area of ​​the surveillance system to which it is applied.

[0016] The target air vehicles to be detected by the air vehicle detection system 1 are determined in advance by a system designer or the like, taking into consideration, for example, the type of facility monitored by the applicable monitoring system and its surrounding environment. Examples of air vehicles set as the target of detection include unmanned aerial vehicles (unmanned aerial vehicles) that can be flown by remote control or automatic pilot, such as unmanned airplanes, unmanned rotorcraft, and unmanned airships, as well as those flown by humans using specific aviation equipment, such as balloons, hang gliders, and paragliders. In this way, various air vehicles can be set as the target of detection, but in the first embodiment, a so-called drone (unmanned aerial vehicle) is set as the target of detection.

[0017] The imaging devices constituting the flying object detection system 1 are of multiple types with different angles of view. One type of imaging device employed in the flying object detection system 1 is a telephoto type imaging device equipped with a telephoto lens. Another type of imaging device employed in the flying object detection system 1 is a wide-angle type imaging device equipped with a wide-angle lens. The lenses equipped in the imaging devices can be classified into three types depending on the angle of view: a "standard lens," a "wide-angle lens," and a "telephoto lens." A "standard lens" is a lens with an angle of view of approximately 45 to 50 degrees, which is said to be close to the human field of view. A "wide-angle lens" is a lens with a wider angle of view than a "standard lens," for example, 60 degrees or more. A "telephoto lens" is a lens with a narrower angle of view than a "standard lens," for example, 30 degrees or less.

[0018] In the following description, the wide-angle type imaging device will also be referred to as a wide-angle camera 2, and the telephoto type imaging device will also be referred to as a telephoto camera 3.

[0019] In the flying object detection system 1, the wide-angle camera 2 and the telephoto camera 3 are installed in a common installation location where they can capture images of the detection area, with the capture direction fixed. For example, the wide-angle camera 2 and the telephoto camera 3 are installed at a height H(y) of approximately 10 meters, allowing them to view the detection area as shown in FIG. 2 . FIG. 2 shows a schematic diagram of an example of the capture range of each detection area of ​​the wide-angle camera 2 and the telephoto camera 3 installed in this manner, viewed from a direction along the ground surface. In FIG. 2 , the capture range portion of the field of view of the wide-angle camera 2 capturing the detection area is represented by the hatched portion Zw. Furthermore, the capture range portion of the detection area captured by the telephoto camera 3 is represented by the lightly shaded portion Zt. Note that unmanned aerial vehicles (e.g., drones) are subject to flight altitude restrictions, such as prohibiting flight in airspace above 150 meters above the ground. The example in FIG. 2 shows a detection area that takes into account the altitude restrictions for such flying objects (unmanned aerial vehicles).

[0020] Telephoto camera 3 captures an image of a distant portion of the detection area that is far from the installation location of telephoto camera 3. For example, the camera settings (magnification, etc.) of telephoto camera 3 are configured so that the image is captured of a distant portion of the detection area, such as a distance D(z) of about 800 meters from the installation location of telephoto camera 3. Here, the image captured by telephoto camera 3 is also referred to as a telephoto image. The telephoto image is used by detection device 5 to detect a flying object that is a detection target in the distant portion of the detection area.

[0021] Meanwhile, the telephoto camera 3 can capture a magnified image of a distant object. In other words, the field of view of the telephoto camera 3 is narrow. Therefore, the range of distant areas that the telephoto camera 3 can capture is limited. FIG. 3 shows a schematic diagram of an example of the capture ranges in the detection areas of the wide-angle camera 2 and the telephoto camera 3, as seen from the zenith. In FIG. 3, as in FIG. 2, the portion of the field of view of the wide-angle camera 2 that captures the detection area is represented by the hatched portion Zw. Furthermore, the portion of the detection area that is captured by the telephoto camera 3 is represented by the lightly shaded portion Zt.

[0022] As shown in FIG. 3 , a single telephoto camera 3 may not be able to capture the entire distant portion of the detection area. In this case, in the flying object detection system 1, multiple telephoto cameras 3 are installed in the same installation location, and the imaging directions of the multiple telephoto cameras 3 are set so that the imaging ranges of the telephoto cameras 3 are shifted and the entire distant portion is captured by the multiple telephoto cameras 3. In other words, the distant portion of the detection area is captured by multiple telephoto type imaging devices with mutually shifted fields of view. Note that in the example of FIG. 3 , the imaging directions of the multiple telephoto cameras 3 are set so that the imaging ranges are shifted in a direction along the ground surface. However, depending on the height H(d) of the detection area from the ground surface and the width of the field of view of the telephoto cameras 3, the imaging directions of the multiple telephoto cameras 3 may be set so that the imaging ranges are shifted in the vertical direction. Furthermore, the number of telephoto cameras 3 installed depends on the width of the distant portion of the detection area and the width of the field of view of the telephoto cameras 3, and is not limited.

[0023] Wide-angle camera 2 has a field of view that compensates for portions of the detection area that are not captured by telephoto camera 3. Here, the image captured by wide-angle camera 2 is also referred to as a wide-angle image. The wide-angle image is used to detect flying objects in a portion of the detection area that is closer to the installation location than the distant portion captured by telephoto camera 3 (a detection area portion other than the distant portion (hereinafter also referred to as the portion covered by the wide-angle camera)). For this reason, the camera settings of wide-angle camera 2 are configured so that flying objects in the portion of the detection area covered by the wide-angle camera are clearly captured. Note that, depending on the size of the detection area, multiple wide-angle cameras 2 may be installed in the same installation location, and the shooting directions of these wide-angle cameras 2 may be set so that the shooting ranges of these wide-angle cameras 2 are shifted and the entire detection area is captured by the multiple wide-angle cameras 2.

[0024] Assume that the wide-angle camera 2 and the telephoto camera 3 capture images of the same size of target flying objects flying in the wide-angle camera's portion and the distant portion of the detection area. In this target flying object detection system 1, the cameras are configured so that the difference between the size of the target flying object captured in the telephoto image and the size of the target flying object captured in the wide-angle image falls within a predetermined range (tolerance). Preferably, the cameras are configured so that the size of the target flying object captured in the telephoto image is similar to the size of the target flying object captured in the wide-angle image. This camera configuration reduces the load of the detection process for the target flying object in the detection device 5. Taking into consideration the reduction of the load of the detection process, a tolerance range related to the difference in size of the target flying object captured in the telephoto image and the wide-angle image is determined.

[0025] The detection device 5 is a computer device that detects the target flying object by analyzing the wide-angle image captured by the wide-angle camera 2 and the telephoto image captured by the telephoto camera 3. That is, as shown in FIG. 1 , the detection device 5 is directly or indirectly connected to each of the wide-angle camera 2 and the telephoto camera 3, and includes a calculation device 50 and a storage device 40. The storage device 40 includes a storage medium for storing data and a computer program (hereinafter also referred to as a program) 41. There are multiple types of storage devices, such as magnetic disk drives and semiconductor memory devices. Furthermore, there are multiple types of semiconductor memory devices, such as RAM (Random Access Memory) and ROM (Read Only Memory). A computer device may be equipped with multiple types of storage devices depending on their intended use, but these storage devices will be collectively referred to as the storage device 40 herein without distinction. Furthermore, the type and number of storage devices 40 included in the detection device 5 are not limited, and a description thereof will be omitted. The detection device 5 may also be connected to a database 6, which is a storage device. In this case, the detection device 5 may write information to the database 6 or read information from the database 6, but in order to avoid complicating the explanation, we will omit the explanation of such cases here.

[0026] The arithmetic device 50 is configured with a processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The arithmetic device 50 can have functions based on a program 41 by reading and executing the program 41 stored in the storage device 40. Here, the arithmetic device 50 has an acquisition unit 51, a detection unit 52, a generation unit 53, and an output unit 54 as functional units related to detecting an air vehicle.

[0027] The acquisition unit 51 acquires wide-angle images captured by the wide-angle camera 2 and telephoto images captured by the telephoto camera 3. Here, the method by which the acquisition unit 51 acquires the wide-angle images and telephoto images is not limited, and for example, the acquisition unit 51 may acquire captured images (wide-angle images and telephoto images) from the wide-angle camera 2 and the telephoto camera 3, respectively, or may acquire wide-angle images and telephoto images from the wide-angle camera 2 and the telephoto camera 3 by reading them from a database (not shown) that has temporarily stored the wide-angle images and telephoto images.

[0028] The detection unit 52 detects target flying objects in the distant part of the detection area from the telephoto image and detects target flying objects in the part of the detection area covered by the wide-angle camera (the part of the detection area other than the distant part) from the wide-angle image. Various methods for detecting target flying objects from captured images (wide-angle images or telephoto images) have been proposed, and one example, although not limited thereto, is a detection method using AI (artificial intelligence) technology. In this case, a detection model generated by AI technology is used. This detection model is generated by learning images of the target flying object as training data. Input information to the detection model is the captured image (wide-angle image or telephoto image), and output information from the detection model includes information indicating the presence or absence of the target flying object in the input captured image and, if the target flying object is detected, information indicating the position of the detected flying object in the captured image (hereinafter also referred to as detection position information).

[0029] As described above, in this flying object detection system 1, the wide-angle camera 2 and the telephoto camera 3 are configured so that when a flying object of the same size is photographed in the wide-angle camera portion and the distant portion of the detection area, the size of the image of the flying object captured in each of the wide-angle and telephoto images falls within a predetermined range (tolerance). In other words, the cameras are configured so that the image analysis of the wide-angle image and the telephoto image can be performed using the same detection model. As a result, in the detection process by the detection unit 52, the same detection model is used for the image analysis of the wide-angle image and the telephoto image. Note that, if multiple types of flying objects are set as the flying object to be detected, a detection model that can distinguish and detect each of the multiple types of flying objects is generated by learning images of those multiple types of flying objects. The detection unit 52 may use this detection model. In this case, the detection unit 52 can also output information indicating the type of flying object detected.

[0030] The detection unit 52 uses such a detection model to detect the target flying object from each of the wide-angle image and the telephoto image. Furthermore, if the detection unit 52 detects a flying object from the telephoto image, the flying object should also appear in the wide-angle image, and therefore the detection unit 52 associates the flying object between the wide-angle image and the telephoto image. This process uses positional relationship data between the telephoto image and the wide-angle image, which associates image portions that show the same real space. This data is generated in advance and stored in the storage device 40.

[0031] Information representing the detection results by the detection unit 52 is stored in the storage device 40 in association with, for example, information identifying the photographing device (wide-angle camera 2 or telephoto camera) that captured the captured image that has been subjected to the detection process, information representing the frame number of the captured image that has been subjected to the detection process, and information on the time of capture.

[0032] As described above, in the flying object detection system 1, the detection process for the target flying object is performed using the same detection model for both wide-angle images and telephoto images. Therefore, compared to when wide-angle images and telephoto images are processed using different detection models, the processing load on the detection device 5 can be reduced and the time required for the detection process for the flying object can be shortened. This contributes to real-time detection of flying objects using images captured by the wide-angle camera 2 and telephoto camera 3.

[0033] The generation unit 53 generates a detection result reflection image. The detection result reflection image is an image in which information representing the position of the detected flying object in the image is superimposed on the wide-angle image. Here, the information representing the position of the detected flying object in the image is represented, for example, by a graphic. The graphic representing the position of the detected flying object in the image (hereinafter also referred to as the flying object detection graphic) is not limited here and may be, for example, a circle, triangle, or rectangle, a symbol, or a mark resembling an flying object, and is set appropriately by a system designer, etc. Information used in the process of generating the detection result reflection image includes information on the position of the flying object in the wide-angle image detected from the wide-angle image (i.e., detection position information) and information on the position of the flying object in the wide-angle image detected from the telephoto image by a process of correlating the telephoto image and the wide-angle image. Furthermore, the wide-angle image and the telephoto image are synchronized so that their capture times match. As a result, the flying object detection graphic representing the position of the flying object in the image detected from the telephoto image is superimposed on the wide-angle image captured at the same capture time as the telephoto image in which the flying object was detected.

[0034] 4 and 5 each show an example of a detection result reflection image. In the example of FIG. 4, the detection result reflection image is an image in which a flying object detection graphic 8, which is a double circle, is superimposed on a wide-angle image. The example of FIG. 5 is an example of a detection result reflection image when multiple types of flying objects are detected. The detection result reflection image is an image in which a star-shaped flying object detection graphic 8 corresponding to the type of detected flying object and a double circle flying object detection graphic 8 corresponding to another type of detected flying object are superimposed on a wide-angle image. Note that in FIGS. 4 and 5, the detected flying object is indicated by the symbol "9." Furthermore, when multiple types of flying objects are detected, the same flying object detection graphic 8 may be superimposed on the wide-angle image regardless of the type of the detected flying object.

[0035] In addition, the alert level may vary depending on the flight location and type of the detected flying object. In such cases, the type of flying object detection graphic 8 corresponding to the alert level may be predetermined, and the generation unit 53 may generate an image reflecting the detection result by superimposing the type of flying object detection graphic 8 corresponding to the alert level on the wide-angle image. Here, the alert level is calculated, for example, by the detection unit 52. That is, when different alert levels are set depending on the distance from the installation location of the wide-angle camera 2 and the telephoto camera 3 (camera installation location), data relating to the distance from the camera installation location to the flying object and the alert level is stored in the storage device 40 as alert level determination information used to determine the alert level. When the alert level is set depending on the type of flying object to be detected, data relating to the type of flying object and the alert level is stored in the storage device 40 as alert level determination information. When the alert level is set based on the combination of the distance from the camera installation location to the flying object and the type of flying object, data relating to the alert level is stored in the storage device 40 as alert level determination information. When a target flying object is detected, the detection unit 52 calculates the alert level of the detected flying object using one or both of the distance from the camera installation location to the detected flying object and the type of flying object, and the alert level determination information stored in the storage device 40. The generation unit 53 then generates an image reflecting the detection results by superimposing an flying object detection graphic 8 of a type corresponding to the calculated alert level on the wide-angle image as described above. Note that the method for calculating the distance between the detected flying object and the camera installation location from the wide-angle image or telephoto image is not limited here, and therefore a description thereof will be omitted.

[0036] Furthermore, the image reflecting the detection result generated by the generation unit 53 may further include text indicating information such as the type of aircraft detected and the alert level.

[0037] The output unit 54 outputs the generated detection result reflection image. An example of an output destination is the display device 7 shown in FIG. 1. The display device 7 is a device that notifies (provides) information by displaying the information on a screen using text and images. Here, the display device 7 receives the detection result reflection image from the output unit 54 and provides the detection result reflection image to, for example, a user of the flying object detection system (monitoring system). Note that the display device 7 may be, for example, a display device of a terminal device carried by a user of the flying object detection system (monitoring system).

[0038] In this case, the captured images (wide-angle image, telephoto image) output by the wide-angle camera 2 and the telephoto camera 3 are moving images, and the detection process for the flying object is performed using frame images selected from the multiple frame images that make up the captured images. The detection result-reflecting image generated by reflecting the results of this detection process is, for example, included in the moving image, which is a wide-angle image, by replacing the frame image of the original wide-angle image, and is output to the output destination by the output unit 54. The display device 7 displays the moving image (wide-angle image) output by the output unit 54 in this manner. In other words, the output unit 54 can be said to control the display operation of the display device 7.

[0039] For example, when a user inputs a request (enlarged display request) to the detection device 5 to enlarge an image of the detected flying object on the display device 7, the output unit 54 may perform display control to enlarge the flying object 9 in response to the request, as shown in FIG. 6 . The method by which the user inputs the enlarged display request is not limited here, but an example of such a method is to input the enlarged display request to the detection device 5 by clicking the flying object detection figure 8 with a cursor 71 as shown in FIG. 6 . The manner in which the flying object is enlarged is not limited here and may be an appropriately set display manner. For example, as shown in FIG. 6 , a window image in which the flying object is enlarged may be superimposed on the wide-angle image, or the wide-angle image and the enlarged display image of the flying object may be displayed side by side.

[0040] The detection device 5 of the first embodiment is configured as described above. Next, an example of the operation of the detection device 5 to detect an airborne object will be described with reference to Fig. 7. Fig. 7 is a flowchart illustrating an example of the operation of the detection device 5 to detect an airborne object. Fig. 7 can also be considered to be a diagram illustrating a method for detecting an airborne object in the detection device 5.

[0041] For example, it is assumed that the storage device 40 in the detection device 5 stores various information (data) used for processing by the aforementioned arithmetic unit (processor) 50. The acquisition unit 51 of the detection device 5 acquires wide-angle images captured by the wide-angle camera 2 and telephoto images captured by the telephoto camera 3 (step 101 in FIG. 7). In the following description, it is assumed that the wide-angle images and telephoto images are moving images.

[0042] The detection unit 52 executes a detection process for detecting the target flying object for each of the acquired wide-angle image and telephoto image (step 102). Here, the detection process is executed on frame images selected for each predetermined number of frames from among the frame images constituting each of the wide-angle image and the telephoto image.

[0043] Then, the detection unit 52 determines whether the target flying object has been detected in at least one of the wide-angle image and the telephoto image through the detection process (step 103). If the target flying object has been detected, the detection unit 52 calculates the position of the image of the detected flying object in the wide-angle image or the telephoto image in which the flying object was detected as detected photographing information. Furthermore, if the photographed image in which the flying object was detected is a telephoto image, the detection unit 52 calculates the position of the flying object detected from the telephoto image in the wide-angle image through a process of associating the telephoto image with the wide-angle image.

[0044] Thereafter, the generation unit 53 uses information from the detection process performed by the detection unit 52 to superimpose an air vehicle detection graphic 8 representing the position of the detected air vehicle in the image onto the wide-angle image, thereby generating an image reflecting the detection result (step 104). The output unit 54 then outputs the image reflecting the detection result (step 105). Here, since the wide-angle image is a moving image, the output unit 54 outputs the wide-angle image including the detection result reflection image (frame image) to, for example, the display device 7. That is, the above-described air vehicle detection process is sequentially performed on each of a plurality of frame images selected from a plurality of frame images in each of the wide-angle image and the telephoto image. When the detection unit 52 detects an air vehicle, the generation unit 53 generates the image reflecting the detection result. When the detection result reflection image (frame image) is generated, the output unit 54 outputs the wide-angle image including the detection result reflection image to, for example, the display device 7.

[0045] On the other hand, if the flying object to be detected is not detected from the selected frame image, the flying object detection process for that frame image is terminated, and preparations are made for the flying object detection process for the next frame image to be selected.

[0046] As described above, the detection device 5 of the first embodiment and the flying object detection system 1 including the detection device 5 use telephoto images to detect target flying objects in the distant portion of the detection area. The telephoto images can capture the target flying object at a detectable size even if it is far from the camera installation location. Therefore, by using telephoto images, the flying object detection system 1 can prevent a situation in which the target flying object captured in the captured image is too small to be detected due to its distance from the camera installation location, resulting in the flying object being missed.

[0047] Furthermore, the flying object detection system 1 uses wide-angle images to detect target flying objects in the detection area other than the distant portion. A wide-angle type imaging device can capture parts of the detection area that are not clearly visible in a telephoto image (in other words, parts of the detection area other than the distant portion). By using wide-angle images captured by such a wide-angle type imaging device, the flying object detection system 1 can detect target flying objects in the detection area other than the distant portion while preventing missed detections.

[0048] Therefore, by using wide-angle images and telephoto images, the detection device 5 and the flying object detection system 1 equipped with the detection device 5 can prevent the detection accuracy of flying objects in the detection area from decreasing due to the distance from the imaging device that photographs the detection area.

[0049] <Other Embodiments> The present disclosure is not limited to the first embodiment and may be embodied in various ways. For example, in the first embodiment, the output unit 54 outputs a detection result reflection image based on a wide-angle image. In addition, for example, when a user of the flying object detection system 1 requests the provision of a telephoto image, the output unit 54 may output the telephoto image to the requestor in response to the request. In this case, for example, the output unit 54 outputs a telephoto image synchronized with the detection result reflection image to be output. The telephoto image output in this manner may be displayed on the display device 7 in place of the detection result reflection image, or the detection result reflection image and the telephoto image may be displayed side by side on the display device 7.

[0050] Furthermore, when a telephoto image is output, the generation unit 53 may generate a telephoto image with the following information. In other words, information on the detection results detected from the wide-angle image cannot be known by simply looking at the telephoto image. A telephoto image with information is a telephoto image on which the detection results detected from such a wide-angle image are superimposed. The form of the information representing the detection results from the wide-angle image superimposed on the telephoto image may be, for example, text as shown in FIG. 8 or an image of an aircraft detected from the wide-angle image. Such a telephoto image with information can also provide information on the detection results from the wide-angle image that is not captured in the telephoto image.

[0051] Furthermore, in the example of Fig. 3, two telephoto cameras 3 are used to capture images of the distant part of the detection area, but depending on the size of the distant part of the detection area and the field of view of the telephoto cameras 3, three or more telephoto cameras 3 may be used to capture images of the distant part, as shown in Fig. 9. Similar to Fig. 3, Fig. 9 is a schematic diagram showing an example of the capture ranges in the respective detection areas of the wide-angle camera 2 and the telephoto camera 3, as viewed from the zenith.

[0052] Furthermore, although the first embodiment illustrates an example in which the flying object detection system is applied to a surveillance system, application of the flying object detection system of the present disclosure is not limited to surveillance systems. For example, in a traffic control system for unmanned aircraft such as drones in an area surrounding a logistics hub, the flying object detection system of the present disclosure may be applied to detect unmanned aircraft that are subject to management as flying objects to be detected.

[0053] Furthermore, in the first embodiment, an air vehicle that is a man-made object is used as an example of an air vehicle to be detected. However, for example, a bird may be set as the air vehicle to be detected. That is, if a bird collides with an aircraft, a serious accident such as the aircraft crashing may occur. If a bird flies near an aircraft, such a risk may arise, and there is a concern that the operation of the unmanned aircraft may be hindered. For this reason, for example, when the air vehicle detection system of the present disclosure is applied to an unmanned aircraft traffic management system, birds may also be detected as the air vehicle to be detected. Then, when a bird (air vehicle to be detected) is detected around a managed aircraft that is the air vehicle to be detected, information indicating that a bird is flying nearby and that there is a risk posed by the bird may be superimposed on the wide-angle image, thereby generating and outputting an image reflecting the detection result, such as that shown in FIG. 10 .

[0054] Furthermore, in the first embodiment, the output unit 54 outputs an image reflecting the detection result to the display device 7 using the generation unit 53. In addition, when the detection unit 52 detects an airborne object, the output unit 54 may output information indicating that the airborne object has been detected (airborne object detection information) to a predetermined notification destination other than the display device. One example of the notification destination is a computer device of a monitoring system to which the airborne object detection system is applied. Examples of the airborne object detection information include text information of a message indicating that an airborne object has been detected, and control information for an alarm sound to notify the user of the detection of the airborne object.

[0055] Furthermore, the flying object detection device may also have a configuration such as that shown in FIG. 11 . That is, the flying object detection device 10 is, for example, a computer device, and includes an acquisition unit 11, a detection unit 12, a generation unit 13, and an output unit 14 as functional units realized by executing a computer program. The acquisition unit 11 acquires telephoto images, which are images captured by a telephoto-type imaging device that captures a distant portion of the detection area that is far from the installation location, among multiple types of imaging devices with different angles of view installed at a common installation location for capturing images of the detection area. The acquisition unit 11 also acquires wide-angle images, which are images captured by a wide-angle-type imaging device among the multiple types of imaging devices.

[0056] The detection unit 12 detects the target flying object in the distant part of the detection area from the telephoto image, and also detects the target flying object in the part of the detection area other than the distant part from the wide-angle image.

[0057] The generation unit 13 generates an image reflecting the detection result by superimposing information indicating the position of the detected flying object in the image on the wide-angle image. The output unit 14 outputs the image reflecting the detection result. Note that the acquisition unit 51, detection unit 52, generation unit 53, and output unit 54 of the detection device 5 in the first embodiment described above are examples of the acquisition unit 11, detection unit 12, generation unit 13, and output unit 14.

[0058] The flying object detection device 10 has the above-described configuration. The flying object detection device 10 constitutes a flying object detection system together with the image capture devices 20 and 30 as shown by the dotted lines in FIG.

[0059] Next, an example of the operation of the flying object detection device 10 will be described with reference to Fig. 12. Fig. 12 is a flowchart explaining an example of the operation of the flying object detection device 10. Fig. 12 can also be said to be a diagram explaining an example of a flying object detection method of the flying object detection device 10.

[0060] For example, when the acquisition unit 11 acquires a wide-angle image and a telephoto image (step 201), the detection unit 12 executes a detection process to detect a target flying object from each of the wide-angle image and the telephoto image (step 202). If the target flying object is detected, the generation unit 13 generates an image reflecting the detection result by superimposing information representing the position of the detected flying object in the image on the wide-angle image (step 203). The output unit 14 outputs the generated image reflecting the detection result (step 204).

[0061] As described above, the flying object detection device 10 performs detection processing of the target flying object using a telephoto image (i.e., an image captured by a telephoto type imaging device) for the distant part of the detection area. Therefore, even if the target flying object is flying in the distant part and located far from the installation location of the imaging device, the flying object detection device 10 can prevent the target flying object from being missed due to appearing small in the captured image because the target flying object is enlarged and captured in the telephoto image.

[0062] Furthermore, for detection area portions other than the distant portion (in other words, detection area portions closer to the installation location of the camera than the distant portion), the flying object detection device 10 performs detection processing of the target flying object using wide-angle images, which are images captured by a wide-angle type camera. In other words, because a telephoto type camera focuses on the distant portion, detection area portions closer to the installation location of the camera than the distant portion are out of focus or have large blind spots, making it difficult to capture the flying object in the telephoto image. In contrast, a wide-angle type camera has a wide field of view that can compensate for the blind spots of a telephoto type camera, and by focusing on detection area portions closer to the installation location of the camera than the distant portion, it is possible to clearly capture the target flying object in that detection area. By using wide-angle images captured by such a wide-angle type imaging device, the flying object detection device 10 can compensate for the disadvantages of using telephoto images by using wide-angle images, thereby preventing the detection accuracy of flying objects in the detection area from decreasing due to the distance from the imaging device that captures the detection area.

[0063] Some or all of the above embodiments may be described as, but are not limited to, the following supplementary notes: [Supplementary Note 1] An flying object detection device comprising: an acquisition unit that acquires telephoto images, which are images captured by a telephoto-type imaging device that captures a distant portion of a detection area that is far from the installation location, among multiple types of imaging devices with different angles of view that are installed at a common installation location for capturing images of the detection area, and wide-angle images, which are images captured by a wide-angle-type imaging device among the multiple types of imaging devices; a detection unit that detects a detection target flying object in the distant portion of the detection area from the telephoto image, and detects a detection target flying object in a portion of the detection area other than the distant portion from the wide-angle image; a generation unit that generates an image reflecting the detection result by superimposing information indicating the position of the detected flying object in the image on the wide-angle image; and an output unit that outputs the detection result reflecting image. [Supplementary Note 2] The flying object detection device of Supplementary Note 1, wherein the distant portion of the detection area is photographed by a plurality of telephoto type imaging devices with fields of view offset from one another, the acquisition unit acquires telephoto images from each of the plurality of telephoto type imaging devices, and the detection unit detects the target flying object in the distant portion of the detection area using the telephoto images from each of the plurality of telephoto type imaging devices. [Supplementary Note 3] The flying object detection device of Supplementary Note 1, wherein the detection unit distinguishes between and detects a plurality of types of target flying objects. [Supplementary Note 4] The flying object detection device of Supplementary Note 1, wherein the generation unit further generates an information-attached telephoto image by superimposing information representing the detection result detected from the wide-angle image by the detection unit on the telephoto image. [Supplementary Note 5] The flying object detection device of Supplementary Note 1, wherein the output unit performs display control to enlarge and display the image of the detected flying object in response to a request to enlarge and display the image of the detected flying object on a display device. [Supplementary Note 6] The flying object detection device according to Supplementary Note 1, wherein the output unit outputs information indicating that the flying object has been detected to a predetermined notification destination other than the display device.[Supplementary Note 7] The flying object detection device described in Supplementary Note 1, wherein the acquisition unit acquires telephoto images and wide-angle images taken by the telephoto type imaging device and the wide-angle type imaging device, whose cameras are set so that when a flying object to be detected of the same size is captured in a telephoto image and a wide-angle image in a distant part of the detection area and in a part of the detection area other than the distant part, the difference in size of the flying object to be detected captured in the telephoto image and the wide-angle image falls within a predetermined range, and the detection unit performs flying object detection processing for each of the telephoto image and the wide-angle image using the same detection model generated by learning the captured image in which the flying object to be detected is captured. [Supplementary Note 8] An air vehicle detection system comprising: a telephoto type camera, among multiple types of camera devices with different angles of view installed at a common installation location for photographing a detection area, which captures a distant portion of the detection area that is far from the installation location; a wide-angle type camera, among the multiple types of camera devices, which captures images including portions of the detection area that are not photographed by the telephoto type camera device; and an air vehicle detection device according to Supplementary Note 1, which uses images photographed by the telephoto type camera device and the wide-angle type camera device, respectively. [Supplementary Note 9] A flying object detection method, using a computer, which acquires telephoto images, which are images taken by a telephoto type camera device that captures a distant part of the detection area that is far from the installation location, among multiple types of camera devices with different angles of view that are installed in a common installation location to capture images of the detection area, and wide-angle images, which are images taken by a wide-angle type camera device among the multiple types of camera devices, detects a flying object to be detected in the distant part of the detection area from the telephoto image, and detects a flying object to be detected in a part of the detection area other than the distant part from the wide-angle image, generates an image reflecting the detection result by superimposing information representing the position of the detected flying object in the image on the wide-angle image, and outputs the image reflecting the detection result.[Supplementary Note 10] A program storage medium storing a computer program that causes a computer to execute the following processes: a process of acquiring a telephoto image, which is an image taken by a telephoto type imaging device that images a distant part of a detection area that is far from the installation location, among multiple types of imaging devices with different angles of view that are installed in a common installation location for imaging the detection area, and a wide-angle image, which is an image taken by a wide-angle type imaging device among the multiple types of imaging devices; a process of detecting a target flying object in the distant part of the detection area from the telephoto image, and detecting a target flying object in a part of the detection area other than the distant part from the wide-angle image; a process of generating an image reflecting the detection result by superimposing information indicating the position of the detected flying object in the image on the wide-angle image; and a process of outputting the image reflecting the detection result.

[0064] Note that some or all of the configurations described in Supplementary Notes 2 to 7, which are dependent on Supplementary Note 1, may also be dependent on Supplementary Notes 8 to 10 in the same dependent relationship as Supplementary Notes 2 to 7. Furthermore, not limited to Supplementary Notes 1 and 8 to 10, some or all of the configurations described as Supplements may be made dependent on various hardware, software, various recording means for recording software, or systems, within the scope of each of the above-mentioned embodiments.

[0065] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0066] This application claims priority based on Japanese Patent Application No. 2024-015839, filed February 5, 2024, the disclosure of which is incorporated herein in its entirety by reference.

[0067] REFERENCE SIGNS LIST 1 Flying object detection system 2 Wide-angle camera 3 Telephoto camera 5 Detection device 10 Flying object detection device 11, 51 Acquisition unit 12, 52 Detection unit 13, 53 Generation unit 14, 54 Output unit

Claims

1. An air vehicle detection device comprising: an acquisition means for acquiring telephoto images, which are images taken by a telephoto type camera device that captures a distant part of the detection area that is far from the installation location, among multiple types of camera devices with different angles of view installed at a common installation location for photographing the detection area, and wide-angle images, which are images taken by a wide-angle type camera device among the multiple types of camera devices; a detection means for detecting an air vehicle to be detected in the distant part of the detection area from the telephoto image, and for detecting an air vehicle to be detected in a part of the detection area other than the distant part from the wide-angle image; a generation means for generating an image reflecting the detection result by superimposing information representing the position of the detected air vehicle in the image on the wide-angle image; and an output means for outputting the image reflecting the detection result.

2. The flying object detection device of claim 1, wherein the distant part of the detection area is photographed by a plurality of telephoto type imaging devices with mutually shifted fields of view, the acquisition means acquires telephoto images from each of the plurality of telephoto type imaging devices, and the detection means detects the target flying object in the distant part of the detection area using the telephoto images from each of the plurality of telephoto type imaging devices.

3. The flying object detection device according to claim 1 or claim 2, wherein the detection means detects each of a plurality of types of target flying objects separately.

4. A flying object detection device as claimed in any one of claims 1 to 3, wherein the generating means further generates an information-attached telephoto image by superimposing information representing the detection results detected from the wide-angle image by the detecting means onto the telephoto image.

5. A flying object detection device as claimed in any one of claims 1 to 4, wherein the output means performs display control to enlarge and display an image of the detected flying object in response to a request to enlarge and display the image of the flying object on a display device.

6. A flying object detection device as claimed in any one of claims 1 to 5, wherein the output means outputs information indicating that a flying object has been detected to a predetermined notification destination other than a display device.

7. A flying object detection device as claimed in any one of claims 1 to 6, wherein the acquisition means acquires telephoto and wide-angle images taken by the telephoto type imaging device and the wide-angle type imaging device, the cameras of which are set so that when a flying object to be detected of the same size is captured in a telephoto image and a wide-angle image in both the distant part of the detection area and the part of the detection area other than the distant part, the difference in size of the flying object to be detected captured in the telephoto and wide-angle images is within a predetermined range, and the detection means performs flying object detection processing for each of the telephoto and wide-angle images using the same detection model generated by learning the captured images in which the flying object to be detected is captured.

8. A flying object detection system comprising: a telephoto type camera, among multiple types of camera devices with different angles of view installed at a common installation location for photographing a detection area, which captures distant parts of the detection area that are far from the installation location; a wide-angle type camera, among the multiple types of camera devices, which captures images including parts of the detection area that are not photographed by the telephoto type camera device; and a flying object detection device described in any one of claims 1 to 7, which uses images photographed by the telephoto type camera device and the wide-angle type camera device, respectively.

9. A flying object detection method, which comprises: acquiring, by a computer, telephoto images which are images taken by a telephoto type camera device that captures a distant part of the detection area that is far from the installation location of multiple types of camera devices with different angles of view that are installed in a common installation location for photographing the detection area; and wide-angle images which are images taken by a wide-angle type camera device among the multiple types of camera devices; detecting a target flying object in the distant part of the detection area from the telephoto image; detecting a target flying object in a part of the detection area other than the distant part from the wide-angle image; generating an image reflecting the detection result by superimposing information representing the position of the detected flying object in the image on the wide-angle image; and outputting the image reflecting the detection result.

10. A program storage medium storing a computer program that causes a computer to execute the following processes: a process of acquiring a telephoto image, which is an image taken by a telephoto type camera that captures a distant part of the detection area that is far from the installation location, among multiple types of camera devices with different angles of view installed at a common installation location for capturing images of the detection area, and a wide-angle image, which is an image taken by a wide-angle type camera device among the multiple types of camera devices; a process of detecting a target flying object in the distant part of the detection area from the telephoto image, and detecting a target flying object in a part of the detection area other than the distant part from the wide-angle image; a process of generating an image reflecting the detection result by superimposing information representing the position of the detected flying object in the image on the wide-angle image; and a process of outputting the image reflecting the detection result.

11. A method for detecting flying objects as described in claim 9, wherein the distant portion of the detection area is photographed by a plurality of telephoto type imaging devices with mutually shifted fields of view, the computer further acquires telephoto images from each of the plurality of telephoto type imaging devices, and the computer detects the target flying object in the distant portion of the detection area using the telephoto images from each of the plurality of telephoto type imaging devices.

12. A method for detecting an aircraft as claimed in claim 9 or claim 11, wherein when detecting an aircraft to be detected, a plurality of types of aircraft to be detected are detected separately.

13. A method for detecting flying objects according to claim 9, claim 11 or claim 12, wherein the computer further generates an information-added telephoto image by superimposing information representing the detection results detected from the wide-angle image onto the telephoto image.

14. A flying object detection method as claimed in any one of claims 9 or 11 to 13, wherein the computer performs display control to enlarge and display an image of the detected flying object in response to a request to enlarge and display the image of the flying object on a display device.

15. A method for detecting an airborne object as claimed in claim 9 or any one of claims 11 to 14, wherein the computer further outputs information indicating that the airborne object has been detected to a predetermined notification destination other than a display device.

16. A program storage medium as described in claim 10, further storing a computer program that causes a computer to execute the following processes: the distant portion of the detection area is photographed by a plurality of telephoto type photographing devices with mutually shifted fields of view, and a process of acquiring telephoto images by each of the plurality of telephoto type photographing devices; and a process of detecting the target flying object in the distant portion of the detection area using the telephoto images by each of the plurality of telephoto type photographing devices.

17. A program storage medium as claimed in claim 10 or claim 16, wherein the process of detecting the target flying object detects each of a plurality of types of target flying object separately.

18. A program storage medium as claimed in claim 10, claim 16 or claim 17, further storing a computer program that causes a computer to execute a process of generating an information-added telephoto image by superimposing information representing the detection result detected from the wide-angle image onto the telephoto image in the process of generating the detection result-reflecting image.

19. A program storage medium as claimed in claim 10 or any one of claims 16 to 18, further storing a computer program that causes a computer to execute a process for performing display control to enlarge and display an image of a detected flying object in response to a request to enlarge and display the image of the flying object on a display device.

20. A program storage medium as claimed in claim 10 or any one of claims 16 to 19, further storing a computer program that causes a computer to execute a process of outputting information indicating that an air vehicle has been detected to a predetermined notification destination other than a display device.

Images