Flying body detection device, flying body detection system, flying body detection method, and program storage medium

Abstract

In order to detect a flying object to be detected in a monitoring area while suppressing the number of imaging devices to be installed, this flying object detection device has the following functions. The flying object detection device acquires captured images captured by imaging devices which each capture an image of a monitoring area. The flying object detection device extracts, from the captured images, image regions related to a plurality of air areas which are different in distances from the imaging devices, and generates an analysis target image by downsizing the image regions extracted from the captured images by downsizing degrees determined using the distances between the air areas related to the image regions and the imaging devices. The flying object detection device detects a flying object to be detected from the analysis target image. The flying object detection device generates a detection result reflection image by superimposing a graphic representing the position of the detected flying object on a captured image in which the flying object detected from the analysis target image is captured. The flying object detection device outputs the detection result reflection image.

Description

Flying object detection device, flying object detection system, flying object detection method, and program storage medium

[0001] The present disclosure relates to an airborne object detection device, an airborne object detection system, an airborne object detection method, and a program storage medium for detecting an airborne object.

[0002] No-fly zones (for example, the airspace above airports and important facilities and their surrounding areas) are designated for unmanned aerial vehicles (small unmanned aircraft), also known as drones or UAVs (Unmanned Aerial Vehicles), and flight permission is required for unmanned aircraft to fly in these no-fly zones. However, with the increasing use of unmanned aircraft, there are concerns that an increasing number of unmanned aircraft will enter these no-fly zones without permission.

[0003] Therefore, technologies for detecting unmanned aerial vehicles (UAVs) flying in no-fly zones have been studied. For example, Patent Literature 1 (JP 2018-101987 A) discloses a technology for detecting UAVs that appear in a monitored area using audio data collected by a microphone that picks up audio in the monitored area.

[0004] JP 2018-101987 A

[0005] One possible method for detecting unmanned aerial vehicles (UAVs) flying in prohibited airspace is to use images captured by a camera. For example, in a detection method using captured images, UAVs are detected from images captured by a camera by analyzing the images of the prohibited airspace. When using this method, there are cases where it is desirable to detect UAVs in prohibited airspace while limiting the number of camera devices used due to factors such as the installation location and cost of the camera devices that capture the prohibited airspace.

[0006] The present disclosure has been devised to solve the above-mentioned problems. That is, a main objective of the present disclosure is to provide a technology for detecting a target flying object in a surveillance area while reducing the number of installed imaging devices.

[0007] In order to achieve the above-mentioned object, one aspect of the flying object detection device of the present disclosure comprises: an acquisition unit that acquires a photographed image taken by an imaging device that photographs a monitored area; an extraction unit that extracts from the photographed image an image area associated with each of a plurality of airspaces that are at different distances from the imaging device; a reduction unit that generates an image to be analyzed by reducing each of the image areas extracted from the photographed image by a reduction rate determined using the distance between the airspace associated with the image area and the imaging device; a detection unit that detects the flying object to be detected from the image to be analyzed; a generation unit that generates an image reflecting the detection result by superimposing a figure representing the position of the detected flying object on the photographed image in which the flying object detected from the image to be analyzed is photographed; and an output unit that outputs the image reflecting the detection result.

[0008] In addition, one aspect of the flying object detection system of the present disclosure includes: an imaging device that captures an image of a monitoring area; and the above-mentioned flying object detection device that uses the image captured by the imaging device.

[0009] Furthermore, one aspect of the flying object detection method of the present disclosure includes acquiring an image captured by an imaging device that captures an area to be monitored, extracting from the captured image an image area associated with each of a plurality of airspaces at different distances from the imaging device, generating an image to be analyzed by reducing each of the image areas extracted from the captured image by a reduction rate determined using the distance between the airspace associated with the image area and the imaging device, detecting the flying object to be detected from the image to be analyzed, generating an image reflecting the detection result by superimposing a figure representing the position of the detected flying object on the captured image in which the flying object detected from the image to be analyzed is captured, and outputting the image reflecting the detection result.

[0010] Furthermore, in one aspect, the program storage medium of the present disclosure stores a computer program that causes a computer to execute the following processes: acquiring an image captured by an imaging device that captures an area to be monitored; extracting from the captured image an image area associated with each of a plurality of airspaces at different distances from the imaging device; generating an image to be analyzed by reducing each of the image areas extracted from the captured image by a reduction rate determined using the distance between the imaging device and the airspace associated with the image area; detecting an aircraft to be detected from the image to be analyzed; generating an image reflecting the detection result by superimposing a figure representing the position of the detected aircraft on the captured image in which the aircraft detected from the image to be analyzed is captured; and outputting the image reflecting the detection result.

[0011] According to the present disclosure, it is possible to detect target flying objects in a monitoring area while reducing the number of installed imaging devices.

[0012] FIG. 1 is a diagram illustrating an example of an imaging device used in a flying object detection system. FIG. 2 is a diagram illustrating an example of an imaging device together with FIG. 1. FIG. 3 is a diagram illustrating an example of a configuration of a flying object detection device. FIG. 4 is a diagram illustrating an example of the relationship between a plurality of airspaces at different distances from the imaging device and image areas (flying object appearance areas) in a captured image that relate to each of the airspaces. FIG. 5 is a diagram schematically illustrating processing performed by the flying object detection device on a captured image. FIG. 6 is a diagram illustrating an example of an image reflecting the detection result generated by the flying object detection device. FIG. 7 is a diagram illustrating another example of an image reflecting the detection result. FIG. 8 is a diagram illustrating a further display example of an image reflecting the detection result displayed on a display device. FIG. 9 is a flowchart illustrating an example of operation related to the detection of a flying object in the flying object detection device. FIG. 10 is a diagram illustrating another example of the configuration of the flying object detection device. FIG. 11 is a diagram illustrating yet another example of the configuration of the flying object detection device. FIG. 12 is a flowchart illustrating another example of operation related to the detection of a flying object in the flying object detection device.

[0013] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0014] First Embodiment An airborne object detection system according to a first embodiment of the present disclosure is a system that detects a target airborne object in a monitored area using a photographing device, and in the first embodiment, is applied to a monitoring system. The target airborne object is predetermined by a system designer or the like. Examples of the target airborne object include unmanned aerial vehicles (unmanned aerial vehicles) that can be flown by remote control or automatic pilot, such as unmanned airplanes, unmanned rotorcraft, and unmanned airships, as well as objects flown by humans using specific aviation equipment, such as balloons, hang gliders, and paragliders. While various airborne objects can be set as the target airborne object, in the first embodiment, a so-called drone (unmanned aerial vehicle) is set as the target airborne object.

[0015] The surveillance system to which the flying object detection system of the first embodiment is applied is a system that monitors the intrusion of a target flying object into a surveillance area. The surveillance area here includes no-fly zones where the flight of unmanned aerial vehicles is prohibited by law, such as the airspace above and surrounding areas of airports and important facilities.

[0016] As shown in FIG. 1 , the flying object detection system 1 of the first embodiment includes an flying object detection device 2 and an imaging device 4. The imaging device 4 is a device (e.g., a camera) that captures images of the monitored area, and in the first embodiment, only one imaging device 4 is installed. Because only one imaging device 4 is installed, it is necessary to capture the entire monitored area with only one imaging device 4. For this reason, the imaging device used as the imaging device 4 is selected taking into consideration the size of the monitored area and the device's performance. For example, if the monitored area is large, such as the airspace above an airport or its surrounding area, the imaging device used as the imaging device 4 is preferably capable of capturing images of flying objects in distant airspace, such as approximately one kilometer away, and has a wide field of view. More specifically, for example, a 4K camera with high resolution, a focal length f of 29 millimeters, a horizontal angle of view of approximately 25 degrees, and a vertical angle of view of approximately 17 degrees, is used as the imaging device 4. Note that the horizontal direction here refers to the direction along the ground surface, and the vertical direction refers to the direction perpendicular to the ground surface.

[0017] When the image capture device 4 of the specific example described above is used, the field of view of the image capture device 4 at a distance D(z) = 800 meters along the optical axis from the image capture device 4 has a vertical height (length) H(d) of 239 meters. However, in the example of Figure 1, part of the field of view is below the ground surface, so the vertical height (length) from the ground surface of the field of view of the image capture device 4 at a distance D(z) = 800 meters from the image capture device 4 is shorter than H(d) = 239 meters. Figure 2 is a schematic diagram of the field of view of the image capture device 4 as seen from the zenith. The width W(d) of the field of view of the image capture device 4, along the ground surface and perpendicular to the optical axis, is 354 meters.

[0018] The image capturing device 4 is installed with its field of view facing the monitoring area, for example, at a height H(y)=10 meters from the ground surface, with its optical axis aligned along the ground surface.

[0019] The flying object detection device 2 is a computer device that detects a target flying object by analyzing images captured by the imaging device 4. That is, the flying object detection device (hereinafter also referred to as the detection device) 2 is directly or indirectly connected to the imaging device 4 and includes, for example, a computing device 20 and a storage device 30, as shown in FIG. 3 . The storage device 30 includes a storage medium for storing data and a computer program (hereinafter also referred to as a program) 31. There are multiple types of storage devices, such as magnetic disk drives and semiconductor memory devices. Furthermore, there are multiple types of semiconductor memory devices, such as RAM (Random Access Memory) and ROM (Read Only Memory). A computer device may be equipped with multiple types of storage devices depending on their intended use, but these storage devices will be collectively referred to as the storage device 30. Furthermore, the types and number of storage devices 30 included in the detection device 2 are not limited, and a description thereof will be omitted. The detection device 2 may also be connected to a database 6, which is a storage device. In this case, the detection device 2 may write information to the database 6 or read information from the database 6, but in order to avoid complicating the explanation, we will omit the explanation of such cases here.

[0020] The arithmetic device 20 is configured with a processor such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit). The arithmetic device 20 can have functions based on a program 31 by reading and executing the program 31 stored in the storage device 30. Here, the arithmetic device 20 has an acquisition unit 21, an extraction unit 22, a reduction unit 23, a detection unit 24, a generation unit 25, and an output unit 26 as functional units related to detecting an air vehicle.

[0021] The acquisition unit 21 acquires the photographed images captured by the photographing device 4. Here, the method by which the acquisition unit 21 acquires the photographed images is not limited, and for example, the acquisition unit 21 may acquire the photographed images from the photographing device 4, or may acquire the photographed images of the photographing device 4 by reading out the photographed images temporarily stored in a database (not shown) from the photographing device 4.

[0022] The extraction unit 22 extracts the following multiple image regions from the image captured by the camera device 4. As described above, in the first embodiment, the image of a wide monitoring area captured by the camera device 4 is used to detect a target flying object in the monitoring area. In other words, the image captured by the camera device 4 captures airspaces from close to the camera device 4 to far away. In such a captured image, the size of the captured image of the flying object and the area in which the flying object appears vary depending on the distance between the camera device 4 and the flying object. FIG. 4 is a schematic illustration of an image captured by the camera device 4 when flying objects 5 of the same size appear in airspaces corresponding to positions A, B, and C in the field of view of the camera device 4. Position A is the closest position to the camera device 4 among positions A, B, and C. For convenience, the airspace corresponding to position A is also referred to as short-distance airspace A. Position C is the farthest position from the camera device 4 among positions A, B, and C. For convenience, the airspace corresponding to position C is also referred to as long-distance airspace C. Furthermore, the airspace corresponding to position B between positions A and C is also referred to herein as intermediate airspace B for convenience.

[0023] As shown in the example of Figure 4, although the flying object 5 is the same size, the image of the flying object 5 captured in the image captured by the camera 4 becomes smaller as it moves farther away from the camera 4. Furthermore, since the field of view of the camera 4 widens as it moves farther away from the camera 4, the image area in which the flying object 5 appears in the image captured by the camera 4 also widens as it moves farther away from the camera 4. Note that unmanned aerial vehicles (drones, etc.) are subject to flight altitude restrictions, such as being prohibited from flying in airspace above 150 meters above the ground. The example of Figure 4 shows the appearance area of ​​the flying object 5 taking into account the flight altitude restrictions of such flying objects (unmanned aerial vehicles).

[0024] 4 , the range of the appearance area of ​​the flying object 5 in the photographed image (hereinafter also referred to as the flying object appearance area) changes depending on the distance from the camera device 4. The range of the flying object appearance area in the photographed image can be calculated using the angle of view of the camera device 4 and the distance from the camera device 4. The present inventors have noticed that the range of the flying object appearance area in the photographed image changes depending on the distance from the camera device 4.

[0025] That is, here, multiple airspaces of interest are defined with different distances from the camera device 4 for detecting flying objects in the monitored area. The number and locations of the airspaces of interest can be set as appropriate by a system designer or the like if there are multiple airspaces of interest. For example, the number and locations of airspaces of interest are set such that detecting flying objects in the multiple airspaces of interest can be considered to have detected flying objects in the entire monitored area. The size of the monitored area, the size of the flying object to be detected, the expected movement speed of the flying object, and other factors are taken into consideration when setting the number and locations of the airspaces of interest. Note that information indicating the location of each of the multiple airspaces of interest that have been defined (e.g., location information represented by the distance from the camera device 4) is stored in the storage device 30.

[0026] For each of the multiple airspaces of interest thus determined, an aircraft appearance area in the captured image is calculated. For example, assume that the aforementioned short-distance airspace A, intermediate airspace B, and long-distance airspace C are set as the airspaces of interest. In this case, in the image captured by the image capture device 4, the range of the aircraft appearance area N for the short-distance airspace A is calculated, the range of the aircraft appearance area M for the intermediate airspace B is calculated, and the range of the aircraft appearance area F for the long-distance airspace C is calculated, as shown in FIG. 4 . Furthermore, information representing the range of such aircraft appearance areas is associated with airspace identification information identifying the airspace of interest related to the aircraft appearance area, and this information is generated as extracted area information for each aircraft appearance area. The extracted area information is stored in the storage device 30 of the detection device 2.

[0027] The extraction unit 22 in the detection device 2 references the extraction area information stored in the storage device 3 and extracts multiple image areas from the captured image acquired by the acquisition unit 21, as shown in Figure 5. The multiple extracted image areas are image areas respectively associated with multiple predetermined airspaces of interest as described above. In the example of Figure 5, multiple aircraft appearance areas N, M, and F associated with the airspaces of interest, short-distance airspace A, intermediate airspace B, and long-distance airspace C, are respectively extracted from the captured image. Note that Figure 5 is an illustration assuming that the same aircraft 5 is flying in each of the short-distance airspace A, intermediate airspace B, and long-distance airspace C, and does not take into account overlapping of aircraft 5 in the captured image.

[0028] The reduction unit 23 generates an image to be analyzed by reducing the image area extracted by the extraction unit 22 from the captured image. The reduction degree of the image area is determined in advance using the distance between the airspace of interest associated with the image area and the camera device 4 and performance information about the camera device 4. That is, as described above, the closer an aircraft is to the camera device 4, the larger it becomes, and conversely, the further it is from the camera device 4, the smaller it becomes. In other words, even for aircraft 5 of the same size, the size of the aircraft 5 in the captured image captured by the camera device 4 varies depending on the distance from the camera device 4. Assume here that aircraft 5 of the same size are captured in multiple different image areas (e.g., multiple aircraft appearance areas N, M, F as shown in FIG. 5 ) extracted from the same captured image. In this case, the reduction degree of each of these image areas (e.g., aircraft appearance areas N, M, F) is a reduction degree that results in the same size of the same aircraft 5 captured in each reduced image area. Here, the reduction degree of each of the aircraft appearance areas N, M, and F is determined so that the size of the aircraft 5 in the aircraft appearance areas N and M is the same as the size of the aircraft 5 in the aircraft appearance area F, which captures the aircraft 5 flying in the airspace of interest far from the image capture device 4. Note that, as in the case of the aircraft appearance area F shown in Fig. 5, there are cases where the image is not substantially reduced by the image reduction process performed by the reduction unit 23. The reduction degree in this case is assumed to be a reduction degree of no reduction.

[0029] The detection unit 24 detects the target flying object from the analysis target image. Various methods for detecting the target flying object from an image have been proposed, and one example, although not limited thereto, is a detection method using AI (Artificial Intelligence) technology. In this case, a detection model generated by AI technology is used. This detection model is generated by learning using images of the target flying object as training data. The input information to the detection model is the analysis target image, and the output information from the detection model includes information indicating the presence or absence of the target flying object in the input analysis target image, and, if the target flying object is detected, information indicating the position of the detected flying object in the analysis target image (hereinafter also referred to as analysis position information).

[0030] The detection unit 24 uses such a detection model to detect an airborne object from the image to be analyzed. Furthermore, when an airborne object is detected from the image to be analyzed, the detection unit 24 calculates the position in the captured image in which the image of the detected airborne object appears as detected image information. This calculation process uses, for example, analysis position information output from the detection model and information representing the position (range) in the captured image from which the image area of ​​the analysis object image before reduction associated with the analysis position information is extracted. Note that, when multiple types of airborne objects are set as the detection target airborne object, a detection model that distinguishes and detects each of the multiple types of airborne objects may be generated by learning images of the multiple types of airborne objects, and the detection unit 24 may use the detection model. In this case, the detection unit 24 may also output information representing the type of airborne object detected.

[0031] In the first embodiment, the image to be analyzed is an image that has been subjected to image reduction processing so that the size of the flying objects 5 to be detected is uniform regardless of the distance from the image capturing device 4. Therefore, the detection processing by the detection unit 24 does not need to take into account that the size of the flying object 5 varies depending on the distance from the image capturing device 4. This allows the detection device 2 to reduce the load of the detection processing for detecting flying objects from the image and also shorten the time required for the detection processing of flying objects. This contributes to real-time detection of flying objects using images captured by the image capturing device 4.

[0032] The generation unit 25 generates an image reflecting the detection results by superimposing a graphic representing the position of the detected flying object on a captured image in which the flying object detected from the image to be analyzed is captured. The graphic representing the position of the detected flying object (hereinafter also referred to as the flying object detection graphic) is not limited here and may be, for example, a circle, triangle, or square, or a symbol or a mark resembling an flying object, and is set appropriately by a system designer or the like. The position at which the flying object detection graphic is superimposed on the captured image is the position represented by the detection and capture information calculated by the detection unit 24.

[0033] 6 and 7 each show an example of a detection result reflection image. In the example of FIG. 6, the detection result reflection image is an image in which a star-shaped flying object detection graphic 8 is superimposed on the captured image. The example of FIG. 7 is an example of a detection result reflection image when multiple types of flying objects are detected. The detection result reflection image is an image in which a star-shaped flying object detection graphic 8 corresponding to the type of flying object detected and a circular flying object detection graphic 8 corresponding to the type of another type of flying object detected are superimposed on the captured image. Note that when multiple types of flying objects are detected, the same flying object detection graphic 8 may be superimposed on the captured image regardless of the type of flying object detected.

[0034] In addition, the alert level may vary depending on the flight location and type of the detected flying object. In such cases, the type of flying object detection graphic 8 corresponding to the alert level may be predetermined, and the generation unit 25 may generate an image reflecting the detection result by superimposing the type of flying object detection graphic 8 corresponding to the alert level on the captured image. Note that here, the alert level is calculated, for example, by the detection unit 24. That is, when an alert level is set for each flying object appearance area associated with each of the multiple airspaces of interest as described above, relationship data between the flying object appearance area and the alert level is stored in the storage device 30 as alert level judgment information used to determine the alert level. When the alert level is set based on the type of flying object to be detected, relationship data between the type of flying object and the alert level is stored in the storage device 30 as alert level judgment information. When the alert level is set based on the combination of the flying object appearance area and the type of flying object, relationship data between the combination of the flying object appearance area and the type of flying object and the alert level is stored in the storage device 30 as alert level judgment information. When a target flying object is detected, the detection unit 24 calculates the alert level of the detected flying object using one or both of the flying object appearance area where the flying object is detected and the type of flying object, and the alert level determination information stored in the storage device 30. The generation unit 25 generates an image reflecting the detection result by superimposing a flying object detection graphic 8 of a type corresponding to the calculated alert level on the captured image as described above.

[0035] Furthermore, the image reflecting the detection result generated by the generation unit 25 may further include text indicating information such as the type of aircraft detected and the alert level.

[0036] The output unit 26 outputs the generated detection result reflection image. An example of an output destination is a display device 7 as shown in FIG. 3. The display device 7 is a device that notifies (provides) information by displaying the information on a screen using text and images. Here, the display device 7 receives the detection result reflection image from the output unit 26 and provides the detection result reflection image to, for example, a user of the flying object detection system (monitoring system). Note that the display device 7 may be, for example, a display device of a terminal device carried by a user of the flying object detection system (monitoring system).

[0037] In this case, the captured image output by the image capture device 4 is a moving image, and a frame image selected from the multiple frame images constituting the captured image is used to perform the above-described series of processes for detecting the flying object. The detection result reflection image generated by these processes is, for example, included in the moving image, which is the captured image, by replacing the original frame image, and is output to the output destination by the output unit 26. The display device 7 displays the moving image (captured image) output by the output unit 26 in this manner. In other words, the output unit 26 can be said to control the display operation of the display device 7.

[0038] For example, when a user inputs a request (enlarged display request) to the detection device 2 to enlarge and display an image of the detected flying object 5 on the display device 7, the output unit 26 may perform display control to enlarge and display the flying object 5 in response to the request, as shown in FIG. 8. The method by which the user inputs the enlarged display request is not limited here, but an example is a method in which the enlarged display request is input to the detection device 2 by clicking the flying object detection figure 8 with the cursor 11 as shown in FIG. 8. Furthermore, the manner in which the flying object 5 is enlarged is not limited here and may be an appropriately set display manner. For example, as shown in FIG. 8, a window image in which the flying object 5 is enlarged may be superimposed on the captured image, or the captured image and the enlarged display image of the flying object 5 may be displayed side by side.

[0039] The detection device 2 of the first embodiment is configured as described above. Next, an example of the operation of the detection device 2 to detect an airborne object will be described with reference to Fig. 9. Fig. 9 is a flowchart illustrating an example of the operation of the detection device 2 to detect an airborne object. Fig. 9 can also be considered a diagram illustrating a method for detecting an airborne object in the detection device 2.

[0040] For example, it is assumed that the storage device 30 in the detection device 2 stores various information (data) used for processing by the aforementioned arithmetic unit (processor) 20. The acquisition unit 21 of the detection device 2 acquires captured images from the image capture device 4 (step 101 in FIG. 9 ). In the following description, it is assumed that the captured images acquired from the image capture device 4 are moving images.

[0041] The extraction unit 22 extracts multiple image regions from the acquired captured image (step 102). The captured image from which the extraction unit 22 extracts image regions is, for example, frame images selected for each predetermined number of frames from multiple frame images constituting the captured image (video) acquired from the image capture device 4. Furthermore, the multiple image regions extracted from the frame images to be subjected to extraction processing by the extraction unit 22 are, as described above, image regions (aircraft appearance regions) associated with multiple airspaces of interest that are at different distances from the image capture device 4. This extraction process uses the aforementioned extraction region information (information including information indicating the range of the aircraft appearance region in the captured image) stored in the storage device 30.

[0042] The reduction unit 23 reduces each of the extracted image regions (aircraft appearance regions) by a predetermined reduction rate to generate analysis target images related to each of the different airspaces of interest (step 103).The detection unit 24 then performs a detection process to detect the target aircraft for each of the analysis target images, and determines whether the target aircraft has been detected in any of the analysis target images related to the image regions (aircraft appearance regions) extracted from the same frame image (step 104).If the target aircraft is detected, the detection unit 24 calculates the position of the image of the detected aircraft in the frame image (photographed image) as detection and photography information.

[0043] Furthermore, when the target flying object is detected, the generation unit 25 generates an image reflecting the detection result by superimposing the flying object detection graphic 8 on the photographed image in which the detected flying object is photographed from the image to be analyzed (step 105). The superimposition position of the photographed image on which the flying object detection graphic 8 is superimposed is the position represented by the detection photographing information calculated by the detection unit 24.

[0044] Then, the output unit 26 outputs the detection result reflection image (step 106). Here, since the image captured by the image capture device 4 is a moving image, the captured image including the detection result reflection image (frame image) is output by the output unit 26 to, for example, the display device 7. That is, here, a series of processes for detecting the flying object as described above is executed sequentially for each of a plurality of frame images selected from a plurality of frame images constituting the image captured by the image capture device 4 (moving image). When the detection unit 24 detects an flying object, the generation unit 25 generates the detection result reflection image. Then, when the detection result reflection image (frame image) is generated, the captured image including the detection result reflection image is output by the output unit 26 to, for example, the display device 7.

[0045] On the other hand, if the target flying object is not detected from the frame image, the process for detecting the flying object for that frame image is terminated, and a series of processes for detecting the flying object for the next frame image is prepared.

[0046] As described above, the detection device 2 of the first embodiment and the flying object detection system 1 including the detection device 2 are configured to be able to detect target flying objects in a wide monitoring area using a single imaging device 4. That is, in this flying object detection system 1, a high-resolution imaging device with a telephoto function that can capture images of distant flying objects so that they can be detected is used as the imaging device 4. This allows the flying object detection system 1 to detect target flying objects in a wide monitoring area, such as an airport and its surrounding area, even with a single imaging device 4.

[0047] Furthermore, if the detection process for an airborne object is performed on the entire area of ​​the captured image (frame image) by the image capture device 4, the captured image is a high-resolution image, and therefore the load on the device due to the detection process involving image analysis becomes large. In contrast, the detection device 2 of the first embodiment can reduce the load on the device. That is, the detection device 2 does not perform image analysis on the entire area of ​​the captured image, but extracts an airborne object appearance area from the captured image and performs detection processing using image analysis on that image area. Furthermore, multiple image areas (airborne object appearance areas) are extracted from a common captured image, and in this case, the load on the device is reduced by performing a reduction process on the extracted image areas. By reducing the processing load on the detection device 2 in this way, the detection device 2 can shorten the time required for the detection process for an airborne object.

[0048] Furthermore, in the first embodiment, the reduction degree of each of the multiple image regions (aircraft appearance regions) extracted from the captured image is such that, assuming that air vehicles of the same size are captured in each of those image regions, the sizes of the air vehicles captured in each image region after reduction processing (i.e., the image to be analyzed) are uniform. Because the analysis target image is generated by reducing the image at such a reduction degree, the size of the detected air vehicles captured in the analysis target image will be the same regardless of their distance from the camera device 4. Therefore, for example, the same detection model can be used to perform air vehicle detection processing for analysis target images associated with multiple airspaces of interest that are at different distances from the camera device 4. In other words, it is not necessary to prepare separate detection models for each of the multiple airspaces of interest. This reduces the effort and time required for generating detection models, thereby enabling the air vehicle detection system 1 to be quickly launched.

[0049] Second Embodiment A second embodiment of the present disclosure will be described below. In the description of the second embodiment, components having the same names as those in the description of the first embodiment will be denoted by the same reference numerals, and duplicate descriptions thereof will be omitted.

[0050] In the second embodiment, the detection device 2 has the same configuration as in the first embodiment, and further includes a calculation unit 27 as shown in Fig. 10. Like the extraction unit 22 and the reduction unit 23, the calculation unit 27 is also a functional unit of a computer that is realized by a processor executing a computer program.

[0051] The calculation unit 27 calculates the range of the aircraft appearance area related to the airspace of interest in the image captured by the image capture device 4 (in other words, the extracted range of the image area extracted from the captured image by the extraction unit 22). For example, a calculation formula for calculating the range of the aircraft appearance area in the captured image is stored in advance in the storage device 30. The calculation formula is a formula for calculating the range of the aircraft appearance area in the captured image using the distance between the image capture device 4 and the airspace of interest, information representing the field of view (field of view angle) of the image capture device 4, and information about the height at which the image capture device 4 is installed.

[0052] The calculation unit 27 acquires, for example, information necessary for calculating the range of the aircraft appearance area in the captured image from a system designer or the like. The information necessary for calculation is, for example, information necessary for calculation using the above-mentioned formula, and includes information such as the distance between the image capture device 4 and the airspace of interest, information representing the field of view (field of view angle) of the image capture device 4, and information about the height at which the image capture device 4 is installed. The method for acquiring such information is not limited here, but for example, the information is acquired by displaying, on a display device viewed by an information inputter such as a system designer, an input field for inputting the information to be acquired and a message prompting the input of information into the input field.

[0053] The calculation unit 27 acquires a calculation formula and information necessary for calculations using the calculation formula, and calculates the range of the aircraft appearance area in the captured image using the acquired calculation formula and information. Here, since multiple airspaces of interest are set, information about each of the multiple airspaces of interest is acquired, and the calculation unit 27 calculates the range of the aircraft appearance area associated with each of the multiple airspaces of interest in the captured image. The information representing the calculated range of the aircraft appearance area is associated with airspace identification information that identifies the airspace of interest associated with the aircraft appearance area, and is stored in the storage device 30 as the extraction area information described in the first embodiment. This extraction area information is used in the extraction process of the extraction unit 22.

[0054] The configuration of the detection device 2 other than that described above is the same as that of the first embodiment.

[0055] The detection device 2 and flying object detection system 1 of the second embodiment have the same configuration as those of the first embodiment, and can therefore achieve the same effects as those of the first embodiment. Furthermore, in the second embodiment, the detection device 2 has a calculation unit 27, which has a function of calculating the extraction range of the image area (aircraft appearance area) extracted from the captured image by the extraction unit 22. This eliminates the need for a system designer or the like to calculate the range of the aircraft appearance area in the captured image.

[0056] <Other Embodiments> The flying object detection device and flying object detection system of the present disclosure are not limited to the first and second embodiments and may be implemented in various ways. For example, although the first and second embodiments illustrate examples in which the flying object detection system is applied to a surveillance system, the application of the flying object detection system of the present disclosure is not limited to surveillance systems. For example, in a traffic control system for unmanned aircraft such as drones in an area surrounding a logistics hub, the flying object detection system of the present disclosure may be applied to detect unmanned aircraft that are subject to management as flying objects to be detected.

[0057] In the first and second embodiments, the output unit 26 outputs an image reflecting the detection result to the display device 7 via the generation unit 25. In addition, when the detection unit 24 detects an airborne object, the output unit 26 may output information indicating that the airborne object has been detected (airborne object detection information) to a predetermined notification destination other than the display device. One example of the notification destination is a computer device of a monitoring system to which the airborne object detection system is applied. Examples of the airborne object detection information include text information of a message indicating that an airborne object has been detected, and control information for an alarm sound to notify the user of the detection of the airborne object.

[0058] Furthermore, the reduction degree of the image area by the reduction unit 23 shown in the first and second embodiments is merely an example, and is not limited to the reduction degree shown in the first and second embodiments.

[0059] Furthermore, the flying object detection device may also have a configuration such as that shown in FIG. 11 . Specifically, the flying object detection device 50 is, for example, a computer device, and includes functional units implemented by executing a computer program, such as an acquisition unit 51, an extraction unit 52, a reduction unit 53, a detection unit 54, a generation unit 55, and an output unit 56. The acquisition unit 51 acquires photographed images captured by a photographing device that captures the monitored area. The extraction unit 52 extracts image regions associated with multiple airspaces at different distances from the photographing device from the photographed images. The reduction unit 53 generates an analysis target image by reducing each of the extracted image regions from the photographed images by a reduction rate determined using the distance between the photographing device and the airspace associated with the image region. The detection unit 54 detects the target flying object from the analysis target image. The generation unit 55 generates a detection result reflection image by superimposing a graphic representing the position of the detected flying object on the photographed image in which the flying object detected from the analysis target image is captured. The output unit 56 outputs the detection result reflection image. In addition, the acquisition unit 21, extraction unit 22, reduction unit 23, detection unit 24, generation unit 25 and output unit 26 of the detection device 2 in the first embodiment described above are examples of the acquisition unit 51, extraction unit 52, reduction unit 53, detection unit 54, generation unit 55 and output unit 56.

[0060] The flying object detection device 50 has the above-described configuration. The flying object detection device 50 can be used in conjunction with an imaging device 60 as shown by the dotted line in Fig. 11 to form a flying object detection system.

[0061] Next, an example of the operation of the flying object detection device 50 will be described with reference to Fig. 12. Fig. 12 is a flowchart illustrating an example of the operation of the flying object detection device 50. Fig. 12 can also be said to be a diagram illustrating an example of a flying object detection method of the flying object detection device 50.

[0062] For example, when the acquisition unit 51 acquires an image captured by a photographing device (step 201), the extraction unit 52 extracts image areas associated with each of a plurality of airspaces at different distances from the photographing device from the captured image (step 202).

[0063] The reduction unit 53 then reduces each image region extracted from the captured image by a reduction factor determined using the distance between the airspace associated with that image region and the image capture device, thereby generating an image to be analyzed (step 203). When the detection unit 54 detects the target aircraft by executing a process to detect the target aircraft from the image to be analyzed (step 204), the generation unit 55 generates an image reflecting the detection result (step 205). That is, the generation unit 55 generates the detection result reflecting image by superimposing a graphic representing the position of the detected aircraft on the captured image in which the aircraft detected from the image to be analyzed is captured. The output unit 56 then outputs the detection result reflecting image (step 206).

[0064] As described above, the flying object detection device 50 is configured to extract, from the same photographed image, image regions associated with each of a plurality of airspaces at different distances from the photographing device, and to execute processing related to the detection of flying objects for each of the extracted image regions. Therefore, the flying object detection device 50 can detect flying objects in each of a plurality of different airspaces using a common photographed image, thereby reducing the number of photographing devices that photograph the monitoring area.

[0065] Furthermore, the flying object detection device 50 can reduce the processing load required to detect flying objects. In other words, the image area of ​​the captured image related to an airspace close to the imaging device, where the flying object is captured in a large image, can still detect the flying object even if the image is coarser than the image area of ​​the captured image related to an airspace far from the imaging device. Using this, the flying object detection device 50 reduces the image area extracted from the captured image by a reduction factor determined using the distance between the imaging device and the airspace related to the image area. The flying object detection device 50 then performs the flying object detection process using the analysis target image generated by reducing the image. In this way, the processing load can be reduced by performing image analysis on the image area (analysis target image) reduced by a reduction factor for each image area that takes into account the distance from the imaging device.

[0066] Some or all of the above embodiments may be described as, but are not limited to, the following supplementary notes: [Supplementary Note 1] An airborne object detection device comprising: an acquisition unit that acquires an image captured by an imaging device that captures an area to be monitored; an extraction unit that extracts from the captured image an image region associated with each of a plurality of airspaces at different distances from the imaging device; a reduction unit that generates an image to be analyzed by reducing each of the image regions extracted from the captured image by a reduction rate determined using the distance between the imaging device and the airspace associated with the image region; a detection unit that detects an airborne object to be detected from the image to be analyzed; a generation unit that generates an image reflecting the detection result by superimposing a figure representing the position of the detected airborne object on the captured image in which the airborne object detected from the image to be analyzed is captured; and an output unit that outputs the image reflecting the detection result. [Supplementary Note 2] The flying object detection device of Supplementary Note 1, wherein, assuming that flying objects of the same size are photographed in each of image regions associated with a plurality of airspaces at different distances from the imaging device, the reduction degree of each of these image regions is such that the sizes of the flying objects photographed in each of the images to be analyzed generated by reduction are uniform. [Supplementary Note 3] The flying object detection device of Supplementary Note 1, further comprising a calculation unit that calculates an extraction range of the image region to be extracted by the extraction unit in the photographed image using the distance between the imaging device and the airspace associated with the image region to be extracted from the photographed image, information representing the field of view of the imaging device, and information about the height at which the imaging device is installed. [Supplementary Note 4] The flying object detection device of Supplementary Note 1, wherein the detection unit detects each of a plurality of types of flying objects to be detected separately. [Supplementary Note 5] The flying object detection device of Supplementary Note 1, wherein the output unit performs display control to enlarge and display the image of the detected flying object in response to a request to enlarge and display the image of the detected flying object on a display device. [Supplementary Note 6] The flying object detection device described in Supplementary Note 1, wherein when a target flying object is detected from the image to be analyzed, the output unit outputs information indicating that the flying object has been detected to a predetermined notification destination other than the display device.[Supplementary Note 7] The flying object detection device according to Supplementary Note 1, comprising a detection model generated by learning from photographed images of an air vehicle to be detected, wherein the detection unit performs air vehicle detection processing for an image to be analyzed that is associated with each of a plurality of image regions extracted from the photographed images using the same detection model. [Supplementary Note 8] An air vehicle detection system comprising: an imaging device that images a monitored area; and the flying object detection device according to Supplementary Note 1, which uses the photographed images taken by the imaging device. [Supplementary Note 9] A flying object detection method, comprising: acquiring photographed images taken by the imaging device that images a monitored area; extracting from the photographed images image regions associated with each of a plurality of airspaces that are at different distances from the imaging device; generating an image to be analyzed by reducing each of the image regions extracted from the photographed image by a reduction rate determined using the distance between the imaging device and the airspace associated with the image region; detecting the flying object to be detected from the image to be analyzed; generating an image reflecting the detection result by superimposing a graphic representing the position of the detected flying object on the photographed image of the air vehicle detected from the image to be analyzed; and outputting the image reflecting the detection result. [Supplementary Note 10] A program storage medium that stores a computer program that causes a computer to execute the following processes: a process of acquiring an image captured by an imaging device that captures an area to be monitored; a process of extracting from the captured image an image area associated with each of a plurality of airspaces at different distances from the imaging device; a process of generating an image to be analyzed by reducing each of the image areas extracted from the captured image by a reduction rate determined using the distance between the imaging device and the airspace associated with that image area; a process of detecting an aircraft to be detected from the image to be analyzed; a process of generating an image reflecting the detection result by superimposing a figure representing the position of the aircraft detected on the captured image in which the aircraft detected from the image to be analyzed is captured; and a process of outputting the image reflecting the detection result.

[0067] Note that some or all of the configurations described in Supplementary Notes 2 to 7, which are dependent on Supplementary Note 1, may also be dependent on Supplementary Notes 8 to 10 in the same dependent relationship as Supplementary Notes 2 to 7. Furthermore, not limited to Supplementary Notes 1 and 8 to 10, some or all of the configurations described as Supplements may be made dependent on various hardware, software, various recording means for recording software, or systems, within the scope of each of the above-mentioned embodiments.

[0068] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0069] This application claims priority based on Japanese Patent Application No. 2024-015838, filed February 5, 2024, the disclosure of which is incorporated herein in its entirety by reference.

[0070] REFERENCE SIGNS LIST 1 Flying object detection system 2 Detection device 4, 60 Photography device 7 Display device 21, 51 Acquisition unit 22, 52 Extraction unit 23, 53 Reduction unit 24, 54 Detection unit 25, 55 Generation unit 26, 56 Output unit 27 Calculation unit

Claims

1. An aircraft detection device comprising: an acquisition means for acquiring an image captured by an imaging device that captures an area to be monitored; an extraction means for extracting from the captured image an image area associated with each of a plurality of airspaces at different distances from the imaging device; a reduction means for generating an image to be analyzed by reducing each of the image areas extracted from the captured image by a reduction rate determined using the distance between the imaging device and the airspace associated with that image area; a detection means for detecting an aircraft to be detected from the image to be analyzed; a generation means for generating an image reflecting the detection result by superimposing a figure representing the position of the detected aircraft on the captured image in which the aircraft detected from the image to be analyzed is captured; and an output means for outputting the image reflecting the detection result.

2. The flying object detection device of claim 1, wherein, assuming that flying objects of the same size are photographed in each of the image areas associated with a plurality of airspaces at different distances from the photographing device, the degree of reduction of each of those image areas is such that the size of the flying objects photographed in each of the images to be analyzed generated by image reduction is uniform.

3. A flying object detection device as described in claim 1 or claim 2, further comprising a calculation means for calculating an extraction range of an image area to be extracted by said extraction means in a photographed image, using the distance between said photographing device and the airspace related to the image area to be extracted from the photographed image, information representing the field of view of said photographing device, and information about the height at which said photographing device is installed.

4. The flying object detection device according to any one of claims 1 to 3, wherein the detection means detects each of a plurality of types of target flying objects separately.

5. A flying object detection device as claimed in any one of claims 1 to 4, wherein the output means performs display control to enlarge and display an image of the detected flying object in response to a request to enlarge and display the image of the flying object on a display device.

6. A flying object detection device as claimed in any one of claims 1 to 5, wherein the output means, when a target flying object is detected from the image to be analysed, outputs information indicating that the flying object has been detected to a predetermined notification destination other than the display device.

7. A flying object detection device as claimed in any one of claims 1 to 6, which is provided with a detection model generated by learning from photographed images in which the flying object to be detected is photographed, and the detection means performs flying object detection processing using the same detection model for images to be analyzed that are related to each of multiple image areas extracted from the photographed images.

8. An airborne object detection system comprising: an imaging device that photographs a monitored area; and an airborne object detection device according to any one of claims 1 to 7 that uses the images captured by the imaging device.

9. A flying object detection method comprising: acquiring, by a computer, images taken by an imaging device that photographs a monitored area; extracting from the captured images image areas associated with a plurality of airspaces at different distances from the imaging device; generating an image to be analyzed by reducing each of the image areas extracted from the captured images by a reduction rate determined using the distance between the airspace associated with the image area and the imaging device; detecting a flying object to be detected from the image to be analyzed; generating an image reflecting the detection result by superimposing a figure representing the position of the detected flying object on the captured image in which the flying object detected from the image to be analyzed is captured; and outputting the image reflecting the detection result.

10. A program storage medium storing a computer program that causes a computer to execute the following processes: a process of acquiring an image taken by a camera that photographs a monitored area; a process of extracting from the image a region associated with each of a plurality of airspaces at different distances from the camera; a process of generating an image to be analyzed by reducing each of the image regions extracted from the image by a reduction rate determined using the distance between the camera and the airspace associated with that region; a process of detecting an aircraft to be detected from the image to be analyzed; a process of generating an image reflecting the detection result by superimposing a figure representing the position of the aircraft detected on the image in which the aircraft detected from the image to be analyzed is photographed; and a process of outputting the image reflecting the detection result.

11. A flying object detection method as described in claim 9, wherein, assuming that flying objects of the same size are photographed in each of the image areas associated with a plurality of airspaces at different distances from the photographing device, the degree of reduction of each of those image areas is such that the sizes of the flying objects photographed in each of the images to be analyzed generated by image reduction are uniform.

12. A method for detecting flying objects as described in claim 9 or claim 11, further comprising using the distance between the camera and the airspace related to the image area to be extracted from the photographed image, information representing the field of view of the camera, and information about the height at which the camera is installed to calculate by a computer an extraction range of the image area to be extracted from the photographed image for each of a plurality of airspaces at different distances from the camera.

13. A method for detecting an air vehicle as claimed in claim 9, claim 11 or claim 12, in which when detecting an air vehicle to be detected, a plurality of types of air vehicles to be detected are detected separately.

14. A flying object detection method as claimed in any one of claims 9 or 11 to 13, wherein the computer performs display control to enlarge and display an image of the detected flying object in response to a request to enlarge and display the image of the flying object on a display device.

15. A flying object detection method as claimed in claim 9 or any one of claims 11 to 14, wherein, when a target flying object is detected from the image to be analysed, information indicating that the flying object has been detected is further output to a predetermined notification destination other than the display device.

16. A program storage medium as described in claim 10, wherein, assuming that an aircraft of the same size is photographed in each of the image areas associated with a plurality of airspaces at different distances from the photographing device, the degree of reduction of each of those image areas is such that the size of the aircraft photographed in each of the images to be analyzed generated by reducing the image is uniform.

17. A program storage medium as claimed in claim 10 or claim 16, further storing a computer program that causes a computer to execute a process of further calculating an extraction range of an image area to be extracted from a photographed image, using the distance between the photographing device and the airspace related to the image area to be extracted from the photographed image, information representing the field of view of the photographing device, and information about the height at which the photographing device is installed, to extract image areas related to each of a plurality of airspaces at different distances from the photographing device.

18. A program storage medium as set forth in claim 10, claim 16 or claim 17, wherein the process of detecting the target flying object detects each of a plurality of types of target flying object separately.

19. A program storage medium as claimed in claim 10 or any one of claims 16 to 18, further storing a computer program that causes a computer to execute a process for performing display control to enlarge and display an image of a detected flying object in response to a request to enlarge and display the image of the flying object on a display device.

20. A program storage medium as claimed in claim 10 or any one of claims 16 to 19, further storing a computer program that causes a computer to execute a process of outputting information indicating that a target flying object has been detected to a predetermined notification destination other than a display device when the target flying object is detected from the image to be analysed.

Images