Object detection device

Abstract

To provide an object detection apparatus capable of providing an image useful for finding an object from an aerial image.SOLUTION: An object detection apparatus (1) for detecting an object from an aerial image comprises imaging apparatuses (10, 20) for imaging a first aerial image, a control apparatus (40), and a display apparatus (30). The control apparatus divides the first aerial image into a plurality of first images, calculates probability with which the object is included in each of the plurality of first images, and causes the display apparatus to display a first discrimination image capable of discriminating between the degree of the probability and a portion of the first aerial image corresponding to the probability.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to an object detection device that detects an object from an aerial image. [Background technology]

[0002] There is a known technology for automatically searching for targets such as missing persons using aerial images captured by aircraft such as helicopters and drones. Patent Document 1 proposes a system for quickly and accurately detecting target objects from aerial images. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2020-144922 A Summary of the Invention [Problem to be solved by the invention]

[0004] When using aerial images, it is possible to search for objects from a wider area than when using images captured on the ground. This reduces the number of times images need to be captured after changing the shooting direction in order to detect the object, thereby improving the efficiency of detecting the object.

[0005] However, as the altitude of the flying object increases when the aerial image is captured, the size of the object included in the aerial image decreases. Therefore, when the aerial image is used as is to detect the object, the detection accuracy of the object decreases, and there is a possibility that the system will output an erroneous detection result.

[0006] An object of the present disclosure is to provide an object detection device capable of providing images useful for discovering objects from aerial images. [Means for solving the problem]

[0007] The object detection device according to the present disclosure is an object detection device that detects an object from an aerial image, and includes an imaging device that captures a first aerial image, a control device, and a display device. The control device divides the first aerial image into a plurality of first images, calculates a probability that the object is included in each of the plurality of first images, and causes the display device to display a first discrimination image that is capable of discriminating between the degree of the probability and a portion of the first aerial image that corresponds to the probability. Effect of the Invention

[0008] According to the present disclosure, it is possible to provide images that are useful for discovering objects from aerial images. [Brief description of the drawings]

[0009] [Figure 1] 1 is a block diagram showing a schematic configuration of an object detection device; [Diagram 2] FIG. 2 is a diagram showing an overview of processing performed by the object detection device. [Diagram 3] 6 is a flowchart showing the contents of a setting process executed by the object detection device. [Figure 4] 4 is a flowchart showing the contents of an object detection process executed by the object detection device. [Diagram 5] FIG. 13 is a diagram showing an example of setting the number of divisions of an aerial image according to an imaging distance. [Figure 6] FIG. 2 is a diagram showing an example of an aerial image captured by the object detection device. [Figure 7] 1 is a diagram showing an example of a discrimination image displayed by the object detection device; [Figure 8] 1A to 1C are diagrams showing various examples of discrimination images. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the drawings, the same or corresponding parts are designated by the same reference characters and their description will not be repeated.

[0011] 1 is a block diagram showing a schematic configuration of an object detection device 1. The object detection device 1 is mounted on an air vehicle such as a helicopter 200. The object detection device 1 may also be mounted on a drone. The object detection device 1 includes a first camera 10, a second camera 20, a display device 30, and a control device 40.

[0012] The first camera 10 is configured as a wide-angle camera, and the second camera 20 is configured as a telephoto camera. The first camera 10 and the second camera 20 capture aerial images in front of the helicopter 200. The first camera 10 and the second camera 20 may be configured as thermal cameras.

[0013] The display device 30 is configured, for example, by a liquid crystal display, a plasma display, an organic EL (Electro Luminescence) display, or the like. The display device 30 includes a touch panel that functions as an operation unit 31. The display device 30 displays an aerial image captured by the first camera 10 or the second camera 20. Furthermore, the display device 30 displays a discrimination image generated by the control device 40 based on the aerial image. The discrimination image is used to find an object, such as a missing person, from the aerial image.

[0014] The control device 40 is connected to the first camera 10, the second camera 20, and the display device 30. The control device 40 acquires aerial images from the first camera 10 and the second camera 20. The control device 40 instructs the first camera 10 and the second camera 20 on the imaging direction and imaging magnification. The control device 40 outputs a video signal to the display device 30. An operation signal output by the operation unit 31 of the display device 30 is input to the control device 40.

[0015] The control device 40 includes a processor 50 and a memory 60. The processor 50 is typically an arithmetic processing unit such as a CPU (Central Processing Unit) or an MPU (Multi-Processing Unit). The processor 50 may be configured with a processing circuitry. The processor 50 realizes various types of processing by reading and executing programs stored in the memory 60.

[0016] The memory 60 is realized by a non-volatile memory such as a random access memory (RAM), a read only memory (ROM), and a flash memory. The memory 60 stores an estimation model 61 used to detect objects such as victims from an aerial photograph. Furthermore, the memory 60 stores various data required for the control device 40 to control the first camera 10, the second camera 20, and the display device 30, in addition to the programs executed by the processor 60.

[0017] In addition, memory 60 may be a CD-ROM (Compact Disc - Read Only Memory), a DVD-ROM (Digital Versatile Disk - Read Only Memory), a USB (Universal Serial Bus) memory, a memory card, a FD (Flexible Disk), a hard disk, an SSD (Solid State Drive), a magnetic tape, a cassette tape, an MO (Magnetic Optical Disc), an MD (Mini Disc), an IC (Integrated Circuit) card (excluding memory cards), an optical card, a mask ROM, or an EPROM, as long as it is capable of non-temporarily recording a program in a format readable by processor 50.

[0018] Processor 50 functions as an input / output unit 51, a reception unit 52, a division unit 53, a determination unit 54, and an output unit 55. Input / output unit 51 is connected to first camera 10 and second camera 20. This connection is realized by wire or wirelessly. Aerial images captured by first camera 10 and second camera 20 are input to input / output unit 51. Reception unit 52 receives commands based on the operation of operation unit 31. Reception unit 52 outputs the commands based on the operation of operation unit 31 to first camera 10 or second camera 20 via input / output unit 51.

[0019] The estimation model 61 stored in the memory 60 is learned by the learning device 100. The control device 40 stores the estimation model 61 learned by the learning device 100 in the memory 60.

[0020] Here, a learning phase for learning the estimation model 61 will be described. The estimation model 61 is learned by the learning device 100. The learning device 100 includes an input unit 101, a determination unit 102, and a learning unit 103. The learning device 100 is typically a computer including a processor and a memory.

[0021] The estimation model 61 is trained by the learning device 100 to detect an object from an aerial image 111.

[0022] For example, a supervised learning algorithm is used to train the estimation model 61. The learning device 100 trains the estimation model 61 by supervised learning using a large amount of training data. The training data is composed of an aerial image 111 and answer data 112 corresponding to the aerial image 111.

[0023] The aerial image 111 is prepared in advance for learning the estimation model 61. The aerial image 111 includes an object to be searched for, such as a person. The supervised data 112 includes a determination result corresponding to the aerial image 111. The supervised data 112 is, for example, data indicating the position of the object included in the aerial image 111.

[0024] An aerial photographed image 111 is input to the input unit 101 of the learning device 100. The designer prepares in advance a plurality of such aerial photographed images 111. The designer inputs learning data to the learning device 100, thereby causing the estimation model 61 to learn.

[0025] The estimation model 61 includes a neural network 62 and parameters 63 used by the neural network 62. The neural network 62 is a known neural network used in image recognition processing by deep learning, such as a convolution neural network (CNN), a recurrent neural network (RNN), or a long short term memory network (LSTM network). The estimation model 61 performs deep learning by using the neural network 62. The parameters 63 include weighting coefficients used in calculations by the neural network 62, and the like.

[0026] A single aerial image 111 is not input as is to the input unit 101, but an aerial image 111 that has been divided into a plurality of images in advance is input. For example, an aerial image 111 divided into a matrix of 5 rows and 8 columns is input to the input unit 101. The determination unit 102 determines for each divided image whether or not an object is included in the aerial image 111, based on the image input from the input unit 101 and an estimation model 61 including a neural network 62.

[0027] The determination unit 102 may perform various types of image processing, such as enhancing contrast or edges, in advance on the aerial photographed image 111. The determination unit 102 acquires the determination result obtained by the estimation model 61 together with the accuracy of the determination result.

[0028] The learning unit 103 trains the estimation model 61 based on the determination result obtained by the determination unit 102 and the correct answer data corresponding to the aerial photographed image 111 to be determined. Specifically, the learning unit 103 trains the estimation model 61 by adjusting the parameters 63 (e.g., weighting coefficients) so that the determination result obtained by the estimation model 61 approaches the correct answer data.

[0029] In this way, the estimation model 61 is trained to calculate the probability that an object is included in each divided image based on learning data including images divided from the aerial image 111 and the results of the object being present in the divided images.

[0030] The number of divisions when dividing the aerial image 111 can be set in various ways. The size and shape of the image when dividing the aerial image 111 can also be set in various ways. By inputting the aerial image 111 divided in various ways to the input unit 101, the estimation model 61 can be trained based on the image divided in various ways. Furthermore, when an image is divided in various ways, the part to cut out the image shifts in various ways. Therefore, the amount of database that stores the learning results of the image can be increased. As a result, the accuracy rate of the judgment result obtained based on the estimation model 61 can be increased.

[0031] For example, the estimation model 61 may be trained using an aerial image 111 divided into a matrix of 2 rows and 8 columns, an aerial image 111 divided into a matrix of 5 rows and 8 columns, and an aerial image 111 divided into a matrix of 5 rows and 16 columns.

[0032] The object detection device 1 stores the estimation model 61 learned by the learning device 100 in the memory 60. The object detection device 1 may download the estimation model 61 from the learning device 100 and store the acquired estimation model 61 in the memory 60.

[0033] Fig. 2 is a diagram showing an overview of the processing of the object detection device 1. With reference to Fig. 2, an overview of the processing of the control device 40 of the object detection device 1 in the operation phase will be described.

[0034] An object detection device 1 mounted on a helicopter 200 or the like captures an aerial image 151 using a first camera 10 and a second camera 20. The first camera 10 and the second camera 20 may output the captured aerial image 151 to the control device 40 at the timing when an instruction is received from the control device 40.

[0035] For example, assume that an aerial image 151 is captured by the first camera 10, which is a wide-angle camera. In this case, the division unit 53 divides the aerial image 151 into a plurality of images 152. The determination unit 54 uses the estimation model 61 to determine whether or not a target object is included in each divided image 152. The result of this determination includes the accuracy of the determination.

[0036] The output unit 55 generates a discrimination image 153 including the determination result of the determination unit 54, and outputs the discrimination image 153 to the display device 30. The discrimination image 153 includes a plurality of divided images 152 and frames separating each of the plurality of images 152. Each of the plurality of images 152 included in the discrimination image 153 displays the probability that an object exists. Furthermore, in the discrimination image 153, a frame surrounding the image 152 whose probability exceeds the threshold is drawn in a bold frame. As a result, the portion of the image 152 whose probability exceeds the threshold is displayed in an emphasized manner.

[0037] A searcher searching for an object can identify an area where the object is likely to exist by looking at the accuracy included in image for discrimination 153. When the searcher wants to check in more detail an area of ​​image for discrimination 153 where the object is likely to exist, the searcher touches with his / her finger the part of the display corresponding to that area. Operation unit 31, which is configured with a touch panel, detects the operation. Operation unit 31 outputs an operation signal to reception unit 52.

[0038] Reception unit 52 outputs a signal specifying the imaging direction and lens magnification to first camera 10 or second camera 20 so that the area of ​​aerial image 151 specified by the operation signal is enlarged and captured.

[0039] For example, when discrimination image 153 is an image captured by first camera 10, reception unit 52 may output a signal to second camera 20 specifying the imaging direction and lens magnification.

[0040] First camera 10 or second camera 20 captures a new aerial image 151 in accordance with an instruction from reception unit 52. Division unit 53 divides newly captured aerial image 151 into a plurality of images. Thereafter, determination unit 54, output unit 55, and reception unit 52 repeat the above-mentioned process.

[0041] This causes the discrimination image 153, which is an enlarged area where the object is likely to exist, to be displayed on the display device 30. As a result, it becomes easier for the searcher to find the object.

[0042] 3 is a flowchart showing the contents of the setting process executed by the object detection device 1. The setting process based on this flowchart is executed by the control device 40 included in the object detection device 1.

[0043] First, the control device 40 judges whether or not an operation to select the setting of the division number has been detected (step S1). The object detection device 1 according to the present embodiment is configured to allow the selection of whether the number of divisions when the aerial photographed image is divided by the division unit 53 is set manually or automatically.

[0044] If an operation to select the number of divisions is not detected, the control device 40 proceeds to step S6. If an operation to select the number of divisions is detected, the control device 40 determines whether or not manual setting is selected (step S2). If manual setting is selected, the control device 40 waits until the number of divisions is input (step S3). If the number of divisions is input, the control device 40 stores the number of divisions according to the input in the memory 60 (step S4).

[0045] When the manual setting is not selected and the auto setting is selected in step S2 (NO in step S2), the control device 40 stores the auto setting flag in the memory 60 (step S5). In this case, the control device 40 automatically determines the number of divisions according to the imaging distance.

[0046] Here, with reference to FIG. 5, the procedure for the control device 40 to determine the number of divisions when the auto setting flag is set will be described.

[0047] FIG. 5 is a diagram showing an example of setting the number of divisions of the aerial image according to the imaging distance. As shown in FIG. 5, when the setting of the number of divisions is auto setting, the control device 40 determines the number of divisions of the aerial image 151 according to the imaging distance when the aerial image 151 is captured. In the memory 60 of the control device 40, thresholds L1 and L2 related to the determination of the number of divisions are stored in advance (L1 < L2).

[0048] The flight altitude when the aerial image 151 is captured may be adopted as the imaging distance, or the distance from the lens to the ground when the central axis of the lens of the first camera 10 or the second camera 20 when the aerial image 151 is captured is extended to the ground may be adopted as the imaging distance. For example, the flight altitude may be specified based on the data of the altimeter provided in the flying object equipped with the object detection device 1.

[0049] When the imaging distance H satisfies L1 ≤ H ≤ L2, the control device 40 divides the aerial image 151 into a 5-row and 8-column matrix. When the imaging distance H satisfies H < L1, the control device 40 divides the aerial image into a 2-row and 8-column matrix. When the imaging distance H satisfies L2 < H, the control device 40 divides the aerial image 151 into a 5-row and 16-column matrix.

[0050] In this way, when the imaging distance is long, the number of divisions is set to be larger than when the imaging distance is short. Therefore, when the object reflected in the aerial image 151 becomes small, the object can be searched in a finer divided area.

[0051] Returning to FIG. 3, the explanation of the flow chart of the setting process will be continued. In step S6, the control device 40 judges whether or not an operation to set a threshold P of the certainty for highlighting is detected (step S6). Highlighting refers to displaying a part of the image in the discrimination image 153 shown in FIG. 2, whose certainty is higher than the threshold P, surrounded by a thick frame. If an operation to set the threshold P is not detected, the control device 40 ends the process. In this case, a value previously inputted is used as the threshold P.

[0052] When an operation to set the threshold P is detected, the control device 40 waits until the threshold P is input (step S7). When an operation to set the threshold P is detected, the control device 40 stores the threshold P according to the input in the memory 60 (step S8), and ends the process.

[0053] Various operations in the flowcharts described above are accepted by operation unit 31 of display device 30, for example.

[0054] According to the setting process based on this flowchart, the number of divisions can be changed between manual and automatic settings. With the manual setting, the searcher can search for the target using a discrimination image divided by the desired number of divisions. With the automatic setting, the searcher can search for the target using a discrimination image appropriately divided according to the imaging distance.

[0055] Furthermore, according to the setting process based on this flowchart, the searcher can set the threshold value P of the probability of highlighting to any value. This allows the searcher to set an appropriate threshold value P depending on the search situation.

[0056] Fig. 4 is a flowchart showing the contents of the object detection process of the object detection device 1. Fig. 6 is a diagram showing an example of an aerial image captured by the object detection device 1. Fig. 7 is a diagram showing an example of a discrimination image displayed by the object detection device 1. Here, the flowchart shown in Fig. 4 will be explained with reference to the images shown in Figs. 6 and 7 as appropriate. Note that the process based on this flowchart is executed by the control device 40 provided in the object detection device 1.

[0057] First, the control device 40 acquires an aerial image from the first camera 10 (step S21). Fig. 6 shows an example of an aerial image 151 captured by the first camera 10. Since the aerial image 151 is an image captured from the sky, objects such as a victim are displayed small in the image.

[0058] Next, the control device 40 determines whether the division number is automatically set (step S22). More specifically, it determines whether an automatic setting flag is stored in the memory 60.

[0059] When the division number is set to the automatic setting, the control device 40 determines the division number based on the imaging distance H (step S23). This determination method has already been described using Fig. 5. The control device 40 divides the aerial photographed image based on the determined division number (step S25).

[0060] If the division number is not automatically set, the control device 40 reads out the division number from the memory 60 (step S24). The control device 40 divides the aerial photographed image based on the read division number (step S25).

[0061] Next, the control device 40 sequentially judges the divided images based on the estimation model 61 and calculates the accuracy (step S26). Next, the control device 40 specifies, from among the judged images, images having an accuracy greater than a threshold value P (step S27). The threshold value P is stored in the memory 60. Here, it is assumed that the threshold value P is set to 80%, for example.

[0062] Next, the control device 40 generates an image for discrimination based on the determination result (step S28). Next, the control device 40 outputs the generated image for discrimination to the display device 30 (step S29). FIG. 7 shows an example of the image for discrimination 153.

[0063] 7, image for discrimination 153 is divided into a plurality of images by frames. The images divided by frames correspond to the images divided by the process of step S25. Image for discrimination 153 displays the probability (%) that an object exists for each image divided by frames. Images with a probability exceeding threshold P (80%) are surrounded by a thick frame 161. This allows the searcher to easily identify the portion where the probability exceeds threshold P. Note that in FIG. 7, probabilities below 80% are uniformly indicated as x%, but in reality, the probability displayed corresponds to the portion of the image.

[0064] By touching the screen of the display device 30 with a finger, the searcher can select a portion of the discrimination image 153 that he or she wishes to check in more detail.

[0065] Returning to Fig. 4, the description of the flowchart will continue. The control device 40 judges whether or not an operation of selecting any part of the discrimination image is detected (step S30). If such an operation is detected before the waiting time T has elapsed, the control device 40 identifies a position on the ground corresponding to the selected image (step S31).

[0066] The control device 40 outputs a command to the second camera 20, which is a telephoto camera, to specify the imaging direction and lens magnification so that the position corresponding to step S31 is imaged (step S32). The second camera 20 captures a new aerial image in response to this command.

[0067] The control device 40 acquires an aerial image from the second camera 20 (step S33). Next, the control device 40 returns to step S22. Thereafter, the control device 40 repeats the processes related to dividing the aerial image, judging the image, generating the image for discrimination, and outputting the image for discrimination (steps S22 to S29). As a result, the image for discrimination based on the aerial image captured by the second camera 20 is displayed on the display device 30.

[0068] Here, when the aerial image captured first by the first camera 10 is referred to as the first aerial image, the aerial image captured by the second camera 20 can be referred to as the second aerial image. Also, when the determination image generated based on the first aerial image is referred to as the first determination image, the determination image generated based on the second aerial image can be referred to as the second determination image.

[0069] If necessary, the searcher can specify a portion of the second judgment image that he or she wants to enlarge and check. In this case, the searcher's operation is detected in step S30.

[0070] In this way, a searcher can easily find the target object by checking the first judgment image, the second judgment image, and, if necessary, the third judgment image.

[0071] When the control device 40 determines that an operation to select any part of the discrimination image has not been detected before the waiting time T has elapsed (YES in step S34), it determines whether or not an operation to end the process has been detected (step S35). If an operation to end the process has not been detected, the control device 40 returns the process to step S21. This causes processing to be performed based on a new aerial image captured by the first camera 10.

[0072] Therefore, if the target object cannot be found in the aerial image captured the first time by the first camera 10, the processing of steps S21 to S33 according to this flowchart is repeatedly executed based on a new aerial image captured again by the first camera 10.

[0073] 8 is a diagram showing various examples of discrimination image 153. As explained above, in discrimination image 153, images whose accuracy exceeds a threshold may be surrounded by a thick frame 161. In this case, the accuracy may be displayed only for images whose accuracy exceeds the threshold among the images surrounded by the frame ((A)). In this way, it is possible to prevent the display of the accuracy from impeding the visibility of the aerial photographed image.

[0074] Only the part of the image whose accuracy exceeds the threshold may be surrounded by a thick frame 161, and the other parts of the image may not be surrounded by a frame ((B)). In this case, the accuracy may not be displayed on the discrimination image 153.

[0075] Depending on the degree of likelihood of exceeding the threshold, the color of the frame surrounding the image corresponding to that likelihood may be changed ((C)). For example, as shown in (C), when the threshold is 80% or more and less than 90%, a blue frame 171 is used, and when the threshold is 90% or more and less than 95%, Yellow frame 172 If the threshold is between 95% and 100%, Red frame 173 may be adopted.

[0076] <Modification> A modification of this embodiment will now be described.

[0077] In this embodiment, the determination unit 54 determines the presence of an object using an estimation model 61 trained by supervised learning. However, the object may be identified from the segmented image using a pattern matching technique.

[0078] A thermal camera using infrared rays may be used as the first camera 10 and the second camera 20. When a thermal camera is used, the temperature can be visualized on the screen. Therefore, for example, a victim who is adrift on the sea at night can be identified by using the temperature difference between the water temperature and the body temperature. In this case, aerial images taken by a thermal camera may be used as learning data. A camera taking still images and a camera using infrared rays may be combined. In this case, the learning data uses aerial images taken by both cameras. Also, if there is a place that needs to be enlarged and captured, the place may be enlarged and captured by both the camera taking still images and the camera using infrared rays, or the place may be enlarged and captured by one of the cameras.

[0079] When the imaging distance of the aerial image is short, the object such as a missing person is displayed large in the aerial image. In this case, if the number of images divided and displayed by the discrimination images is large, it may be difficult to see. For this reason, the object detection device 1 increases the division number of the aerial image when the imaging distance is long, and decreases the division number of the aerial image when the imaging distance is short. Furthermore, the display device 30 displays the discrimination image according to the division number on the display device 30. However, conversely, the object detection device 1 may increase the division number of the aerial image when the imaging distance is short, and decrease the division number of the aerial image when the imaging distance is long. When the imaging distance becomes short, the object approaches, and at this time, it may be effective to increase the division number in order to confirm the location of the object in more detail.

[0080] The control device 40 may display the discrimination image 153 on a head mounted display worn by the searcher.

[0081] First camera 10 and second camera 20 may be cameras that capture moving images. Moving images captured by first camera 10 and second camera 20 may be continuously displayed on display device 30. In this case, display device 30 may switch the display on the screen to the discrimination image when it receives the discrimination image from control device 40.

[0082] As described above, the object detection device 1 does not detect an object using the entire aerial image, but divides the aerial image into multiple parts and detects the object using each of the divided images. Furthermore, the object detection device 1 calculates the accuracy for each divided image and displays a discrimination image including the accuracy on the display device 30. Therefore, the object detection device 1 can provide an image useful for finding an object from an aerial image.

[0083] It is sufficient to display information that allows the degree of accuracy to be determined in the discrimination image. For example, as described with reference to Fig. 8, the accuracy may be displayed in the discrimination image, or information that allows the degree of accuracy to be determined (for example, a thick frame 161, a blue frame 171, a yellow frame 172, a red frame 173) may be displayed without displaying the accuracy in the discrimination image.

[0084] [Aspects] It will be understood by those skilled in the art that the above-described embodiment and its modified examples are specific examples of the following aspects.

[0085] (Item 1) An object detection device according to one embodiment is an object detection device that detects an object from an aerial image, and includes an imaging device that captures a first aerial image, a control device, and a display device. The control device divides the first aerial image into a plurality of first images, calculates a probability that the object is included for each of the plurality of first images, and causes the display device to display a first discrimination image that is capable of discriminating between the degree of probability and the part of the first aerial image that corresponds to the probability.

[0086] According to the object detection device described in paragraph 1, it is possible to provide images that are useful for discovering objects from aerial images.

[0087] (Item 2) In the object detection device described in item 1, the first discrimination image includes a plurality of first images and an indication of a degree of certainty corresponding to each of the plurality of first images.

[0088] According to the object detection device described in paragraph 2, a searcher can check the degree of correspondence between the plurality of first images.

[0089] (Item 3) In the object detection device described in item 1 or 2, the first discrimination image includes an image for highlighting and displaying a portion of the first aerial image in which the accuracy exceeds a threshold value.

[0090] According to the object detection device described in paragraph 3, a searcher can easily identify a portion of the first aerial image whose accuracy exceeds a threshold value.

[0091] (Item 4) In the object detection device described in item 3, the control device changes the threshold value in response to a predetermined operation.

[0092] According to the object detection device described in paragraph 4, it becomes possible to display a portion of the first aerial image in an emphasized manner based on a threshold value desired by a searcher.

[0093] (Item 5) In the object detection device described in any one of items 1 to 4, when the control device receives an operation to specify any portion of the first discrimination image, it outputs an instruction to the imaging device to capture an image of the location corresponding to the specified portion.

[0094] According to the object detection device described in paragraph 5, a searcher can check an aerial image corresponding to a location where the object is thought to be present.

[0095] (Item 6) In the object detection device described in Item 5, the imaging device includes a first imaging device and a second imaging device capable of imaging the subject at a higher magnification than the first imaging device, and the control device outputs an instruction to the second imaging device to image a location corresponding to a specified portion.

[0096] According to the object detection device described in paragraph 6, it is possible to confirm a detailed aerial image in which a location where an object is thought to exist is enlarged.

[0097] (Item 7) In the object detection device described in Item 6, the control device divides the second aerial image captured in response to an instruction to capture an image into a plurality of second images, calculates the probability that the object is included in each second image, and displays on the display device a second discrimination image that is capable of discriminating between the degree of accuracy and the part of the second aerial image that corresponds to the degree of accuracy.

[0098] According to the object detection device described in paragraph 7, it is possible to provide images that are useful for finding an object from an aerial image of a location where the object is thought to exist.

[0099] (Item 8) In the object detection device described in any one of Items 1 to 7, the control device receives an operation to designate the number of divisions when dividing the first aerial photographed image into a plurality of first images.

[0100] According to the object detection device described in paragraph 8, it becomes possible to divide the first aerial photographed image based on the number of divisions desired by the searcher.

[0101] (Clause 9) In the object detection device described in any one of clauses 1 to 8, the control device calculates a likelihood that an object is included in each of the multiple first images, using a trained model for inferring the presence of an object in each of the multiple first images.

[0102] According to the object detection device described in paragraph 9, the accuracy of detecting an object can be improved by using a highly accurate trained model.

[0103] (Item 10) In the object detection device described in Item 9, the trained model is trained to calculate the probability that an object is included in each divided image based on training data including images divided from an aerial photograph and results showing that an object is present in the divided images.

[0104] According to the object detection device described in paragraph 10, it is possible to improve the accuracy of determining the presence of an object based on the divided images.

[0105] The embodiments disclosed herein should be considered to be illustrative and not restrictive in all respects. The scope of the present invention is defined by the claims, not by the description of the embodiments described above, and is intended to include all modifications within the meaning and scope of the claims. [Explanation of symbols]

[0106] 1 object detection device, 10 first camera, 20 second camera, 30 display device, 31 operation unit, 40 control device, 50 processor, 51 input / output unit, 52 reception unit, 53 division unit, 54, 102 judgment unit, 55 output unit, 60 memory, 61 estimation model, 62 neural network, 63 parameters, 100 learning device, 101 input unit, 103 learning unit, 111, 151 aerial image, 112 correct answer data, 152 image, 153 image for discrimination, 161 thick frame, 171 blue frame, 172 yellow frame, 173 red frame, 200 helicopter.

Claims

1. An object detection device for detecting an object from an aerial image, An imaging device that captures a first aerial image; A control device; a display device; The control device includes: Dividing the first aerial image into a plurality of first images; Calculating a likelihood that the object is included in each of the plurality of first images; displaying on the display device a first discrimination image capable of discriminating between a degree of accuracy and a portion of the first aerial photographed image corresponding to the accuracy; When the control device receives an operation to designate any portion of the first discrimination image, the control device outputs an instruction to the imaging device to image a location corresponding to the designated portion; the imaging device includes a first imaging device and a second imaging device capable of imaging a subject at a higher magnification than the first imaging device; The control device outputs an instruction to the second imaging device to capture an image of a location corresponding to the specified portion; The control device includes: Dividing the second aerial image captured in response to the instruction to capture the image into a plurality of second images; Calculating a likelihood that the object is included in each of the second images; An object detection device that displays on the display device a second discrimination image that is capable of discriminating between a degree of accuracy and a portion of the second aerial image that corresponds to the degree of accuracy.

2. The object detection device according to claim 1 , wherein the first discrimination image includes an indication of the plurality of first images and a degree of certainty corresponding to each of the plurality of first images.

3. The object detection device according to claim 1 or 2, wherein the first discrimination image includes an image for highlighting and displaying a portion of the first aerial image in which the accuracy exceeds a threshold value.

4. The object detection device according to claim 3 , wherein the control device changes the threshold value in response to a predetermined operation.

5. The object detection device according to any one of claims 1 to 4, wherein the control device accepts an operation for specifying the number of divisions when dividing the first aerial image into the plurality of first images.

6. The control device calculates a likelihood that the object is included in each of the plurality of first images using a trained model for inferring the presence of the object in each of the plurality of first images.

7. The object detection device according to claim 6, wherein the trained model is trained to calculate a probability that the object is included in each of the divided images based on training data including images obtained by dividing the aerial image and results indicating that the object is present in the divided images.

Images