Position estimation device, position estimation method, and program

Abstract

A position estimation device (10) comprises an image acquisition unit (110) and a position estimation unit (120). The image acquisition unit (110) acquires a plurality of images generated by an image capture device that captures images of a space of interest in which a flying object is flying. The position estimation unit (120) estimates the position of the flying object by processing the plurality of images. The position estimation unit (120) tentatively sets the type of the flying object and estimates the distance to and the speed of the flying object. The position estimation unit (120) then updates the type of the flying object using this speed and the rate of change in the area of the flying object in the images. The position estimation unit (120) repeats the processes described above.

Description

Position estimation device, position estimation method, and program

[0001] The present invention relates to a position estimation device, a position estimation method, and a program.

[0002] In recent years, large structures, such as wind power generation equipment, have been installed in various locations. There is a possibility that flying objects, such as birds, may collide with these structures. In response to this, the device described in Patent Document 1 recognizes the arrival of birds, calculates their positions, and estimates the locations of the arriving birds after a predetermined time, based on two sets of image data obtained by two camera modules. It then supplies sound signals to multiple speaker modules so that the sound reaches the estimated locations.

[0003] Japanese Patent Application Publication No. 10-4858

[0004] In the technology described in Patent Document 1, at least two image capturing devices are required to estimate the position of a flying object. In view of the above-mentioned problems, one example of the object of the present invention is to make it possible to estimate the position of a flying object using a single image capturing device.

[0005] According to one aspect of the present invention, there is provided an image acquisition unit that acquires a plurality of images generated by an image capture device that captures an image of a target space in which a flying object is flying, and a position estimation unit that processes the plurality of images to estimate a position of the flying object, wherein the position estimation unit: (1) provisionally sets a type of the flying object; (2) specifies three or more images arranged in chronological order as a target image group; (3) for each of the plurality of images included in the target image group, estimates a distance from the image capture device to the flying object using the provisionally set type and the area of ​​the flying object in the image belonging to the target image group; (4) calculates a rate of change in the area of ​​the flying object between the plurality of images included in the target image group; (5) estimates a speed of the flying object using the distance to the flying object and the position within the image in each of the plurality of images included in the target image group; and (6) updates the type of the flying object using the speed and the rate of change in the area. (7) Using the type updated in (6), the processes shown in (2) to (6) are performed again until a predetermined criterion is met, and a position estimation device is provided in which the estimated position of the flying object at the time the image was taken is indicated by the position of the flying object in the image and the distance calculated based on the image.

[0006] According to another aspect of the present invention, there are also provided a method for estimating the position of a flying object using the position estimation device described above, and a program for realizing the position estimation device described above.

[0007] According to one aspect of the present invention, the position of a flying object can be estimated using a single imaging device.

[0008] 5 is a diagram for explaining a first example of a usage environment of a position estimation device according to an embodiment. FIG. 6 is a diagram for explaining a second example of a usage environment of a position estimation device. FIG. 7 is a diagram for explaining an example of a functional configuration of a position estimation device. FIG. 8 is a diagram for explaining an example of a hardware configuration of a position estimation device. FIG. 9 is a diagram for explaining an example of processing performed by a position estimation device. FIG. 10 is a diagram for explaining step S30. FIG. 11 is a diagram for explaining the processing shown in steps S20 to S60 of FIG. 5. FIG. 12 is a diagram for explaining a detailed example of step S90. FIG. 13 is a diagram for explaining a detailed example of step S90. FIG. 14 is a diagram for explaining a detailed example of step S90. FIG. 15 is a diagram for explaining a usage environment of a position estimation device according to a modified example. FIG. 16 is a diagram for explaining an example of a method for specifying an estimated position of a flying object by a position estimation unit.

[0009] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, like components are designated by like reference numerals, and the description thereof will be omitted as appropriate.

[0010] (Embodiment) FIG. 1 is a diagram illustrating a first example of a usage environment of a position estimation device 10 according to an embodiment. In the example shown in this figure, the position estimation device 10 is a device that detects flying objects present around a structure 20 and estimates the position of the flying objects. For this detection and estimation, the position estimation device 10 uses images generated by a single imaging device 30. The imaging range of the imaging device 30 includes a target space. The target space includes an area into which flying objects should not enter. An example of this area is the area around the structure 20. The target space may or may not include the structure 20. Note that the target space includes both the structure 20 and a space within 2 km of the structure 20, for example. However, depending on the type of imaging device 30, the target space may also include a space within 10 km of the structure 20.

[0011] A typical example of a projectile is an animal such as a bird, but the projectile may also be another object, such as an airplane, a helicopter, or an object lifted by the wind.

[0012] The structure 20 is, for example, a wind power generation device, but may also be other equipment such as a steel tower. If the structure 20 has a mechanical or electrical structure, this structure is controlled by a control device. If the structure 20 is a wind power generation device, this structure is a blade. If the position estimation device 10 determines that the position or progression of the flying object meets a predetermined criterion and there is a possibility of collision with the structure 20, it transmits predetermined information (control information, described below) to the control device to stop or slow the movement of the mechanical or electrical structure. The control device then slows or stops the movement of the above-mentioned structure.

[0013] The image capturing device 30 is a monocular camera. The image capturing device 30 preferably has tilt, pan, and zoom functions (hereinafter referred to as TPZ functions). The installation position of the image capturing device 30 is preferably fixed. The frame rate of the images generated by the image capturing device 30 is optional. The image capturing device 30 may transmit the settings of the image capturing device 30 at the time of generating the image, such as the focal length, together with the generated image to the position estimation device 10.

[0014] The installation position of the image capture device 30 may be determined, for example, using a simulation. In this case, by inputting the angle of view, installation position, and camera direction of the image capture device 30 into a simulation device, the range that the image capture device 30 can capture can be calculated. Then, using the calculation result, an appropriate installation position for the image capture device 30 can be estimated.

[0015] The position estimation device 10 estimates the current position of a flying object and also estimates the future position of the flying object. If the flying object is an animal such as a bird, the position estimation device 10 controls the speaker 40 so that sound reaches the animal at the timing when the animal will arrive at its future position. In other words, the position estimation device 10 also functions as a control device for the speaker 40. In this case, the position estimation device 10 outputs a sound from the speaker 40 to convey predetermined information to the animal.

[0016] An example of a sound that the position estimation device 10 outputs from the speaker 40 is a sound for conveying predetermined information from the same species of the detected animal to the animal, or a sound for conveying predetermined information from a closely related species of the detected animal to the animal.

[0017] An example of the predetermined information conveyed by sound is information that can change the behavior of the animal, such as a sound that indicates that a natural enemy is approaching or a sound that indicates that food is in a different location or direction. For example, if the detected animal is a prey animal, the sound output by the speaker 40 is a sound that indicates that a predator is approaching. On the other hand, if the detected animal is a predator, the sound output by the speaker 40 is a sound that indicates that a complementary animal is in a different location. In other words, the position estimation device 10 changes the sound depending on the type of animal detected.

[0018] The speaker 40 preferably has directionality. An example of the speaker 40 is a parametric speaker. The orientation of the speaker 40 is variable. The position estimation device 10 controls the orientation of the speaker 40 so that the emission direction of the speaker 40 is oriented toward the future position of the animal.

[0019] FIG. 2 is a diagram illustrating a second example of a usage environment of the position estimation device 10. In the example shown in this figure, the position estimation device 10 predicts the future location of an animal that is about to invade a specific piece of land (hereinafter referred to as the target area) and controls the speaker 40 to prevent the animal from invading the target area. In the example shown in this figure, the target animals are, for example, wild boars, deer, monkeys, rats, or their related species, which are animals that cause so-called animal damage. In the example shown in this figure, "flying object" will be read as "animal" in the following description.

[0020] 3 is a diagram showing an example of the functional configuration of the position estimation device 10. The position estimation device 10 includes an image acquisition unit 110 and a position estimation unit 120. The image acquisition unit 110 acquires a plurality of images captured by the imaging device 30. The position estimation unit 120 processes the plurality of images to estimate the position of the flying object.

[0021] The position estimation unit 120 performs the following processes: (1) provisionally sets the type of flying object; (2) identifies three or more images arranged in chronological order as a target image group; (3) for each of the multiple images included in the target image group, estimates the distance from the shooting device to the flying object using the provisionally set type and the area of ​​the flying object in the images belonging to the target image group; (4) calculates the rate of change in the area of ​​the flying object between the multiple images included in the target image group; (5) estimates the speed of the flying object using the distance to the flying object and its position in each of the multiple images included in the target image group; (6) updates the type of flying object using the rate of change in speed and area; and (7) performs the processes described in (2) to (6) again using the type updated in (6) until a predetermined criterion is met. The estimated position of the flying object at the time the image was captured is then indicated by the position of the flying object in the image and the distance calculated based on the image.

[0022] In the above (3), it is preferable that the position estimation unit 120 uses the focal length when the image capturing device 30 generates the image.

[0023] In addition, in (3), the position estimation unit 120 calculates the average value of the area occupied by the flying object between the currently processed image and at least one other image belonging to the target image group, and estimates the distance using the average value. At this time, the position estimation unit 120 also generates motion information indicating the motion state of the flying object. The motion information includes at least one of the radius of rotation, angular velocity, acceleration, and angular acceleration when the trajectory of the flying object is approximated as an arc. The center of the radius of rotation is the position of the image capture device 30. The position estimation unit 120 then estimates the future position of the flying object. The future position is an estimation result of the position of the flying object at a first timing in the future. The first timing is a predetermined time from the present. This predetermined time is, for example, a value between 1 / 120 seconds and 5 seconds, but may be longer. When estimating the future position, the position estimation unit 120 uses the position, velocity, and motion information of the flying object, as well as time-series information on at least one of these.

[0024] The processing performed by the position estimation unit 120 will be described in detail with reference to other figures.

[0025] The position estimation device 10 also functions as a control device that controls the speaker 40 , and therefore includes a control unit 130 and a storage unit 140 .

[0026] If the flying object is an animal, the control unit 130 controls the speaker 40 to output a sound for conveying predetermined information to the animal so that the animal reaches its future position at a first timing. Specific examples of the predetermined information are as described with reference to FIG. 1.

[0027] The storage unit 140 stores data that serves as the sound source for the sound output by the speaker 40. Hereinafter, this data will be referred to as sound data. The storage unit 140 stores sound data by type of animal, for example, by species. The control unit 130 then reads out from the storage unit 140 the sound data that corresponds to the type of animal detected by the position estimation unit 120, and outputs the sound data from the speaker 40.

[0028] 1, the speaker 40 has directionality and its direction is variable. The control unit 130 controls the direction of the speaker 40. As an example, the control unit 130 controls the direction of the speaker 40 so that the emission direction of the speaker 40 is oriented toward the future position of the animal.

[0029] 4 is a diagram showing an example of the hardware configuration of the position estimation device 10. The position estimation device 10 includes a bus 1010, a processor 1020, a memory 1030, a storage device 1040, an input / output interface 1050, and a network interface 1060.

[0030] The bus 1010 is a data transmission path for transmitting and receiving data among the processor 1020, memory 1030, storage device 1040, input / output interface 1050, and network interface 1060. However, the method of connecting the processor 1020 and the like to each other is not limited to bus connection.

[0031] The processor 1020 is implemented by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or the like.

[0032] The memory 1030 is a main storage device realized by a RAM (Random Access Memory) or the like.

[0033] The storage device 1040 is an auxiliary storage device realized by removable media such as a hard disk drive (HDD), a solid state drive (SSD), or a memory card, or a read-only memory (ROM). The storage device 1040 stores program modules that realize each function of the position estimation device 10 (e.g., the image acquisition unit 110, the position estimation unit 120, and the control unit 130). The processor 1020 loads each of these program modules into the memory 1030 and executes them, thereby realizing each function corresponding to the program module. The storage device 1040 also functions as the memory unit 140.

[0034] The input / output interface 1050 is an interface for connecting the position estimation device 10 to various input / output devices. For example, the position estimation device 10 may communicate with at least one of the image capturing device 30 and the speaker 40 via the input / output interface 1050.

[0035] The network interface 1060 is an interface for connecting the position estimation device 10 to a network. This network is, for example, a local area network (LAN) or a wide area network (WAN). The network interface 1060 may be connected to the network wirelessly or by wire. The position estimation device 10 may communicate with at least one of the image capture device 30 and the speaker 40 via the network interface 1060.

[0036] 5 is a diagram illustrating an example of processing performed by the position estimation device 10. In the processing shown in this figure, the imaging device 30 repeatedly generates images. Each time the imaging device 30 generates an image, it immediately transmits the image to the position estimation device 10. The image acquisition unit 110 of the position estimation device 10 acquires these multiple images and stores them in a storage unit, for example, the storage unit 140.

[0037] Each time the image acquisition unit 110 acquires a new image, the position estimation unit 120 determines whether or not the image contains a flying object. For example, this process involves determining whether or not a moving object exists in the image. If a flying object is found to be present, the process shown in this figure is initiated.

[0038] First, the position estimation unit 120 provisionally sets the type of the flying object. This process corresponds to (1) described with reference to Fig. 2. When an animal is provisionally set as the flying object, it is preferable that the type set here be specified down to the species of animal (step S10).

[0039] Next, the position estimation unit 120 selects a predetermined number of images from the images stored in the storage unit 140. The predetermined number is at least three, but may be more. Hereinafter, the selected images will be referred to as a target image group. The images constituting the target image group are consecutive in time series and all contain flying objects. For example, if the target image group is composed of three images, the target image group generated initially will be composed of the image that first contained the flying object and the two images generated afterwards (step S20). This process corresponds to (2) described using FIG. 2.

[0040] Next, the position estimation unit 120 estimates the distance from the image capture device 30 to the flying object at the time when each image was captured for each of the multiple images that make up the target image group (step S30). The estimated position of the flying object at the time when each image was generated is indicated by the position of the flying object in that image and the distance calculated based on that image. This process corresponds to (3) described using Figure 2.

[0041] FIG. 6 is a diagram illustrating step S30. Once the type of flying object has been identified, the absolute size of the flying object falls within a predetermined range set for that type, even taking into account individual differences. Therefore, the proportion of the area (e.g., the number of pixels) occupied by the flying object in the image varies depending on the distance from the image capture device 30 to the flying object. Specifically, as the distance from the image capture device 30 to the flying object increases, the proportion of the area occupied by the flying object in the image decreases. The position estimation unit 120 uses this relationship to estimate the distance from the image capture device 30 to the flying object.

[0042] The proportion of the area of ​​the image occupied by the flying object also varies depending on the focal length of the image capturing device 30. Therefore, if the focal length of the image capturing device 30 is variable, the position estimation unit 120 preferably uses the focal length of the image capturing device 30 at the time the image was generated when calculating the distance from the image capturing device 30 to the flying object.

[0043] Returning to Fig. 5, after step S30, the position estimation unit 120 uses the position of the flying object at each timing when each image was captured to estimate the velocity of the flying object, the radius of rotation when the trajectory of the flying object is approximated to an arc, the angular velocity, the acceleration, and the angular acceleration (step S40). This process corresponds to (3) and (5) described using Fig. 2.

[0044] Two images are required to estimate the velocity and angular velocity, and three or more images are required to estimate the acceleration and angular acceleration. Specifically, the position of the flying object calculated in step S30 indicates the trajectory of the flying object. Using this trajectory, the radius of rotation of the flying object can be estimated. Furthermore, using the amount of change in the position of the flying object between at least two adjacent images on the time axis, the velocity and acceleration of the flying object can be estimated. Furthermore, using the radius of rotation of the flying object and the amount of change in the position of the flying object between the above-mentioned two images, the angular velocity and angular acceleration of the flying object can be estimated. The position estimation unit 120 can perform the estimation shown in step S40 on, for example, images other than the first image on the time axis among the multiple images constituting the target image group.

[0045] Next, the position estimation unit 120 calculates the rate of change of the area occupied by the flying object calculated in step S30, and calculates the average value of this area (step S50). This process corresponds to (4) described with reference to FIG. 2.

[0046] For example, the position estimation unit 120 performs the following process on each of the multiple images that make up the target image group, excluding the last image on the time axis. First, the ratio of the area of ​​the flying object in that image to the area of ​​the flying object in the next image is taken as the area change rate. Then, the average value of these two areas is calculated. These area change rates and average values ​​are set to be midway between the timings at which these two images were generated on the time axis. Note that when step S50 is performed, the number of data items on the time axis is reduced by one.

[0047] The position estimation unit 120 repeats the process from step S30 to step S50 using the average area calculated in step S50 until a first criterion is met (step S60). One example of the first criterion is that the process from step S30 to step S50 is repeated a predetermined number of times.

[0048] FIG. 7 is a diagram for explaining the processing shown in steps S20 to S60 in FIG. 5. In the example shown in this diagram, the target image group is composed of 10 images. FIG. 7(A) shows the movement path (trajectory) of the flying object on the image, i.e., the trajectory projected onto the xy plane, and each diagram in FIG. 7(B) shows the distance from the image capture device 30 to the flying object, i.e., the estimated path of the flying object projected onto the xz plane. The x-axis and y-axis represent the two axes that make up the image, and the z-axis represents the direction away from the image capture device 30, as shown in FIG. 1.

[0049] As described above with respect to step S50, when step S50 is performed, the number of data points on the time axis is reduced by one. Therefore, each time the processes shown in steps S20 to S50 are repeated, the number of measurement points indicating the estimated movement path, i.e., the measurement timings, is reduced by one. Instead, the accuracy of the distance estimation, i.e., the accuracy of the flying object position estimation, is improved.

[0050] Returning to FIG. 6 . If the first criterion is met (step S60: Yes), the position estimation unit 120 determines whether or not the target image group needs to be updated (step S70). An example of a case where the target image group needs to be updated is when there are images remaining that have not yet been selected as part of the target image group. Another example of a case where the target image group needs to be updated is when the image acquisition unit 110 has acquired new images from the imaging device 30. If the target image group needs to be updated (step S70: Yes), the position estimation unit 120 updates the target image group (step S20) and repeats the processes shown in steps S30 to S70.

[0051] The target image group is updated using a first-in, first-out method. That is, the position estimation unit 120 removes the oldest image in chronological order from the target image group among the images included in the current target image group. Then, the position estimation unit 120 identifies the most recent image among the images included in the current target image group, and adds the image that is next in chronological order to the target image group. That is, each target image group corresponds to the timing when the most recent image in that target image group was captured.

[0052] Thereafter, the position estimation unit 120 calculates a moving average of each value calculated in steps S40 to S50 (step S80). This process corresponds to the process performed between steps (5) and (6) shown in Figure 2. The position estimation unit 120 then estimates the future position of the flying object using the moving averages of the position and velocity (step S90).

[0053] 8 to 11 are diagrams for explaining detailed examples of step S90.

[0054] As shown in FIG. 8 , the position estimation unit 120 first calculates a K-point moving average of the area of ​​the flying object, a minimum value within the K points, and a maximum value within the K points for each group of target images. Specifically, the position estimation unit 120 calculates a K-point moving average of the area of ​​the flying object and associates the calculated average with the most recent image in chronological order among the images used to calculate the moving average. The position estimation unit 120 also recognizes the area of ​​the flying object in each of the K images used to calculate the moving average. The position estimation unit 120 then associates each of the K minimum and maximum area values ​​with the most recent image in chronological order among the images used to calculate the moving average. In this way, the minimum and maximum area values ​​are associated with each of the multiple images.

[0055] Then, as shown in FIG. 9 , the position estimation unit 120 identifies, for each group of target images, an estimated movement path of the flying object based on the minimum value of the area (hereinafter referred to as the first estimated movement path) and an estimated movement path based on the maximum value of the area (hereinafter referred to as the second estimated movement path). The first estimated movement path indicates the movement path when the estimated distance from the image capture device 30 to the flying object is the maximum value of the estimation range (i.e., when the flying object is farthest from the image capture device 30). On the other hand, the second estimated movement path indicates the movement path when the estimated distance from the image capture device 30 to the flying object is the minimum value of the estimation range (i.e., when the flying object is closest to the image capture device 30). The range between these two estimated movement paths indicates the estimated range of the flying object's movement path. The flying object is likely to fly within this estimated range. Therefore, the estimated movement path of the flying object can be displayed as a band.

[0056] Although not shown, the position estimation unit 120 identifies an estimated movement path (hereinafter referred to as a third estimated movement path) based on the moving average of the area. The third estimated movement path is located between the first estimated movement path and the second estimated movement path.

[0057] As shown in Figure 10, the position estimation unit 120 calculates, for each target image group, a radius of rotation indicating the first estimated movement path (minimum area), a radius of rotation indicating the second estimated movement path (maximum area), and a radius of rotation indicating the third estimated movement path (moving average area).

[0058] 11, the position estimation unit 120 superimposes estimated movement paths based on the plurality of target image groups. Each of the plurality of target image groups is calculated based on the time (t ０ , t １ , t ２ ...). Each estimated movement path is shown as an arc centered on the position of the image capture device 30 and having a radius based on the third estimated movement path. The position estimation unit 120 then estimates the future movement path of the flying object using the progress of these estimated movement paths, and estimates the future position of the flying object using this movement path. As an example, the position estimation unit 120 can estimate the future position of the flying object using a Kalman filter. In this case, the future position of the flying object can be predicted even if there are missing values. Furthermore, the robustness of the prediction of the future position is improved.

[0059] Note that the calculation of the estimated path of travel uses the moving averages of the position, speed, and movement information of the flying object, so that the prediction of the future position of the flying object essentially uses the moving averages of the position, speed, and movement information of the flying object.

[0060] 11 , the future position of the flying object may be estimated based on a first estimated latitudinal route (i.e., the route farthest from the image capture device 30) and a second estimated latitudinal route (i.e., the route closest to the image capture device 30) for each estimated movement route. In this case, the position estimation unit 120 may use these two future positions to calculate the future position to be actually used. As an example, the position estimation unit 120 may use an average position of these two future positions as the future position to be actually used.

[0061] Returning to FIG. 5 . After step S90, the position estimation unit 120 estimates the type of flying object using the speed and area change rate of the flying object (step S100). Both the speed and area change rate indicate the characteristics of the flying object's movement. For example, if the flying object is a bird, the speed range is determined by the bird's species, and the characteristics of the wing movement when flapping and the movement when turning vary depending on the bird's species. Therefore, the position estimation unit 120 can identify the type of flying object using the speed and area change rate. Here, if the flying object is an animal, it is preferable that the position estimation unit 120 estimates the species as well. This process corresponds to (6) described using FIG. 2 .

[0062] If the second criterion is not satisfied (step S110: No), the location estimation unit 120 returns to step S30. This process corresponds to (7) described with reference to Fig. 2. Here, the second criterion is, for example, either (a) or (b) below.

[0063] (a) The loop from step S30 to step S110 is repeated a preset number of times. This criterion is suitable when it is necessary to speed up the processing by the position estimation unit 120 to some extent.

[0064] (b) The difference between the estimated position of the flying object at the first timing and the future position of the flying object that was generated before the first timing (the position where the flying object is expected to be at the first timing) is below a reference value. This criterion indicates that the difference between the predicted result of the future position (corresponding to the future position generated before the first timing) and the actual measured value (corresponding to the estimated position at the first timing) is below a reference value, which indicates that the accuracy has become sufficiently high.

[0065] It is preferable that the second criterion be switchable in accordance with, for example, an input from the user.

[0066] Thereafter, if the type of flying object estimated in step S100 is an animal, the control unit 130 reads sound data corresponding to the species of animal from the storage unit 140. The control unit 130 also orients the speaker 40 toward the future position identified in step S90. The control unit 130 then causes the speaker 40 to output the sound data read from the storage unit 140 (step S120). At this time, the control unit 130 controls the timing of sound emission so that the sound reaches the future position at the timing when the flying object is predicted to arrive at the future position.

[0067] As explained with reference to FIG. 1, the sound output from the speaker 40 is a sound for conveying predetermined information to the animal. When the animal recognizes the information contained in this sound, it changes its direction of movement in accordance with that information. This prevents the animal from entering areas where it should not. In this case, there is no need to intimidate the animal with the volume of the sound. Therefore, the volume of the sound output from the speaker 40 can be smaller than that of the prior art.

[0068] As described above, according to this embodiment, the current and future positions of a flying object can be estimated using a single imaging device. Furthermore, if the flying object is an animal such as a bird, even if the sound output from the speaker 40 is small, it is possible to prevent the animal from entering an area where it should not enter.

[0069] (Modification) Fig. 12 is a diagram illustrating a usage environment of a position estimation device 10 according to this modification. In the above-described embodiment, the position estimation device 10 is used together with one imaging device 30. In this modification, the position estimation device 10 is used together with multiple imaging devices 30, preferably three or more imaging devices 30. The multiple imaging devices 30 are arranged apart from each other and include the target space (or target area) in their imaging ranges. In other words, the multiple imaging devices 30 capture images of the target space from different directions.

[0070] It is preferable that all of the plurality of image capturing devices 30 are monocular cameras, which reduces the cost of the image capturing devices 30.

[0071] The image acquisition unit 110 of the position estimation device 10 acquires multiple images from multiple imaging devices 30, and the position estimation unit 120 of the position estimation device 10 calculates information indicating the estimated position of the flying object for each of the multiple imaging devices 30, and uses this information for each of the imaging devices 30 to identify the estimated position of the flying object.

[0072] An example of information indicating the estimated position of the flying object is the radius of rotation described with reference to Fig. 10. In this case, the position estimation unit 120 performs processing up to calculating the radius of rotation for each of the multiple image capture devices 30. Then, the position estimation unit 120 uses the calculation results of these multiple radii of rotation to identify the estimated position of the flying object.

[0073] 13 is a diagram illustrating an example of a method for identifying the estimated position of a flying object by the position estimation unit 120. In the example shown in this figure, the position estimation unit 120 calculates the radius of rotation for each of the multiple image capture devices 30. Then, for each of the multiple image capture devices 30, the position estimation unit 120 draws a circle centered at the position of that image capture device 30 and with a radius equal to the radius of rotation corresponding to that image capture device 30. The position estimation unit 120 then estimates that the flying object is present where these multiple circles overlap.

[0074] In addition, the position estimation unit 120 may use the distance calculated in step S30 and the position of the flying object in the image to calculate the position of the flying object in real space for each of the multiple imaging devices 30, and the average of these calculation results may be used as the estimated position of the flying object.

[0075] 9 for each of the image capturing devices 30, and estimates that the flying object is located in an area where these estimated paths overlap, thereby specifying the estimated path to be actually used. In this case, the accuracy of the estimated path is increased, and therefore the accuracy of the estimation of the future position of the flying object is also increased.

[0076] The position estimation unit 120 then determines the trajectory of the flying object using the transition of the estimated position, and estimates the future position using this trajectory. The control unit 130 then directs the speaker 40 toward the future position, and emits sound from the speaker 40.

[0077] According to this embodiment, the position estimation device 10 estimates the position of the flying object using images captured by the multiple image capture devices 30. This increases the accuracy of estimating the position of the flying object. Furthermore, the multiple image capture devices 30 can all use monocular cameras. This reduces the cost of the image capture devices 30 while increasing the accuracy of estimating the position of the flying object.

[0078] Although the embodiments of the present invention have been described above with reference to the drawings, these are merely examples of the present invention, and various other configurations can also be adopted.

[0079] In addition, although the flowcharts used in the above description show multiple steps (processes) in a sequential order, the order of steps executed in each embodiment is not limited to the order shown. In each embodiment, the order of the steps shown in the drawings can be changed as long as it does not cause any problems in terms of content. Furthermore, the above-described embodiments can be combined as long as the content is not contradictory.

[0080] REFERENCE SIGNS LIST 10 Position estimation device 20 Structure 30 Photography device 40 Speaker 110 Image acquisition unit 120 Position estimation unit 130 Control unit 140 Storage unit

Claims

1. an image acquisition unit that acquires a plurality of images generated by an image capture device that captures an image of a target space in which a flying object is flying; a position estimation unit that estimates a position of the flying object by processing the plurality of images; Equipped with The position estimation unit is (1) provisionally setting the type of the flying object; (2) identifying three or more of the images arranged in chronological order as a group of target images; (3) for each of the plurality of images included in the target image group, estimating a distance from the image capture device to the flying object using the provisionally set type and an area of ​​the flying object in the images belonging to the target image group; (4) calculating a rate of change in area of ​​the flying object between a plurality of the images included in the target image group; (5) estimating a speed of the flying object using a distance to the flying object and a position within the image in each of the plurality of images included in the target image group; (6) updating the type of the flying object using the rate of change of the velocity and the area; (7) repeating the processes described in (2) to (6) above using the type updated in (6) above until a predetermined criterion is met; A position estimation device in which an estimated position of the flying object at the time the image was captured is indicated by the position of the flying object in the image and the distance calculated based on the image.

2. 2. The position estimation device according to claim 1, The position estimation device according to (3), wherein the position estimation unit uses a focal length when the photographing device generates the image.

3. 3. The position estimation device according to claim 1, In (3), the position estimation unit calculates the average value of the area between the image and at least one other image belonging to the target image group, and estimates the distance using the average value.

4. 4. The position estimation device according to claim 3, The position estimation unit is In the method (3), estimating motion information of the flying object, which is at least one of a radius of rotation, an angular velocity, an acceleration, and an angular acceleration; A position estimation device that estimates a future position of the flying object using the position, the velocity, and the motion information of the flying object.

5. 5. The position estimation device according to claim 4, The position estimation unit is After performing the processes shown in (1) to (6) at least once, each time the image acquisition unit acquires a new image, the target image group is updated in (2), and the process in (3) is performed on the updated target image group; Between (5) and (6), Calculating a moving average of each of the position, the velocity, and the motion information of the flying object, which are repeatedly calculated in (3), A position estimation device that estimates the future position using the moving average.

6. 5. The position estimation device according to claim 4, A position estimation device, wherein the specified criterion is that the difference between the estimated position at a first timing and the future position at the first timing generated before the first timing is less than or equal to a reference value.

7. 3. The position estimation device according to claim 1, The target space is photographed by a plurality of the photographing devices from different directions, the image acquisition unit acquires the images from each of the plurality of photographing devices; The position estimation unit calculates information indicating a position of the flying object for each of the multiple imaging devices, and estimates the position of the flying object using the information.

8. 3. The position estimation device according to claim 1, The position estimation device, wherein the imaging device is a monocular camera.

9. 3. The position estimation device according to claim 1, A position estimation device, wherein the predetermined criteria is selectable.

10. The computer an image acquisition process for acquiring a plurality of images generated by an image capturing device that captures a target space in which a flying object is flying; a position estimation process for estimating a position of the flying object by processing the plurality of images; Do the following: In the position estimation process, the computer (1) provisionally setting the type of the flying object; (2) identifying three or more of the images arranged in chronological order as a group of target images; (3) for each of the plurality of images included in the target image group, estimating a distance from the image capture device to the flying object using the provisionally set type and an area of ​​the flying object in the images belonging to the target image group; (4) calculating a rate of change in area of ​​the flying object between a plurality of the images included in the target image group; (5) estimating a speed of the flying object using a distance to the flying object and a position within the image in each of the plurality of images included in the target image group; (6) updating the type of the flying object using the rate of change of the velocity and the area; (7) repeating the processes described in (2) to (6) above using the type updated in (6) above until a predetermined criterion is met; A position estimation method, wherein an estimated position of the flying object at the time the image was captured is indicated by the position of the flying object in the image and the distance calculated based on the image.

11. On the computer, An image acquisition function for acquiring a plurality of images generated by an image capturing device that captures a target space in which a flying object is flying; a position estimation function for estimating a position of the flying object by processing the plurality of images; Let them have The position estimation function includes: (1) provisionally setting the type of the flying object; (2) identifying three or more of the images arranged in chronological order as a group of target images; (3) for each of the plurality of images included in the target image group, estimating a distance from the image capture device to the flying object using the provisionally set type and an area of ​​the flying object in the images belonging to the target image group; (4) calculating a rate of change in area of ​​the flying object between a plurality of the images included in the target image group; (5) estimating a speed of the flying object using a distance to the flying object and a position within the image in each of the plurality of images included in the target image group; (6) updating the type of the flying object using the rate of change of the velocity and the area; (7) repeating the processes described in (2) to (6) above using the type updated in (6) above until a predetermined criterion is met; The program indicates an estimated position of the flying object at the time the image was captured based on the position of the flying object in the image and the distance calculated based on the image.

Images