Information processing device, flying creature extermination system, pest capture system, information processing method, and program

Abstract

To provide an information processing device, a flying creature extermination system, a noxious animal capturing system, an information processing method and a program which can predict a position of a moving object even when the moving object moves at a high speed.SOLUTION: An information processing device comprises: a derivation unit which derives time-series state information of a moving object including at least a position on the basis of point group information expressing the moving object extracted from each of a plurality of images obtained by imaging the moving object in time series; and a prediction unit which predicts a future position of the moving object on the basis of the time-series state information and peripheral environmental information of the moving object.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an information processing device, a flying creature extermination system, a pest capture system, an information processing method, and a program. [Background technology]

[0002] Conventionally, there are known techniques for measuring the positions of moving objects such as insects and animals. For example, Patent Document 1 describes a technique for acquiring stereo images associated with the 3-D movement-related behavior of a laboratory animal and processing the acquired image frames to obtain 3-D movement parameters of the laboratory animal. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2009-500042 Summary of the Invention [Problem to be solved by the invention]

[0004] The technology described in Patent Document 1 measures the position of a moving object, but does not predict the position of the moving object. In particular, when the moving object is a fast-moving pest or the like, the measured position may differ significantly from the actual position due to the time lag caused by reading images captured by a camera and performing calculations. Therefore, there is a need for a technology that can predict the position of a moving object even when the moving object is moving at high speed.

[0005] The present invention has been made taking these circumstances into consideration, and one of its objects is to provide an information processing device, a flying creature extermination system, a pest capture system, an information processing method, and a program that can predict the position of a moving object even when the moving object is moving at high speed. [Means for solving the problem]

[0006] An information processing device according to one aspect of the present invention includes a derivation unit that derives time-series state information of a moving object, including at least a position, based on point cloud information representing the moving object extracted from each of a plurality of images of the moving object taken in time series, and a prediction unit that predicts the future position of the moving object based on the time-series state information and information about the surrounding environment of the moving object.

[0007] The prediction unit may predict the future position by repeating a process of correcting a state value of the moving object estimated in advance using the observed value of the time-series state information to obtain a state value of the moving object estimated retroactively.

[0008] The prediction unit may predict the future position by inputting the time-series state information and the surrounding environment information into a trained model that takes the time-series state information and surrounding environment information of a moving object as input and is trained to output the future position of the moving object.

[0009] The surrounding environment information may include at least one of information regarding wind direction, wind speed, air temperature, temperature, precipitation, position of a light source, spectral intensity of a light source, season, and time of day in the environment where the moving object is located.

[0010] the moving object is a flying pest, The time-series state information may further include at least one of the roll angle, pitch angle, wing flapping frequency, wing direction, wing angle, abdomen direction, abdomen twist, abdomen bending, head direction, head twist, antenna direction, difference in movement of both wings, leg direction, and abdomen size of the pest.

[0011] the moving object is an animal belonging to the avian family, The time-series state information may further include at least one of the animal's wing flapping frequency, wing orientation, wing angle, abdomen orientation, head orientation, leg orientation, and abdomen size.

[0012] the moving object is a quadrupedal animal, The time-series state information may further include at least one of the orientation of the animal's abdomen, the orientation of its head, the orientation of its legs, and the size of its abdomen. [Effects of the Invention]

[0013] According to the present invention, it is possible to predict the position of a moving object even when the moving object is moving at high speed, and further, it is possible to predict the position of a moving object even when the moving object frequently changes speed or moves non-linearly. [Brief explanation of the drawings]

[0014] [Figure 1] 1 is a diagram showing an example of a usage environment and configuration of an information processing device 100 according to a first embodiment. [Figure 2] 10 is a diagram showing an example of a three-dimensional point cloud extracted by a point cloud extraction unit 120. FIG. [Figure 3] 10 is a diagram for explaining a method in which the state derivation unit 130 extracts state information relating to wings of flying pests. FIG. [Figure 4] 10 is a diagram for explaining a method by which the state derivation unit 130 extracts state information relating to the abdomen of a flying pest. FIG. [Figure 5] 10 is a diagram for explaining a method by which the state derivation unit 130 extracts state information relating to the head of a flying pest. FIG. [Figure 6] 10 is a diagram for explaining a method by which the state derivation unit 130 extracts state information relating to the antennae of flying pests. FIG. [Figure 7] 10 is a diagram for explaining a method by which the state derivation unit 130 extracts state information relating to the legs of flying pests. FIG. [Figure 8] FIG. 1 is a diagram for explaining an outline of a method for predicting future positions of flying pests by Kalman filter processing. [Figure 9] This is a diagram for explaining a method for predicting the future positions of flying pests using a deep neural network. [Figure 10]The flow of processing executed by the information processing device 100 according to the first embodiment will be described. [Figure 11] FIG. 10 is a diagram illustrating an example of a usage environment and configuration of an information processing device 100 according to a second embodiment. [Figure 12] The flow of processing executed by the information processing device 100 according to the second embodiment will be described. DETAILED DESCRIPTION OF THE INVENTION

[0015] [First embodiment] An information processing device 100 according to a first embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a diagram showing an example of a usage environment and configuration of the information processing device 100 according to the first embodiment. The information processing device 100 operates in cooperation with, for example, a camera 10 and a laser irradiation device 50. The combination of the information processing device 100 and the laser irradiation device 50 is an example of a "flying creature extermination system."

[0016] The camera 10 is, for example, a 3D camera such as a stereo camera that can capture images of an object from different directions to obtain not only two-dimensional (vertical and vertical) information but also depth information of the object. Alternatively, the camera 10 may be a 3D laser scanner such as a LIDAR (light detection and ranging) scanner. The camera 10 is installed in an area where flying pests such as common cutworms live, facing in any direction (e.g., horizontal or vertical with respect to the ground). The camera 10 is communicably connected to the information processing device 100 via a wired cable or a wireless network. The camera 10 captures images of the flying pests in time series in response to an instruction from the information processing device 100 or periodically, and transmits the captured 3D images (hereinafter, sometimes simply referred to as "images") to the information processing device 100. FIG. 1 illustrates, as an example, an example in which the camera 10 is installed horizontally with respect to the ground, and the 3D images are recognized with the vertical direction of the camera 10 as the X-axis, the horizontal direction as the Y-axis, and the height direction as the Z-axis. Flying pests are an example of a "moving object."

[0017] The laser irradiation device 50 is, for example, a high-power laser device that outputs a laser capable of killing flying pests, and is communicably connected to the information processing device 100 via a wired cable or a wireless network. More specifically, when the laser irradiation device 50 receives the future position of the flying pest predicted by the position prediction unit 140 of the information processing device 100 (described later), it controls the built-in galvanometer mirror to irradiate the laser toward the future position, and outputs the laser. This makes it possible to exterminate the flying pests.

[0018] The information processing device 100 is a computer device such as a personal computer or a tablet terminal. The information processing device 100 includes, for example, an image acquisition unit 110, a point cloud extraction unit 120, a state derivation unit 130, a position prediction unit 140, and a storage unit 150. Each of the image acquisition unit 110, the point cloud extraction unit 120, the state derivation unit 130, and the position prediction unit 140 is realized by, for example, a hardware processor such as a CPU (Central Processing Unit) executing a program (software). Some or all of these components may be realized by hardware (including circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a GPU (Graphics Processing Unit), or may be realized by a combination of software and hardware. The program may be stored in advance in a storage device such as a hard disk drive (HDD) or flash memory (a storage device having a non-transitory storage medium), or may be stored in a removable storage medium (a non-transitory storage medium) such as a DVD or CD-ROM, and installed by inserting the storage medium into a drive device. The storage unit 150 is realized by a storage device such as a HDD, flash memory, or RAM (Random Access Memory).

[0019] The image acquisition unit 110 acquires images of flying pests photographed in time series by the camera 10 from the camera 10 and stores them in the storage unit 150.

[0020] The point cloud extraction unit 120 extracts, as point cloud information, a three-dimensional point cloud representing a flying pest from each of a plurality of images of the flying pest captured in time series. Fig. 2 is a diagram showing an example of a three-dimensional point cloud extracted by the point cloud extraction unit 120. As shown in the left part of Fig. 2, when the point cloud extraction unit 120 extracts a three-dimensional point cloud representing the flying pest from the image, the state derivation unit 130 calculates the position P of the flying pest by taking the average value of each point constituting the three-dimensional point cloud.

[0021] As shown in the right part of Fig. 2, the state derivation unit 130 further calculates a regression plane RP that approximates the 3D point cloud by, for example, applying the least squares method to the 3D point cloud. The state derivation unit 130 calculates the attitude angles (roll angle and pitch angle) of the flying pest based on the tilt of the regression plane RP in the XYZ space. Alternatively, the state derivation unit 130 may use a template matching technique to compare the 3D point cloud with shape data and angle data of the flying pest pre-stored in the storage unit 150, thereby determining the attitude angles of the flying pest.

[0022] The state derivation unit 130 further derives state information such as the wing flapping frequency, wing direction, wing angle, abdomen direction, abdomen twist, abdomen bending, head direction, head twist, antenna direction, difference in movement of both wings, leg direction, and abdomen size of the flying pest in time series based on the point cloud information of each image.

[0023] FIG. 3 is a diagram illustrating a method by which the state derivation unit 130 extracts state information related to the wings of a flying pest. The state derivation unit 130 first applies techniques such as clustering and pattern matching to the 3D point cloud to extract, from the 3D point cloud, a partial point cloud corresponding to the wings of the flying pest as a point cloud of interest. Next, the state derivation unit 130 calculates a representative vector RV representing the wings of the flying pest by averaging vectors whose starting point is the position P of the flying pest and whose ending point is each point in the point cloud of interest. For example, as shown in the lower right of FIG. 3, the state derivation unit 130 can calculate the wing flapping frequency by measuring the temporal progression of the representative vector RV and counting the moment when the representative vector RV matches the vector at the initial time as one flapping.

[0024] The state derivation unit 130 further calculates the wing orientation and wing angle based on the target point cloud representing the wings of the flying pest. More specifically, for example, the state derivation unit 130 first applies a method such as clustering or pattern matching to the 3D point cloud to extract a partial point cloud corresponding to the abdomen of the flying pest from the 3D point cloud. Next, the state derivation unit 130 calculates a regression plane RP that approximates the abdomen of the flying pest by applying, for example, the least squares method to the partial point cloud corresponding to the abdomen of the flying pest. The state derivation unit 130 calculates the angle between the representative vector RV representing the wings of the flying pest and the regression plane RP that approximates the abdomen, and determines this as the wing angle.

[0025] The state derivation unit 130 further calculates a regression surface RS that approximates the wings of the flying pest, for example, by applying the least squares method to the partial point cloud corresponding to the wings of the flying pest. The state derivation unit 130 determines the normal vector NV of the calculated regression surface RS as the orientation of the wing.

[0026] The state derivation unit 130 further determines the difference between the movements of the two wings by calculating, for example, the difference vector between the representative vector RV representing the left wing of the flying pest and the representative vector RV representing the right wing of the flying pest. Through the above processing, the flapping frequency, wing direction, wing angle, and differences in the movements of the two wings are determined.

[0027] FIG. 4 is a diagram illustrating a method by which the state derivation unit 130 extracts state information related to the abdomen of a flying pest. The state derivation unit 130 applies a method such as clustering or pattern matching to the 3D point cloud to extract a partial point cloud corresponding to the abdomen of the flying pest as a point cloud of interest, and derives state information such as the orientation, twist, curvature, and size of the abdomen. More specifically, for example, the state derivation unit 130 first calculates an approximated surface RS that approximates the abdomen of the flying pest by applying a method such as the least squares method to the partial point cloud corresponding to the abdomen of the flying pest. The state derivation unit 130, for example, calculates the normal vector NV of the approximated surface RS as the orientation of the abdomen, calculates the curvature of the approximated surface RS as the curvature of the abdomen, and calculates the torsion rate (torsion tensor) of the approximated surface RS as the torsion of the abdomen.

[0028] Furthermore, the state derivation unit 130 calculates a closed surface that approximates the partial point cloud corresponding to the abdomen of the flying pest using a technique such as fitting, and calculates the volume of the area surrounded by the closed surface as the size of the abdomen. Through the above processing, the orientation, twist, curvature, and size of the abdomen are determined.

[0029] FIG. 5 is a diagram illustrating a method by which the state derivation unit 130 extracts state information related to the head of a flying pest. The state derivation unit 130 applies techniques such as clustering and pattern matching to the 3D point cloud to extract a partial point cloud corresponding to the head of the flying pest as a point cloud of interest, and derives state information such as the twist and orientation of the head. More specifically, for example, the state derivation unit 130 applies a technique such as the least squares method to the partial point cloud corresponding to the head of the flying pest, as in the case of the abdomen, to calculate an approximated surface RS that approximates the head of the flying pest. For example, the state derivation unit 130 calculates the torsion rate (torsion tensor) of the approximated surface RS as the twist of the head, and calculates the normal vector NV of the approximated surface RS as the orientation of the head. Through the above processing, the twist and orientation of the head are determined.

[0030] FIG. 6 is a diagram illustrating a method by which the state derivation unit 130 extracts state information related to the antennae of a flying pest. The state derivation unit 130 applies techniques such as clustering and pattern matching to the 3D point cloud to extract a partial point cloud corresponding to the antennae of the flying pest as a point cloud of interest, and derives state information such as the direction of the antennae. More specifically, for example, the state derivation unit 130 applies a technique such as the least squares method to the partial point cloud corresponding to the antennae of the flying pest to calculate regression lines RL1 and RL2 that approximate the left and right antennae of the flying pest. The state derivation unit 130 calculates, for example, directional vectors V1 and V2 of the regression lines RL1 and RL2 as the directions of the left and right antennae. The directions of the antennae are determined by the above process.

[0031] FIG. 7 is a diagram illustrating a method by which the state derivation unit 130 extracts state information related to the legs of a flying pest. The state derivation unit 130 applies a method such as clustering or pattern matching to the 3D point cloud to extract a partial point cloud corresponding to the legs of the flying pest as a point cloud of interest, and derives state information such as the orientation of the legs. More specifically, for example, the state derivation unit 130 applies a method such as the least squares method to the partial point cloud corresponding to the legs of the flying pest to calculate a regression line RC that approximates the legs of the flying pest for each leg. The state derivation unit 130 calculates, for example, a directional vector V of the regression line RL as the orientation of the leg. The orientation of the leg is determined by the above process.

[0032] The above describes a method in which the state derivation unit 130 determines state information such as the wing flapping frequency, wing direction, wing angle, abdomen direction, abdomen twist, abdomen curvature, head direction, head twist, antenna direction, difference in wing movement, leg direction, and abdomen size of a flying pest. However, the above-described method is merely an example, and any other method may be used to calculate these values.

[0033] The position prediction unit 140 predicts the future position of the flying pest based on the state information derived in time series by the state derivation unit 130 and predetermined surrounding environment information. Here, the surrounding environment information is information that represents the surrounding conditions of the environment in which the camera 10, the laser irradiation device 50, and the information processing device 100 are installed, and includes at least one of information regarding, for example, wind direction, wind speed, air temperature, temperature, amount of precipitation, location of the host plant, type of host plant, growth stage of the host plant, odor components of the host plant, location of a light source (such as a street lamp), spectral intensity of the light source, location of the moon, spectral intensity of the moon, location of bats, presence or absence of ultrasonic waves from bats, location of female flying pests (if the flying pests are male), season, and time of day.

[0034] For example, when the observation start time of the status information is represented by 0 and the current time is represented by t, the position prediction unit 140 predicts the future position of the flying pest at a time t+n n steps ahead (n is any positive integer) according to equation (1).

[0035]

number

[0036] In equation (1), Y k (k=0, ,t-2,t-1,t) is the state information observed from time 0 to time t, and θ t is the surrounding environment information at the preset time t, and Y^ t+n is the state information of flying pests at time t+n, and f θt is the surrounding environment information θ t are internal parameters, Y k For input (k=0,···,t-2,t-1,t), Y^ t+n is a linear or nonlinear function that outputs Y t and θ t is a vector represented by equations (2) and (3).

[0037]

number

[0038]

number

[0039] In equation (2), x t , y t , z t are the x, y, and z components of the flying pest's position P, respectively. t is the roll angle of the flying pest, and θ t is the pitch angle of the flying pest, and ω t is the flapping frequency of the wings. Although not shown in equation (2), Y t may further include at least some of wing orientation, wing angle, abdomen orientation, abdomen twist, abdomen curvature, head orientation, head twist, antenna orientation, difference in wing movement, leg orientation, and abdomen size.t is at least the position of the flying pest x t , y t , z t Anything that includes the above is acceptable.

[0040] In equation (3), a t , b t , c t , d t , e t ... includes at least some of the information regarding wind direction, wind speed, air temperature, temperature, precipitation, location of host plant, type of host plant, growth stage of host plant, odor components of host plant, location of light source (e.g., street light), spectral intensity of light source, position of the moon, spectral intensity of the moon, location of bats, presence or absence of bat ultrasound, location of female flying pests (if the flying pest is male), season, and time of day, etc.

[0041] Under the above assumptions, the position prediction unit 140 constructs a state space model including a system model and an observation model for the flying pest, and predicts the future position of the flying pest at time t+n by performing Kalman filtering. More specifically, the system model and the observation model for the flying pest are defined by equations (4) and (5), respectively.

[0042]

number

[0043]

number

[0044] In equation (4), X t is the state variable of the flying pest at time t, and is defined by equation (6).

[0045]

number

[0046] In equation (6), x t , y t , z t are the x, y, and z components of the position P of the flying pest, respectively, and x · t , y · t , z · t are the velocities obtained by differentiating the x, y, and z components of the flying insect's position P, respectively, and φ t is the roll angle of the flying pest, and θ t is the pitch angle of the flying pest, and ω t is the wing flapping frequency.

[0047] Returning to the explanation of equation (4), A θ is the system matrix with surrounding environment information as an internal parameter, and V t is the system noise that follows a normal distribution with mean 0 and variance Q, and b is a constant vector. t is the observed number of flying pests at time t, and is defined by equation (7).

[0048]

number

[0049] In equation (7), x t , y t , z t are the x, y, and z components of the flying pest's position P, respectively, and φ t is the roll angle of the flying pest, and θ t is the pitch angle of the flying pest, and ω t is the wing flapping frequency.

[0050] Returning to the explanation of equation (5), C is the state variable X of flying pests. t The observed value Y t is a transformation matrix that transforms W tis the observation noise that follows a normal distribution with mean 0 and variance R. The position prediction unit 140 predicts the future position of the flying pest at time t+n by performing Kalman filter processing on the state space model constructed as described above.

[0051] FIG. 8 is a diagram for explaining an outline of a method for predicting a future position by Kalman filtering. k (k=0, ,t-2,t-1,t) is sequentially input to the Kalman filter consisting of a prediction step and a filtering step, and the Kalman gain G t and update the n-step ahead forecast value Y^ t+n More specifically, the position prediction unit 140 calculates the observed value Y k (k=0,···,t-2,t-1,t) to estimate the state value x^ of flying pests in advance. - t The state value x^ estimated a posteriori after correcting t By repeating the process to obtain the Kalman gain G t and update the n-step ahead forecast value Y^ t+n We obtain a model to calculate

[0052] FIG. 9 is a diagram illustrating a method for predicting the future positions of flying pests using a deep neural network. As shown in Figure 9, the information processing device 100 uses as input data some or all of the following information: the three-dimensional position, three-dimensional speed, roll angle, pitch angle, wing flapping frequency, wing direction, wing angle, abdomen direction, abdomen twist, abdomen bending, head direction, head twist, antennae direction, difference in wing movement, leg direction, and abdomen size of the flying pest at times 0, 1, . . . , t-2, t-1, and t, as well as the wind direction, wind speed, temperature, humidity, precipitation, host plant position, host plant type, host plant growth stage, host plant odor components, light source position, light source spectral intensity, moon position, moon spectral intensity, bat position, presence or absence of bat ultrasound, female position (for males), season, and time of day.The information processing device 100 generates a trained model trained, for example, using a deep neural network, so as to output the future position of the flying pest at time t+n. The position prediction unit 140 can predict the future position of flying pests at time t+n by inputting the above data at time points 0, 1, . . . , t-2, t-1, t into the trained model.

[0053] Next, the flow of processing executed by the information processing device 100 according to the first embodiment will be described with reference to Fig. 10. Fig. 10 is a flowchart showing an example of the flow of processing executed by the information processing device 100 according to the first embodiment.

[0054] First, the image acquisition unit 110 acquires images of flying pests taken in time series by the camera 10 from the camera 10 (step S100). Next, the point cloud extraction unit 120 extracts point cloud information representing the flying pests in time series from each acquired image (step S101). Next, the state derivation unit 130 derives state information of the flying pests in time series based on the point cloud information extracted in time series (step S102).

[0055] Next, the position prediction unit 140 predicts the future position of the flying pest using Kalman filter processing or a trained model generated using a deep neural network, based on the flying pest status information derived in time series by the status derivation unit 130 and the surrounding environment information (step S103). Next, the information processing device 100 transmits the predicted future position to the laser irradiation device 50, and the laser irradiation device 50 outputs a high-power laser toward the predicted future position (step S104). This ends the processing of this flowchart.

[0056] According to the first embodiment described above, the future positions of flying pests are predicted based on the status information of the flying pests derived in time series and the surrounding environment information. This makes it possible to predict the position of a moving object even if the moving object moves at high speed. Furthermore, it is possible to predict the position of a moving object even if the moving object frequently changes speed or moves non-linearly.

[0057] In the first embodiment, an example has been described in which the future position of flying pests is predicted and a high-power laser is output toward the predicted future position. However, the configuration of the first embodiment can also be applied to, for example, birds such as crows. In this case, the state information derived in time series may include, in addition to position information, at least one of the animal's wing flapping frequency, wing direction, wing angle, abdomen direction, head direction, leg direction, and abdomen size. Similar to pests, extermination is performed by outputting a high-power laser toward the predicted future position.

[0058] [Second embodiment] In the first embodiment, the present invention is applied to a case where the moving object whose future position is predicted is a flying pest. However, the present invention is not limited to such a configuration and can be applied to predicting the future position of any moving object. In the second embodiment, as another example, the present invention is applied to predicting the future position of a deer. Below, only the configuration that differs from the first embodiment will be described, and a description of the common configuration will be omitted.

[0059] 11 is a diagram showing an example of the usage environment and configuration of the information processing device 100 according to the second embodiment. The information processing device 100 operates in cooperation with, for example, a camera 10 and an animal capture device 50B. The combination of the information processing device 100 and the animal capture device 50B is an example of a "vermin capture system."

[0060] The animal capture device 50B is, for example, a drop net that has a mechanism of hanging a net in the air through trees in a forest and dropping the net when a deer comes under the net to capture the deer. The animal capture device 50B is controlled by the information processing device 100, and operates to capture the deer when it is determined that the device's own position is at the future position predicted by the position prediction unit 140.

[0061] As in the first embodiment, the position prediction unit 140 predicts the future position of the deer based on the deer's state information derived in time series by the state derivation unit 130 and the surrounding environment information, using Kalman filter processing or a trained model generated using a deep neural network.

[0062] In that case, in the above formula (2), for example, x t , y t , z t are the x, y, and z components of the deer's position P, respectively, and φ t is the orientation of the deer's abdomen, and θ t is the orientation of the deer's head, and ω t is the direction of the deer's legs. Instead of or in addition to these parameters, the size of the deer's abdomen may be set as state information. More generally, Y t is at least the position of the flying pest x t , y t , z t Anything that includes the above is acceptable.

[0063] Furthermore, in the above-mentioned equation (3), the surrounding environment information θ tFor example, the information includes at least one of information regarding wind direction, wind speed, air temperature, temperature, precipitation, topography, tree position, light source position, light source spectral intensity, sound source position, sound source intensity, season, and time of day.

[0064] Furthermore, in the above formula (6), x t , y t , z t are the x, y, and z components of the deer's position P, respectively, and x · t , y · t , z · t are the velocities obtained by differentiating the x, y, and z components of the deer's position P, respectively, and φ t is the orientation of the deer's abdomen, and θ t is the orientation of the deer's head, and ω t is the direction of the deer's feet.

[0065] Furthermore, in the above formula (7), x t , y t , z t are the x, y, and z components of the deer's position P, respectively, and φ t is the orientation of the deer's abdomen, and θ t is the orientation of the deer's head, and ω t is the direction of the deer's feet. Under the above conditions, the position prediction unit 140 can predict the future position of the deer by Kalman filter processing, as in the first embodiment.

[0066] Next, the flow of processing executed by the information processing device 100 according to the second embodiment will be described with reference to Fig. 12. Fig. 12 is a flowchart showing an example of the flow of processing executed by the information processing device 100 according to the second embodiment.

[0067] First, the image acquisition unit 110 acquires images of flying pests taken in time series by the camera 10 from the camera 10 (step S100). Next, the point cloud extraction unit 120 extracts point cloud information representing the flying pests in time series from each acquired image (step S101). Next, the state derivation unit 130 derives state information of the flying pests in time series based on the point cloud information extracted in time series (step S102).

[0068] Next, the position prediction unit 140 predicts the future position of the flying pest using Kalman filter processing or a trained model generated using a deep neural network based on the flying pest state information derived in time series by the state derivation unit 130 and the surrounding environment information (step S103). Next, the information processing device 100 transmits the predicted future position to the animal trapping device 50B, and the animal trapping device 50B operates when it determines that its own position is at the future position predicted by the position prediction unit 140 (step S104). This ends the processing of this flowchart.

[0069] According to the second embodiment described above, the future position of a deer is predicted based on the deer's state information derived in time series and the surrounding environment information. This makes it possible to predict the position of a moving object even if the moving object is moving at high speed. [Explanation of symbols]

[0070] 10 Camera 50 Laser irradiation device 50B Animal capture device 100 Information processing device 110 Image acquisition unit 120 Point cloud extraction part 130 State derivation part 140 Position Prediction Unit 150 Storage section

Claims

1. a derivation unit that derives time-series state information of the moving object, including at least a position, based on point cloud information representing the moving object extracted from each of a plurality of images obtained by capturing the moving object in time series; a prediction unit that predicts a future position of the moving object based on the time-series state information and surrounding environment information of the moving object, the prediction unit predicts the future position by repeating a process of correcting a state value of the moving object estimated in advance using the observed value of the time-series state information to obtain a state value of the moving object estimated ex post. Information processing device.

2. A derivation unit that derives time-series state information of a moving object, including at least a position, based on point cloud information representing the moving object extracted from each of a plurality of images obtained by capturing the moving object in time series; a prediction unit that predicts a future position of the moving object based on the time-series state information and surrounding environment information of the moving object, the prediction unit predicts the future position by inputting the time-series state information and the surrounding environment information into a trained model that receives the time-series state information and the surrounding environment information of a moving object as input and is trained to output a future position of the moving object; Information processing device.

3. the surrounding environment information includes at least one of information regarding wind direction, wind speed, air temperature, temperature, amount of precipitation, position of a light source, spectral intensity of a light source, season, and time of day in the environment where the moving object is located; 3. The information processing device according to claim 1.

4. the moving object is a flying pest, The time-series state information further includes at least one of the roll angle, pitch angle, wing flapping frequency, wing direction, wing angle, abdomen direction, abdomen twist, abdomen bending, head direction, head twist, antenna direction, difference in movement of both wings, leg direction, and abdomen size of the pest. The information processing device according to claim 1 .

5. the moving object is an animal belonging to the avian family, the time-series state information further includes at least one of a wing flapping frequency, a wing orientation, a wing angle, an abdomen orientation, a head orientation, a leg orientation, and an abdomen size of the animal; The information processing device according to claim 1 .

6. the moving object is a quadrupedal animal, The time-series state information further includes at least one of an orientation of the abdomen, an orientation of the head, an orientation of the legs, and a size of the abdomen of the animal. The information processing device according to claim 1 .

7. The information processing device according to claim 4 or 5; a laser irradiation device that is controlled by the information processing device and that irradiates a laser toward the future position; A flying creature extermination system equipped with:

8. The information processing device according to claim 6 ; an animal capture device controlled by the information processing device and operable when the location of the device itself is determined to be at the future location; A pest capture system comprising:

9. The computer deriving time-series state information of the moving object, including at least a position, based on point cloud information representing the moving object extracted from each of a plurality of images obtained by capturing the moving object in time series; predicting a future position of the moving object based on the time-series state information and surrounding environment information of the moving object; The prediction predicts the future position by repeating a process of correcting a state value of the moving object estimated in advance using the observed value of the time-series state information to obtain a state value of the moving object estimated ex post. Information processing methods.

10. On the computer, deriving time-series state information of the moving object, including at least a position, based on point cloud information representing the moving object extracted from each of a plurality of images obtained by capturing the moving object in time series; predicting a future position of the moving object based on the time-series state information and surrounding environment information of the moving object; The prediction predicts the future position by repeating a process of correcting a state value of the moving object estimated in advance using the observed value of the time-series state information to obtain a state value of the moving object estimated ex post. program.

Images