Noxious animal extermination system and noxious animal extermination program

Abstract

To provide a noxious animal extermination system capable of driving away a noxious animal even being indoors.SOLUTION: A noxious animal extermination system 1 includes: a camera 2 for imaging a warehouse R; a drone 3 that flies in the warehouse R, and a controlling device 5. The controlling device 5 comprises: an animal detection unit 51 that uses an animal detection model 56 created using machine learning to detect a mouse A from a camera image 50 imaged by the camera 2; an appearance position identification unit 52 that uses output data from the animal detection model 56 and a 3D map 61 of the warehouse R to identify a detected location 34 of the mouse A in the warehouse R; a flight route determination unit 53 that uses the 3D map 61 to determine an optimal flight route 62 from a standby location 30, at which a drone 3 is subjected to stand by, toward the detected location 34; and a drone controlling unit 54 that transmits, to the drone 3, a flight instruction for flying through the optimal flight route 62 and a threat instruction for causing the drone 3 to perform previously-set threat operation.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a pest control system and a pest control program for exterminating pests such as rats using unmanned aerial vehicles such as drones. [Background technology]

[0002] Damage caused by small animals such as rats that have invaded warehouses and other buildings has long been a problem. Methods of exterminating rats include using rat repellents (scent) or battery-operated rat extermination devices (sound) to drive them away, and blocking their entry routes with netting or sticky sheets.

[0003] Meanwhile, in recent years, technology has been developed to scare away harmful animals using small flying devices such as drones. Patent Document 1 describes a system (unmanned flying device control system) that uses a drone to scare away harmful animals (e.g., deer, wild boar, monkeys, birds, etc.) when the harmful animals are detected.

[0004] The unmanned flying device control system of Patent Document 1 installs a camera in a monitored area and detects vermin from the camera images using a machine learning model. When a vermin is detected, one of the positions (e.g., animal trails) where the vermin can easily leave the area it is invading is set as an exit position based on the current position of the vermin. Then, flight instructions are sent to the drone to direct the vermin to the exit position. For example, the drone is flown to follow the vermin from the position where the vermin was detected to the exit position, or to guide the vermin toward the exit position. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 6704979 Summary of the Invention [Problem to be solved by the invention]

[0006] The system in Patent Document 1 is intended for use in open outdoor spaces, and is not intended for use in spaces where flight paths are limited, such as warehouses where various materials are stored. In addition, the method for determining the flight path of the drone is not specifically disclosed.

[0007] When operating a drone-based system to scare away pests indoors, it is difficult to fly along the shortest route while avoiding obstacles. Also, if the drone detects obstacles on its flight path and flies while making evasive maneuvers each time, it is not possible to quickly approach the detected animals (pests). As a result, there is a risk that the pests will not be scare away.

[0008] In view of the above, an object of the present invention is to provide a pest control system that can repel pests even indoors. [Means for solving the problem]

[0009] In order to solve the above problems, the pest extermination system of the present invention comprises a camera that photographs a monitored space, an unmanned aerial vehicle that flies in the monitored space, and a control device capable of communicating with the unmanned aerial vehicle and the camera, the control device comprising an animal detection unit that detects animals to be exterminated from camera images taken by the camera using an animal detection model created by machine learning, and an appearance position identification unit that identifies the appearance position of the animal in the monitored space using output data from the animal detection model and three-dimensional data of the monitored space. The system is characterized by comprising a flight route determination unit that uses the three-dimensional data to determine an optimal flight route from the waiting position where the unmanned aerial vehicle is waiting to the appearance position, and an unmanned aerial vehicle control unit that transmits to the unmanned aerial vehicle a flight command to fly along the optimal flight route and a threatening command to cause the unmanned aerial vehicle to perform a predetermined threatening action.

[0010] In addition, the pest control program of the present invention is characterized in that it has a single or multiple computers perform the following steps: an animal detection process that uses an animal detection model created by machine learning to detect animals to be exterminated from camera images taken by a camera installed in the monitored space; an appearance position identification process that uses output data from the animal detection model and three-dimensional data of the monitored space to identify the appearance position of the animal in the monitored space; a flight route determination process that uses the three-dimensional data to determine an optimal flight route from a waiting position where an unmanned aerial vehicle flying in the monitored space is waiting to the appearance position; and an unmanned aerial vehicle control process that transmits to the unmanned aerial vehicle a flight command to fly the optimal flight route and a threatening command to cause the unmanned aerial vehicle to perform a predetermined threatening action.

[0011] According to the present invention, when an animal to be exterminated appears in a monitored space, the animal can be detected from a camera image and an unmanned aerial vehicle can be directed to the location of the animal's appearance to scare it away. Therefore, the animal to be exterminated can be chased away by automatic processing. In this case, the location of the animal's appearance can be identified using three-dimensional data of the monitored space, and the optimal flight route determined using the three-dimensional data of the monitored space can be used as the flight route of the unmanned aerial vehicle. Therefore, even in an indoor space where the flight path is limited, the unmanned aerial vehicle can be quickly directed to the location of the animal's appearance. Therefore, the animal that has invaded the monitored space can be chased away.

[0012] In the present invention, the unmanned aerial vehicle waits at each of the plurality of waiting positions that are different from each other and that are set in the space to be monitored, and the flight route determination unit preferably divides the three-dimensional data into a plurality of target areas, and determines the flight route associated with the target area including the appearance position as the optimal flight route from a candidate route data set in which a flight route from any of the plurality of waiting positions is associated with each of the plurality of target areas. In this way, when an animal appears, it is only necessary to select a flight route associated with the target area including the appearance position from the candidate route data set that is preset based on the appearance position. Therefore, the load required for the optimal flight route determination process can be reduced.

[0013] In the present invention, the flight route included in the candidate route data set is preferably determined based on at least one of the flight time and the energy consumption when flying from the waiting position to the target area. For example, by setting a flight route with the shortest flight time, it is possible to scare off animals in the shortest time. Alternatively, by setting a flight route with the smallest energy consumption, it is possible to minimize the consumption of the battery of the unmanned aerial vehicle. Alternatively, a flight route can be set in consideration of the balance between flight time and energy consumption.

[0014] In the present invention, it is preferable that the flight time of the flight route included in the candidate route data set is shorter than a preset reference flight time. In this way, it is possible to scare off animals within the reference flight time. In addition, it is also possible to include a plurality of flight routes having different combinations of flight time and consumed energy among flight routes that can reach the appearance position within the reference flight time in the candidate route data set. In this case, it is possible to determine whether to give priority to flight time or remaining battery power depending on the situation and select a flight route.

[0015] In the present invention, the flight route included in the candidate route data set is the third It is preferable to set a no-fly space on the original data and to set a route that connects route points arranged in a grid pattern in an area excluding the no-fly space. In this way, a flight route that does not enter the no-fly space can be easily set. In addition, by setting a route that connects route points with a simple shape (for example, a straight line or a circular arc), it is easy to generate flight commands. Alternatively, by setting a flight route that connects distant route points with a straight line, it is possible to set a flight route that consumes less flight time or energy. In addition, it is possible to detect the self-position for each route point and modify the flight program.

[0016] In the present invention, it is preferable that the animal detection unit extracts an image portion in which the extermination target animal is captured from the camera image using the animal detection model, generates a bounding box of a size surrounding the extracted image portion, and the appearance position identification unit identifies three-dimensional coordinates indicating the appearance position from the coordinates of the bounding box in the camera image. In addition, when converting the camera image into three-dimensional coordinates, processing such as correcting distortion of the camera image is performed in order to identify accurate coordinates in real space. In this way, appropriate coordinates can be obtained as the appearance position of the animal.

[0017] In the present invention, it is preferable that the appearance position identification unit transmits three-dimensional coordinates indicating the appearance position to a cloud server, and the three-dimensional data and the candidate route data set are stored in the cloud server. In this way, it is easy to build a pest extermination system because there is no need to build a dedicated server or communication path.

[0018] In the present invention, it is preferable that the output data from the animal detection model is stored in an animal detection history database provided in the cloud server, so that information useful for future measures against pests can be accumulated in the cloud server.

[0019] In the present invention, it is preferable that the cameras are arranged along the walls of the space to be monitored, facing the floor to capture images, and the optimal flight route is a route that flies along the floor of the space to be monitored. In this way, when a vermin such as a rat running around on the floor of the space to be monitored (e.g., a warehouse) appears, the camera is likely to capture the rat's figure. Therefore, the appearance of the vermin is easily detected. In addition, the unmanned aerial vehicle can be flown close to the vermin, so that the vermin can be effectively chased away.

[0020] In the present invention, the camera is preferably an infrared camera, which makes it possible to detect an emergent pest even when the monitored space is dark.

[0021] In the present invention, it is preferable to perform a flip motion as the intimidating motion, since the intimidating motion is performed using a motion generally performed by unmanned aerial vehicles, it is easy to control the intimidating motion and it is possible to effectively scare off harmful animals. Effect of the Invention

[0022] According to the present invention, when an animal to be exterminated appears in a monitored space, the animal can be detected from a camera image and an unmanned aerial vehicle can be directed to the location of the animal's appearance to scare it away. Therefore, the animal to be exterminated can be chased away by automatic processing. In this case, the location of the animal's appearance can be identified using three-dimensional data of the monitored space, and the optimal flight route determined using the three-dimensional data of the monitored space can be used as the flight route of the unmanned aerial vehicle. Therefore, even in an indoor space where the flight path is limited, the unmanned aerial vehicle can be quickly directed to the location of the animal's appearance. Therefore, the animal that has invaded the monitored space can be chased away. [Brief description of the drawings]

[0023] [Figure 1] 1 is a schematic diagram of a pest control system to which the present invention is applied. [Diagram 2] FIG. 1 is a perspective view showing a schematic 3D map of a warehouse. [Diagram 3]1 is a block diagram showing a functional configuration of a pest control system. [Figure 4] 1 is a schematic flow chart illustrating a method for creating an animal detection model. [Diagram 5] 1 is a schematic flow chart illustrating a method for detecting mice from camera images using an animal detection model. [Figure 6] 1 is a schematic flowchart showing a method for creating a 3D map and a method for setting an optimal flight route. [Figure 7] This is an explanatory diagram showing an example of setting no-fly spaces, route points, and optimal flight routes on a plan view of a 3D map. [Figure 8] FIG. 1 is a sequence diagram of rat extermination using a pest control system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0024] Hereinafter, an embodiment of a pest control system to which the present invention is applied will be described with reference to the drawings. Note that the embodiment described below is an example of the present invention, and the present invention is not limited to the embodiment described below.

[0025] (Overall composition) The pest extermination system 1 is designed to monitor an indoor space such as a warehouse, and assumes that the pests to be exterminated are small animals such as mice. A small unmanned aerial vehicle is used to drive away the pests. As the unmanned aerial vehicle, for example, a multicopter (a so-called drone) with three or more rotors or a radio-controlled aircraft can be used. Below, an example will be described in which the space to be monitored is a warehouse R, the pest to be exterminated is a mouse A, and the unmanned aerial vehicle that drives away the pests is a drone 3.

[0026] FIG. 1 is a schematic diagram of a pest control system 1 to which the present invention is applied. FIG. 2 is a perspective view that typically shows a 3D map 61 of a warehouse R. FIG. 3 is a block diagram showing the functional configuration of the pest control system 1. As shown in FIGS. 1 and 2, the pest control system 1 includes a camera 2 that photographs a space to be monitored (in this embodiment, the warehouse R), a drone 3 as an unmanned aerial vehicle, and a control device 5 that acquires data from the camera 2 via a router 4 and controls the flight of the drone 3. The control device 5 is capable of communicating with a cloud server 6 via a network such as the Internet.

[0027] 1, in this embodiment, a router 4 is installed in a warehouse R in which a camera 2 is installed, and a control device 5 is installed in a building separate from the warehouse R. For example, the control device 5 is installed in an administration office that manages the warehouse R. As the control device 5, for example, a general-purpose computer is used.

[0028] The pest control system 1 performs image analysis on an image (hereinafter, referred to as camera image 50) captured by the camera 2 of the interior space of the warehouse R. If it detects that a mouse A is included in the camera image 50, it determines that the mouse A has invaded the warehouse R, and identifies the position where the mouse A appeared based on a 3D map 61, which is three-dimensional data of the warehouse R, and the detection result of the mouse A by image analysis. Then, it flies the drone 3 to the position where the mouse A appeared, and causes the drone 3 to perform a threatening action to drive away the invading mouse A. In this embodiment, the 3D map 61 and data required for determining the flight path of the drone 3 are stored in the cloud server 6.

[0029] The camera 2 is connected to the router 4 by wireless communication or wired communication. The camera image 50 is transmitted to the control device 5 via the router 4. The camera 2 is a fixed surveillance camera installed in the warehouse R. As shown in FIG. 2, multiple cameras 2 are installed in the warehouse R. The multiple cameras 2 have overlapping shooting angles of view between adjacent cameras 2, and cover the entire warehouse R. It will be installed so that people can enjoy barbecue.

[0030] As the camera 2, for example, an infrared camera is used. The position and orientation of the camera 2 are appropriately set so that the area that serves as the passageway of the mouse A is captured. For example, as shown in FIG. 2, the camera 2 installed on the upper part of the wall surface 31 of the warehouse R captures an image diagonally downward. If the mouse A often invades from a position close to the ground, installing the camera 2 in such an orientation makes it easier for the invading mouse A to be captured in the camera image 50. Also, by capturing an image diagonally downward from the upper part of the wall surface 31, the area that serves as the passageway of the mouse A is less likely to be blocked by the materials 32 accumulated in the warehouse R. Therefore, the invading mouse A is easier to capture in the camera image 50.

[0031] It is also possible to use a movable camera 2 that can change the shooting direction to shoot images in sequence of different shooting areas. The arrangement and number of cameras 2 can be changed as appropriate. For example, if there is an area where it is desired to focus on eradicating mice A, the camera 2 can be arranged to monitor only that area.

[0032] The drone 3 receives flight commands from the control device 5 via wireless communication through the router 4. A plurality of waiting positions 30 for the drones 3 are set in the warehouse R, and the drones 3 are made to wait at each of the waiting positions 30. A drone take-off and landing station equipped with a wireless power supply device is provided at each waiting position 30. The drone 3 that lands at the drone take-off and landing station is connected to the wireless power supply device and begins charging. The number and arrangement of the drones 3 are set appropriately according to the size and shape of the space to be monitored.

[0033] (Control device) As shown in Fig. 3, the control device 5 includes an animal detection unit 51, an appearance position identification unit 52, a flight route determination unit 53, a drone control unit 54 as an unmanned aerial vehicle control unit, and a storage unit 55. The storage unit 55 includes non-volatile or volatile memory such as RAM, ROM, flash memory, EPROM, and EEPROM. The storage unit 55 stores programs such as software or firmware, and data used in processing the programs. The control device 5 also includes a calculation unit (not shown) such as a CPU or processor that reads out and executes the programs stored in the storage unit 55.

[0034] In this embodiment, a pest extermination program is stored in the memory unit 55 of the control device 5. The pest extermination program causes the computer to perform animal detection processing, appearance position identification processing, flight route determination processing, drone control processing (unmanned aerial vehicle control processing), and the like. The contents of each process will be described later. The control device 5 realizes functions such as the animal detection unit 51, appearance position identification unit 52, flight route determination unit 53, and drone control unit 54 by the calculation unit reading and executing the pest extermination program stored in the memory unit 55.

[0035] The animal detection unit 51 performs an animal detection process to detect an animal to be exterminated (in this embodiment, a mouse A) from the camera image 50 using a trained animal detection model 56 created by machine learning. Details of the animal detection model 56 will be described later.

[0036] When a mouse A is detected from the camera image 50, the appearance position identification unit 52 performs an appearance position identification process to identify three-dimensional coordinates indicating the appearance position 34 of the detected mouse A based on the output data from the animal detection model 56 and the 3D map 61 of the warehouse R.

[0037] When the flight route determination unit 53 identifies the three-dimensional coordinates indicating the appearance position 34 of the mouse A, the flight route determination unit 53 performs a flight route determination process to determine an optimal flight route 62 from the waiting position 30 where the drone 3 is waiting to the appearance position 34 where the mouse A appeared. The optimal flight route 62 is determined based on a 3D map 61 of the warehouse R.

[0038] When the optimal flight route 62 is determined, the drone control unit 54 performs drone control processing to transmit a flight command to the drone 3 to fly along the optimal flight route 62, and also transmits a threatening command to cause the drone 3 to perform a preset threatening action. The threatening command causes the drone 3 to perform the threatening action when it reaches the appearance position 34 of the mouse A. Alternatively, the threatening command may be a command to cause the drone 3 to perform the threatening action on the way to the appearance position 34, rather than when it reaches the appearance position 34.

[0039] (Cloud server) The cloud server 6 stores a 3D map 61 of the warehouse R and an optimal flight route 62 set based on the 3D map 61. As will be described later, the optimal flight route 62 is set for each of a plurality of target areas RA, and data associating the optimal flight route 62 with each target area RA is stored in the cloud server 6 as a candidate route data set 66. The cloud server 6 also stores an animal detection history database 64. The data set output from the animal detection model 56 is accumulated in the animal detection history database 64 in association with information (e.g., camera number) of the camera that captured the camera image 50 in which the mouse A was detected.

[0040] (Animal detection model) The trained animal detection model 56 stored in the control device 5 is an object detection model consisting of a machine learning algorithm such as a convolutional neural network (CNN) that utilizes AI, and for example, the widely used YOLO series can be used.

[0041] FIG. 4 is a schematic flowchart showing a method for creating the animal detection model 56. The creation procedure is general, but a data set including a captured image of a mouse A is used as training data for the animal detection model 56. As described above, in this embodiment, an infrared camera is used as the camera 2 to monitor the mouse A, which is a nocturnal pest. Therefore, an image of the infrared camera is used as training data to create a trained model corresponding to nighttime monitoring. For this purpose, training mouse images were taken by changing the position, angle, distance, brightness, individual mouse and its state of the camera (ST11), and a training data set was created (ST12). These were input as training data, and deep learning was performed so as to obtain the desired detection result (ST13), and the animal detection model 56, which is a trained model capable of extracting mice with high accuracy, was obtained (ST14).

[0042] The inventors created multiple trained models by deep learning and verified the detection accuracy and the load when performing the detection process. In the pest control system 1, the trained model to be adopted as the animal detection model 56 was determined in consideration of the specifications of the control device 5, which is a computer in which the animal detection model 56 is installed. In this embodiment, three types of trained models, namely trained model A (low accuracy, low load), trained model B (high accuracy, high load), and trained model C (medium accuracy, low load), were created by deep learning, and trained model C, which has the best balance between accuracy and load, was adopted as the animal detection model 56.

[0043] (Animal detection processing) FIG. 5 is a schematic flow chart showing a method for detecting mouse A from camera images 50 using animal detection model 56. Pest control system 1 takes images using camera 2 installed in warehouse R (ST21). Image data from camera 2 is transmitted in real time to control device 5 via router 4. Control device 5 detects when a moving object enters the field of view of camera 2 (ST22). For example, a moving object is detected by detecting a difference in camera images 50. Note that the method for detecting a moving object is not limited to detecting a difference in images, and other methods may be used.

[0044] The control device 5 uses the animal detection model 56 (trained model) to analyze the camera image 50 when a moving object is detected (ST23 to 25). That is, the camera image 50 is input to the animal detection model 56 (trained model) (ST23), the feature amount of each image part in the camera image 50 is extracted (ST24), and the image part in which the mouse A appears is extracted based on the feature amount, thereby detecting the mouse A (ST25).

[0045] 5, the animal detection model 56 generates a bounding box 58 (rectangular frame) of a minimum size capable of containing the image portion showing the mouse A. The generated bounding box 58 can be displayed on the camera image 50 to be analyzed. For example, the control device 5 can display the bounding box 58 surrounding the mouse A on the camera image 50 on the monitor as a detection result.

[0046] The dataset output by the animal detection model 56 includes the following data: - Position of bounding box 58 (x, y coordinates of the upper left corner) -Size of bounding box 58 (width w, height h) -Time stamp of the photo taken (yyyy-mm-dd(T)HH.MM.SS) Probability value (probability that the extracted image part is Mouse A) When the probability value output by the animal detection model 56 exceeds a certain value, the animal detection unit 51 determines that the mouse A has been detected. Then, every time the mouse A is detected from the camera image 50 (i.e., every time an image part where the probability value of being the mouse A exceeds a certain value is detected), the animal detection unit 51 transmits the above data set together with other data (e.g., the camera number) to the cloud server 6 and accumulates them in the animal detection history database 64.

[0047] (3D map, location identification processing) In the pest control system 1, a 3D map 61 of the warehouse R is created and used to control the flight of the drone 3. When a mouse A is detected in the camera image 50, the appearance position identification unit 52 uses the data set output by the animal detection model 56 and the 3D map 61 to identify the three-dimensional coordinates of the appearance position 34 of the mouse A on the 3D map 61.

[0048] For example, in this embodiment, the multiple cameras 2 installed in the warehouse R are fixed, and the two-dimensional coordinates on the camera image 50 captured by each camera 2 can be converted into three-dimensional coordinates on the 3D map 61. When creating the 3D map 61, correspondence data between the two-dimensional coordinates on the camera image 50 captured by each of the multiple cameras 2 and the three-dimensional coordinates on the 3D map 61 is created as an appearance position identification data set 57. At that time, the two-dimensional camera image 50 is processed to correct distortion, and the coordinates on the corrected two-dimensional image are associated with the three-dimensional coordinates on the 3D map. In this way, an appearance position identification data set is created in which the coordinates on the camera image 50 are associated with accurate coordinates in real space. The appearance position identification data set 57 is stored in the storage unit 55 of the control device 5 together with the pest extermination program. The appearance position identification data set 57 may be stored in the cloud server 6 together with the 3D map 61. In this case, the control device 5 accesses the cloud server 6 and refers to the appearance position identification data set 57 to perform the appearance position identification process.

[0049] When mouse A is detected from camera image 50, appearance position identifying unit 52 identifies the three-dimensional coordinates of appearance position 34 of mouse A by converting the x, y coordinates of the center position of bounding box 58 into three-dimensional coordinates on 3D map 61 using appearance position identifying dataset 57. The identified three-dimensional coordinates are transmitted to cloud server 6. Note that the method of identifying appearance position 34 is not limited to this method, and other methods may be used.

[0050] (Setting the optimal flight route) FIG. 6 is a schematic flow chart showing a method for creating a 3D map 61 and a method for setting an optimal flight route 62. When the pest control system 1 is installed in a space to be monitored, a 3D map 61, which is three-dimensional data of the space to be monitored, is created. For example, a LiDAR camera or a video camera for 3D modeling is used to scan or capture images while moving around the warehouse R. Thereafter, a three-dimensional shaping process is performed using the scan data or the captured data, and the 3D map 61 is obtained (ST31). For example, SfM (Structure from Motion) is used as a method for three-dimensional shaping from a two-dimensional image. This generates point cloud data, which is a collection of points having three-dimensional coordinates.

[0051] Next, the no-fly space R1 and the route points P are set on the generated 3D map 61 (ST32 to 33). FIG. 7(a) is an explanatory diagram showing an example of setting the no-fly space R1, the route points P, and the optimal flight route 62 on a plan view of the 3D map 61. For example, a noise removal process is performed on the 3D map 61 (point cloud data) generated by the above-mentioned three-dimensional shaping process, and at a position where an object that becomes an obstacle when flying the drone 3 (for example, materials 32 in the warehouse R (see FIG. 2)) exists, a three-dimensional shape such as a rectangular parallelepiped that contains the object is set as the no-fly space R1 (ST32). The space on the 3D map 61 excluding the no-fly space R1 is set as the flyable space R0, and the lattice-shaped route points P are set in the flyable space R0 (ST33). The interval between the route points P can be, for example, 1 m, but is not limited to 1 m.

[0052] Next, in order to determine the optimal flight route 62 no matter where the mouse A appears in the warehouse R, the target area RA is set on the 3D map 61 by dividing the flight space R0 into a grid, and the waiting position 30 of the drone 3 is set on the 3D map 61 (ST34). As shown in FIG. 7(a), each target area RA is set to include a route point P. The size of the target area RA may be set appropriately taking into consideration the interval between the route points P and the accuracy of the position control of the drone 3. Then, for each of the set target areas RA, a flight route from one of the multiple waiting positions 30 is determined as the optimal flight route 62, and a candidate route data set 66 in which the optimal flight route 62 is associated with all the target areas RA is created (ST35).

[0053] (Optimal Route Generation Algorithm) The optimal flight route 62 for each target area RA is determined by an optimal route generation algorithm. The optimal route generation algorithm optimizes the flight route using the equation of motion of the drone 3. In order to effectively scare away the mouse A, the optimal route generation algorithm determines the optimal flight route 62 by prioritizing shortening the flight time of the drone 3 until it arrives at the appearance position 34 of the mouse A. By prioritizing the flight time, it is possible to make the drone 3 arrive at the appearance position 34 of the mouse A as quickly as possible to scare the mouse A away.

[0054] In this embodiment, the standby position 30 is set as the starting point, and a path point P in the target area RA is set as the target point PT, and a flight route is set that connects the starting point (standby position 30) and the target point PT. The flight route is set to pass through path point P. Specifically, as shown in FIG. 7(a), the flight route is set so that one of the multiple path points P arranged to surround the standby position 30 is set as the starting point, and adjacent path points P are connected by a straight line or a circular arc from the starting point to the target point PT. Alternatively, the flight route is set so that distant path points P are connected by a straight line or an arbitrary curve in at least a part of the flight route.

[0055] As illustrated in Fig. 7(a), for each target area RA, a plurality of flight routes F1, F2 starting from different standby positions 30 can be set. The optimal route generation algorithm determines the optimal flight route 62 by selecting flight routes from all the standby positions 30 set on the 3D map 61 as options. Also, with the same standby position 30 as the starting point and passing through different route points P various flight routes are used as options to determine the optimal flight route 62. As shown in Fig. 7(b), the optimal route generation algorithm uses the motion equation of the drone 3 to calculate the flight time until reaching the target point PT for each flight route, and determines the flight route F1 with the minimum flight time (flight time = t1) as the optimal flight route 62.

[0056] (Modification example of the optimal route generation algorithm) Here, the optimal route generation algorithm can optimize a plurality of objective values including not only one objective value (flight time) but also other objective values (for example, energy consumption). When flying the drone 3, the flight time and the energy consumption may be in a trade-off relationship. Therefore, when optimizing the flight time and the energy consumption, the optimal route generation algorithm uses the motion equation of the drone 3 to calculate the flight time and the energy consumption of each flight route, and obtains a set of optimal solutions (Pareto solutions) from the combinations of the flight time and the energy consumption. Then, one optimal solution is selected from them.

[0057] The Pareto solutions are, for example, when t1 < t2 < t3 < t4 and e1 < e2 < e3 < e4, a set of solutions as follows. · Solution 1 (flight time t1, energy consumption e4) · Solution 2 (flight time t2, energy consumption e3) · Solution 3 (flight time t3, energy consumption e2) · Solution 4 (flight time t4, energy consumption e1) The optimal route generation algorithm selects one of the Pareto solutions, and determines the flight route corresponding to the selected solution as the optimal flight route 62. For example, from the Pareto solutions, a solution whose flight time is equal to or less than a preset reference flight time and whose energy consumption is the smallest is selected. The reference flight time can be determined by considering, for example, how short the time from when the mouse A is detected until the drone 3 reaches the location is required. In this way, the drone 3 can perform a threatening action at the appearance position 34 of the mouse A within the required time, and the remaining battery power of the drone 3 can be secured as much as possible.

[0058] Among flight routes that can reach the appearance position 34 within the reference flight time, a plurality of flight routes with different combinations of flight time and consumed energy can be included in the candidate route data set 66. In this case, an algorithm can be employed that determines whether to give priority to flight time or remaining battery power depending on the situation and selects a flight route.

[0059] Alternatively, the optimal route generation algorithm may assume that mouse A will appear frequently and determine the optimal flight route 62 by prioritizing reducing energy consumption in order to preserve the battery life of drone 3.

[0060] In this embodiment, the waiting position 30 of the drone 3 is a position close to the floor of the warehouse R. The flight route of the drone 3 includes vertical movement within the warehouse R. The optimal path generation algorithm determines the optimal flight route 62 from among routes that fly along the floor, rather than routes that rise to a high position away from the floor. When the mouse A moves to a position close to the floor, the mouse A can be effectively intimidated by flying at a low position. In addition, when setting the flight route, the drone is set to fly at a position at least a certain distance away from the wall 31 of the warehouse R and the no-fly space R1.

[0061] (Example of rat extermination) 8 is a sequence diagram of rat extermination by the pest extermination system 1. When the installation of the pest extermination system 1 configured as described above and the creation of necessary data are completed and monitoring of the warehouse R is started, the camera 2 starts taking pictures and the captured data is transmitted to the control device 5 in real time. (ST41, 51). As described above, the control device 5 detects mouse A from the camera image 50 when a moving object is detected, using the animal detection model 56 (ST52). When mouse A is detected, the three-dimensional coordinates of the appearance position 34 of mouse A are immediately identified (ST53). In this way, the intrusion of mouse A into warehouse R is detected in real time, and the appearance position 34 is identified.

[0062] Next, the control device 5 determines an optimal flight route for flying the drone 3 to the specified coordinates (appearance position 34) (ST54). As described above, in this embodiment, a target area RA including the appearance position 34 is determined, and an optimal flight route 62 associated with the target area RA is selected from the candidate route data set. For this purpose, the cloud server 6 is accessed to refer to necessary data. Also, data on mouse detection history is accumulated in the cloud server 6 (ST61). Next, a flight command and a threat command for flying the determined optimal flight route 62 are transmitted to the drone 3, and flight control is performed (ST55).

[0063] The drone 3 is charged and waiting at a drone takeoff and landing station installed in the warehouse R (ST71). The drone 3 scrambles and takes off from the drone takeoff and landing station based on a flight command from the control device 5 (ST72) and flies along the optimal flight route 62 (ST73). For example, a combination of multiple flight commands is sent from the control device 5 as a flight command. The drone 3 drives actuators such as rotors in accordance with the flight commands and flies to the target area RA while performing feedback control of the position of the drone 3 as a control object as necessary to correct any deviation from the optimal flight route 62.

[0064] As a means for correcting the deviation of the current position of the drone 3 from the optimal flight route 62, a program can be used that attaches a marker to the drone 3, constantly grasps the current position, and corrects the trajectory. For example, the navigation lights (infrared LEDs) equipped on the drone 3 are used as the markers. In this case, the navigation lights (infrared LEDs) captured in the image data of the camera 2 are detected, and a process is performed to obtain the three-dimensional coordinates of the detected navigation lights, and the flight commands are corrected by the program so as to eliminate the deviation between the obtained three-dimensional coordinates and the optimal flight route 62.

[0065] When the drone 3 reaches the target area RA, it performs a threatening action in accordance with the threatening command (ST74). If the threatening command is a command to perform a threatening action on the way to the appearance position 34, the drone 3 performs the threatening action on the way to the appearance position 34. As the threatening action, for example, a flip action (an action of making the drone 3 do a one-time somersault). Alternatively, the threatening action may be to generate a threatening sound or to emit light. After the threatening action is completed, the drone 3 is flown along a return route and returned to the waiting position 30 (ST75). The return route may be the same as the optimal flight route 62, or another flight route. For example, a flight route with the minimum energy consumption may be used. The drone 3 starts charging by returning to the waiting position 30, and waits until the next flight command.

[0066] (Main effects of this embodiment) As described above, the pest control system 1 of this embodiment has a camera 2 that photographs the warehouse R, which is a space to be monitored, a drone 3 as an unmanned aerial vehicle that flies over the warehouse R, and a control device 5 that can communicate with the drone 3 and the camera 2. The control device 5 has an animal detection unit 51 that detects mouse A from a camera image 50 taken by the camera 2 using an animal detection model 56 created by machine learning, an appearance position identification unit 52 that identifies the appearance position 34 of mouse A in the warehouse R using output data from the animal detection model 56 and a 3D map 61 of the warehouse R, a flight route determination unit 53 that determines an optimal flight route 62 from the waiting position 30 where the drone 3 is waiting to the appearance position 34 using the 3D map 61, and transmits to the drone 3 a flight command to fly along the optimal flight route 62 and a threatening command to cause the drone 3 to perform a preset threatening action. The drone control unit 54 controls the drone.

[0067] In addition, the pest control program of this embodiment causes the control device 5, which is a general-purpose computer, to perform the following operations: animal detection processing, which uses an animal detection model 56 created by machine learning to detect rat A from a camera image 50 taken by a camera 2 installed in warehouse R; appearance position identification processing, which uses output data from the animal detection model 56 and a 3D map 61 of warehouse R to identify the appearance position 34 of rat A in warehouse R; flight route determination processing, which uses the 3D map 61 to determine an optimal flight route 62 for the drone 3 flying over warehouse R from a waiting position 30 where the drone 3 is waiting to the appearance position 34; and drone control processing, which transmits to the drone 3 a flight command to fly along the optimal flight route 62 and a threatening command to cause the drone 3 to perform a predetermined threatening action.

[0068] According to this embodiment, when a mouse A appears in the warehouse R, the mouse A can be detected from the camera image 50, and the drone 3 can be directed to the appearance position 34 to scare the mouse A away. Therefore, the mouse A can be driven away by automatic processing. In this case, the appearance position 34 of the mouse A is specified using the 3D map 61 of the warehouse R, and the optimal flight route 62 determined using the 3D map 61 is used as the flight route of the drone 3, so that the drone 3 can quickly arrive at the appearance position 34 even in an indoor space where the flight route is limited. Therefore, the mouse A that has invaded the warehouse R can be driven away effectively.

[0069] In this embodiment, the drone 3 waits at each of a plurality of mutually different waiting positions 30 set in the warehouse R, and the flight route determination unit 53 divides the 3D map 61 into a plurality of target areas RA, and determines a flight route corresponding to the target area RA including the appearance position 34 as the optimal flight route 62 from a candidate route data set 66 in which a flight route from one of the waiting positions 30 corresponds to each of the plurality of target areas RA. In this way, when the mouse A appears, it is only necessary to select a flight route corresponding to the target area RA including the appearance position 34 from the candidate route data set 66. Therefore, the load required for the determination process of the optimal flight route 62 can be reduced.

[0070] In this embodiment, the flight routes included in the candidate route data set 66 are determined based on the flight time required to fly from the waiting position 30 to the target area RA, and the flight route with the shortest flight time is determined for each target area RA. Therefore, it is possible to intimidate the mouse A in a short time.

[0071] As described above, the flight route included in the candidate route data set 66 may be determined based on both the flight time and the consumed energy, or the flight route may be determined taking into consideration the balance between the flight time and the consumed energy. Alternatively, the flight route may be determined based on the consumed energy in order to give top priority to ensuring the remaining battery power of the drone 3.

[0072] When taking into consideration the balance between flight time and energy consumption, as described above, a flight route included in the candidate route data set 66 can be selected on the condition that the flight time is shorter than a preset reference flight time. In this way, it is possible to intimidate Mouse A within the desired time.

[0073] In this embodiment, the flight routes included in the candidate route data set 66 are routes that set a no-fly space R1 on the 3D map 61 and connect route points P arranged in a grid pattern in the area excluding the no-fly space R1. In this way, it is easy to set a flight route that does not enter the no-fly space R1. Also, for example, it is possible to set a route that connects the route points P with a simple shape (for example, a straight line or a circular arc), in which case the flight program can be simplified, making it easier to control the drone 3. Alternatively, it is possible to set a route that connects distant route points P with a straight line. In this case, it is easy to create a flight route with less flight time or energy consumption. In addition, it is possible to detect the self-position for each route point P and correct the flight program.

[0074] In this embodiment, the animal detection unit 51 uses the animal detection model 56 to extract an image portion in which the extermination target mouse A appears from the camera image 50, and generates a bounding box 58 of a size that surrounds the extracted image portion, and the appearance position identification unit 52 identifies three-dimensional coordinates indicating the appearance position 34 from the coordinates of the bounding box 58 in the camera image 50. By performing such processing, it is possible to obtain appropriate coordinates as the appearance position 34 of the mouse A.

[0075] In this embodiment, the appearance position identification unit 52 transmits three-dimensional coordinates indicating the appearance position 34 to the cloud server 6, and the 3D map 61 and the candidate route data set 66 are stored in the cloud server 6. In this way, by making the system such that data is stored on the cloud, there is no need to build a dedicated server or communication path. Therefore, the pest extermination system 1 can be easily built.

[0076] In this embodiment, the output data from the animal detection model 56 is stored in the animal detection history database 64 provided in the cloud server 6, so that information useful for future measures against pests can be accumulated in the cloud server 6. For example, this is useful for analyzing the behavioral patterns and intrusion routes of mouse A.

[0077] In this embodiment, the cameras 2 are arranged along the wall surface 31 of the warehouse R, and take pictures facing the floor, and the optimal flight route 62 is a route that flies along the floor of the warehouse R. If the camera 2 takes pictures in this direction, when a pest such as a mouse A running around on the floor of the warehouse R appears, the figure of the pest is likely to be captured by the camera 2. Therefore, there is little risk that the appearance of the mouse A will not be detected. Furthermore, if such a flight route is selected, the drone 3 can be flown close to the mouse A, so that the mouse can be effectively driven away.

[0078] In this embodiment, an infrared camera is used as the camera 2, so that the mouse A can be detected even if the warehouse R to be monitored is dark. Therefore, the mouse A can be chased away at night when the mouse A is most likely to appear, so that the damage caused by the mouse A can be reduced.

[0079] In this embodiment, the mouse A is intimidated using a motion (flip motion) that is generally performed by the drone 3. Therefore, the control for performing the intimidating motion is easy, and the mouse A can be effectively scare away harmful animals.

[0080] (Other embodiments) (1) In the above embodiment, the pest control program is installed in a single computer, the control device 5, but the processing by the pest control program may be configured to be performed in multiple computers. For example, some of the animal detection processing, appearance position identification processing, flight route determination processing, and drone control processing may be configured to be performed in a computer other than the control device 5 (for example, the cloud server 6).

[0081] (2) In the above embodiment, the system is constructed using a cloud server 6, but the system may also be configured to store the 3D map 61, the optimal flight route 62, and the animal detection history database 64 in the control device 5 without using the cloud server 6.

[0082] (3) In the above embodiment, the data set 57 for identifying the appearance position is stored in the control device 5. However, the data set 57 for identifying the appearance position may be stored in the cloud server 6 together with the 3D map 61.

[0083] (4) The above embodiment is a system that monitors at night to drive away mice, but it can be a system that monitors both day and night. In order to make the system compatible with day and night, a camera that can take pictures during the day and at night is installed as a surveillance camera. For example, a camera that takes pictures in visible light mode during the day and automatically takes pictures in infrared mode at night is installed. Then, as an animal detection model, a trained model that can be used for day and night is created by performing machine learning using a dataset that includes both images of mice taken in infrared mode and images of mice taken in visible light mode as training data. Then, the system synchronizes with the camera's shooting mode to detect mice both day and night.

[0084] (5) In the above embodiment, a method of detecting a marker such as an infrared LED captured in an image from the camera 2 is used as a means for grasping the current position of the drone 3 in order to correct deviation from the flight route, but the marker is not limited to an infrared LED. For example, an infrared reflective tape can be used. Also, when performing daytime surveillance, a visible light LED can be used. Alternatively, an AR marker can be used as a daytime marker, and a configuration can be made in which the current position is confirmed using a tool other than an AR marker.

[0085] (6) As a means for determining the current location of drone 3, a method can be used in which a drone detection model is created using machine learning and the area in which drone 3 is captured is extracted from camera image 50 using AI to determine the current location of drone 3.

[0086] (7) In the above embodiment, one drone is sent when mouse A is detected, but multiple drones 3 may be sent to intimidate mouse A. In this case, machine learning is performed to create a drone detection model so that the multiple drones 3 installed in warehouse R can be individually identified. This makes it possible to identify each drone 3 individually and confirm its current location even when multiple drones 3 are sent. [Explanation of symbols]

[0087] 1...pest control system, 2...camera, 3...drone, 4...router, 5...control device, 6...cloud server, 30...waiting position, 31...wall, 32...supplies, 34...appearance position, 50...camera image, 51...animal detection unit, 52...appearance position identification unit, 53...flight route determination unit, 54...drone control unit, 55...memory unit, 56...animal detection model, 57...dataset for identifying appearance position, 58...bounding box, 61...3D map, 62...optimal flight route, 64...animal detection history database, 66...candidate route dataset, A...mouse, P...route point, R...warehouse, R0...flyable space, R1...prohibited space, RA...target area

Claims

1. A camera that captures the monitored space; An unmanned aerial vehicle that flies in the space to be monitored; a control device capable of communicating with the unmanned aerial vehicle and the camera; The control device includes: an animal detection unit that detects animals to be exterminated from camera images taken by the camera using an animal detection model created by machine learning; an appearance position identification unit that identifies an appearance position of the animal in the monitored space using output data from the animal detection model and three-dimensional data of the monitored space; a flight route determination unit that determines an optimal flight route from a waiting position where the unmanned aerial vehicle is waiting to the appearance position using the three-dimensional data; A pest control system characterized by comprising an unmanned aircraft control unit that transmits to the unmanned aircraft a flight command to fly the optimal flight route and a threatening command to cause the unmanned aircraft to perform a predetermined threatening action.

2. The unmanned aerial vehicle waits at each of a plurality of different waiting positions set within the monitored space; The pest control system described in claim 1, characterized in that the flight route determination unit divides the three-dimensional data into a plurality of target areas, and determines, from a candidate route data set in which a flight route from any one of the plurality of waiting positions is associated with each of the plurality of target areas, the flight route associated with the target area including the appearance position as the optimal flight route.

3. The pest control system described in claim 2, characterized in that the flight route included in the candidate route data set is determined based on at least one of the flight time and energy consumed when flying from the waiting position to the target area.

4. The pest control system according to claim 3 , wherein the flight route included in the candidate route data set has a flight time that is shorter than a preset reference flight time.

5. The pest control system described in claim 3, characterized in that the flight routes included in the candidate route data set are routes that set up no-fly spaces on the three-dimensional data and connect route points arranged in a lattice pattern in the area excluding the no-fly spaces.

6. The animal detection unit uses the animal detection model to extract an image portion in which the extermination target animal is captured from the camera image, and generates a bounding box having a size that surrounds the extracted image portion; The pest control system according to claim 1 , wherein the appearance position identification unit identifies three-dimensional coordinates indicating the appearance position from the coordinates of the bounding box in the camera image.

7. the appearance position identification unit transmits three-dimensional coordinates indicating the appearance position to a cloud server; The pest control system of claim 2 , wherein the three-dimensional data and the candidate route data set are stored in the cloud server.

8. The pest control system according to claim 7 , characterized in that output data from the animal detection model is stored in an animal detection history database provided in the cloud server or the control device.

9. The cameras are arranged along the walls of the space to be monitored, and are directed toward the floor to capture images; 2. The pest control system according to claim 1, wherein the optimal flight route is a route that flies along the floor of the space to be monitored.

10. 2. The pest control system according to claim 1, wherein the camera is an infrared camera.

11. The pest control system according to claim 1, wherein the intimidating action is a flip action.

12. An animal detection process for detecting animals to be exterminated from camera images taken by a camera installed in a monitored space using an animal detection model created by machine learning; an appearance position identification process for identifying an appearance position of the animal in the monitored space using output data from the animal detection model and three-dimensional data of the monitored space; a flight route determination process for determining an optimal flight route from a waiting position where the unmanned aerial vehicle flying in the monitored space is waiting to the appearance position using the three-dimensional data; A pest control program characterized by having a single or multiple computers perform an unmanned aircraft control process that sends to the unmanned aircraft a flight command to fly the optimal flight route and a threatening command to cause the unmanned aircraft to perform a predetermined threatening action.

13. The unmanned aerial vehicle waits at each of a plurality of different waiting positions set within the monitored space; The pest control program described in claim 12, characterized in that the flight route determination process is a process of dividing the three-dimensional data into a plurality of target areas, and determining, from a candidate route data set in which a flight route from any one of the plurality of waiting positions is associated with each of the plurality of target areas, the flight route associated with the target area including the appearance position as the optimal flight route.

14. The pest control program according to claim 13, characterized in that the flight route included in the candidate route data set is determined based on at least one of the flight time and energy consumption when flying from the waiting position to the target area.

15. 15. The pest control program according to claim 14, wherein the flight route included in the candidate route data set has a flight time that is shorter than a preset reference flight time.

16. The pest control program described in claim 13, characterized in that each of the flight routes included in the candidate route data set is a route that sets up no-fly spaces on the three-dimensional data and connects route points arranged in a lattice pattern in the area excluding the no-fly spaces.

17. The animal detection process includes a process of extracting an image portion in which the extermination target animal is captured from the camera image using the animal detection model, and generating a bounding box having a size that surrounds the extracted image portion; 13. The pest control program according to claim 12, wherein the appearance position specifying process includes a process of specifying three-dimensional coordinates indicating the appearance position from coordinates of the bounding box in the camera image.

Images