Drone operation support system and drone operation support method

Abstract

To enable an operator to be guided to a suitable position to operate a working drone.SOLUTION: In a drone-work support system 1 for doing one or more work using one or more working drones 7, a work plan creation part 51 of a control server 5 is configured so that: a moving route of the working drone 7 is determined based on the location information of an actual work area necessary for actual work in a predetermined field; a moving route along which the operator should move is determined according to the moving route of the working drone 7; and a work support screen including the moving route of the working drone 7 and the moving route of the operator is displayed on an operation terminal 6. The work plan creation part 51 of the control server 5 may determine the moving route of the operator so that the distance between the operator and the working drone 7 moving along the moving route of the working drone 7 becomes close in range.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a system for supporting work such as surveying, inspecting, maintaining, processing, destroying, or constructing artificial or natural objects such as facilities, buildings, land, fields, and forests, and relates to, for example, a drone work support system that supports work using drones. In this specification, the term "drone" is used to refer to a mobile object that can be operated without a crew member (for example, an aircraft, running vehicle, surface vessel, submarine vehicle, etc. that can be remotely controlled wirelessly or autonomously). [Background technology]

[0002] Drones are now being used for a variety of purposes, such as spraying pesticides and fertilizers in the agricultural field.

[0003] In the past, it was common to spray pesticides and fertilizers by uniformly spreading a specified amount per unit area over the entire field. However, as farmland becomes more consolidated and intensive, problems have become apparent, such as the increasing cost of spreading the specified amount over the entire field, and the shortage of human resources to carry out agricultural work as the area under management increases. Drones are beginning to be used to overcome these problems.

[0004] When used for purposes such as spraying pesticides or fertilizer, a farmer with the experience and skills of a master craftsman first checks the condition of the field and determines the necessary agricultural work. The method and extent of the work, the type of pesticide to be used, and the amount of pesticide to be used are then decided by a human, and the work is carried out by manually operating the work drone in accordance with these decisions.

[0005] Manually operating a work drone requires skilled techniques and is not easy for many people, so there is a demand for technology that can automatically move work drones along a predetermined route.

[0006] In order for a work drone to move automatically along a predetermined route, a map with sufficient positioning accuracy is required when determining the route the work drone will take. However, it is not always easy to obtain a map with sufficient positioning accuracy. For example, publicly available maps cannot be used as is because their positioning accuracy is not guaranteed.

[0007] For example, Patent Documents 1 and 2 disclose a technology that uses an airborne beacon equipped with a GNSS receiver as a technology that can be used to improve the positional accuracy of a map when setting a navigation route. [Prior art documents] [Patent documents]

[0008] [Patent Document 1] Japanese Patent Application Publication No. 2018-146546 [Patent Document 2] International Patent Publication No. 2017 / 024358 Summary of the Invention [Problem to be solved by the invention]

[0009] When using a work drone to spray pesticides or fertilizer, the operator operates the work drone from the ground, so they cannot check the condition of the field while operating the work drone.As a result, even if the area that needs work is only part of the field, it is difficult to maneuver the drone to a position suitable for operation, and the reality is that they work uniformly over the entire field.

[0010] For example, when a work drone is moved automatically, there may be a situation where the operator must switch over to manual control of the work drone. There may also be cases where it is necessary to give the work drone various instructions other than control (e.g., start moving, stop moving, start working, end working, etc.).

[0011] Since work drones are often used in relatively large areas, in some cases the work drone may be moved far away from the operator. In such cases, the operator may be unable to visually check the situation around the work drone, making it difficult to operate it properly.

[0012] There is also a possibility that a work drone will finish a task, move far away from the operator's location, stop, and it may take a long time before it can start another task.

[0013] The present invention has been made in consideration of the above circumstances, and aims to achieve at least one of the following first to third objectives. The first objective is to provide technology that can guide an operator to a position suitable for operating a work drone. The second objective is to provide technology that can easily and appropriately improve the positioning accuracy of a work drone. The third objective is to provide technology that can improve the work efficiency of a work drone. [Means for solving the problem]

[0014] In order to achieve the above-mentioned first object, the drone work support system according to the first aspect is a drone work support system for performing one or more tasks using one or more drones, and includes a drone movement path determination means for determining the movement path of the drone based on position information of an actual work area where actual work is required in a specified target area, an operator movement path determination means for determining the movement path of the operator along the movement path of the drone, and a display control means for displaying a work support screen including the movement path of the drone and the movement path of the operator on an operation terminal.

[0015] With this drone operation support system, the operator can easily and clearly understand how to move when the drone is moving.

[0016] In the drone operation support system, the operator movement path determination means may determine a movement path for the operator that keeps the distance to the drone short, following the movement path of the drone. This drone operation support system allows the operator to move along a path that keeps the distance to the drone short, making it possible to properly grasp the status of the drone and, for example, to properly respond when operation of the drone is required.

[0017] In addition, in the above drone work support system, the drone movement path determination means determines a movement path to a position where energy can be replenished, passing through one or more actual work areas within the range where the drone can move using the energy it has, the operator movement path determination means determines the operator's movement path to be a movement path to the position where energy can be replenished, and the display control means may display the position where energy can be replenished on the work support screen, as well as information indicating energy replenishment.

[0018] This drone operation support system allows operators to know in advance when energy replenishment is required and where energy can be replenished.

[0019] In the drone operation support system, the display control means may display information indicating the position of the operation terminal on the operation support screen based on the position information of the operation terminal. With this drone operation support system, the operator can appropriately grasp the positional relationship between the drone and the operator's own position (operation terminal).

[0020] In addition, in the above drone work support system, the display control means may be configured to display a line on the work support screen indicating a range of a predetermined distance centered on the position of the operation terminal or the position of the drone.

[0021] This drone operation support system allows operators to properly determine whether they are in the right position relative to the drone, and also allows them to easily estimate where to move if they are far away from the drone.

[0022] In order to achieve the above-mentioned first object, the drone work support method according to the second aspect is a drone work support method using a drone work support system for performing one or more tasks using one or more drones, which determines a movement path for the drone based on position information of an actual work area where actual work is required in a specified target area, determines a movement path for the operator to move along with the movement path of the drone, and displays a work support screen including the movement path of the drone and the movement path of the operator on the operation terminal.

[0023] According to this drone operation support method, the operator can easily and clearly understand how to move when the drone is moving.

[0024] Furthermore, in order to achieve the second object, the drone work support system according to a third aspect is a drone work support system for performing one or more tasks using one or more drones, and includes: a map information storage means for storing a high-precision map including high-precision position information of a plurality of positions that define the shape of a predetermined target area; a target area identification means for identifying the target area from an aerial image including an image of the target area; a position information matching means for matching the position of the target area in the aerial image with the position of the target area on the high-precision map based on the shape of the target area in the aerial image and the shape of the target area on the high-precision map; an actual work area identification means for identifying an actual work area in the target area where actual work is required based on the aerial image; a position information identification means for identifying high-precision position information of the actual work area based on the correspondence between the position of the target area in the aerial image identified by the actual work area identification means and the position of the target area on the high-precision map; and a drone movement path determination means for determining a movement path for the drone based on the position information of the actual work area.

[0025] With this drone operation support system, even if the positional accuracy of the aerial image containing an image of the target area is low, the position of the target area in the aerial image can be identified with high accuracy based on a high-precision map.

[0026] In the above drone work support system, the actual work area identification means performs edge extraction on the aerial image to identify the shape of the target area, the location information association means deforms the aerial image so that the shape of the target area in the aerial image matches the shape of the target area based on the high-precision map, and associates the corresponding positions of the target area in the deformed aerial image with the target area based on the high-precision map as being the same position, and the actual work area identification means may identify the actual work area based on the deformed aerial image.

[0027] This drone operation support system makes it possible to associate a deformed aerial image containing an image of the target area with a high-precision map, making it possible to grasp a specific position in the deformed image with high accuracy.

[0028] Furthermore, in order to achieve the second objective, a drone work support method according to a fourth aspect is a drone work support method using a drone work support system for performing one or more tasks using one or more drones, which method includes: storing a high-precision map including high-precision position information of multiple positions that define the shape of a predetermined target area; identifying the target area from an aerial image including an image of the target area; matching the position of the target area in the aerial image with the position of the target area on the high-precision map based on the shape of the target area in the aerial image and the shape of the target area in the high-precision map; identifying an actual work area in the target area where actual work is required based on the aerial image; identifying high-precision position information of the actual work area based on the matching between the position of the identified target area in the aerial image and the position of the target area on the high-precision map; and determining a movement route for the drone based on the position information of the actual work area.

[0029] According to this drone operation support method, even if the positional accuracy of the aerial image containing an image of the target area is low, the position of the target area in the aerial image can be identified with high accuracy based on a high-precision map.

[0030] Furthermore, in order to achieve the third objective, the drone work support system according to the fifth aspect is a drone work support system for performing one or more tasks using one or more drones, and comprises: an actual work area identification means for identifying one or more actual work areas in a target area where actual work is required based on an aerial image including an image of the target area; a work plan creation means for creating a work plan including, in the case of multiple actual work areas, a travel path for moving linearly from the end point of the actual work for one actual work area to the start point of the actual work for the next actual work area, and the work content for performing the actual work between the start point of the actual work for each actual work area and the end point of the actual work; and a control information transmission means for transmitting control information for executing the work plan to the drone.

[0031] This drone work support system can shorten the travel time of drones when performing work, thereby improving processing efficiency.

[0032] Furthermore, in order to achieve the third objective, the drone work support method according to the sixth aspect is a drone work support method using a drone work support system for performing one or more tasks using one or more drones, which identifies one or more actual work areas in a target area where actual work is required based on an aerial image including an image of the target area, and if there are multiple actual work areas, creates a work plan including a movement path for moving in a straight line from the end point of the actual work for one actual work area to the start point of the actual work for the next actual work area, and work content for performing the actual work between the start point of the actual work for each actual work area and the end point of the actual work, and transmits control information for executing the work plan to the drone.

[0033] This drone work support method can shorten the drone's travel time when performing work, thereby improving processing efficiency. [Brief explanation of the drawings]

[0034] [Figure 1] FIG. 1 is a diagram illustrating the overall configuration of a drone operation support system according to one embodiment. [Figure 2] FIG. 2 is a diagram illustrating the configuration and processing of a GNSS airborne beacon according to an embodiment. [Figure 3] FIG. 3 is a configuration diagram of a survey drone according to one embodiment. [Figure 4] FIG. 4 is a configuration diagram of a work drone according to one embodiment. [Figure 5] FIG. 5 is a flowchart of processing by the drone operation support system according to one embodiment. [Figure 6] FIG. 6 is a diagram showing a work support screen according to an embodiment. [Figure 7] FIG. 7 is a flowchart of processing by a drone operation support system according to a modified example. [Figure 8] FIG. 8 is a first diagram illustrating the high-precision map generation process and the farm field condition analysis process according to the modified example. [Figure 9] FIG. 9 is a second diagram illustrating the high-precision map generation process and the farm field condition analysis process according to the modified example. [Figure 10] FIG. 10 is a diagram illustrating the work plan creation process. DETAILED DESCRIPTION OF THE INVENTION

[0035] The following description of the embodiments will be given with reference to the drawings. Note that the embodiments described below do not limit the scope of the invention as claimed, and not all of the elements and combinations thereof described in the embodiments are necessarily essential to the solution of the invention.

[0036] In the following description, an embodiment of the present invention will be explained using as an example an application in which an aerial drone configured as an unmanned aerial vehicle is used for inspecting and maintaining a farm field (target area) (for example, identifying areas where disease or physiological disorders have occurred and spraying pesticides or fertilizers thereon). However, this application is merely an example for the purpose of explanation, and is not intended to limit the application of the embodiment to other applications.

[0037] FIG. 1 is a diagram illustrating the overall configuration of a drone operation support system according to one embodiment.

[0038] The drone operation support system 1 includes a GNSS (Global Navigation Satellite System) airborne beacon 2, a survey drone 3, an image analysis and map generation server 4, a control server 5, an operation terminal 6, and a work drone 7.

[0039] The GNSS airborne beacon 2 is an airborne beacon that can be photographed from the sky, receives signals (GNSS signals) from GNSS satellites, and transmits them to the image analysis and map generation server 4. Details of the GNSS airborne beacon 2 will be described later.

[0040] The survey drone 3 photographs an area including a farm field from above and outputs the photographed images. In this embodiment, when photographing an area including a farm field, it is necessary to place GNSS anti-aircraft markers 2 in advance, for example, at the four corners and the center of the area including the farm field, and after installing the GNSS anti-aircraft markers 2, they must be placed for several tens of minutes to several hours, and GNSS data must be continuously collected during that time. Details of the survey drone 3 will be described later.

[0041] The image analysis and map generation server 4 is configured by, for example, a computer (for example, a PC (Personal Computer) or a server computer) equipped with a processor, memory, and interface. The image analysis and map generation server 4 includes an interface 41, a data storage and search unit 42, a map generation unit 43, a field condition analysis unit 44, and an interface 45.

[0042] The interface 41 communicates data via the network with the GNSS airborne beacon 2, the survey drone 3, a weather data distribution service, a ground sensor observation data distribution service, a GNSS correction data distribution service, etc. Specifically, the interface 41 receives GNSS data from the GNSS airborne beacon 2, receives images including the farm field from the survey drone 3, receives weather-related observation data from the weather data distribution service, receives ground sensor observation data from the ground sensor observation data distribution service, and receives GNSS correction data used for correction to improve the accuracy of the GNSS data from the GNSS correction data distribution service.

[0043] The data storage and retrieval unit 42 stores and manages data received via the interface 41. Specifically, the data storage and retrieval unit 42 stores and manages GNSS data from the GNSS airborne beacon 2, images including the farm field from the survey drone 3, meteorological observation data from a meteorological data distribution service, observation data by ground sensors from a ground sensor observation data distribution service, and GNSS correction data used for correction to improve the accuracy of the GNSS data from a GNSS correction data distribution service.

[0044] The map generation unit 43 generates map data (high-precision map data) by identifying position information (e.g., latitude and longitude) of each position in the image based on the image received from the survey drone 3 stored in the data storage and search unit 42 and the GNSS data received from the GNSS airborne beacon 2 and associating the information with each position. Specifically, the map generation unit 43 identifies the airborne beacons of multiple GNSS airborne beacons 2 in the image, identifies the position information (e.g., latitude and longitude) of each GNSS airborne beacon 2 based on the GNSS data acquired from each GNSS airborne beacon 2 in the image, and associates the identified position coordinates with the position of the GNSS airborne beacon 2 in the image. Note that positions other than those corresponding to the GNSS airborne beacons 2 in the image can be identified based on the position information of multiple GNSS airborne beacons 2. Identifying position information using the GNSS airborne beacons 2 in this manner can achieve high positional accuracy, for example, on the order of centimeters to 10 centimeters.

[0045] The field condition analysis unit 44 analyzes images of a specific field captured by the survey drone 3 to identify an area where actual work is required (actual work area). For example, if the actual work is spraying pesticides on the field, the field condition analysis unit 44 identifies an area where disease is likely to have occurred as the actual work area based on the color and shape of the crops and leaves in the image of the target field. The area where disease is likely to have occurred can be identified from the field image by, for example, using a neural network model that has previously learned about images of disease. The field condition analysis unit 44 may also identify the details of the actual work to be performed on the actual work area (e.g., the pesticide to be sprayed and the amount to be sprayed, etc.).

[0046] The interface 45 performs data communication with the control server 5. The interface 45 transmits to the control server 5 the high-precision map data generated by the map generation unit 43 and the analysis results identified by the field condition analysis unit 44 (information identifying the actual work area and the actual work content).

[0047] The control server 5 is configured by, for example, a computer (for example, a PC (Personal Computer), a server computer) that includes a processor, a memory, and an interface.

[0048] The control server 5 has a work plan creation unit 51 that creates a work plan for the work drone 7. The work plan creation unit 51 is an example of a drone movement path determination means, an operator movement path determination means, and a display control means. The work plan creation unit 51 is a functional unit configured by a processor executing a program. The work plan creation unit 51 receives high-precision map data and analysis results from the image analysis and map generation server 4. Based on the high-precision map data and analysis results, the work plan creation unit 51 determines a work plan for the work drone 7 that satisfies preset conditions. Here, the work plan includes a flight path (an example of a movement path) from a takeoff position to a landing position via one or more actual work areas, and the work content to be performed along the flight path. The takeoff position and landing position may be selected from preset candidate positions. The preset condition may be a work plan that can be realized within the range of flight possible with the battery charge of the work drone 7. Therefore, if the battery capacity of the work drone 7 is not enough to pass through all of the actual work area, the landing position will be selected from positions where the battery can be replaced (refillable positions). The battery capacity of the work drone 7 may be the standard battery capacity of the battery installed in the work drone 7, or if the actual battery capacity of the work drone 7 can be known, the actual battery capacity may be used.

[0049] The work plan may include information about the flight speed of the work drone 7. For example, if the work mechanism 72 of the work drone 7 always sprays pesticides or the like at a constant speed, the flight speed may be determined according to the amount sprayed in the actual work area. In addition, a condition may be that the flight path does not cross a specified road (for example, a public road). In addition, the work plan creation unit 51 transmits the created work plan (specifically, control information for the work plan) to the work drone 7.

[0050] The work plan creation unit 51 also determines the operator's travel route from the takeoff position to the landing position on the flight route in the created work plan. Here, the operator's travel route may be determined based on conditions such as the operator passing through a road that the operator can travel on and the road being close to the flight route of the work drone 7 (for example, a road within a predetermined distance that allows the work drone 7 to be operated appropriately, for example, a road within a maximum of approximately 100 to 150 meters). If the road that the operator can travel on is not within a predetermined distance from the flight route of the work drone 7, the road closest to the flight route may be used. Note that the road that the operator can travel on may be determined by previously associating information about the road that the operator can travel on with map data, or by recognizing whether the road is a road that the operator can travel on from a map or image.

[0051] Furthermore, the work plan creation unit 51 generates a work support screen 100 (see FIG. 6) based on the created work plan and the created movement route of the operator, and transmits it to the operation terminal 6. Details of the work support screen 100 will be described later.

[0052] The operation terminal 6 is a terminal for using the work assistance system 1 and is carried and used by, for example, an operator of the work drone 7. The operation terminal 6 may be, for example, a general-purpose portable information processing terminal (such as a mobile phone, smartphone, tablet terminal, mobile PC, or laptop personal computer). The operation terminal 6 has functions for performing data communication with the control server 5 via a communication network such as the Internet, displaying various information, and receiving signals from GPS satellites. In this embodiment, the operation terminal 6 displays a work assistance screen 100 (see FIG. 6 ) transmitted from the control server 5. The work assistance screen 100 may be displayed, for example, by a web browser on the operation terminal 6. The operation terminal 6 may also have a function for performing data communication with the work drone 7 wirelessly.

[0053] The work drone 7 performs certain tasks to be performed in the field (hereinafter referred to as actual tasks), such as selectively spraying pesticides on areas where disease has occurred in a vast field (actual work area). Details of the work drone 7 will be described later.

[0054] FIG. 2 is a diagram illustrating the configuration and processing of a GNSS airborne beacon according to an embodiment.

[0055] The GNSS airborne beacon 2 receives GNSS data transmitted from a plurality of GNSS satellites 9 .

[0056] The GNSS satellite 9 includes a data generation unit 91 and a GNSS transmitter 92. The data generation unit 91 generates GNSS data to be transmitted. The GNSS transmitter 92 transmits the GNSS data generated by the data generation unit 91.

[0057] The GNSS airborne beacon 2 includes a GNSS receiver 21, a data storage unit 22, and a transmitter 23. The GNSS receiver 21 receives GNSS data transmitted from a plurality of GNSS satellites 9. The data storage unit 22 stores the GNSS data received by the GNSS receiver 21. The transmitter 23 transmits the GNSS data stored in the data storage unit 22 to the image analysis and map generation server 4. The transmitter 23 may transmit data via communication in accordance with 5G (fifth generation mobile communication system).

[0058] FIG. 3 is a configuration diagram of a survey drone according to one embodiment.

[0059] The survey drone 3 includes an ESC (Electronic Speed ​​Controller) 31, a flight mechanism 32, a controller 33, a transmitter 34, a camera 35, and a GPS receiver 36.

[0060] The flight mechanism 32 is a mechanism for flying the survey drone 3, and has, for example, multiple sets of motors and propellers. The ESC 31 controls the rotation speed of the motor of the flight mechanism 32 according to instructions from the controller 33. The GPS receiver 36 receives GPS data from GPS satellites, measures the three-dimensional position (latitude, longitude, altitude) of the survey drone 3, and passes it to the control unit 33. The camera 35 takes images from the sky. The transmitter 34 transmits the images taken by the camera 35 to the image analysis and map generation server 4.

[0061] The controller 33 controls the ESC 31, the transmitter 34, the camera 35, and the GPS receiver 36. Specifically, the controller 33 controls the flight according to an input flight plan. The flight plan is information indicating the flight route to photograph the field where work will be performed. The controller 33 also adds the three-dimensional position of the survey drone 3 at the time the image was captured to the image captured by the camera 35, and passes the image to the transmitter 34.

[0062] The survey drone 3 is equipped with a wireless controller (so-called radio transmitter) (not shown) that allows an operator to wirelessly control the drone from a remote location. The wireless controller can communicate wirelessly with the controller 33, and transmits various control commands to the controller 33 in response to various user operations on the wireless controller.

[0063] FIG. 4 is a configuration diagram of a work drone according to one embodiment.

[0064] The work drone 7 includes an ESC 31, a flight mechanism 32, a controller 71, a work mechanism 72, a transmitter 34, a camera 35, and a high-precision GNSS receiver 73. Note that functional parts similar to those of the survey drone 3 are given the same reference numerals and descriptions thereof will be omitted.

[0065] The high-performance GNSS receiver 73 receives GNSS data and measures the three-dimensional position of the work drone 7 with higher accuracy than the GPS receiver 36. The GNSS receiver 73 may be a receiver that measures the three-dimensional position using technologies such as DGPS (Differential GPS), D-RTK (Differential Real Time Kinematic), or dual-frequency GNSS. The work mechanism 72 performs work on the actual work area of ​​the field. The work mechanism 72 sprays, for example, liquid pesticides or fertilizers or powder pesticides or fertilizers. The work mechanism 72 may spray a fixed amount of pesticide or fertilizer per unit time, or the amount sprayed per unit time may be adjustable. The controller 71 controls the ESC 31, the work mechanism 72, the transmitter 34, the camera 35, and the high-precision GNSS receiver 73. The controller 71 controls flight and work according to the received work plan. Specifically, the controller 71 controls the ESC 31 in accordance with the flight path of the work plan, and controls the work mechanism 72 in accordance with the work content of the work plan.

[0066] The work drone 7 also comes with a wireless controller (so-called radio transmitter) (not shown) that allows the user to remotely and wirelessly control it. The wireless controller can communicate wirelessly with the controller 71, and transmits various control commands to the controller 71 in response to various user operations on the wireless controller.

[0067] Next, the processing operations performed by the drone operation support system 1 will be described.

[0068] FIG. 5 is a flowchart of processing by the drone operation support system according to one embodiment.

[0069] The survey drone 3 takes an image from above an area including a farm field, adds the three-dimensional position measured by the GPS receiver 36 to the image (step S1), and transmits the image to the image analysis and map generation server 4 (step S2).

[0070] Meanwhile, a plurality of GNSS airborne beacons 2 placed in the field receive the GNSS data (step S3) and transmit the received GNSS data to the image analysis and map generation server 4 (step S4). The GNSS airborne beacons 2 execute the processes of steps S3 and S4 for a period of time that is longer than that required for highly accurate position analysis.

[0071] The image analysis and map generation server 4 analyzes the highly accurate position of each GNSS airborne marker 2 based on the GNSS data transmitted from the multiple GNSS airborne markers 2 (step S5). Next, the image analysis and map generation server 4 extracts the GNSS airborne markers 2 from the image of the farm field transmitted from the survey drone 3 (step S6). Next, the image analysis and map generation server 4 generates a highly accurate map by associating the positions of the GNSS airborne markers 2 in the image with the positions of the corresponding GNSS airborne markers 2 obtained by analysis in step S5 (step S7).

[0072] Next, the image analysis and map generation server 4 analyzes the condition of the field from the image of the field transmitted from the survey drone 3 (step S8). Specifically, the image analysis and map generation server 4 identifies the actual work area where work such as spraying pesticides or fertilizer will be performed. Next, the image analysis and map generation server 4 transmits the high-precision map generated in step S7 and the analysis result of step S8 (information indicating the identified actual work area: actual work area information) to the control server 5 (step S9).

[0073] The control server 5 receives the high-precision map and actual work area information sent from the image analysis / map generation server 4 (step S10), and creates a work plan based on the high-precision map and actual work area information, and creates a work support screen 100 including the work plan and operator route (step S11). Next, the control server 5 sends the work plan (control information for executing the work plan) to the work drone 7, and sends the work support screen 100 to the operation terminal 6 (step S12).

[0074] Thereafter, the work drone 7 moves and performs work in accordance with the work plan received from the control server 5 (step S13). Meanwhile, the operation terminal 6 displays the work support screen 100 received from the control server 5 on the display unit (liquid crystal panel, etc.) (step S14).

[0075] FIG. 6 is a diagram showing a work support screen according to an embodiment.

[0076] An image 101 including a farm field is displayed on the work support screen 100. In the example of Fig. 6, the image 101 includes two farm fields 104 and a road (e.g., a public road) 103. Note that the farm field 104 may be made up of multiple small farm fields.

[0077] The work support screen 100 displays, on an image 101, a flight path 110 of the work drone 7 and a movement path 120 of the operator.

[0078] The flight path 110 includes a start point 111 indicating the takeoff location of the work drone 7, an end point 112 indicating the landing location, and one or more actual work areas 113, with a line connecting the start point 111 to the end point 112 via one or more actual work areas 113. Note that the work support screen 100 in Figure 6 shows an example of a flight path 110 where the condition is that the work drone 7 does not fly across the road 103, and therefore the actual work area 113 in the field 104 beyond the road 103 is not included in the same flight path 110.

[0079] Furthermore, in this embodiment, a drone mark 115 indicating the actual position of the work drone 7 is displayed on the flight path 110. Furthermore, at the end point 112, a warning message 114 is displayed as information indicating that the battery of the work drone 7 needs to be replaced. This warning message 114 allows the operator to know in advance that a battery replacement is required at the end point 112, so a replacement battery can be arranged for the end point 112 in advance.

[0080] The operator's movement route 120 is a route from the start point 111 to the end point 112 along roads that the operator can move along.

[0081] The work support screen 100 also displays an operator mark 121 indicating the current position of the operator (strictly speaking, the operation terminal 6), and an appropriate range 122 indicating a predetermined distance (for example, a distance of 100 to 150 m) suitable for operating the work drone 7, centered on the operator mark 121. Depending on whether the drone mark 115 is present within this appropriate range 122, the operator can easily determine whether the work drone 7 is at an appropriate distance for operation. In the example of FIG. 6, the appropriate range is displayed centered on the operator's position (the position of the operator mark 121 in FIG. 6). However, for example, the appropriate range may be a predetermined distance range centered on the position of the work drone 7 (the position of the drone mark 115 in FIG. 6). In this case, depending on whether the operator is present within the appropriate range, it is easy to determine whether the distance to the work drone 7 is appropriate for operation.

[0082] Here, we consider how to enable the work drone 7 to work in an appropriate position.

[0083] In order for the work drone 7 to work in the appropriate position, the position of the work drone 7 needs to be controlled with high precision, and the positional accuracy of the work drone 7 is key.

[0084] The position error Δ occurring in the work drone 7 is It can be expressed as Δ=Δrobot+Δbasemap+Δsesing_data.

[0085] Here, Δrobot indicates the error caused by the positional accuracy of the work drone 7 itself, and indicates the degree of accuracy with which the work drone 7 can move to a specified point when instructed to move to that point.

[0086] Δbasemap indicates the error due to the positional accuracy of the map used when issuing work instructions to the work drone 7, and indicates the degree of accuracy with which the locations of features on the map are managed. This map is used, for example, when working the entire field, when moving from a parking lot to a field, or when moving between fields.

[0087] Δsensing_data indicates the error resulting from the data acquired to determine the latest field conditions. Aerial photographs are a typical example of this data. Many aerial photographs store GNSS data at the time of capture, but this often contains an error of around ±10 m due to the positional accuracy of the survey drone. Furthermore, in many cases, the orientation (direction) of the survey drone 3 at the time of capture is not stored, making it difficult to properly overlay the aerial photograph on a map. Furthermore, due to large altitude errors, the size and distance of features are often not displayed correctly when the image is overlaid on a map.

[0088] There are technologies such as DGPS, D-RTK, and dual-frequency GNSS that can improve Δrobot, which can achieve positioning accuracy on the order of centimeters.

[0089] As a method for improving the Δbasemap, there are technologies such as ortho generation processing using GNSS airborne landmarks, which can achieve positional accuracy on the order of cm to 10 cm.

[0090] One way to improve Δsensing_data is to implement technologies such as DGPS, D-RTK, and dual-frequency GNSS in survey drones, just like with the Δrobot. However, these technologies are not implemented in commonly used survey drones, and developing a dedicated drone capable of using this technology would be expensive, so the number of users of dedicated drones is limited.

[0091] In contrast to this, there is a method of using GNSS aerial beacons 2 (as with Δbasemap) as shown in the example above when acquiring sensing data (for example, aerial images). With this method, it is necessary to keep the GNSS aerial beacons 2 in place at the site for several tens of minutes to several hours after installation to acquire GNSS data, and it is also necessary to install the GNSS aerial beacons 2 at, for example, the four corners and the center of the imaging area, which may make it difficult to use when sensing a wide area.

[0092] In response to this, the positioning accuracy of work drones can be improved by using a drone operation support system according to the following modified example. For convenience, the drone operation support system according to the modified example will be described with reference to FIG.

[0093] In the drone work support system of the modified example, if polygon information (for example, latitude and longitude information of the vertices of the field outline) that can identify the position and shape of a field with high accuracy can be prepared in advance, the position information of a specified position in the captured image can be identified with high accuracy based on this polygon information and the image captured by the survey drone 3.

[0094] In the drone operation support system, polygon information about farm fields is stored in the data storage and search unit 42 of the image analysis and map generation server 4. The polygon information about farm fields can be obtained, for example, from a farm field information providing service. The data storage and search unit 42 is an example of map information storage means.

[0095] FIG. 7 is a flowchart of processing by a drone operation support system according to a modified example.

[0096] The survey drone 3 takes an image from above the area including the farm field, adds the three-dimensional position measured by the GPS receiver 36 to the image (step S15), and transmits the image to the image analysis / map generation server 4 (step S16).

[0097] The image analysis and map generation server 4 generates a high-precision map based on the polygon information stored in the data storage and search unit 42 and the image of the farm field transmitted from the survey drone 3 (step S17: high-precision map generation process). The high-precision map generation process will be described in detail later.

[0098] Next, the image analysis and map generation server 4 analyzes the image of the field to identify an actual work area where work such as spraying pesticides or fertilizer will be performed, and extracts the position information of this actual work area based on the high-precision map (step S18: field condition analysis processing).The image analysis and map generation server 4 then transmits the high-precision map generated in step S17 and the position information of the actual work area identified in step S18 (actual work area information) to the control server 5 (step S19).

[0099] The control server 5 receives the high-precision map and actual work area information sent from the image analysis / map generation server 4 (step S20), and creates a work plan based on the high-precision map and actual work area information, and creates a work support screen 100 including the work plan and operator route (step S21). Next, the control server 5 sends the work plan to the work drone 7, and sends the work support screen 100 to the operation terminal 6 (step S22).

[0100] Thereafter, the work drone 7 moves and performs work in accordance with the work plan received from the control server 5 (step S23). Meanwhile, the operation terminal 6 displays the work support screen 100 received from the control server 5 on the display unit (step S24).

[0101] FIG. 8 is a first diagram illustrating the high-precision map generation process (step S17 in FIG. 7) and the field condition analysis process (step S18 in FIG. 7) according to the modified example, and FIG. 9 is a second diagram illustrating the high-precision map generation process and the field condition analysis process according to the modified example.

[0102] First, the data storage and search unit 42 of the image analysis and map generation server 4 registers the image of the farm field transmitted from the survey drone 3 (step A).

[0103] Next, the map generation unit 43 (an example of a target area identification means and a position information association means) of the image analysis and map generation server 4 performs edge detection processing on the registered image 200 to extract an edge (field edge) 201 that is assumed to be a field (Step B). Next, the map generation unit 43 identifies polygon information corresponding to the field edge 201 from the polygon information about the field stored in the data storage and search unit 42, and superimposes a polygon 210 based on the polygon information on the field edge 201 on the field edge 201 (Step C). The polygon information corresponding to the field edge 201 is polygon information that has position information similar to the position information assigned to the image 200 from which the field edge 201 was extracted, and is polygon information that has a shape similar to that of the field edge 201 (for example, the same or a similar shape).

[0104] Next, the map generating unit 43 adjusts the angle of the field edge 201, that is, rotates the field edge 201, so that each side of the field edge 201 and each side of the polygon 210 are parallel to each other (step D).

[0105] Next, the map generating unit 43 adjusts the magnification rate of the field edge 201 so that each side of the field edge 201 coincides with each side of the polygon 201 (step E).

[0106] Next, the map generation unit 43 rotates the image 200 of the field transmitted from the survey drone 3 by the adjustment angle in step C relative to the field edge 201, adjusts the magnification in step D, and associates the position of the polygon information with the adjusted image 202 (step F). This makes it possible to generate a highly accurate map in which the polygon 210 of the field is superimposed and associated with the image 202 based on the image 200 of the field transmitted from the survey drone 3. This makes it possible to identify the position (latitude, longitude) of a predetermined point in the image 202 with high accuracy based on the position information of the polygon information.

[0107] Next, the field condition analysis unit 44 (an example of an actual work area identification means and a position information identification means) analyzes the adjusted image 202 to extract an actual work area 220 where, for example, work such as spraying pesticides or fertilizer will be performed (step G), identifies polygon information (e.g., the latitude (lat) and longitude (lon) of each vertex indicating the position of the actual work area 220) corresponding to the extracted actual work area 220 (e.g., 220A, 220B), and outputs the polygon information (position information) of the actual work area 220 together with the adjusted image 202 and the associated polygon information to the control server 5 (step H). Here, in FIG. 9 , the polygon information of the actual work area 220A is shown as Polygon 1, and the polygon information of the actual work area 220B is shown as Polygon 2. This process enables highly accurate position information for the actual work area 220 to be output to the control server 5.

[0108] According to the modified drone work support system, even if the survey drone 3 is not equipped with a high-precision GNSS receiver, and even if a GNSS aerial beacon 2 is not placed and photographed, it is possible to create a high-precision map and obtain high-precision location information of the actual work area.

[0109] Next, a specific example of the work plan creation process will be described. This process corresponds to step S11 in FIG. 5 and step S22 in FIG.

[0110] FIG. 10 is a diagram illustrating the work plan creation process.

[0111] The work plan creation unit 51 (an example of a work plan creation means and a control information transmission means) creates a work plan as shown in Figure 10(a) when work is to be performed uniformly over the entire surface of a field. Specifically, the work plan creation unit 51 continues work while moving in a first direction (horizontal direction in the drawing), and when it reaches the end of the first direction, it moves in a second direction (vertical direction in the drawing) that intersects with the first direction without performing work, and after moving, it moves in the first direction and performs work. Here, the movement path shown in Figure 10(a) is called the entire surface work path.

[0112] In addition, when the work plan creation unit 51 receives polygon information corresponding to the actual work area 220 (for example, the latitude and longitude of each vertex of the actual work area 220) from the control server 5, it creates a work plan using one of methods 1 (Figure 10(c)) to 3 (Figure 10(e)).

[0113] The work plan according to method 1 is a work plan in which the work drone 7 moves along the same route as the full-area work route and performs work only within the range of the actual work area 220.

[0114] The work plan according to method 2 is a work plan in which the work drone 7 moves along the entire work route, omitting the route in the first direction where the actual work area 220 does not exist, and the work is performed only within the range of the actual work area 220.

[0115] The work plan according to Method 3 is a movement plan in which the work drone 7 is moved linearly to a position (work start point) in the actual work area 220 where work will begin, to perform the actual work, and after completing the work in that actual work area 220, is moved linearly to the work start point of the next actual work area 220. The order in which the work in the actual work area 220 is performed may be determined so as to create the shortest movement path.

[0116] According to the above-mentioned work plan creation process, regardless of which method is used to create the work plan, processing efficiency can be improved compared to when work is done uniformly over the entire field. In particular, work planning using method 3 can shorten the travel distance of the work drone 7, resulting in the highest processing efficiency.

[0117] The present invention is not limited to the above-described embodiment, and can be modified appropriately without departing from the spirit of the present invention.

[0118] For example, in the high-precision map generation process and field condition analysis process shown in Figures 8 and 9 described above, the actual work area 220 is extracted from the adjusted image 202, but the present invention is not limited to this. For example, the actual work area may be extracted from the pre-adjustment image 200, adjustments may be made to the actual work area in steps C and D, and the position information of the adjusted actual work area may be determined based on the polygon information of the field.

[0119] Furthermore, in the above embodiment and modified example, the control server 5 and the image analysis / map generation server 4 are configured as separate server computers, but the present invention is not limited to this, and they may be configured as a single processing server configured by the same server computer, or multiple functional means realized by the image analysis / map generation server 4 may be executed by more server computers.

[0120] In addition, in the above embodiment, a work drone that moves using battery power as energy has been shown as an example, but the present invention is not limited to this, and a work drone that moves using energy based on fuel such as gasoline, diesel, etc. In this case, the above-mentioned battery replacement point can be read as a fuel replenishment point. [Explanation of symbols]

[0121] 1...Drone work support system, 2...GNSS aerial beacon, 3...Survey drone, 4...Image analysis and map generation server, 5...Control server, 6...Operation terminal, 7...Work drone

Claims

1. A drone operation support system for performing one or more operations using one or more drones, an actual work area specifying means for specifying one or more actual work areas in a predetermined target area where actual work is required based on an aerial image including an image of the target area; a work plan creation means for creating a work plan including a movement path for moving linearly from an end point of an actual work along an entire work path when work is to be performed uniformly over the entire surface of the target area for a certain actual work area to a start point that is a point on the entire work path where actual work for the next actual work area is performed, whichever is closer to the start point or end point of the path on the entire work path where actual work for each actual work area is performed, and work content for executing the actual work between the start point and the end point that is a point on the entire work path where actual work is performed for each actual work area; and a control information transmitting means for transmitting control information for executing the work plan to the drone. Drone work support system.

2. A drone operation support method using a drone operation support system for performing one or more tasks using one or more drones, Identifying one or more actual work areas in the target area that require actual work based on an aerial image including an image of the target area; When there are a plurality of said actual work areas, a work plan is created which includes a movement path for moving linearly from an end point of an actual work along an entire work path when work is performed uniformly over the entire surface of the target area for a certain actual work area to a start point which is a point on the entire work path where actual work for the next actual work area is performed, which is a point closer to the start point or end point of the path on the entire work path where actual work for each actual work area is performed, and work content for executing the actual work between the end point which is a point farther from the start point of the path on the entire work path where actual work is performed for each actual work area, Transmitting control information to the drone to execute the work plan Drone work support methods.

Images