Display systems and methods for work vehicles and unmanned aerial vehicles.

Abstract

This display system causes a display device to display the respective positions of a work vehicle and one or more unmanned aerial vehicles in the vicinity of the work vehicle. The display system comprises a processing device that: acquires position information of the work vehicle and each of the unmanned aerial vehicles; and on the basis of the position information, causes the display devices to display, in a field shown thereon, the respective positions of the work vehicle and the unmanned aerial vehicles.

Description

Technical Field

[0001] The present disclosure relates to a display system and a display method for a work vehicle and an unmanned aerial vehicle.

Background Art

[0002] An unmanned aerial vehicle (UAV) is an aircraft that cannot be boarded by a person due to its structure and can fly by remote control or automatic control. A rotary-wing unmanned aerial vehicle is an unmanned aerial vehicle that obtains lift using a propeller that rotates around an axis, that is, a rotary wing. A small unmanned aerial vehicle (Multi-Rotor UAV) equipped with a plurality of rotary wings is also called a "drone", "multi-rotor", or "multi-copter" and is widely used in applications such as aerial photography, surveying, logistics, and agricultural chemical spraying.

[0003] Patent Document 1 describes an unmanned aerial vehicle (unmanned flying body) that changes its flight position in conjunction with the operation of agricultural machinery. For example, it is described that an agricultural tractor transmits information (such as the position of the field, the position of the tractor, the vehicle speed, the up / down state of the lifting device that connects the working device) to the unmanned flying body, and the unmanned flying body changes its flight position based on this information. Also, it is described that the unmanned flying body transmits information such as the size and position of an object detected using a sensing device to the tractor, and the tractor changes the detection area for detecting obstacles based on this information.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] This disclosure provides a display system and display method that enable effective information display in a system including work vehicles and unmanned aerial vehicles that perform ground-based work such as agricultural or construction work. [Means for solving the problem]

[0006] In exemplary and non-limiting embodiments, the display system of this disclosure is a system for displaying the positions of a work vehicle and one or more unmanned aerial vehicles flying around the work vehicle on a display device. The display system includes a processing device that acquires position information of the work vehicle and the unmanned aerial vehicle and displays the positions of the work vehicle and the unmanned aerial vehicle on a field shown on the display device based on the position information. [Effects of the Invention]

[0007] According to embodiments of this disclosure, the positions of the work vehicle and one or more unmanned aerial vehicles in its vicinity can be displayed on a field shown on the display device. This makes it possible for the user to quickly understand the positional relationship between the work vehicle and the unmanned aerial vehicle. [Brief explanation of the drawing]

[0008] [Figure 1A] This is a schematic block diagram showing several examples of rotary drive devices for rotating rotors in an unmanned aerial vehicle equipped with multiple rotors. [Figure 1B] This is a schematic plan view illustrating one basic configuration example of an unmanned aerial vehicle equipped with multiple rotors. [Figure 1C] This is a schematic side view illustrating one basic configuration example of an unmanned aerial vehicle equipped with multiple rotors. [Figure 1D] This is a schematic plan view illustrating another basic configuration example of an unmanned aerial vehicle equipped with multiple rotors. [Figure 2A] This block diagram shows an example of the basic configuration of a battery-powered multirotor. [Figure 2B]This block diagram shows an example of the basic configuration of a series hybrid drive multicopter. [Figure 2C] This is a block diagram showing an example of the basic configuration of a parallel hybrid drive type multicopter. [Figure 3] This figure shows an example of a system including a multirotor and an agricultural work vehicle. [Figure 4] Figure 3 shows an example of the system configuration. [Figure 5] This flowchart shows an example of a communication method performed by the control device of a work vehicle or multirotor. [Figure 6] The first figure illustrates an example of a communication method in a system where a work vehicle and a multicopter operate in conjunction. [Figure 7] The second figure illustrates an example of a communication method in a system where a work vehicle and a multicopter operate in conjunction. [Figure 8] This flowchart shows an example where the communication mode changes in three stages depending on the distance between the work vehicle and the multicopter. [Figure 9] Block diagram showing an example of the hardware configuration of a control system for a work vehicle. [Figure 10] This flowchart shows an example of a display method executed by the processor (processing unit) in the control system of a work vehicle. [Figure 11] This figure shows an example of a map image displayed on a display device. [Modes for carrying out the invention]

[0009] Unmanned aerial vehicles equipped with multiple rotors are equipped with a rotational drive device that rotates the rotors (hereinafter sometimes referred to as "propellers"). Hereafter, such unmanned aerial vehicles will be referred to as "multicopters."

[0010] There are various configurations for the rotary drive systems that multirotors are equipped with. Figure 1A is a schematic block diagram showing four examples of the rotary drive system 3 in this disclosure.

[0011] The first rotational drive device 3A shown in FIG. 1A includes a plurality of electric motors (hereinafter referred to as "motors") 14 that rotate a plurality of rotors 2, and a battery 52 that stores electric power supplied to each motor 14. The battery 52 is a secondary battery such as a polymer lithium-ion battery, for example. Each rotor 2 is connected to the output shaft of the corresponding motor 14 and is rotated by the motor 14. In order to increase the payload and / or flight time, it is necessary to increase the power storage capacity of the battery 52. The power storage capacity of the battery 52 can be increased by increasing the size of the battery 52, but increasing the size of the battery 52 causes an increase in weight.

[0012] The second rotational drive device 3B shown in FIG. 1A includes a power transmission system 23 that is mechanically connected to the rotor 2, and an internal combustion engine 7a that applies a driving force (torque) to the power transmission system 23. The power transmission system 23 includes mechanical components such as gears or belts, for example, and transmits the torque of the output shaft of the internal combustion engine 7a to the rotor 2. The internal combustion engine 7a can efficiently generate mechanical energy by burning fuel. Examples of the internal combustion engine 7a can include a gasoline engine, a diesel engine, and a hydrogen engine. Also, the number of internal combustion engines 7a included in the rotational drive device 3B is not limited to one.

[0013] The third rotational drive device 3C shown in FIG. 1A includes a plurality of motors 14, a power buffer 9 that stores power supplied to each motor 14, a power generation device 8 such as an alternator that generates power, and an internal combustion engine 7a that provides mechanical energy for power generation to the power generation device 8. A typical example of the power buffer 9 is a battery such as a secondary battery, but it may also be a capacitor. In the third rotational drive device 3C, even when the storage capacity of the power buffer 9 is not large, the power generation device 8 generates power using the driving force (mechanical energy) of the internal combustion engine 7a, making it possible to increase the payload and / or flight time. Such a type of drive is called "series hybrid drive". The power generation device 8 and the internal combustion engine 7a in the series hybrid drive are called "range extenders" to extend the flight distance of the multicopter.

[0014] The fourth rotational drive device 3D shown in FIG. 1A includes a plurality of motors 14, a power buffer 9 that stores power supplied to each motor 14, a power generation device 8 such as an alternator that generates power, an internal combustion engine 7a that provides a driving force for power generation to the power generation device 8, and a power transmission system 23 that transmits the driving force generated by the internal combustion engine 7a to the rotor 2 to rotate the rotor 2. At least one of the plurality of rotors 2 is rotated by the internal combustion engine 7a, and the other rotors 2 are rotated by the motors 14. In the fourth rotational drive device 3D, the mechanical energy generated by the internal combustion engine 7a can be used for the rotation of the rotor 2 without converting it into electric power, making it possible to improve the energy utilization efficiency. Such a type of drive is called "parallel hybrid drive".

[0015] FIG. 1B is a plan view schematically showing one basic configuration example of the multicopter 10. The configuration example in FIG. 1B includes the first rotational drive device 3A shown in FIG. 1A as the rotational drive device 3. That is, the rotational drive device 3 (3A) in this example includes a motor 14 and a battery 52. FIG. 1C is a side view schematically showing the multicopter 10.

[0016] The multicopter 10 shown in Figures 1B and 1C comprises multiple rotors 2, a main body 4, and a frame 5 that supports the rotors 2 and the main body 4. The frame 5 supports the main body 4 at its center and rotatably supports the multiple rotors 2 with multiple arms 5A extending outward from the center. A motor 14 for rotating the rotors 2 is provided near the tip of each arm 5A. The main body 4 and the frame 5 are sometimes collectively referred to as the "aircraft 11".

[0017] In the example shown in Figure 1B, the multicopter 10 is a quadcopter equipped with four rotors 2. Rotors 2 located on one diagonal rotate in the same direction (clockwise or counterclockwise), while rotors 2 located on different diagonal lines rotate in opposite directions.

[0018] The aircraft body 4 includes a control device 4a that controls the operation of devices and components mounted on the multicopter 10, a group of sensors 4b connected to the control device 4a, a communication device 4c connected to the control device 4a, and a battery 52.

[0019] The control device 4a may include, for example, a flight control device such as a flight controller and a higher-level computer (companion computer). The companion computer can perform advanced computational processing such as image processing, obstacle detection, and obstacle avoidance based on sensor data acquired by the sensor group 4b.

[0020] The sensor group 4b may include an acceleration sensor, an angular velocity sensor, a geomagnetic sensor, a barometric pressure sensor, an altitude sensor, a temperature sensor, a flow rate sensor, an imaging device, a laser sensor, an ultrasonic sensor, an obstacle contact sensor, and a GNSS (Global Navigation Satellite System) receiver. The acceleration sensor and angular velocity sensor may be mounted on the aircraft body 4 as components of an IMU (Inertial Measurement Unit), for example. Examples of laser sensors may include a laser rangefinder used to measure the distance to the ground, and a two-dimensional or three-dimensional LiDAR (light detection and ranging).

[0021] The communication device 4c may include a wireless communication module for transmitting and receiving signals to and from a transmitter or ground control station (GCS) on the ground via an antenna, and a mobile communication module that utilizes a cellular communication network. The communication device 4c may receive signals such as control commands transmitted from the ground and transmit sensor data such as image data acquired by the sensor group 4b as telemetry information. The communication device 4c may also have the function of communicating with other multirotors and the function of satellite communication. The control device 4a can be connected to a computer on the cloud by the communication device 4c. Some or all of the functions of the companion computer may be performed by the computer on the cloud.

[0022] The battery 52 is a rechargeable battery that can store power by charging and supply power to the motor 14 by discharging. Through the action of the battery 52 and the multiple motors 14, multiple rotors 2 are driven to rotate, making it possible to generate the desired thrust.

[0023] Each of the multiple rotors 2 generally has multiple blades with a fixed pitch angle and generates thrust through rotation. The pitch angle may be variable. Not all of the multiple rotors 2 need to have the same diameter (propeller diameter); one or more rotors 2 may have a larger diameter than the others. The thrust generated by the rotating rotors 2 (static thrust) is generally proportional to the cube of the rotor's diameter. For this reason, when rotors 2 with different diameters are provided, the rotor 2 with a relatively larger diameter may be called the "main rotor," and the rotor 2 with a relatively smaller diameter may be called the "sub-rotor." Regardless of the size of the diameter, the configuration of the rotary drive unit 3 may include a rotor 2 with a relatively large thrust and a rotor 2 with a relatively small thrust. In that case, the rotor 2 with a relatively large thrust may be called the "main rotor," and the rotor 2 with a relatively small thrust may be called the "sub-rotor." For example, a rotor 2 that generates a relatively large thrust per revolution may be called the "main rotor," and a rotor 2 that generates a relatively small thrust per revolution may be called the "sub-rotor." In one example, the main rotor may be positioned inward of the sub-rotors. In other words, each rotor 2 may be positioned such that the distance from the center of the aircraft to the axis of rotation of each main rotor is shorter than the distance from the center of the aircraft to the axis of rotation of each sub-rotor.

[0024] In this example, the rotary drive unit 3 has a plurality of motors 14. As mentioned above, the rotary drive unit 3 may also include an internal combustion engine 7a.

[0025] Figure 1D is a schematic plan view showing a basic configuration example of a multicopter 10 equipped with a second rotary drive unit 3B as the rotary drive unit 3. In the example shown in Figure 1D, the internal combustion engine 7a is supported by the aircraft body 4. In this example, the driving force generated by the internal combustion engine 7a is transmitted to multiple rotors 2 by multiple power transmission systems 23, causing each rotor 2 to rotate. The control device 4a can change the rotational speed of each rotor 2 by controlling each power transmission system 23. The rotary drive unit 3B may include a mechanism for changing the pitch angle of each blade of the multiple rotors 2. In that case, the control device 4a may adjust the lift generated by each rotor 2 by controlling this mechanism to change the pitch angle of the blades.

[0026] In a "parallel hybrid drive" configuration, where some of the multiple rotors 2 are rotated by an internal combustion engine 7a and other rotors 2 are rotated by a motor 14, the internal combustion engine 7a and battery 52 are supported by the main body 4. At least one of the multiple rotors 2 is connected to the internal combustion engine 7a via a power transmission system 23, and the other rotors 2 are connected to the motor 14.

[0027] In such a parallel hybrid drive, the diameter of one or more rotors 2 rotated by the internal combustion engine 7a may be larger than the diameter of the other rotors 2 rotated by the motor 14. In other words, the internal combustion engine 7a may be used to rotate the main rotor, and the motor 14 may be used to rotate the sub-rotor. In such a case, the main rotor is mainly used for thrust generation, and the sub-rotor is used for thrust generation and attitude control. The main rotor may be called the "booster rotor" and the sub-rotor the "attitude control rotor".

[0028] In parallel hybrid drive systems, the internal combustion engine is used for both thrust generation and power generation. By selectively transmitting the driving force (torque) generated by the internal combustion engine to either the rotor or the power generator, or both, it is possible to achieve a balanced combination of thrust generation and power generation.

[0029] The inclusion of an internal combustion engine in a multirotor, which generates thrust and / or electricity, contributes to increased payload and flight time. Attitude control of a multirotor is preferably performed by rotating the propellers with motors that have superior response characteristics to internal combustion engines. Therefore, in applications requiring precise attitude control of a multirotor, employing a parallel hybrid drive or series hybrid drive is desirable to increase payload and flight time. Furthermore, if the rotary drive unit 3 includes a mechanism for changing the pitch angle of each blade of the multiple rotors 2, attitude can also be adjusted by changing the pitch angle of each blade.

[0030] Increased payload and flight time could further expand the applications of multicopters. For example, in agriculture, multicopters are currently being used for pesticide spraying or monitoring crop growth, but by attaching various ground implements (hereinafter sometimes simply referred to as "implements") to the multicopter, it will be possible to perform various agricultural tasks from the air. Agricultural implements are sometimes called "implements." Examples of implements may include sprayers for spraying pesticides on crops, mowers, seeders, spreaders, rakes, balers, harvesters, plows, harrows, or rotary tillers. Work vehicles such as tractors are not included in the definition of "implements" in this disclosure.

[0031] In the example shown in Figure 1C, the multicopter 10 is coupled with an implement 200 capable of spraying, for example, pesticides or fertilizers onto or within a field. Increased payload and flight time allow for larger and / or more multifunctional implements 200. For example, by changing the implement 200 coupled to the multicopter 10, it becomes possible to perform a variety of ground operations (agricultural work), including liquid and granular application, fertilization, thinning, weeding, transplanting, direct seeding, and harvesting. The implement 200 may be equipped with mechanisms such as a robotic hand. In that case, one implement 200 can perform a variety of ground operations. If the implement 200 has enough space to accommodate materials, it can also be used to transport agricultural materials or harvested produce over a wide area. There are various ways in which the implement 200 is coupled to the multicopter 10. The multicopter 10 may suspend and tow the work machine 200 by a cable. The work machine 200 towed by the multicopter 10 can perform ground work while being towed, while the multicopter 10 is flying or hovering. The work machine 200 may be in the air or on the ground while performing work.

[0032] In the example shown in Figure 1C, the multicopter 10 is equipped with a power supply device 76. The power supply device 76 is a device that supplies power to the work implement 200 from a drive energy source such as a battery 52 or a power generator 8 provided by the multicopter 10. Various functions of the work implement 200 can be performed by this power. The work implement 200 is equipped with actuators such as motors that are operated by the power obtained from the power supply device 76 of the multicopter 10. Preferably, the work implement 200 is equipped with a battery for storing power. 。

[0033] Figure 2A is a block diagram showing an example of the basic configuration of a battery-powered multicopter 10. The battery-powered multicopter 10 includes multiple rotors 12, multiple motors 14 that rotate each of the multiple rotors 12, multiple ESCs (Electric Speed ​​Controllers) 16 each having a motor drive circuit that drives each of the multiple motors 14, a battery 52 that supplies power to the corresponding motors 14 via each ESC 16, a control device 4a for controlling the attitude and performing flight by controlling the multiple ESCs 16, a sensor group 4b, a communication device 4c, and a power supply device 76 that is electrically connected to the battery 52. ​​In Figure 2A, for simplicity, the rotors 12, motors 14, and ESCs 16 are each shown as one block, but the number of rotors 12, motors 14, and ESCs 16 is multiple. This is also true for Figures 2B and 2C. ESC16 may be included in the control unit 4a.

[0034] The control device 4a can wirelessly receive control commands from, for example, a ground station 6 located on the ground via the communication device 4c. The number of ground stations 6 is not limited to one, but may be distributed across multiple locations. The communication device 4c can also wirelessly receive control commands from the control device of a pilot on the ground. The control device 4a may have the function of automatically or autonomously performing takeoff, flight, obstacle avoidance, and landing operations based on sensor data obtained from the sensor group 4b. The control device 4a may communicate with the work machine 200 connected to the power supply device 76 and be configured to obtain signals from the work machine 200 indicating the status of the work machine 200. The control device 4a may also provide signals to the work machine 200 to control its operation. Furthermore, the work machine 200 may generate signals instructing the operation of the multicopter 10 and transmit them to the control device 4a. Such communication between the control device 4a and the work machine 200 can be wired or wireless.

[0035] Figure 2B is a block diagram showing an example of the basic configuration of a series hybrid drive multicopter 10. Similar to a battery-powered multicopter 10, the series hybrid drive multicopter 10 comprises multiple rotors 12, multiple motors 14, multiple ESCs 16, a control device 4a, a sensor group 4b, and a communication device 4c. The illustrated series hybrid drive multicopter 10 further includes an internal combustion engine 7a, a fuel tank 7b for storing fuel for the internal combustion engine 7a, a power generator 8 driven by the internal combustion engine 7a to generate electricity, a power buffer 9 for temporarily storing the electricity generated by the power generator 8, and a power supply device 76 electrically connected to the power buffer 9. The power buffer 9 is, for example, a battery such as a secondary battery. The electricity generated by the power generator 8 is supplied to the motors 14 via the power buffer 9 and the ESCs 16. The electricity generated by the power generator 8 can also be supplied to the work machine 200 via the power supply device 76.

[0036] Figure 2C is a block diagram showing an example of the basic configuration of a parallel hybrid drive multicopter 10. Similar to a series hybrid drive multicopter 10, the parallel hybrid drive multicopter 10 includes a plurality of rotors 12, a plurality of motors 14 that drive each of the plurality of rotors 12, a plurality of ESCs 16, a control device 4a, a sensor group 4b, a communication device 4c, an internal combustion engine 7a, a fuel tank 7b, a power generator 8, a power buffer 9, and a power supply device 76. The parallel hybrid drive multicopter 10 further includes a drivetrain 27 that transmits the driving force of the internal combustion engine 7a, and a rotor 22 that rotates by receiving the driving force from the internal combustion engine 7a via the drivetrain 27. One of the rotors 12 and rotor 22 may be called the "first rotor" and the other the "second rotor" to distinguish them from each other. There may be one rotor 22 that is connected to the drivetrain 27 and rotates, or there may be two or more rotors 22.

[0037] In a parallel hybrid drive multicopter 10, the internal combustion engine 7a not only drives the power generator 8 to generate electricity, but also mechanically transmits energy to the rotor 22 to rotate it. On the other hand, in a series hybrid drive multicopter 10, all rotors 12 rotate using the electricity generated by the power generator 8. Therefore, in a series hybrid drive multicopter 10, if the power generator 8 is, for example, a fuel cell, the internal combustion engine 7a is not an essential component.

[0038] As described above, the configuration of the multicopter 10 is diverse. The multicopter 10 can, for example, spray pesticides, fertilizers, or seeds in a field, or suspend ground-level work equipment to perform tasks such as mowing grass. The multicopter 10 can also be used in conjunction with industrial machinery that performs ground-level work (e.g., agricultural machinery or construction machinery) to support ground-level work performed by industrial machinery. Agricultural machinery includes, for example, agricultural work vehicles such as tractors, combines, rice transplanters, and riding cultivators. Construction machinery includes, for example, work vehicles for construction and civil engineering work such as backhoes, wheel loaders, and carriers. Ground-level work is work performed on the ground and includes, for example, agricultural work such as tilling, sowing, pest control, fertilizing, planting crops, and harvesting, as well as construction and civil engineering work such as excavating the ground.

[0039] Figure 3 shows an example of a system including a multicopter 10 and an agricultural work vehicle 100. In this example, the work vehicle 100 is an agricultural tractor. The work vehicle 100 may be other agricultural machinery or construction machinery. Figure 3 also shows a server 300 that communicates with the multicopter 10 and the work vehicle 100. The server 300 may be a server computer on the cloud, for example, installed in a data center. The server 300 can communicate with the work vehicle 100 and the multicopter 10 via relay equipment (for example, multiple routers and switches in the network). Indirect communication via the server 300 and direct wireless communication are possible between the work vehicle 100 and the multicopter 10. Although Figure 3 illustrates one work vehicle 100 and one multicopter 10, the number of work vehicles 100 and multicopters 10 may be two or more.

[0040] Figure 4 is a block diagram showing an example configuration of the system shown in Figure 3. In the example in Figure 4, the multicopter 10 includes multiple rotors 12, multiple motors 14 that drive each of the multiple rotors 12, a battery 52 for storing power, a control device 4a for controlling the flight of the multicopter 10, a communication device 4c, and a sensor group 4b, similar to the example shown in Figure 2A. Note that the power supply device 76 and the work machine 200 shown in Figure 2A are omitted from Figure 4. In Figure 4, for simplicity, the rotors 12, motors 14, and ESCs 16 are each shown by one block, but the number of rotors 12, motors 14, and ESCs 16 is multiple. The multicopter 10 may also include at least one second rotor 22 driven by an internal combustion engine 7a, as shown in Figure 2B or Figure 2C. In that case, either a "series hybrid" or "parallel hybrid" drive configuration may be adopted.

[0041] Figure 4 shows an example of sensor group 4b, which includes a GNSS receiver 41, an IMU 42, an altitude sensor 43, an imaging device 44, and a LiDAR sensor 45. The GNSS receiver 41 and IMU 42 function as positioning devices that measure the position and attitude (pose) of the multicopter 10. The altitude sensor 43 measures the altitude of the multicopter 10 and outputs a signal indicating that altitude. Altitude refers to the vertical distance between the aircraft and a reference plane (e.g., the ground surface). The altitude sensor 46 can be implemented, for example, by a barometer, a distance measuring device that measures the distance from the aircraft to the ground, or a combination thereof. The imaging device 44 generates and outputs image data by photographing the area around the multicopter 10. The LiDAR sensor 45 is an example of a distance measuring device that measures the distance to objects present around the multicopter 10. The imaging device 44 and LiDAR sensor 45 are external sensors that sense the environment around the multicopter 10 and output sensor data.

[0042] The control device 4a controls the flight and communication operations of the multicopter 10. The communication device 4c is a communication module that communicates with external devices such as the work vehicle 100 and the server 300. The communication device 4c may be configured to perform wireless communication using, for example, Wi-Fi (Wireless Fidelity, registered trademark), BLE (Bluetooth Low Energy), LPWA (Low Power Wide Area), specified low-power radio, or cellular communication networks such as 4G or 5G. The communication device 4c can communicate directly with the communication device 110 in the work vehicle 100, or indirectly via the network 90 and the server 300.

[0043] The work vehicle 100 in the example in Figure 4 is equipped with a communication device 110, a control device 120, a GNSS receiver 130, an IMU 140, an imaging device 150, and a LiDAR sensor 160. The functions of these devices are similar to those of the corresponding devices in the multicopter 10. The work vehicle 100 is further equipped with a display device 170 and a drive system 180 including an engine and a running gear.

[0044] The communication device 110 can communicate directly with the communication device 4c of the multicopter 10, or indirectly via the network 90 and server 300. The control device 120 controls the operation of the work vehicle 100. The GNSS receiver 130 and IMU 140 function as positioning devices that measure the position and attitude of the work vehicle 100. The imaging device 150 and LiDAR sensor 160 function as external sensors that sense the environment around the work vehicle 100 and output sensor data. The display device 170 displays a map of the area in which the work vehicle 100 travels, as well as position information and speed information of the work vehicle 100 and the multicopter 10, respectively. The display device 170 may be an operating terminal installed on the work vehicle 100, or it may be a portable terminal used by the user of the work vehicle 100.

[0045] The server 300 includes a communication device 4c of the multicopter 10 and a communication device 110 of the work vehicle 100, a communication device 310 that communicates via the network 90, and a processing device 320 that performs processing based on information acquired from the multicopter 10 and the work vehicle 100.

[0046] <Communication System> In the example shown in Figure 4, the multicopter 10 is equipped with a communication system including a communication device 4c and a control device 4a. Similarly, the work vehicle 100 is equipped with a communication system including a communication device 110 and a control device 120. These communication systems enable the multicopter 10 and the work vehicle 100 to operate in conjunction with each other. Below, examples of the operation of the communication systems mounted on the multicopter 10 and the work vehicle 100 will be described.

[0047] The control device 4a in the multicopter 10 controls communication by the communication device 4c. The control device 4a changes the mode of communication with the work vehicle 100 according to the distance between the work vehicle 100 and the multicopter 10. Similarly, the control device 120 in the work vehicle 100 controls communication by the communication device 110. The control device 120 changes the mode of communication with the multicopter 10 according to the distance between the work vehicle 100 and the multicopter 10.

[0048] A change in the communication mode may include, for example, a change in the type of data transmitted, the frequency of communication, the amount of data transmitted, or the communication method. For example, if the distance between the work vehicle 100 and the multicopter 10 is greater than a threshold (i.e., they are far apart), they may share sensing information with each other. Sensing information can be acquired by external sensors such as imaging devices 44, 150 and LiDAR sensors 45, 160. Conversely, if the distance between the work vehicle 100 and the multicopter 10 is below a threshold (i.e., they are close together), they may share information necessary for collision avoidance (e.g., position information, attitude information, and / or altitude information). Position information can be acquired by positioning devices such as GNSS receivers 41, 130. Attitude information can be acquired by attitude detection sensors such as IMUs 42, 140. Altitude information can be acquired by an altitude sensor 43. Alternatively, if they are far apart, they may communicate indirectly via an external computer such as a server 300, and if they are close together, they may communicate directly via wireless communication.

[0049] The distance between the work vehicle 100 and the multicopter 10 can be calculated, for example, based on the position information output from positioning devices (e.g., GNSS receivers 130 and 41) mounted on the work vehicle 100 and the multicopter 10, respectively. That is, the control devices 120 and 4a may be configured to acquire position information of the work vehicle 100 from the positioning device mounted on the work vehicle 100, acquire position information of the multicopter 10 from the positioning device mounted on the multicopter 10, and calculate the distance between the work vehicle 100 and the multicopter 10 based on that position information. Alternatively, the control devices 120 and 4a may acquire information indicating the distance between the work vehicle 100 and the multicopter 10 from a distance measuring device (e.g., LiDAR sensor 160 or 45) mounted on the work vehicle 100 or the LiDAR sensor 45. Alternatively, the distance between the work vehicle 100 and the multicopter 10 may be measured using a beacon transmitter mounted on one of the work vehicle 100 and the multicopter 10, and a beacon receiver mounted on the other of the work vehicle 100 and the multicopter 10.

[0050] The following describes several examples of how to change the communication mode depending on the distance between the work vehicle 100 and the multicopter 10. In the following description, unless otherwise specified, the operating entity is the control device 4a in the multicopter 10. Each example of the communication method described below can also be performed by the control device 120 in the work vehicle 100. This allows information to be shared between the work vehicle 100 and the multicopter 10.

[0051] Figure 5 is a flowchart showing an example of a communication method performed by the control device 4a.

[0052] In step S101, the control device 4a acquires positional information for the work vehicle 100 and the multicopter 10 from their respective positioning devices. The positional information may include, for example, latitude and longitude information measured by the GNSS receiver 41.

[0053] In step S102, the control device 4a calculates the distance between the work vehicle 100 and the multicopter 10 based on the acquired position information. The control device 4a can calculate the distance between the two from the difference between the position of the work vehicle 100 and the position of the multicopter 10.

[0054] Alternatively, instead of performing steps S101 and S102, distance information between the work vehicle 100 and the multicopter 10 may be acquired using a rangefinder or beacon, as described above.

[0055] In step S103, the control device 4a compares the calculated distance with a threshold. The threshold is stored in advance in a memory or other storage device of the control device 4a. The threshold is a predetermined value, such as 5m, 10m, or 20m, and is determined according to the purpose or application of the system. If the distance is greater than the threshold, the process proceeds to step S104. If the distance is less than or equal to the threshold, the process proceeds to step S105.

[0056] In step S104, the control device 4a performs communication with the work vehicle 100 in a first communication mode. In the first communication mode, the control device 4a may be configured to transmit, for example, position information acquired from the positioning device of the multicopter 10 (e.g., GNSS receiver 41) to the work vehicle 100 via the communication device 4c. Alternatively, in the first communication mode, the control device 4a may transmit sensor data acquired from external sensors mounted on the multicopter 10 (e.g., imaging device 44 or LiDAR sensor 45) to the work vehicle 100 via the communication device 4c. In the first communication mode, the control device 4a may transmit both position information and sensor data to the work vehicle 100. By transmitting position information and / or sensor data to the work vehicle 100, the control device 120 of the work vehicle 100 can display information such as the position of the multicopter 10 or images acquired by sensing on the display device 170.

[0057] In step S105, the control device 4a performs communication with the work vehicle 100 in a second communication mode different from the first communication mode. In the second communication mode, the control device 4a may be configured to transmit to the work vehicle 100, via the communication device 4c, attitude information obtained from an attitude detection sensor (e.g., IMU 42) mounted on the multicopter 10 and / or altitude information obtained from an altitude sensor 43, in addition to position information and / or sensor data. By transmitting at least one of the attitude information and altitude information in addition to the position information to the work vehicle 100, the work vehicle 100 becomes more likely to detect the multicopter 10 as an obstacle, making it easier to avoid a collision between the work vehicle 100 and the multicopter 10.

[0058] The communication in the first communication mode in step S104 and the communication in the second communication mode in step S105 may each be performed repeatedly at predetermined time intervals. In this case, the communication in the second communication mode may be performed at shorter time intervals than the communication in the first communication mode. Furthermore, in the communication in the first communication mode in step S104, the control device 4a may be configured to indirectly transmit data to the work vehicle 100 via a server 300, which is an external computer on the cloud, using a public network such as 4G or 5G. This is because, when the distance between the work vehicle 100 and the multicopter 10 is long, there is no possibility of collision between the two, and therefore the speed and frequency of communication between them may be low. Conversely, in the communication in the second communication mode in step S105, the control device 4a may be configured to directly transmit data to the work vehicle 100 using wireless communication such as Wi-Fi®, Bluetooth, or low-power radio. When the distance between the work vehicle 100 and the multicopter 10 is short, it is desirable to increase the speed and frequency of communication between the two to exchange information such as position and attitude at short time intervals in order to avoid collisions.

[0059] In step S106, the control device 4a determines whether or not to terminate the operation. For example, if a pre-programmed flight is completed, or if a command to terminate the operation is received from an external device such as a pilot or remote monitoring device, the control device 4a terminates the operation. The control device 4a repeats the operations from steps S101 to S106 until it determines that it has terminated the operation.

[0060] The operation shown in Figure 5 can be performed not only by the control device 4a of the multicopter 10, but also by the control device 120 of the work vehicle 100. That is, if the distance between the work vehicle 100 and the multicopter 10 is greater than a threshold, the control device 120 of the work vehicle 100 may transmit position information acquired from a positioning device (e.g., GNSS receiver 130) mounted on the work vehicle 100 to the multicopter 10 via the communication device 110. The control device 120 may also transmit sensor data acquired from an external sensor (e.g., imaging device 150 or LiDAR sensor 160) mounted on the work vehicle 100 to the multicopter 10 via the communication device 110 if the distance is greater than a threshold. Conversely, if the distance is less than or equal to a threshold, the control device 120 may transmit attitude information acquired from an attitude detection sensor (e.g., IMU 140) mounted on the work vehicle 100 to the multicopter 100 via the communication device 4c, in addition to the position information. Furthermore, if the distance is greater than a threshold, the control device 120 may communicate with the multicopter 10 via the server 300, and if the distance is less than or equal to the threshold, it may communicate directly with the multicopter 10.

[0061] Next, a more specific example of the communication method according to this embodiment will be described with reference to Figures 6 and 7.

[0062] Figures 6 and 7 show an example of a communication method in a system in which a work vehicle 100 and a multicopter 10 operate in conjunction. Figure 6 shows a state where the distance between the work vehicle 100 and the multicopter 10 is greater than a threshold. Figure 7 shows a state where the distance between the work vehicle 100 and the multicopter 10 is less than or equal to a threshold.

[0063] In this example, the work vehicle 100 travels along a predetermined route in the field 70 while performing agricultural tasks such as spraying fertilizer, pesticides, or seeds, planting crop seedlings, harvesting crops, or mowing grass. The work vehicle 100 shown in Figure 6 is a tractor that performs agricultural tasks by driving implements connected to the work vehicle 100. The work vehicle 100 is not limited to a tractor; it may also be a transplanter such as a rice transplanter, or a harvester such as a combine harvester. Furthermore, the work vehicle 100 is not limited to agricultural machinery; it may also be construction machinery.

[0064] The travel path of the work vehicle 100 shown in Figure 6 is regularly meandering, as indicated by the thick arrows in the figure. The work vehicle 100 may be driven manually by the user or automatically. When the work vehicle 100 is driven automatically, map information of the field 70 and travel path information are recorded in advance in the storage device of the work vehicle 100. The control device 120 can drive the work vehicle 100 along a preset travel path based on the position information and attitude information of the work vehicle 100 output from the positioning device including the GNSS receiver 130 and IMU 140.

[0065] The multicopter 10 flies around the work vehicle 100 and assists the work vehicle 100 in ground operations. For example, if the work vehicle 100 is spraying fertilizer, pesticides, or seeds, or planting crop seedlings, the multicopter 10 may be configured to deliver agricultural materials (hereinafter sometimes referred to as "agricultural materials" or simply "materials") to the work vehicle 100 when the remaining amount of such materials is low. Alternatively, if the work vehicle 100 is a harvester that harvests crops, the multicopter 10 may be configured to receive, for example, the harvested crop stored in the work vehicle 100's tank and transport it to a designated location.

[0066] In the system described above, the work vehicle 100 may be equipped with sensors to measure the remaining amount of materials or the amount of harvested crops. The control device 120 may be configured to control the communication device 110 to send a request to summon the multicopter 10 based on the signals output from the sensors. The control device 4a of the multicopter 10 may be configured to fly the multicopter 10 to the vicinity of the work vehicle 100 in response to a request from the work vehicle 100.

[0067] While the multicopter 10 is flying, the control device 4a performs the operations shown in Figure 5. Specifically, the control device 4a determines whether the distance between the work vehicle 100 and the multicopter 10 is greater than a threshold. If the distance is greater than the threshold, communication is performed using the first communication mode. When the distance falls below the threshold, the communication mode is changed from the first communication mode to the second communication mode. Figure 6 shows an example of communication using the first communication mode, and Figure 7 shows an example of communication using the second communication mode. In the first communication mode shown in Figure 6, the control device 4a of the multicopter 10 transmits position information acquired by the GNSS receiver 41 and sensing information acquired by the imaging device 150 and LiDAR sensor 160 to the work vehicle 100. Communication between the multicopter 10 and the work vehicle 100 is performed indirectly via the server 300. In contrast, in the second communication mode shown in Figure 7, the control device 4a transmits position information acquired by the GNSS receiver 41 and attitude information acquired by the IMU 140 to the work vehicle 100. Furthermore, communication between the multicopter 10 and the work vehicle 100 is performed by direct wireless communication. As a result, when the multicopter 10 approaches the work vehicle 100, position information and attitude information are transmitted to the work vehicle 100 at a high frequency, making it easy for the work vehicle 100 to accurately determine the position and attitude of the multicopter 10 based on this information. This makes it possible for the work vehicle 100 to properly align for the transfer of materials or harvested goods without colliding with the multicopter 10.

[0068] In both the first and second communication modes, information may be transmitted not only from the multicopter 10 to the work vehicle 100, but also from the work vehicle 100 to the multicopter 10. For example, in the first communication mode, the work vehicle 100 and the multicopter 10 may share each other's position information and / or sensing information. In the second communication mode, the work vehicle 100 and the multicopter 10 may share each other's position information and / or attitude information. By transmitting this information from the work vehicle 100 to the multicopter 10, it becomes easier for the multicopter 10 to accurately determine the position and attitude of the work vehicle 100. This prevents the multicopter 10 from colliding with the work vehicle 100. thing This eliminates the need for proper positioning for the transfer of materials or harvested produce.

[0069] Thus, bidirectional communication may occur between the multicopter 10 and the work vehicle 100, or unidirectional communication may occur from one to the other.

[0070] Note that the changes in the information transmitted and the communication method in the first and second communication modes in the examples of Figures 6 and 7 are merely illustrative, and various modifications are possible. For example, the control device 4a of the multicopter 10 may transmit position information in the first communication mode, and in the second communication mode, transmit altitude information acquired by the altitude sensor 43 to the work vehicle 100 in addition to position information and attitude information. By adding altitude information, the work vehicle 100 can more accurately determine the position of the multicopter 10. Furthermore, although the control device 4a or the control device 120 transmits the same type of information in both the first and second communication modes, the communication frequency in the second communication mode may be higher than the communication frequency in the first communication mode. For example, in the first communication mode, information may be transmitted at a first time interval (e.g., 0.1 seconds or more and less than 1 second), and in the second communication mode, information may be transmitted at a shorter second time interval (e.g., 0.01 seconds or more and less than 0.1 seconds).

[0071] In the examples shown in Figures 6 and 7, the multicopter 10 assists the work of the work vehicle 100, but the relationship can also be reversed. For example, the multicopter 10 may drive a work machine 200 as shown in Figure 1C to perform ground work such as spraying agricultural materials such as pesticides, fertilizers, and seeds, harvesting crops, or mowing grass, while the work vehicle 100 assists by transporting agricultural materials, harvested crops, or mowed grass. In this case as well, the aforementioned communication method can be applied similarly.

[0072] In the above example, the multicopter 10 receives position information of the work vehicle 100 measured by the positioning device of the work vehicle 100, and the work vehicle 100 acquires position information of the multicopter 10 measured by the positioning device of the multicopter 10. The position information of the communication partners of the multicopter 10 and the work vehicle 100 is not limited to being obtained from the communication partner. For example, the multicopter 10 and the work vehicle 100 may each estimate the position of their communication partner based on data output from their own imaging device and / or sensing device such as a LiDAR sensor. For example, they may recognize their communication partner and determine its position based on image data output from an imaging device such as a stereo camera and / or distance data or point cloud data output from a laser sensor such as a LiDAR sensor.

[0073] As described above, the communication method of this embodiment allows for the sharing of necessary information between the work vehicle 100 and the multicopter 10 with high accuracy and in real time. This makes it possible for the work vehicle 100 and the multicopter 10 to approach each other without collision and to perform coordinated work such as supplying materials or handing over harvested goods. For example, it is possible for the work vehicle 100 and the multicopter 10 to perform coordinated work while overlapping without contact in a plan view. Such coordinated work is also called "cooperative work".

[0074] In the example above, the communication mode is changed based on the comparison result between the distance between the work vehicle 100 and the multicopter 10 and one threshold, but the communication mode may also be changed in multiple stages based on the comparison result with two or more thresholds.

[0075] Figure 8 is a flowchart illustrating an example in which the communication mode changes in three stages depending on the distance between the work vehicle 100 and the multicopter 10. The operation shown in Figure 8 can be performed by either or both the control device 4a of the multicopter 10 and the control device 120 of the work vehicle 100. Hereinafter, the explanation will assume that the control device 4a of the multicopter 10 performs the operation shown in Figure 8. The operations in steps S201, S202, and S208 in this example are the same as the operations in steps S101, S102, and S106, respectively, and therefore will not be explained.

[0076] In step S203, the control device 4a determines whether the distance between the work vehicle 100 and the multicopter 10 is greater than a first threshold. If the distance is greater than the first threshold, the process proceeds to step S205, where the control device 4a communicates in first communication mode. After step S205, the process proceeds to step S208. If the distance is less than or equal to the first threshold, the process proceeds to step S204.

[0077] In step S204, the control device 4a determines whether the distance between the work vehicle 100 and the multicopter 10 is greater than a second threshold which is less than the first threshold. If the distance is greater than the second threshold, the process proceeds to step S206, where the control device 4a communicates in second communication mode. After step S206, the process proceeds to step S208. If the distance is less than or equal to the second threshold, the process proceeds to step S207, where the control device 4a communicates in third communication mode. After step S207, the process proceeds to step S208.

[0078] As shown in Figure 8, the communication mode can be changed in three stages depending on the distance between the work vehicle 100 and the multicopter 10. The first communication mode may be a mode in which position information is transmitted by indirect communication. The second communication mode may be a mode in which position information and attitude information are transmitted by direct communication. The third communication mode may be a mode in which position information, attitude information, and altitude information are transmitted by direct communication. Sensing information may also be transmitted in each mode. In this way, by transmitting more types of information or increasing the communication speed or transmission frequency as the distance between the work vehicle 100 and the multicopter 10 decreases, information communication for cooperation between the work vehicle 100 and the multicopter 10 can be performed more appropriately. The control devices 4a and 120 may change the communication mode in four or more stages depending on the distance between the work vehicle 100 and the multicopter 10.

[0079] The methods of information and communication described in this disclosure are not limited to the examples above. Other communication methods are described below as examples.

[0080] If the distance between the multicopter 10 and the work vehicle 100 is greater than a threshold, the control devices 4a and 120 may share position information measured by their respective positioning devices, and if the distance falls below the threshold, they may share more detailed relative position information obtained using an imaging device or beacon, etc.

[0081] The control device 4a of the multicopter 10 may switch the sensors, such as LiDAR, used for sensing depending on the relative position to the work vehicle 100. For example, when the multicopter 10 is flying at a height higher than the work vehicle 100, it may transmit sensor data acquired by a sensor sensing below the multicopter 10, and when the multicopter 10 is flying at a height lower than the work vehicle 100, it may transmit sensor data acquired by a sensor sensing to the side of the multicopter 10.

[0082] In addition to the distance between the work vehicle 100 and the multicopter 10, the communication method may be changed according to the relative speed between the two. For example, if the relative speed is below a threshold, communication may be performed via a public line and server 300, and if the relative speed exceeds a threshold, direct communication may be performed between the work vehicle 100 and the multicopter 10.

[0083] <Information Display System> In this embodiment, the control device 120 also functions as a display system that displays the positions of the work vehicle 100 and one or more multicopters 10 flying around the work vehicle 100 on the display device 170. The display system is capable of displaying the relative positions of the work vehicle 100, which is capable of performing ground work while moving, and the flying multicopters 10 on the display device 170.

[0084] The control device 120 includes a processing device that acquires positional information of the work vehicle 100 and the unmanned aerial vehicle 10, and displays the respective positions of the work vehicle 100 and the unmanned aerial vehicle 10 in a field (i.e., display area) shown on the display device 170 based on said positional information. The display device 170 may be included in the display system or may be an external element of the display system.

[0085] Figure 9 is a block diagram showing an example of the hardware configuration of the control device 120. The control device 120 comprises a processor 34, a ROM (Read Only Memory) 35, a RAM (Random Access Memory) 36, a storage device 37, and a communication interface 38. These components are interconnected via a bus 39.

[0086] The processor 34 is one or more semiconductor integrated circuits, also known as a central processing unit (CPU) or microprocessor. The processor performs various processes by sequentially executing computer programs stored in the ROM 35. The term "processor" is broadly interpreted to include FPGAs (Field Programmable Gate Arrays), GPUs (Graphic Processor Units), ASICs (Application Specific Integrated Circuits), or ASSPs (Application Specific Standard Products) equipped with a CPU.

[0087] ROM35 can be, for example, writable memory (e.g., PROM), rewritable memory (e.g., flash memory), or read-only memory. ROM35 stores programs that control the operation of the processor. ROM35 does not need to be a single recording medium; it can be a collection of multiple recording media. Some of these collections may be removable memory.

[0088] RAM36 provides a workspace for temporarily unpacking programs stored in ROM35 during boot-up. RAM36 does not need to be a single storage medium; it can be a collection of multiple storage mediums.

[0089] The communication interface 38 is an interface for communication between the control device 120 and other electronic components or electronic control units (ECUs). For example, the communication interface 38 can perform wired communication compliant with various protocols. The communication interface 38 can also perform wireless communication compliant with the Bluetooth® standard and / or the Wi-Fi® standard. Both standards include wireless communication standards that utilize the 2.4GHz frequency band.

[0090] The storage device 37 may be, for example, a semiconductor memory, a magnetic storage device, or an optical storage device, or a combination thereof. The storage device 37 can store, for example, map data (map information) useful for the autonomous driving of the work vehicle 100, and various sensor data acquired by the work vehicle 100 while it is in motion.

[0091] The control device 4a of the multicopter 10 may also have a hardware configuration similar to that shown in Figure 9. As mentioned above, the control device 4a may include, for example, a flight control device such as a flight controller and a higher-level computer (companion computer). The companion computer may perform the aforementioned processes and provide flight commands to the flight controller based on the results of those processes.

[0092] In this embodiment, the display device 170 is mounted on the work vehicle 100. The display device 170 may also be built into or connected to a computer used by a user who monitors the operation of the work vehicle 100 at a location away from the work vehicle 100. The computer used by the user may be a mobile device such as a smartphone or tablet computer, or a stationary computer such as a personal computer (PC) or workstation.

[0093] Figure 10 is a flowchart showing an example of processing performed by the processor 34 (processing unit) in the control device 120. In this example, the processor 34 acquires map information of the area where the work vehicle 100 is located, generates a map image on which the positions of the work vehicle 100 and the unmanned aerial vehicle 10 are superimposed, and displays the map image in the field of the display device 170.

[0094] In step S301, the processor 34 acquires map information of the area in which the work vehicle 100 travels. The map information is pre-stored in the storage device 37 of the control device 120. The map information may include, for example, location information (e.g., latitude and longitude) of the field in which the work vehicle 100 travels and its surrounding features. The processor 34 acquires the map information from the storage device 37. The processor 34 may also acquire the map information from an external device such as a server 300.

[0095] In step S302, the processor 34 acquires the position information of the work vehicle 100 and the multicopter 10. The processor 34 acquires the position information of the work vehicle 100 output from a positioning device such as a GNSS receiver 130 on the work vehicle 100. The processor 34 acquires the position information of the multicopter 10 output from a positioning device such as a GNSS receiver 41 on the multicopter 10 via the communication devices 4c and 110.

[0096] In step S303, the processor 34 generates a map image in which the respective positions of the work vehicle 100 and the multicopter 10 are superimposed on a map based on the location information and map information. The map image may be, for example, an image in which icons indicating the respective positions of the work vehicle 100 and the multicopter 10 are superimposed on a map of the field. In addition to the location information, the processor 34 may also acquire the respective speed information of the work vehicle 100 and the multicopter 10 and generate a map image that further indicates the respective directions of movement of the work vehicle 100 and the multicopter 10 based on the speed information.

[0097] In step S304, the processor 34 displays the generated map image on the display device 170. The processor 34 may also repeatedly perform the operations shown in steps S301 to S304 to display a moving image on the display device 170 showing the changes in the positions of the work vehicle 100 and the multicopter 10, respectively.

[0098] Figure 11 shows an example of a map image displayed on the display device 170. In this example, two multicopters 10 (drone #1 and drone #2) are flying around a tractor, which is a work vehicle 100. The processor 34 in the work vehicle 100 sequentially receives position information and speed information from each of the two multicopters 10 and displays a map image as shown in Figure 11 on the display device 170. In this example, two multicopters 10 are flying around the work vehicle 100, but one or more multicopters 10 may be flying around the work vehicle 100.

[0099] Communication control may be performed between the work vehicle 100 and each multicopter 10 according to the distance described above, or communication may be performed regardless of distance to share necessary information such as identifiers (IDs) and location information. Information about multicopters 10 that do not perform cooperative work with the work vehicle 100, as well as information about multicopters 10 that do not perform cooperative work with the work vehicle 100, may be transmitted to and displayed on the work vehicle 100. Communication between the work vehicle 100 and each multicopter 10 may be performed by direct wireless communication, or it may be performed via a server 300 on the cloud.

[0100] In the example shown in Figure 11, the processor 34 generates a map image showing the position and direction of movement of each multicopter 10 centered on the position of the work vehicle 100, and displays it on the display device 170. More specifically, the processor 34 generates a map image including icons for the work vehicle 100 and the multicopter 10, and arrows indicating the direction of movement of the work vehicle 100 and the multicopter 10, and displays it on the display device 170. The processor 34 may also acquire flight path information from each multicopter 10, generate a map image including future flight path information, and display it on the display device 170. The processor 34 sequentially acquires position information and speed information for the work vehicle 100 and the multicopter 10, and sequentially updates the display of the position and direction of movement of the work vehicle 100 and the multicopter 10 in the map image. The information transmitted from each multicopter 10 to the work vehicle 100 includes the ID of that multicopter 10. The work vehicle 100 can identify information from individual multicopters 10 based on its ID.

[0101] In the example shown in Figure 11, the processor 34 acquires operational information indicating the operating status of each multicopter 10, and based on this operational information, displays the operating status of each multicopter 10 along with a map image on the display device 170. On the right side of Figure 11, an example of the operating status of two multicopters 10 (drone #1 and drone #2) is shown. The operating status may include, for example, the type of work being performed by the multicopter 10 (spraying, transporting, mowing, etc.), whether the multicopter 10 is flying automatically (unmanned), the remaining flight time (or energy level) of the multicopter 10, and at least one of the remaining energy level of the multicopter 10. In the example in Figure 11, the operational information for each multicopter 10 includes the distance from the work vehicle 100, altitude, type of work, driving mode (automatic or manual), and remaining flight time. In the example shown in Figure 11, the model name (manufacturer and model number, etc.) identified from the ID of each multicopter 10 is also displayed.

[0102] In the example in Figure 11, a dashed circle is shown indicating a range of a predetermined distance from the position of the work vehicle 100. The processor 34 may output a warning (alert) to the display device 170 if any of the multicopters 10 enter within a range of less than or equal to the predetermined distance from the position of the work vehicle 100. In the example in Figure 11, one multicopter 10 (drone #2) is approaching the work vehicle 100 and has entered within a range of less than or equal to the predetermined distance. At this time, the processor 34 displays the message "A drone is approaching." as an alert on the display device 170. If the multicopter 10 is intentionally approaching the work vehicle 100 for coordinated work with the work vehicle 100, the purpose of that work may be displayed. As shown in Figure 11, the processor 34 can display information on the display device 170 indicating the purpose of the multicopter 10's approach (for example, automatic replenishment of chemicals).

[0103] The alert is not limited to being displayed on the display device 170, but may also be expressed by, for example, a warning sound to a speaker, light from a light source, or vibration. The alert may change depending on the distance between the work vehicle 100 and the multicopter 10. For example, the processor 34 may change the interval at which alerts are generated by warning sound, vibration, or light depending on the distance between the work vehicle 100 and the multicopter 10. As an example, when the distance between the multicopter 10 and the work vehicle 100 falls below a first threshold as the multicopter 10 approaches, a long-interval intermittent sound such as "beep, ..., beep, ..." may be output, and when the distance falls below a second threshold which is smaller than the first threshold, a short-interval intermittent sound such as "beep, beep, beep" may be output as an alert. The distance between the work vehicle 100 and the multicopter 10 may be the distance in three-dimensional space or the distance in a planar view.

[0104] In the examples shown in Figures 10 and 11, the processor 34 causes the display device 170 to display a map image on which the positions of the work vehicle 100 and each unmanned aerial vehicle 10 are superimposed on the map, but the display is not limited to this. For example, the positions of the work vehicle 100 and each multicopter 10 may be displayed in other forms, such as radar display.

[0105] As described above, the display system in this embodiment displays an image on the display device 170 showing the relative positional relationship between the ground-operated work vehicle 100 and one or more multicopters 10 located around it (for example, within a certain distance). This makes it possible for the user of the work vehicle 100 (for example, the driver or supervisor) to grasp information such as the position and speed of the multicopters 10 present around the work vehicle 100. If the work vehicle 100 is, for example, a tractor equipped with a cabin, and the user is riding in the tractor, it is generally difficult for the user to accurately grasp the position of the surrounding multicopters 10. By adopting the display system of this embodiment, it becomes easy for the user to immediately grasp the position of each surrounding multicopter 10.

[0106] In the example above, information such as the positions of one or more multicopters 10 around the work vehicle 100 is displayed, but a similar display may be centered on the position of a specific multicopter 10. In that case, an image including the position information of one or more multicopters 10 and one or more work vehicles 100 that are present around the specific multicopter 10 may be displayed on the display device 170.

[0107] In this embodiment, the processor 34 (i.e., processing unit) included in the control device 120 in the work vehicle 100 generates a map image containing information such as the respective positions of the work vehicle 100 and the multicopter 10, but other processing units may perform this processing. For example, the processing unit 320 in the server 300 may receive the necessary information from the work vehicle 100 and the multicopter 10, generate a map image based on that information, and transmit it to the display device 170. Alternatively, the processor included in the display device 170 may generate and display the map image. The display is not limited to the display device 170 in the work vehicle 100, but may also be displayed on other display devices, such as a display terminal for remotely monitoring the work vehicle 100 or the multicopter 10.

[0108] The work vehicles in the above embodiments are not limited to agricultural machinery such as tractors, but may also be construction machinery. For example, the communication method and display method of this disclosure may be applied to a system including one or more construction and civil engineering work vehicles such as backhoes, wheel loaders, and carriers, and one or more unmanned aerial vehicles.

[0109] This specification discloses solutions described in the following sections.

[0110] [Item 1] A display system that displays the positions of a work vehicle and one or more unmanned aerial vehicles flying around the work vehicle on a display device, A processing device that acquires the location information of the work vehicle and the unmanned aerial vehicle, and displays the respective locations of the work vehicle and the unmanned aerial vehicle in the fields shown on the display device based on the location information. A display system equipped with the following features.

[0111] [Item 2] The display system according to item 1, wherein the processing device further acquires speed information of the work vehicle and the unmanned aerial vehicle, and displays the respective directions of movement of the work vehicle and the unmanned aerial vehicle in the field based on the speed information, along with the respective positions of the work vehicle and the unmanned aerial vehicle.

[0112] [Item 3] The display system described in item 2, wherein the processing device displays the respective positions and directions of movement of the unmanned aerial vehicles in the field, with the position of the work vehicle as the center.

[0113] [Item 4] The display system according to item 2 or 3, wherein the processing device displays icons for the work vehicle and the unmanned aerial vehicle, and arrows indicating the respective directions of movement of the work vehicle and the unmanned aerial vehicle, in the field.

[0114] [Item 5] The display system according to any one of items 2 to 4, wherein the processing device sequentially acquires the position information and speed information of the work vehicle and the unmanned aerial vehicle, and sequentially updates the display of the position and direction of movement of the work vehicle and the unmanned aerial vehicle in the field.

[0115] [Item 6] The display system according to any one of items 1 to 5, wherein the processing device further acquires operational information indicating the operational status of each of the unmanned aerial vehicles, and displays the operational status of each of the unmanned aerial vehicles together with the field on the display device based on the operational information.

[0116] [Item 7] The display system according to item 6, wherein the operating status includes at least one of the work being performed by the unmanned aerial vehicle, whether or not the unmanned aerial vehicle is flying autonomously, the remaining flight time of the unmanned aerial vehicle, and the remaining energy level of the unmanned aerial vehicle.

[0117] [Item 8] The display system according to any one of items 1 to 7, wherein the processing device outputs an alert to the display device when any unmanned aerial vehicle enters within a predetermined distance or less from the position of the work vehicle.

[0118] [Item 9] The display system according to any one of items 1 to 8, wherein the processing device acquires map information of the area where the work vehicle is located, generates a map image on which the positions of the work vehicle and the unmanned aerial vehicle are superimposed on the map, and displays the map image in the field.

[0119] [Item 10] The display system according to any one of items 1 to 9, further comprising the aforementioned display device.

[0120] [Item 11] A work vehicle equipped with a display system as described in any one of items 1 through 10.

[0121] [Item 12] A display method in a system including a work vehicle and one or more unmanned aerial vehicles flying around the work vehicle, To acquire the location information of the aforementioned work vehicle and the aforementioned unmanned aerial vehicle, Based on the aforementioned location information, the respective positions of the work vehicle and the unmanned aerial vehicle are displayed in the fields shown on the display device. Display methods that include this. [Industrial applicability]

[0122] The unmanned aerial vehicles of this disclosure can be widely used not only for aerial photography, surveying, logistics, and pesticide spraying, but also for ground work related to agricultural operations, and for transporting harvested crops and agricultural materials. [Explanation of Symbols]

[0123] 2...Rotor (propeller), 3...Rotary drive unit, 4...Aircraft body, 4a...Control device, 4b...Sensor group, 4c...Communication device, 5...Aircraft frame, 10...Multicopter, 12...Rotor, 14...Motor, 16...ESC, 22...Rotor, 41...GNSS receiver, 41...GNSS receiver, 42...IMU, 43...Altitude sensor, 44...Imaging device, 45...LiDAR sensor, 52...Battery, 100...Work vehicle, 110...Communication device, 120...Control device, 130...GNSS receiver, 140...IMU, 150...Imaging device, 160...LiDAR sensor, 170...Display device, 180...Drive device, 300...Server, 310...Communication device, 320...Processing device

Claims

1. A display system that displays the positions of a work vehicle and one or more unmanned aerial vehicles flying around the work vehicle on a display device, The processing device acquires the respective location information of the work vehicle and the unmanned aerial vehicle, and based on the location information, displays icons indicating the respective locations of the work vehicle and the unmanned aerial vehicle, and arrows indicating the respective directions of movement of the unmanned aerial vehicle, in a field shown on the display device. The aforementioned processing apparatus is Further operational information indicating the operating status of each of the aforementioned unmanned aerial vehicles is acquired, and based on the operational information, the operating status of each of the aforementioned unmanned aerial vehicles is displayed on the display device along with the field. The aforementioned operating status includes the content of the work being performed by the unmanned aerial vehicle. Display system.

2. The display system according to claim 1, wherein the processing device further acquires speed information of the work vehicle and the unmanned aerial vehicle, and displays arrows indicating the respective directions of movement of the unmanned aerial vehicle in the field based on the speed information, together with icons indicating the respective positions of the work vehicle and the unmanned aerial vehicle.

3. The display system according to claim 2, wherein the processing device displays the respective positions and directions of movement of the unmanned aerial vehicles in the field, with the position of the work vehicle as the center.

4. The display system according to claim 2, wherein the processing device displays icons for the work vehicle and the unmanned aerial vehicle, and arrows indicating the respective directions of movement of the work vehicle and the unmanned aerial vehicle, in the field.

5. The display system according to claim 2, wherein the processing device sequentially acquires the position information and speed information of the work vehicle and the unmanned aerial vehicle, respectively, and sequentially updates the display of icons indicating the position of the work vehicle and the unmanned aerial vehicle and arrows indicating the direction of movement in the field.

6. The one or more unmanned aircraft is a plurality of unmanned aircraft, The aforementioned processing apparatus is An icon indicating the position of each of the multiple unmanned aerial vehicles and an arrow indicating the direction of movement of each of the multiple unmanned aerial vehicles are displayed in the field. The display system according to claim 1, further comprising: acquiring operational information indicating the operational status of each of the plurality of unmanned aerial vehicles; and displaying the operational status, including the content of the work being performed by each of the plurality of unmanned aerial vehicles, on the display device together with the field, based on the operational information.

7. The display system according to claim 1, wherein the operating status further includes at least one of whether the unmanned aerial vehicle is flying autonomously, the remaining flight time of the unmanned aerial vehicle, and the remaining energy of the unmanned aerial vehicle.

8. The processing device outputs an alert to the display device when any unmanned aerial vehicle enters within a predetermined distance or less from the position of the work vehicle. The display system according to claim 1, wherein the display device displays a message indicating that the unmanned aerial vehicle is approaching and the purpose of the approach of the unmanned aerial vehicle.

9. The display system according to claim 1, wherein the processing device acquires map information of the area where the work vehicle is located, generates a map image on which the positions of the work vehicle and the unmanned aerial vehicle are superimposed on the map, and displays the map image on the field.

10. The display system according to claim 1, further comprising the aforementioned display device.

11. A work vehicle equipped with the display system according to any one of claims 1 to 10.

12. A display method in a system including a work vehicle and one or more unmanned aerial vehicles flying around the work vehicle, To acquire the location information of the aforementioned work vehicle and the aforementioned unmanned aerial vehicle, Based on the aforementioned location information, icons indicating the respective positions of the work vehicle and the unmanned aerial vehicle, and arrows indicating the respective directions of movement of the unmanned aerial vehicle, are displayed in a field shown on the display device. To acquire operational information indicating the operating status of each of the aforementioned unmanned aerial vehicles, Based on the operational information, the operational status of each of the unmanned aerial vehicles is displayed on the display device along with the field, Includes, The aforementioned operating status includes the content of the work being performed by the unmanned aerial vehicle. Display method.

Images