Management System

Abstract

To provide a management system capable of easily grasping an actual turning point of a work vehicle with respect to an actual travel route.SOLUTION: A management system according to an embodiment comprises a work vehicle equipped with a first positioning device and configured to autonomously travel on a preset travel route while measuring positions with the first positioning device, a flying object equipped with a second positioning device and a camera capable of changing the shooting direction according to a result of positioning from the second positioning device, and a terminal device configured to display on a display unit a composite image obtained by merging a virtual route representing a virtual travel route with a camera image on the basis of position information on the flying object based on the second positioning device and the shooting direction of the camera. The terminal device generates a composite image including a turning point displayed on the travel route.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a management system. [Background technology]

[0002] A management system for a work vehicle is known which compares positioning information from a first positioning device mounted on the work vehicle and a second positioning device mounted on the unmanned aerial vehicle, adjusts the shooting direction of the camera mounted on the unmanned aerial vehicle, and simultaneously displays vehicle information including vehicle speed information received from the work vehicle and camera footage received from the unmanned aerial vehicle on a terminal device (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-176418 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, with the conventional technology described above, it was not easy to confirm, even by looking at camera footage, when a work vehicle traveling by automatic driving would turn along its planned travel route.

[0005] The present invention has been made in consideration of the above, and has an object to provide a management system that can easily grasp the actual turning position of a work vehicle relative to the actual travel route. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems and achieve the objective, the management system of the embodiment comprises a work vehicle equipped with a first positioning device and autonomously traveling along a predetermined driving route while being positioned by the first positioning device, an aircraft equipped with a second positioning device and a camera capable of changing the shooting direction based on the positioning result of the second positioning device, and a terminal device that displays on a display unit a composite image that combines a virtual route, which is a virtual driving route, with an image from the camera based on position information of the aircraft based on the second positioning device and the shooting direction of the camera, and the terminal device is characterized in that it generates the composite image including an indication of a turning position on the driving route. Effect of the Invention

[0007] According to the management system of the embodiment, the actual turning position of the work vehicle with respect to the actual travel route can be easily grasped. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a schematic left side view showing a work vehicle according to an embodiment. [Diagram 2] FIG. 2 is a perspective view showing an example of the appearance of the aircraft. [Diagram 3] FIG. 3 is a block diagram showing a management system according to an embodiment. [Figure 4] FIG. 4 is a diagram showing an example of a composite image displayed on a display unit of a mobile terminal device. [Diagram 5] FIG. 5 is a flowchart showing a processing procedure of the management system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of a management system disclosed in the present application will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to the embodiment described below.

[0010] <Overview of Work Vehicles (Tractors)> First, an overview of a work vehicle 1 according to an embodiment will be described with reference to Fig. 1. Fig. 1 is a schematic left side view showing a work vehicle 1 according to an embodiment. In the following, a tractor will be used as an example of the work vehicle 1. The tractor 1, which is a work vehicle, is an agricultural tractor that performs agricultural work in a field while traveling by itself.

[0011] In addition, the tractor 1, which is a work vehicle, is operated by a driver (also called an operator) while traveling within a field to perform specified tasks, and also performs specified tasks while traveling autonomously within the field through control of each part by a control system centered on the control unit 200 (see Figure 3), which will be described later.

[0012] In the following description, the forward / rearward direction is the direction of travel of the tractor 1 when traveling straight, with the forward side of the traveling direction being defined as "front" and the rear side being defined as "rear." The traveling direction of the tractor 1 is the direction from the operator's seat 8 to the steering wheel 9, which will be described later, when the tractor 1 is traveling straight.

[0013] The left-right direction is a direction that is horizontally perpendicular to the front-rear direction. In the following, left and right are defined with respect to the "front" side. That is, when the operator is seated in the operator's seat 8 and facing forward, the left hand side is the "left" and the right hand side is the "right". The up-down direction is the vertical direction. The front-rear direction, the left-right direction, and the up-down direction are perpendicular to each other in three dimensions. In the following description, the tractor 1 or the traveling vehicle body 2 may be referred to as the "machine body".

[0014] As shown in FIG. 1, the tractor 1 includes a traveling body 2 and a work implement 6. The traveling body 2 is capable of traveling in a field and includes front wheels 3 and rear wheels 4. The front wheels 3 are a pair of left and right wheels for steering (steered wheels). The rear wheels 4 are a pair of left and right wheels for driving (drive wheels). The traveling body 2 may include a crawler device instead of the wheels (at least one of the front wheels 3 and the rear wheels 4). In this case, the traveling crawlers are the drive wheels.

[0015] Rotational power generated by an engine E, which is a drive source housed inside a bonnet 5, is transmitted to rear wheels 4, which are drive wheels, after being appropriately reduced in speed by a speed change device (transmission) 121 (see FIG. 3) provided in a power transmission device (mission case) 12. The rear wheels 4 are driven by the rotational power transmitted from the engine E. The speed change device 121 switches the rotational power transmitted from the engine E to one of a plurality of gears (for example, 1st to 8th gears).

[0016] The traveling vehicle body 2 is configured to be able to transmit power generated by the engine E and reduced in speed by the transmission 121 to the front wheels 3 via the 4WD clutch as well. In this case, when the 4WD clutch transmits power, the four wheels, the front wheels 3 and the rear wheels 4, are driven by the power transmitted from the engine E. Also, when the 4WD clutch blocks the transmission of power, only the two wheels, the rear wheels 4, are driven by the power transmitted from the engine E. In this way, the traveling vehicle body 2 is configured to be able to switch between two-wheel drive (2WD) and four-wheel drive (4WD).

[0017] A work implement 6 that performs work in a farm field is connected to the rear of the traveling body 2, and a PTO device 7 having a PTO (Power take-off) shaft 71 that transmits power to drive the work implement 6 is provided. A driver's seat 8 where a driver sits when operating the tractor 1 is provided in the center of the traveling body 2.

[0018] A steering wheel 9, which is a handle for steering the front wheels 3, is provided in front of the cockpit 8. The steering wheel 9 and a drive section for driving the steering wheel 9 constitute a steering device 122 (see FIG. 3). The steering wheel 9 is provided at the upper end of a handle post 10. Various operating pedals 11 (accelerator pedal, brake pedal, clutch pedal) are provided below the handle post 10, near the feet of the pilot when he or she sits in the cockpit 8.

[0019] In addition, a lifting device 13 that raises and lowers the work implement 6 is provided at the rear of the traveling body 2. The lifting device 13 moves the work implement 6 to a non-working position by raising the work implement 6. In addition, the lifting device 13 moves the work implement 6 to a ground work position by lowering the work implement 6. The lifting device 13 includes a hydraulic lifting cylinder 131, a lift arm 132, a lift rod 133, a lower link 134, and a top link 135.

[0020] When hydraulic oil is supplied to the lift cylinder 131, the lift arm 132 rotates about the axis AX to raise the work implement 6, and when hydraulic oil is discharged from the lift cylinder 131, the lift arm 132 rotates about the axis AX to lower the work implement 6. A lift arm sensor that detects the rotation angle of the lift arm 132 is provided at the base of the lift arm 132 (near the axis AX). The height of the work implement 6 is calculated based on the detection value of the lift arm sensor.

[0021] Furthermore, the lift arm 132 is connected to the lower link 134 via the lift rod 133. In this manner, the lifting device 13 connects the work machine 6 to the traveling body 2 by the lower link 134 and the top link 135 so that the work machine 6 can be raised and lowered.

[0022] 1 illustrates an example in which the working machine 6 is a rotary tiller. The rotary tiller tills the field (soil) by rotating the tiller tines 61 using power transmitted from a PTO shaft 71 of a PTO device 7.

[0023] The tractor 1 also includes a control unit 200 (see FIG. 3). The control unit 200 controls the engine E and also controls the traveling speed of the traveling vehicle body 2. The control unit 200 also controls the work implement 6.

[0024] The tractor 1 also includes a positioning device 150. The positioning device 150 is a first positioning device that is provided on the upper part of the traveling body 2, and measures the position of the traveling body 2 at a predetermined period and acquires information (e.g., latitude and longitude) of the self-position of the traveling body 2. The positioning device 150 is, for example, a GNSS (Global Navigation Satellite System), and can receive radio waves from a navigation satellite S orbiting in the sky to perform positioning and timing.

[0025] Furthermore, the tractor 1 allows the operator to set various tasks in a specific field by operating the mobile terminal device 160. The mobile terminal device 160 is, for example, a tablet terminal, and can be connected to a communication network such as the Internet, and can be connected to the work management device via the communication network. In this case, the work management device is a system capable of so-called cloud computing. The mobile terminal device 160 and the work management device are connected, for example, via a wireless LAN (Local Area Network).

[0026] The mobile terminal device 160 includes a storage unit including a hard disk, a ROM (Read Only Memory), a RAM (Random Access Memory), and the like, and a display unit and an operation unit including a touch panel. Note that various keys and buttons may be provided separately as the operation unit. The mobile terminal device 160 may also include a processing unit having a CPU (Central Processing Unit) and the like so as to be able to control each unit by electronic control, similar to the control unit 200 described later.

[0027] The task management device is a processing device having a CPU and the like, a storage device such as a ROM (Read Only Memory), a RAM (Random Access Memory), and a HDD (Hard Disk Drive), and further a computer provided with an input / output device.

[0028] The tractor 1 also includes an azimuth angle acquisition means 170 (see FIG. 2). The azimuth angle acquisition means 170 acquires the azimuth angle of the traveling vehicle body. The azimuth angle acquisition means 170 is, for example, an azimuth angle sensor. Hereinafter, the azimuth angle acquisition means 170 will be referred to as an azimuth angle sensor.

[0029] The azimuth angle sensor 170 detects, for example, the absolute azimuth angle of the traveling direction of the traveling vehicle body 2 (for example, "north" is 0° (360°), "east" is 90°, "south" is 180°, and "west" is 270°). The azimuth angle sensor 170 detects the absolute azimuth angle at regular time intervals and transmits the detected absolute azimuth angle to the control unit 200 or the like. Note that the azimuth angle acquisition means 170 may be, in addition to an azimuth angle sensor, for example, a geomagnetic sensor or the like.

[0030] The tractor 1 is also connected to the flying object 300 so as to be able to communicate with it. The flying object 300 is also connected to the mobile terminal device 160 so as to be able to communicate with it. The flying object 300 is, for example, an unmanned flying object such as a drone, and is controlled to follow the tractor 1. The flying object 300 includes a camera 310 that captures an image of the surroundings of the flying object 300. The flying object 300 includes a positioning device 330 (see FIG. 3). The positioning device 330 is a second positioning device that is provided in the flying object 300, measures the position of the flying object 300 at a predetermined period, and acquires information on the self-position of the flying object 300 (for example, latitude and longitude). The positioning device 330 is, for example, a Global Navigation Satellite System (GNSS), and can receive radio waves from a navigation satellite S orbiting in the sky to perform positioning and timing.

[0031] The flying object 300 adjusts the shooting direction of the camera 310 so as to capture an image of the tractor 1 based on the result of comparing the position of the tractor 1 with the position of the flying object 300. The shooting direction of the camera 310 is controlled by a servo motor 320 as shown in FIG.

[0032] 2 is a perspective view showing an example of the appearance of the flying object 300. As shown in FIG. 2, the camera 310 is installed at the bottom of the flying object 300, and mainly captures an image of the area below the flying object 300. The servo motor 320 changes the shooting direction of the camera 310. Specifically, the servo motor 320 changes the shooting direction by rotating the camera 310 about two rotation axes that are perpendicular to each other. In this way, the flying object 300 can constantly capture images of the tractor 1 by changing the shooting direction depending on the positional relationship with the tractor 1.

[0033] <Work vehicle (tractor) management system> Next, the work vehicle management system 100 according to the embodiment, i.e., a control system of the work vehicle (tractor) 1 centered around the control unit 200, will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the management system 100 according to the embodiment. As shown in Fig. 3, the control unit 200 includes an engine ECU (Electronic Control Unit) 201, a travel system ECU 202, and a work implement lifting system ECU 203.

[0034] The engine ECU 201 controls the rotation speed of the engine E. The traveling system ECU 202 controls the traveling speed of the traveling vehicle body 2 (see FIG. 1) by controlling the rotation of the drive wheels (rear wheels 4). The working machine lifting system ECU 203 controls the lifting device 13 to drive the working machine 6 to lift and lower.

[0035] The control unit 200 is capable of controlling each part through electronic control, and includes a processing unit having a CPU (Central Processing Unit) and the like, as well as a memory unit consisting of, for example, a hard disk, a ROM (Read Only Memory), a RAM (Random Access Memory), etc., in which necessary data such as various programs and the planned driving route (hereinafter referred to as the driving route) of the traveling vehicle body 2 that is set in advance for each field is stored.

[0036] 3, the control unit 200 is connected to a positioning device (GNSS) 150, an azimuth angle sensor 170, an engine rotation sensor 110, a vehicle speed sensor 111, a gear shift sensor 112, a turning angle sensor 113, etc. Also connected to the control unit 200 are an engine E, a gear shift device 121, a steering device 122, an elevator device 13, etc.

[0037] The engine revolution sensor 110 detects the revolution speed of the engine E. The vehicle speed sensor 111 detects the traveling speed (vehicle speed) of the traveling vehicle body 2 (see FIG. 1). The gear shift sensor 112 detects which of a plurality of gear shift stages the transmission 121 is in. The turning angle sensor 113 detects the turning angle of the front wheels 3 (see FIG. 1), which are steered wheels.

[0038] The control unit 200 receives inputs of position (self-position) information of the traveling vehicle body 2 in a field or the like from the positioning device 150, the number of revolutions of the engine E from the engine revolution sensor 110, the vehicle speed of the traveling vehicle body 2 from the vehicle speed sensor 111, the current gear position from the gear shift sensor 112, and the turning angle of the front wheels 3 from the turning angle sensor 113. When the control unit 200 causes the traveling vehicle body 2 to travel autonomously, as described above, the control unit 200 uses the detection value of the turning angle sensor 113 to feed back the turning angle of the front wheels 3, thereby controlling the steering cylinder connected to the steering wheel 9 (see FIG. 1 ), thereby steer- ing the steering wheel 9.

[0039] In the control unit 200, the engine ECU 101 is connected to the engine E, the traveling system ECU 102 is connected to the transmission 121 and the steering system 122, and the work machine lifting system ECU 103 is connected to the lifting device 13. The work machine lifting system ECU 103 lifts and lowers the work machine via the lifting device 13.

[0040] Furthermore, in the control unit 200, when the traveling body 2 is made to travel autonomously, a travel route corresponding to the work content of the work implement 6 is determined in advance for each field, digitized, and stored in the memory unit. Based on the measurement results of the positioning device 150, the control unit 200 controls the engine E, the transmission 121, the steering device 122, the lifting device 13, and the like so that the tractor 1 performs work while traveling along the travel route stored in the memory unit. The travel route is set according to the shape and size of the field, the width, length, and number of ridges formed in the field, and the type of crop. Furthermore, the control unit 200 sets in advance the turning radius of the tractor 1 (traveling body 2) when moving in the field.

[0041] As described above, the control unit 200 is wirelessly connected to, for example, a portable terminal device (tablet terminal) 160 that can be carried by the worker. The control unit 200 controls each part of the tractor 1 based on an instruction signal from the portable terminal device 160 operated by the worker. The control unit 200 may have a machine information database of the tractor 1 and be configured to be able to transmit and receive information such as the model from the portable terminal device 160 or the like.

[0042] As described above, the control unit 200 is wirelessly connected to the flying object 300. The control unit 200 transmits the position information of the tractor 1 measured by the positioning device 150 to the flying object 300.

[0043] Next, the configuration of the flying object 300 will be described. As shown in Fig. 3, the flying object 300 includes a camera 310, a servo motor 320, a positioning device 330 (second positioning device), and a control unit 340. The control unit 340 is capable of controlling each unit by electronic control, and includes a processing unit having a CPU (Central Processing Unit) and the like, as well as a storage unit configured of, for example, a hard disk, a ROM (Read Only Memory), a RAM (Random Access Memory), and the like, in which necessary data such as various programs is stored.

[0044] The control unit 340 compares the position information of the tractor 1 and the flying object 300, and controls the servo motor 320 so that the camera 310 captures an image of the tractor 1. The control unit 340 also transmits the image captured by the camera 310 to the mobile terminal device 160.

[0045] The mobile terminal device 160 displays on the display unit a composite image in which a virtual route, which is a virtual travel route, is combined with the image of the camera 310 based on the position information of the flying object 300 obtained from the positioning device 330 and the shooting direction of the camera 310. The mobile terminal device 160 also acquires various information required for generating the composite image 500 shown in FIG. 4 from the tractor 1 and the flying object 300.

[0046] Fig. 4 is a diagram showing an example of a composite image 500 displayed on the display unit of the mobile terminal device 160. As shown in Fig. 4, the composite image 500 is an image in which a virtual route 520, which is a virtual travel route, is superimposed on an image of the camera 310 including a tractor image 510. This allows a manager or the like who operates the mobile terminal device 160 to easily understand that the tractor 1 is traveling on the travel route.

[0047] Furthermore, the mobile terminal device 160 extracts an area showing the ground (ground area) from the image based on color information (e.g., RGB information) in the image from the camera 310, and changes the display form between the virtual route 520 in the ground area and the virtual route 520 in a non-ground area other than the ground area. For example, the color of the soil of the ground (e.g., brown) is extracted from the image as the ground area, and the area other than the soil color is extracted as the non-ground area.

[0048] 4, the virtual path 520 is displayed in a dashed line in an area (non-ground area) that overlaps with the tractor image 510, and is displayed in a solid line in an area (ground area) that does not overlap with the tractor image 510. Note that the virtual path 520 in the non-ground area may be a dotted line other than the dashed line, such as a dashed line. Alternatively, the virtual path 520 in the non-ground area may not be displayed.

[0049] In this way, by changing the display form of the virtual route 520 in the ground area and the virtual route 520 in the non-ground area, it is possible to display an image (synthetic image) in which the virtual route 520 is actually laid on the ground.

[0050] The virtual path 520 in the ground area and the virtual path 520 in the non-ground area may be displayed with lines of different colors or different thicknesses.

[0051] Furthermore, the mobile terminal device 160 may display a composite image 500 that is composited with a virtual route 520 according to undulation information on the driving route. Specifically, when there are large undulations on the ground, the mobile terminal device 160 composites a virtual route 520 in which the angle of the lines is adjusted according to the inclination of the undulations. This makes it possible to composite a virtual route 520 that does not look unnatural with the actual driving route.

[0052] The undulation information may be information measured in advance and stored in a storage unit (not shown) for use, or the undulation may be measured in real time using LiDAR (Light Detection and Ranging) or the like.

[0053] The display form of the virtual route 520 may be changed between the route already traveled and the route to be traveled. In other words, the display form of the virtual route 520 on the traveling direction side of the tractor image 510 is changed from that of the virtual route 520 on the opposite direction side of the traveling direction.

[0054] 4, only the virtual route 520 corresponding to a part of the travel route of the entire farm field is displayed. In other words, the mobile terminal device 160 generates a composite image 500 in which the virtual route 520 corresponding to the travel route before and after a predetermined time is synthesized with the current position of the tractor 1. This prevents the virtual route 520 that is longer than necessary from being displayed in the composite image 500, making it easier to understand the travel route.

[0055] The composite image 500 also includes a display of a traveling direction 530 that indicates the traveling direction of the travel route. In the example shown in FIG. 4, the traveling direction 530 is indicated by two triangular arrows. In this way, by including the display of the traveling direction 530, the manager can easily determine the traveling direction of the tractor 1. Note that the display of the traveling direction 530 may be expressed by an animation that moves in the traveling direction. This makes it easier to determine the traveling direction.

[0056] The composite image 500 also includes a display of a start position 540a and an end position 540b of a turn on the travel route, which allows the manager to easily grasp the start position 540a and the end position 540b of the next turn.

[0057] In the example shown in FIG. 4, a balloon image 541a indicating the turning direction is displayed at the start position 540a. This allows the turning direction to be easily understood. In the example shown in FIG. 4, balloon images 542a and 541b indicating the lifting and lowering timing (lifting and lowering position) of the work implement 6 are displayed at the start position 540a and the end position 540b. Specifically, a balloon image 542a indicating the timing to lift the work implement 6 is displayed at the start position 540a, and a balloon image 541b indicating the timing to lower the work implement 6 is displayed at the end position 540b. In other words, the start position 540a and the end position 540b are displays that also indicate the lifting and lowering timing of the work implement 6. Note that when the start position 540a and the end position 540b are different from the lifting and lowering timing of the work implement 6, they may be displayed separately (separate points).

[0058] The manager operates the operation button 590 at this timing to raise and lower the work machine 6. This makes it possible to prevent the manager from making a mistake in the timing for raising and lowering the work machine 6 with a high degree of accuracy.

[0059] The composite image 500 also includes display of forward / reverse switching positions 550a, 550b on the travel route. In the example shown in Fig. 4, switching position 550a indicates switching from forward to reverse, and switching position 550b indicates switching from reverse to forward. In the example shown in Fig. 4, balloon images 551a, 551b indicating the switching destination direction are displayed at switching positions 550a, 550b, respectively. This allows the manager to easily understand the switching direction.

[0060] 4, a balloon image 552a indicating the refueling position of the tractor 1 is displayed at the switching position 550a. That is, the mobile terminal device 160 generates a composite image 500 including an indication of the refueling position of the tractor 1 (switching position 550a) based on the remaining fuel information acquired from the tractor 1. Specifically, the mobile terminal device 160 calculates the remaining travelable distance based on the remaining fuel information, and determines the refueling position based on the calculated distance. Note that the refueling position is preferably not on the route along which work is performed, but is a position midway through the turn from the start position 540a to the end position 540b.

[0061] The composite image 500 also includes an indication of a relative position 560 of the flying object 300 (camera 310) with respect to the tractor 1. The example shown in Fig. 4 shows a case where the flying object 300 is positioned behind the tractor 1 and captures an image with the camera 310. This makes it easy to grasp the positional relationship between the flying object 300 and the tractor 1 and the shooting direction of the camera 310.

[0062] The composite image 500 also includes a display of a running state 570 of the tractor 1. The running state 570 displays information such as the vehicle speed, the number of rotations of the PTO shaft, the shifting state of the auxiliary transmission, the remaining fuel, the oil temperature, the total running time, and whether or not automatic driving is enabled. This allows the running state 570 of the tractor 1 to be easily understood.

[0063] The composite image 500 also includes a display of a control state 580 of the tractor 1. In Fig. 4, each control state 580 is displayed by a slide bar, and each control state 580 can be changed by operating the slide bar.

[0064] The composite image 500 also includes a display of operation buttons 590 for remotely controlling the elevation and lowering of the work implement 6. In the example shown in Fig. 4, a button for raising the work implement 6 and a button for lowering it are displayed.

[0065] Next, a processing procedure of the processing executed in the management system 100 according to the embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the processing procedure of the management system 100.

[0066] As shown in FIG. 5, the mobile terminal device 160 acquires information about the set travel route (step S101).

[0067] Next, the mobile terminal device 160 acquires the position information of the flying object 300 and the tractor 1 (step S102).

[0068] Next, mobile terminal device 160 acquires the shooting direction of camera 310 and the image of camera 310 (step S103).

[0069] Next, the mobile terminal device 160 extracts a ground area and a non-ground area based on the image captured by the camera 310 (step S104).

[0070] Next, mobile terminal device 160 generates a composite image by combining the virtual route with the image from camera 310, based on the information acquired (extracted) in steps S101 to S104 (step S105).

[0071] Next, mobile terminal device 160 displays the generated composite image on the display unit (step S106), and ends the process.

[0072] According to the above-described embodiment, the following management system 100 is realized.

[0073] (1) The management system 100 according to the embodiment includes a work vehicle (tractor 1), an aircraft 300, and a terminal device (portable terminal device 160). The tractor 1 includes a first positioning device 150, and autonomously travels along a preset travel route while measuring its position using the first positioning device 150. The aircraft 300 includes a second positioning device 330 and a camera 310 that can change the shooting direction based on the positioning result of the second positioning device 330. The portable terminal device 160 displays on the display unit a composite image 500 that combines a virtual route 520, which is a virtual travel route, with the image of the camera 310, based on the position information of the aircraft 300 based on the second positioning device 330 and the shooting direction of the camera 310.

[0074] This makes it possible to easily grasp the actual traveling position of the tractor 1 with respect to the actual traveling route.

[0075] (2) In (1) above, the mobile terminal device 160 extracts the ground area from the image based on color information in the image of the camera 310, and changes the display form between the virtual route 520 in the ground area and the virtual route 520 in the non-ground area other than the ground area.

[0076] This makes it possible to display a synthetic image 500 in which the virtual route 520 appears as if it were actually laid on the ground.

[0077] (3) In the above (2), the mobile terminal device 160 does not display the virtual route 520 in the non-ground area.

[0078] This makes it possible to display a composite image 500 in which only the driving route that is actually visible in the image is used as the virtual route 520.

[0079] (4) In the above (2), the mobile terminal device 160 displays the virtual route 520 in the ground area with a solid line, and displays the virtual route 520 in the non-ground area with a dotted line.

[0080] This makes it easy to determine whether the travel route overlaps with the tractor 1 or the like.

[0081] (5) In any one of the above (1) to (4), the mobile terminal device 160 displays the composite image 500 that is composited with a virtual route 520 that corresponds to information about the undulations of the travel route.

[0082] This makes it possible to display the virtual route 520 that matches the actual terrain, thereby reducing the sense of incongruity felt by the manager.

[0083] (6) The management system 100 according to the embodiment includes a work vehicle (tractor 1), an aircraft 300, and a terminal device (portable terminal device 160). The tractor 1 includes a first positioning device 150 and autonomously travels along a preset travel route while measuring its position using the first positioning device 150. The aircraft 300 includes a second positioning device 330 and a camera 310 that can change the shooting direction based on the positioning result of the second positioning device 330. The portable terminal device 160 displays a composite image 500 on the display unit, which is a composite image of the camera 310 and a virtual route 520, which is a virtual travel route, based on the position information of the aircraft 300 based on the second positioning device 330 and the shooting direction of the camera 310. The portable terminal device 160 generates a composite image 500 including a display of a turning position (start position 540a, end position 540b) on the travel route.

[0084] This makes it possible to easily grasp the position at which the tractor 1 will turn next.

[0085] (7) In the above (6), the tractor 1 is equipped with a liftable work implement 6 that performs work on the travel route. The mobile terminal device 160 generates a composite image 500 including an indication of the lifted position of the work implement 6 on the travel route.

[0086] This makes it possible to easily grasp the position to which the tractor 1 will next raise or lower the work implement 6.

[0087] (8) In the above (7), the mobile terminal device 160 generates a composite image 500 including a display of an operation button 590 for remotely operating the lifting and lowering of the work machine 6.

[0088] This allows the manager to manually and remotely operate the work implement 6 while viewing the image of the tractor 1.

[0089] (9) In the above (6) to (8), the mobile terminal device 160 generates the composite image 500 including an indication of the refueling position (switching position 550a) of the tractor 1 based on the remaining fuel amount information acquired from the tractor 1.

[0090] This allows the location where refueling is required to be identified while viewing the image of the tractor 1.

[0091] (10) In the above (6) to (9), the mobile terminal device 160 generates a composite image 500 by combining the current position of the tractor 1 with a virtual route 520 corresponding to the travel route before and after a predetermined time.

[0092] This makes it possible to prevent the composite image 500 from becoming complicated due to an increase in the number of display objects.

[0093] (11) In the above (6) to (10), the mobile terminal device 160 synthesizes the composite image 500 including an indication of the traveling direction 530 on the travel route.

[0094] This makes it possible to easily grasp the direction in which the tractor 1 is moving.

[0095] Further advantages and modifications may readily occur to those skilled in the art. Thus, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and equivalents thereof. [Explanation of symbols]

[0096] 1 Tractor 2 Running vehicle 3 Front wheel 4 Rear Wheel 5. Bonnet 6 Work equipment 7 PTO device 8. Cockpit 9. Steering Wheel 10 Handle post 11 Operation pedal 13 Lifting device 61 Cultivating Claw 71 PTO shaft 100 Management Systems 101 Engine ECU 102 Driving system ECU 103 Work equipment lifting system ECU 110 Engine revolution sensor 111 Vehicle speed sensor 112 Gear shift sensor 113 Turn angle sensor 121 Gearbox 122 Steering device 131 Lifting cylinder 132 Lift arm 133 Lift Rod 134 Lower Link 135 Top Link 150 Positioning device 160 Portable terminal device 170 Azimuth Angle Sensor 200 Control section 201 Engine ECU 202 Driving system ECU 203 Work equipment lifting system ECU 300 Flying Objects 310 Camera 320 Servo Motor 330 Positioning Device 340 Control Unit 500 composite images 510 Tractor Images 520 Virtual Route 530 Direction of travel 560 Relative Position 570 Running Condition 580 Control State 590 Operation Button E-Engine S Navigation satellite

Claims

1. A work vehicle that is equipped with a first positioning device and that autonomously travels along a predetermined travel route while measuring its position using the first positioning device; an aircraft including a second positioning device and a camera capable of changing a photographing direction based on a positioning result of the second positioning device; A terminal device including a display unit that displays a composite image obtained by combining a virtual route, which is a virtual travel route, with an image of the camera based on position information of the flying object based on the second positioning device and the shooting direction of the camera; Equipped with The terminal device generating the composite image including an indication of a turning position on the travel path; A ground area is extracted from the image of the camera, and a display form is changed between a virtual route in the ground area and a virtual route in a non-ground area other than the ground area. A management system characterized by:

2. The work vehicle is A liftable work machine that performs work on the travel path is provided, The terminal device generating the composite image including simultaneously a display of a lifting position of the work machine on the travel route and a display of an operation button for remotely operating the lifting and lowering of the work machine; The management system according to claim 1 .

3. The terminal device generating the composite image including an indication of a forward / reverse switch position on the travel route; The management system according to claim 2 .

4. The terminal device generating the composite image including an indication of a refueling position of the work vehicle based on remaining fuel information acquired from the work vehicle; 4. The management system according to claim 1, wherein the control unit is a computer.

5. The terminal device generating the composite image by combining the virtual route corresponding to the travel route before and after a predetermined time with the current position of the work vehicle; 5. The management system according to claim 1, wherein the first and second storage units are connected to the management device.

6. The terminal device generating the composite image including an indication of a direction along the travel route; 6. The management system according to claim 1, wherein the management system comprises:

Images