Display control method, display control program, and display control system

Abstract

Provided are a display control method, a display control program, and a display control system with which it is possible to improve the convenience of an operation terminal that displays information pertaining to a work vehicle that can be automatically driven. A display processing unit (212) of an operation terminal (20) displays a pre-registered work vehicle (10) on a first screen of the operation terminal (20) in a user-selectable manner, and displays prescribed information pertaining to the work vehicle (10) selected by the user on a second screen of the operation terminal (20).

Description

Technical Field

[0001] The present invention relates to a technology for displaying information related to an automatically drivable work vehicle. Background Art

[0002] Conventionally, a technology has been known for automatically driving a work vehicle in a work area according to a preset target path. In addition, a technology for remotely monitoring a work vehicle during automatic driving has also been known (for example, refer to Patent Document 1).

[0003] Patent Document 1: Japanese Patent No. 6253678

[0004] In the prior art, for example, if there are multiple work vehicles to be monitored, there is a problem that it is difficult for an operator to obtain necessary information about each work vehicle, and the convenience of an operation terminal for displaying information related to the work vehicle decreases. Summary of the Invention

[0005] An object of the present invention is to provide a display control method, a display control program, and a display control system that can improve the convenience of an operation terminal for displaying information related to an automatically drivable work vehicle.

[0006] The display control method according to the present invention is a method for controlling display information of an operation terminal, and includes: causing a pre-registered work vehicle to be displayed on a first screen of the operation terminal in a manner selectable by a user; and causing specified information related to the work vehicle selected by the user to be displayed on a second screen of the operation terminal.

[0007] In addition, the display control method according to the present invention is a method for controlling display information of an operation terminal, and includes: causing a pre-registered work vehicle to be displayed on a first screen of the operation terminal; and when an obstacle is detected by the work vehicle during automatic driving, or when the work vehicle is automatically driving in a specific area, causing specified information related to the work vehicle to be displayed on a second screen of the operation terminal.

[0008] In addition, the display control program according to the present invention is a program for controlling display information of an operation terminal, and is used to cause one or more processors to execute: causing a pre-registered work vehicle to be displayed on a first screen of the operation terminal in a manner selectable by a user; and causing specified information related to the work vehicle selected by the user to be displayed on a second screen of the operation terminal.

[0009] In addition, the display control system involved in the present invention is a system for controlling the display information of an operation terminal. The above display control system includes: a first display processing unit that displays a pre-registered work vehicle on a first screen of the operation terminal in a manner selectable by a user; and a second display processing unit that displays specified information related to the work vehicle selected by the user on a second screen of the operation terminal.

[0010] According to the present invention, it is possible to provide a display control method, a display control program, and a display control system that can improve the convenience of an operation terminal for displaying information related to an automatically drivable work vehicle. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 is a block diagram showing the structure of an automatic driving system according to an embodiment of the present invention.

[0012] Figure 2 is an external view showing an example of a work vehicle according to an embodiment of the present invention.

[0013] Figure 3 is a diagram showing an example of a selection screen displayed on an operation terminal according to an embodiment of the present invention.

[0014] Figure 4 is a diagram showing an example of a selection screen displayed on an operation terminal according to an embodiment of the present invention.

[0015] Figure 5 is a diagram showing an example of a selection screen displayed on an operation terminal according to an embodiment of the present invention.

[0016] Figure 6A is a diagram showing an example of an information display screen displayed on an operation terminal according to an embodiment of the present invention.

[0017] Figure 6B is a diagram showing an example of an information display screen displayed on an operation terminal according to an embodiment of the present invention.

[0018] Figure 6C is a diagram showing an example of an information display screen displayed on an operation terminal according to an embodiment of the present invention.

[0019] Figure 7 is a diagram showing an example of a selection screen and an information display screen displayed on an operation terminal according to an embodiment of the present invention.

[0020] Figure 8 is a flowchart showing an example of the sequence of display control processing executed by the automatic driving system according to an embodiment of the present invention.

[0021] Figure 9 This is a diagram showing an example of a selection screen and an information display screen displayed on an operation terminal related to an embodiment of the present invention.

[0022] Figure 10 This is a diagram showing an example of a selection screen and an information display screen displayed on an operation terminal related to an embodiment of the present invention.

[0023] Figure 11 This is a diagram showing an example of an information display screen displayed on an operation terminal related to an embodiment of the present invention.

[0024] Figure 12 This is a diagram showing an example of a selection screen displayed on an operation terminal related to an embodiment of the present invention.

[0025] Figure 13 This is a diagram showing an example of a selection screen displayed on an operation terminal related to an embodiment of the present invention.

[0026] Figure 14 This is a block diagram showing the structure of a remote monitoring system related to an embodiment of the present invention.

[0027] Figure 15 This is a schematic diagram showing a situation of remote operation in a remote monitoring system related to an embodiment of the present invention.

[0028] Figure 16 This is an external view of an operation unit of a remote operation device related to an embodiment of the present invention.

[0029] Figure 17A This is an example of an operation screen of an operation terminal in a remote monitoring system related to an embodiment of the present invention.

[0030] Figure 17B This is an example of an operation screen of an operation terminal in a remote monitoring system related to an embodiment of the present invention.

[0031] Figure 18 This is an example of a display screen of a remote operation device related to an embodiment of the present invention.

[0032] Figure 19 This is an example of a display screen (front image) of a remote operation device related to an embodiment of the present invention.

[0033] Explanation of reference numerals

[0034] 1... Automatic driving system; 10... Work vehicle; 11... Vehicle control device; 12... Storage unit; 15... Obstacle detection device; 20... Operation terminal; 21... Operation control unit; 23... Operation display unit; 53... Camera; 54... Obstacle sensor; 30... Remote operation device (operation terminal); 33... Display unit; 34... Operation unit; 111... Driving processing unit; 211... Setting processing unit; 212... Display processing unit (first display processing unit, second display processing unit); 213... Reception processing unit; A1... Icon image (identification information of work vehicle 10a); A2... Frame image (identification information of work vehicle 10a); B1... Icon image (identification information of work vehicle 10b); B2... Frame image (identification information of work vehicle 10b); C1... Icon image (identification information of operator X); C2... Icon image (identification information of operator Y); M1... Message (prescribed information); P1... Selection screen (first screen); P2... Information display screen (second screen); P3... Instruction screen. Detailed implementation manner

[0035] The following implementation manner is an example that embodies the present invention and does not limit the technical scope of the present invention.

[0036] As Figure 1 shown, the automatic driving system 1 according to the implementation manner of the present invention includes a work vehicle 10 and an operation terminal 20. The work vehicle 10 can be one or multiple. In this implementation manner, an example in which the automatic driving system 1 includes multiple work vehicles 10 (two work vehicles 10a and 10b in Figure 1 ) is taken for illustration. In addition, hereinafter, without distinguishing between the work vehicle 10a and the work vehicle 10b, it is referred to as "work vehicle 10".

[0037] The work vehicle 10 and the operation terminal 20 can communicate via the communication network N1. For example, the work vehicle 10 and the operation terminal 20 can communicate via a mobile phone line network, a packet line network, or a wireless LAN.

[0038] In the present embodiment, the work vehicle 10 is taken as an example of a tractor for illustration. In addition, as other embodiments, the work vehicle 10 may also be a combine harvester, a transplanter, a seeder, a construction machine, a snow removal vehicle, or the like. Further, the work vehicle 10a and the work vehicle 10b may be different types of vehicles respectively, or may be the same type of vehicles. The work vehicle 10 is provided with a structure capable of automatically traveling (autonomous traveling) in the work area of the field according to a preset target path. In addition, the work vehicle 10 can perform a prescribed operation while automatically traveling in the work area. Further, the work vehicle 10 is provided with a structure capable of automatically traveling in a road (connection road) connecting multiple fields according to a preset inter-field path.

[0039] The work vehicle 10 can automatically travel in the field and outside the field (road) according to the preset target path and inter-field path based on the position information of the current position of the work vehicle 10 calculated by the positioning unit 17. In addition, the work vehicle 10 may also automatically travel in a state where an operator is on board.

[0040] For example, if a target path including a work path, a travel start position, and a travel end position is set for the field, the work vehicle 10 automatically travels in the field from the travel start position according to the target path while performing a prescribed operation until the travel end position.

[0041] In addition, if the work vehicle 10 finishes the operation of one field, it can move to other fields to perform the operation. For example, if the work vehicle 10 finishes the operation of the first field, it automatically travels on the preset inter-field path on the road and moves to the second field. When the work vehicle 10 arrives at the second field, it automatically travels in the second field according to the preset target path while performing a prescribed operation.

[0042] The target path in the field is appropriately set according to the work content. In addition, the inter-field path of the road is preset according to the operation (teaching operation) performed by the operator. The inter-field path may be a road dedicated to work vehicles such as an agricultural road, a forest road, a highway, a private road, a motor vehicle lane, or may be a road passable by general vehicles (passenger cars, etc.).

[0043] The operation terminal 20 is an information processing device that displays various information related to the work vehicle 10. For example, for the pre-registered work vehicle 10, the operation terminal 20 displays the position information of the current position, the travel status, the work status, and the like. The user (operator, monitor) of the operation terminal 20 can remotely monitor the work vehicle 10 by confirming the above information.

[0044] [Work vehicle 10]

[0045] AsFigure 1 and Figure 2 As shown in Figure 2 and , the work vehicle 10 includes a vehicle control device 11, a storage unit 12, a traveling device 13, a working machine 14, an obstacle detection device 15, a communication unit 16, a positioning unit 17, etc. The vehicle control device 11 is electrically connected to the storage unit 12, the traveling device 13, the working machine 14, the obstacle detection device 15, the positioning unit 17, etc. In addition, the vehicle control device 11 and the obstacle detection device 15 may be capable of wireless communication, and the vehicle control device 11 and the positioning unit 17 may also be capable of wireless communication.

[0046] The communication unit 16 is a communication interface for connecting the work vehicle 10 to the communication network N1 by wire or wirelessly and performing data communication compliant with a prescribed communication protocol with an external device (such as the operation terminal 20) via the communication network N1.

[0047] The storage unit 12 is a non-volatile storage unit such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive) that stores various kinds of information. The storage unit 12 stores control programs such as an automatic driving program for causing the vehicle control device 11 to execute automatic driving processing. For example, the above-mentioned automatic driving program is non-temporarily recorded on a computer-readable recording medium such as a CD or a DVD, and is read by a prescribed reading device (not shown) and stored in the storage unit 12. In addition, the above-mentioned automatic driving program may be downloaded from a server (not shown) to the work vehicle 10 via the communication network N1 and stored in the storage unit 12. Further, data of a target path, etc. is stored in the storage unit 12.

[0048] The traveling device 13 is a driving unit for causing the work vehicle 10 to travel. As Figure 2 shown in

[0048] and Figure 2 , the traveling device 13 includes an engine 131, front wheels 132, rear wheels 133, a transmission device 134, a front axle 135, a rear axle 136, a steering wheel 137, etc. In addition, the front wheels 132 and the rear wheels 133 are respectively provided on the left and right of the work vehicle 10. Further, the traveling device 13 is not limited to a wheel type having the front wheels 132 and the rear wheels 133, and may be a crawler type having crawlers provided on the left and right of the work vehicle 10.

[0049] The engine 131 is a drive source such as a diesel engine or a gasoline engine that is driven using fuel supplied from a fuel tank (not shown). The traveling device 13 may also include an electric motor as a drive source in addition to, or in place of, the engine 131. Further, the engine 131 is connected to a generator (not shown), and power is supplied from the generator to electrical components such as the vehicle control device 11, the obstacle detection device 15, and the positioning unit 17 provided in the work vehicle 10, as well as to a battery. In addition, the above-mentioned battery is charged by the power supplied from the above-mentioned generator. Moreover, electrical components such as the vehicle control device 11, the obstacle detection device 15, and the positioning unit 17 provided in the work vehicle 10 can also be driven using the power supplied from the above-mentioned battery after the engine 131 stops.

[0050] The driving force of the engine 131 is transmitted to the front wheels 132 via the transmission 134 and the front axle 135, and is transmitted to the rear wheels 133 via the transmission 134 and the rear axle 136. In addition, the driving force of the engine 131 is also transmitted to the work implement 14 via a PTO shaft (not shown). When the work vehicle 10 is traveling automatically, the traveling device 13 performs a traveling operation according to a command from the vehicle control device 11. In addition, the traveling device 13 causes the work vehicle 10 to decelerate or stop according to a command from the vehicle control device 11.

[0051] The work implement 14 is, for example, a tiller, a lawn mower, a plow, a fertilizer applicator, a sprayer (chemical spreader), a harrow, or a seeder, etc., and can be attached to and detached from the work vehicle 10. Thus, the work vehicle 10 can perform various operations using each work implement 14. In Figure 2 it shows the case where the work implement 14 is a tiller.

[0052] The steering wheel 137 is an operation unit operated by an operator or the vehicle control device 11. For example, in the traveling device 13, according to the operation of the steering wheel 137 by the vehicle control device 11, the angle of the front wheels 132 is changed by a hydraulic power steering mechanism (not shown) or the like, and the traveling direction of the work vehicle 10 is changed.

[0053] In addition to the steering wheel 137, the traveling device 13 also includes a shift lever, an accelerator, a brake, etc. (not shown) that are operated by the vehicle control device 11. Moreover, in the traveling device 13, according to the operation of the above-mentioned shift lever by the vehicle control device 11, the gear of the transmission 134 is switched to a forward gear or a reverse gear, etc., and the traveling mode of the work vehicle 10 is switched to forward or reverse, etc. In addition, the vehicle control device 11 operates the above-mentioned accelerator to control the rotational speed of the engine 131. In addition, the vehicle control device 11 operates the above-mentioned brake to brake the rotation of the front wheels 132 and the rear wheels 133 using an electromagnetic brake.

[0054] The positioning unit 17 is a communication device including a positioning control unit 171, a storage unit 172, a communication unit 173, a positioning antenna 174, etc. For example, as Figure 2 shown, the positioning unit 17 is provided above the cab 138 on which the operator rides. In addition, the installation location of the positioning unit 17 is not limited to the cab 138. Also, the positioning control unit 171, the storage unit 172, the communication unit 173, and the positioning antenna 174 of the positioning unit 17 may be dispersedly arranged at different positions in the work vehicle 10. Furthermore, as described above, the positioning unit 17 is connected to the above battery, and the positioning unit 17 can also operate during the stop of the engine 131. Additionally, for example, a mobile phone terminal, a smartphone, a tablet terminal, or a quantum compass may be used instead of the positioning unit 17.

[0055] The positioning control unit 171 is a computer system including one or more processors and storage memories such as a non-volatile memory and a RAM. The storage unit 172 is a non-volatile memory or the like that stores data such as a program for causing the positioning control unit 171 to perform positioning processing, positioning information, and movement information. For example, the above program is non-temporarily recorded on a computer-readable recording medium such as a CD or a DVD, and is read by a prescribed reading device (not shown) and stored in the storage unit 172. In addition, the above program may also be downloaded from a server (not shown) to the positioning unit 17 via the communication network N1 and stored in the storage unit 172.

[0056] The communication unit 173 is a communication interface for connecting the positioning unit 17 to the communication network N1 by wire or wirelessly and performing data communication compliant with a prescribed communication protocol with external devices such as a base station server via the communication network N1.

[0057] The positioning antenna 174 is an antenna that receives radio waves (GNSS signals) transmitted from satellites.

[0058] The positioning control unit 171 calculates the current position of the work vehicle 10 based on the GNSS signals received from satellites by the positioning antenna 174. For example, when the work vehicle 10 is automatically traveling in a field, if the positioning antenna 174 receives radio waves (transmission time, orbital information, etc.) respectively sent from multiple satellites, the positioning control unit 171 calculates the distances between the positioning antenna 174 and each satellite, and calculates the current position (latitude and longitude) of the work vehicle 10 based on the calculated distances. In addition, the positioning control unit 171 may perform positioning in the real-time kinematic mode (RTK - GNSS positioning mode (RTK mode)), and this real-time kinematic mode of positioning calculates the current position of the work vehicle 10 by using correction information corresponding to a base station (reference station) close to the work vehicle 10. In this way, the work vehicle 10 automatically travels by using the positioning information by means of the RTK mode. In addition, the current position of the work vehicle 10 may be the same position as the positioning position (for example, the position of the positioning antenna 174), or may be a position offset from the positioning position. In addition, the positioning control unit 171 may also calculate the current position of the (positioning) work vehicle 10 by using a quantum compass.

[0059] If the work vehicle 10 detects a detection object during automatic driving, the obstacle detection device 15 outputs detection information (measurement information) to the vehicle control device 11. Specifically, the obstacle detection device 15 includes a detection control unit 51, a storage unit 52, a camera 53, an obstacle sensor 54, a communication unit 55, etc. The obstacle detection device 15 may be composed of a single unit and mounted on the work vehicle 10, or multiple constituent elements may be dispersedly arranged on the work vehicle 10.

[0060] The communication unit 55 is a communication interface for connecting the obstacle detection device 15 to the communication network N1 by wire or wirelessly, and performing data communication compliant with a prescribed communication protocol with an external device (such as the operation terminal 20) via the communication network N1.

[0061] The storage unit 52 is a non-volatile storage unit such as an HDD or an SSD that stores various information. The storage unit 52 stores control programs such as an obstacle detection program for causing the obstacle detection device 15 to perform obstacle detection processing. For example, the above-mentioned obstacle detection program is non-temporarily recorded on a computer-readable recording medium such as a CD or a DVD, and is read by a prescribed reading device (not shown) and stored in the storage unit 52. In addition, the above-mentioned obstacle detection program may also be downloaded from a server (not shown) to the obstacle detection device 15 via the communication network N1 and stored in the storage unit 52.

[0062] The camera 53 is a digital camera that captures an image of a subject included in a specified shooting range and outputs it as digital image data. The camera 53 continuously captures the subject at a specified frame rate, generates frame images (captured images) with a specified resolution, and sequentially sends them to the detection control unit 51. In addition, the camera 53 sends the image data of the above-mentioned captured images to the operation terminal 20 via the communication unit 55. The operation terminal 20 can display the above-mentioned captured images on the operation screen of the operation display unit 23 (refer to Figure 6A etc.).

[0063] The camera 53 includes: a front camera 53a that can capture a shooting range in front when observed from the work vehicle 10, a rear camera 53b that can capture a shooting range in the rear when observed from the work vehicle 10, a left camera 53c that can capture a shooting range on the left when observed from the work vehicle 10, and a right camera 53d (not shown) that can capture a shooting range on the right when observed from the work vehicle 10. As Figure 2 shown, the front camera 53a is arranged on the upper front side of the cab 138, the rear camera 53b is arranged on the upper rear side of the cab 138, the left camera 53c is arranged on the upper left side of the cab 138, and the right camera 53d is arranged on the upper right side of the cab 138. In addition, the camera 53 may be composed only of the front camera 53a and the rear camera 53b. In addition, the camera 53 may be composed of one unit and is an omnidirectional camera that can capture all directions around the work vehicle 10.

[0064] The obstacle sensor 54 is a sensor that uses infrared rays, ultrasonic waves, etc. to detect a detection target within a specified detection range. For example, the obstacle sensor 54 may be a radar sensor (distance sensor) that can three-dimensionally measure the distance to the detection target using laser, or a sonar sensor having a plurality of sonars that can measure the distance to the detection target using ultrasonic waves. The obstacle sensor 54 is provided at the central front part, central rear part, etc. of the body of the work vehicle 10, monitors the surroundings of the work vehicle 10, and detects obstacles. In the present embodiment, the following situation is taken as an example for explanation, that is, the obstacle sensor 54 includes: a front obstacle sensor 54a that can detect a detection target within a detection range in front and on the side (left and right) when observed from the work vehicle 10, and a rear obstacle sensor 54b that can detect a detection target within a detection range in the rear and on the side (left and right) when observed from the work vehicle 10. As Figure 2As shown in the figure, the front obstacle sensor 54a is disposed on the upper front side of the cab 138, and the rear obstacle sensor 54b is disposed on the upper rear side of the cab 138. The front obstacle sensor 54a and the rear obstacle sensor 54b can detect detection objects within a 360-degree detection range around the work vehicle 10. It is also possible that the front camera 53a and the front obstacle sensor 54a are formed as one unit, and the rear camera 53b and the rear obstacle sensor 54b are formed as one unit. In addition, the obstacle sensor 54 may also include a left obstacle sensor that can detect detection objects within the detection range on the left side when viewed from the work vehicle 10, and a right obstacle sensor that can detect detection objects within the detection range on the right side when viewed from the work vehicle 10.

[0065] The obstacle sensor 54 measures the distance to each ranging point (measurement object) existing in the measurement range using, for example, laser (near-infrared laser, etc.), and generates a distance image based on the measurement information. The obstacle sensor 54 includes a processing unit composed of an integrated microcontroller, etc., and various control programs, etc., and is connected to the detection control unit 51, the vehicle control device 11, etc. via CAN in a manner that enables mutual communication.

[0066] The obstacle sensor 54 measures the distance from the obstacle sensor 54 to the ranging points within the measurement range by the TOF (Time Of Flight) method. The TOF method measures the distance to the ranging points based on the round-trip time from when the emitted laser reaches the ranging points and returns. The obstacle sensor 54 scans the laser horizontally and vertically at high speed throughout the entire measurement range, and sequentially measures the distances to the ranging points at each scan angle (coordinate), thereby performing three-dimensional measurement within the measurement range. The obstacle sensor 54 sequentially measures the intensity (reflection intensity) of the reflected light from each ranging point obtained when scanning the laser horizontally and vertically at high speed throughout the entire measurement range. The obstacle sensor 54 repeatedly measures the distances to the respective ranging points within the measurement range, the respective reflection intensities, etc. in real time.

[0067] The obstacle sensor 54 generates a distance image based on the measurement information such as the measured distances to the respective ranging points and the scan angles (coordinates) relative to the respective ranging points, extracts a group of ranging points inferred to be obstacles, and sends the measurement information related to the extracted group of ranging points to the detection control unit 51 as measurement information related to the obstacles.

[0068] In addition, a control range corresponding to the distance from the work vehicle 10 is set within the measurement range (detection range) of the obstacle sensor 54. For example, within the measurement range, the range where the distance from the work vehicle 10 is L1 is set as the stop control range, the range from the distance L1 to the distance L2 is set as the deceleration control range, and the range from the distance L2 to the distance L3 is set as the reporting control range (where L1 < L2 < L3). In addition, each control range can be set according to the type, model, work content, vehicle speed, etc. of the work vehicle 10. Further, the control range can also be switched according to the traveling direction of the work vehicle 10. For example, the detection control unit 51 sets the control range on the front side of the vehicle body when the work vehicle 10 is traveling forward, and sets the control range on the rear side of the vehicle body when the work vehicle 10 is traveling backward.

[0069] The detection control unit 51 outputs the measurement information obtained from the obstacle sensor 54 to the vehicle control device 11. During the automatic driving of the work vehicle 10, the detection control unit 51 sequentially outputs the measurement information to the vehicle control device 11 each time the measurement information is obtained from the obstacle sensor 54. As another embodiment, the detection control unit 51 may also determine the type (person, vehicle, structure, material, etc.) of the detection object (measurement object) based on the captured image obtained from the camera 53 and the measurement information obtained from the obstacle sensor 54, and output the determination result to the vehicle control device 11.

[0070] The vehicle control device 11 includes control devices such as a CPU, a ROM, and a RAM. The above-mentioned CPU is a processor that executes various arithmetic processes. The above-mentioned ROM is a non-volatile storage unit that pre-stores control programs such as BIOS and OS for causing the above-mentioned CPU to execute various arithmetic processes. The above-mentioned RAM is a volatile or non-volatile storage unit that stores various information and is used as a temporary storage memory for various processes executed by the above-mentioned CPU. Moreover, the vehicle control device 11 controls the work vehicle 10 by causing the above-mentioned CPU to execute various control programs pre-stored in the above-mentioned ROM or the storage unit 12.

[0071] Specifically, as Figure 1 shown, the vehicle control device 11 includes various processing units such as a travel processing unit 111. In addition, the vehicle control device 11 functions as the above-mentioned various processing units by causing the above-mentioned CPU to execute various processes in accordance with the above-mentioned automatic driving program. Further, a part or all of the above-mentioned processing units may be constituted by electronic circuits. In addition, the above-mentioned automatic driving program may also be a program for causing a plurality of processors to function as the above-mentioned processing units.

[0072] The travel processing unit 111 controls the travel of the work vehicle 10. For example, when the travel mode of the work vehicle 10 is autonomous driving (autonomous driving mode), the travel processing unit 111 causes the work vehicle 10 to perform autonomous driving based on the position information (positioning information) indicating the current position of the work vehicle 10 located by the positioning unit 17. For example, if the work vehicle 10 satisfies the start condition for autonomous driving and receives a travel start instruction from the operator, the travel processing unit 111 causes the work vehicle 10 to start autonomous driving based on the above positioning information. For example, the travel processing unit 111 causes the work vehicle 10 to autonomously travel from the travel start position to the travel end position according to a target path pre-generated and set in the operation terminal 20.

[0073] In addition, when the travel mode of the work vehicle 10 is manual driving (manual driving mode), the work vehicle 10 can be manually driven based on the operation of the operator (manual steering operation). For example, the travel processing unit 111 obtains operation information corresponding to driving operations such as a steering wheel operation, a gear shift operation, a travel direction switching operation, and a brake operation performed by the operator, and causes the travel device 13 to perform a travel action based on this operation information.

[0074] In addition, when the work vehicle 10 detects an obstacle during autonomous driving, the travel processing unit 111 causes the work vehicle 10 to perform a prescribed response process. Specifically, when the travel processing unit 111 detects an obstacle within the above report control range, it causes the work vehicle 10 to report a warning sound. In addition, when an obstacle is detected within the above deceleration control range, the travel processing unit 111 causes the work vehicle 10 in autonomous driving to decelerate from a pre-set vehicle speed. In addition, when an obstacle is detected within the above stop control range, the travel processing unit 111 causes the work vehicle 10 in autonomous driving to temporarily stop (park). In addition, the travel processing unit 111 may also cause the work vehicle 10 to perform a response process corresponding to the type of detection target detected by the obstacle detection device 15.

[0075] In addition, the travel processing unit 111 controls the travel according to an instruction obtained from the operation terminal 20. For example, if a travel start instruction is obtained from the operation terminal 20, the travel processing unit 111 causes the autonomous driving of the work vehicle 10 to start, and if a travel stop instruction is obtained from the operation terminal 20, the travel processing unit 111 causes the autonomous driving of the work vehicle 10 to stop.

[0076] [Operation terminal 20]

[0077] As Figure 1 shown, the operation terminal 20 is an information processing device including an operation control unit 21, a storage unit 22, an operation display unit 23, a communication unit 24, etc. For example, the operation terminal 20 is a smart phone (refer to Figure 2It consists of mobile terminals such as smart phones, tablet terminals, etc. In addition, the operation terminal 20 can also be a fixed (desktop) personal computer.

[0078] The communication unit 24 is a communication interface for connecting the operation terminal 20 to the communication network N1 by wire or wirelessly and performing data communication compliant with a specified communication protocol with external devices such as one or more work vehicles 10 via the communication network N1.

[0079] The operation display unit 23 is a user interface having a display unit such as a liquid crystal display or an organic EL display for displaying various information, and an operation unit such as a touch panel, a mouse, or a keyboard for accepting operations. The above display unit displays a map of a specified area of the field including the work object, specified information related to the work vehicle 10, etc. The above specified information includes at least any one of information related to the current surrounding environment of the work vehicle 10, information related to the current driving condition of the work vehicle 10, and information related to the current work condition of the work vehicle 10. An operator can, at a location away from the work vehicle 10, grasp the driving state of the work vehicle 10 automatically driving in the field, road, etc., the detection result of obstacles, etc. based on the above specified information displayed on the operation terminal 20. That is, the operator can remotely monitor the work vehicle 10 using the operation terminal 20.

[0080] In addition, the above display unit displays an operation screen for giving an instruction to start driving, an instruction to stop driving, an instruction to resume driving, etc. to the work vehicle 10. In addition, the above display unit displays a registration screen for registering work vehicle information, field information, work information, etc.

[0081] The storage unit 22 is a non-volatile storage unit such as an HDD or an SSD for storing various information. The storage unit 22 stores control programs such as a display control program for causing the operation control unit 21 to execute the display control process described later (refer to Figure 8 ). For example, the above control program is non-temporarily recorded on a computer-readable recording medium such as a CD or a DVD, and is read by a specified reading device (not shown) and stored in the storage unit 22. In addition, the above control program can also be downloaded from a server (not shown) to the operation terminal 20 via the communication network N1 and stored in the storage unit 22.

[0082] In addition, the storage unit 22 stores data such as information related to the work vehicle 10, namely work vehicle information, and information related to the target path, namely target path information. The above-mentioned work vehicle information includes information such as vehicle number and model in units of the work vehicle 10. The above-mentioned vehicle number is the identification information of the work vehicle 10. The above-mentioned model is the model of the work vehicle 10. In addition, the storage unit 22 may store the above-mentioned work vehicle information related to one work vehicle 10, or may store the above-mentioned work vehicle information related to multiple work vehicles 10. In the present embodiment, the storage unit 22 stores the above-mentioned work vehicle information related to each of the two work vehicles 10a and 10b.

[0083] The above-mentioned target path information includes information such as path name, field name, address, field area, working time, etc. in units of the target path. The above-mentioned path name is the path name of the target path generated in the operation terminal 20. The above-mentioned field name is the name of the field that is the work object where the above-mentioned target path is set. The above-mentioned address is the address of the above-mentioned field, and the above-mentioned field area is the area of the above-mentioned field. The above-mentioned working time is the time required for the work vehicle 10 to work on the above-mentioned field.

[0084] When the above-mentioned target path is a path corresponding to a road (inter-field path), the above-mentioned target path information includes information such as path name, address, driving distance, driving time, etc. The above-mentioned path name is the name of the road, and the above-mentioned address is the address of the road. The above-mentioned driving distance is the distance that the work vehicle 10 travels on the road, for example, the distance from the first field to the second field. The above-mentioned driving time is the time that the work vehicle 10 travels on the road, for example, the time required for the movement from the first field to the second field.

[0085] In addition, the storage unit 22 may store the above-mentioned target path information related to one target path, or may store the above-mentioned target path information related to multiple target paths. For example, when the operator generates multiple target paths for one or more fields under his own management, the storage unit 22 stores the above-mentioned target path information related to each target path. In addition, one target path may be set for one field, or multiple target paths may be set. In addition, for a group of fields, one inter-field path may be set, or multiple inter-field paths may be set. In the present embodiment, the storage unit 22 stores the target path information corresponding to the target path on which the work vehicle 10a travels in the first field, and the target path information corresponding to the target path on which the work vehicle 10b travels in the second field.

[0086] As other embodiments, part or all of the above-mentioned information such as the work vehicle information and the target path information may also be stored in a server accessible from the operation terminal 20. The operator may also perform operations of registering the work vehicle information and the target path information in the above-mentioned server (such as a personal computer, a cloud server, etc.). In this case, the operation control unit 21 may also obtain the above-mentioned information from the server to execute each process.

[0087] The operation control unit 21 includes control devices such as a CPU, a ROM, and a RAM. The above-mentioned CPU is a processor that executes various arithmetic processes. The above-mentioned ROM is a non-volatile storage unit that pre-stores control programs such as BIOS and OS for causing the above-mentioned CPU to execute various arithmetic processes. The above-mentioned RAM is a volatile or non-volatile storage unit that stores various information and is used as a temporary storage memory (working area) for various processes executed by the above-mentioned CPU. Moreover, the operation control unit 21 controls the operation terminal 20 by causing the above-mentioned CPU to execute various control programs pre-stored in the above-mentioned ROM or the storage unit 22.

[0088] As Figure 1 shown, the operation control unit 21 includes various processing units such as a setting processing unit 211, a display processing unit 212, and a reception processing unit 213. In addition, the operation control unit 21 functions as the above-mentioned various processing units by causing the above-mentioned CPU to execute various processes in accordance with the above-mentioned control programs. In addition, part or all of the above-mentioned processing units may also be constituted by electronic circuits. In addition, the above-mentioned control program may also be a program for causing a plurality of processors to function as the above-mentioned processing units.

[0089] The setting processing unit 211 sets information related to the work vehicle 10 (hereinafter referred to as work vehicle information), information related to the farmland (hereinafter referred to as farmland information), and information related to how to perform the work specifically (hereinafter referred to as work information).

[0090] Specifically, for information such as the model of the work vehicle 10, the position where the positioning antenna 174 is installed in the work vehicle 10, the type of the work implement 14, the size and shape of the work implement 14, the position of the work implement 14 relative to the work vehicle 10, the vehicle speed and engine speed during the operation of the work vehicle 10, and the vehicle speed and engine speed during the turning of the work vehicle 10, the setting processing unit 211 sets this information through the operation of the operator registering in the operation terminal 20.

[0091] In addition, for information such as the position and shape of the farmland, the work start position (travel start position) where the work starts, and the work end position (travel end position) where the work ends, and the work direction, the setting processing unit 211 sets this information through the operation of the operator registering in the operation terminal 20.

[0092] Information on the position and shape of the field can be automatically obtained, for example, by an operator driving the work vehicle 10 in a way that goes around the outer perimeter of the field and recording the change in the position information of the positioning antenna 174 at this time. Additionally, the position and shape of the field can also be obtained based on a polygon formed by an operator specifying multiple points on the map by operating the operation terminal 20 while a map is displayed on the operation terminal 20.

[0093] Furthermore, as operation information, the setting processing unit 211 is configured to be able to set the presence or absence of cooperative operation between the work vehicle 10 (unmanned tractor) and a manned work vehicle 10, the number of work paths skipped when the work vehicle 10 makes a turn at the headland, i.e., the skip count, the width of the headland, and the width of the non-cultivated land, etc.

[0094] Moreover, the setting processing unit 211 generates a target path for the work vehicle 10 to automatically travel in the field based on the above various setting information. Specifically, the setting processing unit 211 generates a target path within the field based on the registered travel start position and travel end position through field setting. For example, the setting processing unit 211 generates a target path including a travel start position, a travel end position, a straight path, and a turning path based on the setting operation of the operator. The setting processing unit 211 associates the generated target path with the field and registers it.

[0095] The display processing unit 212 displays various information on the operation display unit 23. Specifically, the display processing unit 212 displays the pre-registered work vehicle 10 in a manner that can be selected by the operator on the selection screen P1 (the first screen of the present invention). For example, when the operator has registered the work vehicles 10a and 10b to be monitored (managed), the display processing unit 212 makes the work vehicles 10a and 10b displayable for selection on the selection screen P1. The work vehicle 10 displayed on the selection screen P1 can be a work vehicle 10 for which information has been registered in the operation terminal 20, or a work vehicle 10 selected by the operator from among the multiple registered work vehicles 10 for starting work from now on.

[0096] Figure 3 An example of the selection screen P1 is shown. As Figure 3 shown, the display processing unit 212 displays a map of a specified area including the current positions of the pre-registered work vehicles 10a and 10b on the selection screen P1. Additionally, the display processing unit 212 displays the identification information of the work vehicle 10 at a position corresponding to the current position of the work vehicle 10 in the above map. Specifically, as Figure 3As shown, the display processing unit 212 causes the icon image A1 of the work vehicle 10a to be displayed at a position corresponding to the current position of the work vehicle 10a in the above map, and causes the icon image B1 of the work vehicle 10b to be displayed at a position corresponding to the current position of the work vehicle 10b. The display processing unit 212 acquires position information (positioning information) from the work vehicle 10 and causes the icon image of the work vehicle 10 to be displayed on the map. The display processing unit 212 updates the position of the icon image on the map in real time. As another embodiment, the display processing unit 212 may cause the name (text information) of the work vehicle 10a to be displayed on the map instead of the icon image.

[0097] In addition, the display processing unit 212 may also cause the icon image A1 of the work vehicle 10a and the icon image B1 of the work vehicle 10b to be displayed in a recognizable manner. For example, the display processing unit 212 may cause the name of the work vehicle 10a to be displayed near the icon image A1, and the name of the work vehicle 10b to be displayed near the icon image B1. In addition, the display processing unit 212 may cause the identification information (name, etc.) of the work vehicle 10a to be displayed when the operator performs a touch operation or a mouse hover operation on the icon image A1, and cause the identification information (name, etc.) of the work vehicle 10b to be displayed when the operator performs a touch operation or a mouse hover operation on the icon image B1.

[0098] In addition, the display processing unit 212 may also cause the above map to be enlarged or reduced in accordance with an operation of enlarging or reducing the above map on the selection screen P1. In addition, the display processing unit 212 may automatically adjust the display magnification according to the position of the work vehicle 10 to be monitored. For example, the display processing unit 212 determines the display magnification of the above map in such a manner that the icon images of all the work vehicles 10 to be monitored converge on one screen (the same page).

[0099] According to Figure 3 the shown selection screen P1, the operator can grasp the current positions of all the work vehicles 10 to be monitored. In addition, the display processing unit 212 may be configured to also cause the work vehicle 10 that is not in operation and is stored in a storage location (such as a warehouse) to be displayed on the selection screen P1, or may be configured not to display the work vehicle 10 that is not in operation and is stored in a storage location on the selection screen P1. In addition, it may be possible for the operator to set whether or not to display the work vehicle 10 that is not in operation and is stored in a storage location on the selection screen P1.

[0100] In addition, the display processing unit 212 causes the identification information to be displayed in a display form corresponding to the current state of the work vehicle 10 on the above-mentioned map. Specifically, the display processing unit 212 causes the icon image to be displayed on the above-mentioned map in such a way that it can be recognized whether the work vehicle 10 is in autonomous driving or stopped. That is, the display processing unit 212 makes the display forms different when the work vehicle 10 is in autonomous driving and when the work vehicle 10 is parked on the above-mentioned map. In Figure 4 the example shown, when the work vehicle 10a is stopped, the display processing unit 212 causes the icon image A1 to be displayed in a form indicating that it is stopped. In addition, when the work vehicle 10b is in autonomous driving, the display processing unit 212 causes the icon image B1 to be displayed in a form indicating that it is in autonomous driving. In addition, the above-mentioned display form can be the type, color, thickness, etc. of the line of the frame image surrounding the icon image, or the color, size, display method (lighting, flashing), etc. of the icon image. For example Figure 4 as shown, the display processing unit 212 causes the frame image A2 of the icon image A1 and the frame image B2 of the icon image B1 to be displayed in mutually different display forms. In addition, the display form of each icon image can also be set and changed by the operator. For example, the operator can also set the frame image of the icon image indicating autonomous driving to the first display form and the frame image of the icon image indicating stopped to the second display form on the setting screen.

[0101] As another embodiment, the display processing unit 212 may also not display the frame image when the work vehicle 10 is normally in autonomous driving, and display the frame image when the work vehicle 10 has stopped autonomous driving.

[0102] In addition, the display processing unit 212 may also cause a message (text information) indicating a stopped state to be displayed near the icon image A1, and a message indicating an autonomous driving state to be displayed near the icon image B1. In addition, the display processing unit 212 may also cause a message indicating a stopped state to be displayed when the operator performs a touch operation or a mouse hover operation on the icon image A1, and cause a message indicating an autonomous driving state to be displayed when the operator performs a touch operation or a mouse hover on the icon image B1.

[0103] In addition, the work vehicle 10 stops autonomous driving, for example, when an obstacle is detected, when the work is completed, when the remaining amount of the spreading material or fuel is less than the specified amount, when the harvested amount of the harvested product is more than the specified amount, when a regeneration operation of the DPF (Diesel Particulate Filter) is required, when a replenishment operation of the urea water is required, etc.

[0104] As other embodiments, the display processing unit 212 may also display identification information that can identify the detection of an obstacle when an obstacle is detected in the selection screen P1. For example, when the work vehicle 10a detects an obstacle, the display processing unit 212 lights or blinks the icon image A1, or displays the frame image A2 of the icon image A1 in a specified color. In addition, the display processing unit 212 may also display the icon image A1 or the frame image A2 in a display form corresponding to the detection position (control range) when the work vehicle 10a detects an obstacle. For example, when the work vehicle 10a detects an obstacle within the above-mentioned reporting control range, the display processing unit 212 displays the frame image A2 in green; when the work vehicle 10a detects an obstacle within the above-mentioned deceleration control range, the display processing unit 212 displays the frame image A2 in yellow; when the work vehicle 10a detects an obstacle within the above-mentioned stop control range, the display processing unit 212 displays the frame image A2 in red.

[0105] In addition, the display processing unit 212 may also display identification information that can identify the operation status in the selection screen P1. For example, when the work vehicle 10a finishes the operation in the field of the operation target, the display processing unit 212 lights or blinks the icon image A1, and displays the frame image A2 of the icon image A1 in a specified color. Another example is that when the work vehicle 10a is a vehicle (sprayer, etc.) that sprays spraying materials (liquid medicine, water, etc.), the display processing unit 212 lights or blinks the icon image A1 and displays the frame image A2 of the icon image A1 in a specified color when the remaining amount of the spraying material in the work vehicle 10a is less than the specified amount, or when the replenishment timing of the spraying material arrives. Another example is that when the work vehicle 10a is a vehicle (combine harvester, etc.) that harvests the harvested product, the display processing unit 212 lights or blinks the icon image A1 and displays the frame image A2 of the icon image A1 in a specified color when the harvested amount of the harvested product is equal to or more than the specified amount, or when the discharge timing of the harvested product arrives.

[0106] In this way, when the work vehicle 10 stops the automatic driving, the display processing unit 212 can also display in the selection screen P1 in a manner that can identify the stop state of the work vehicle 10. When the work vehicle 10 stops the automatic driving, in the selection screen P1, the display processing unit 212 displays identification information that can determine the reason for the stop of the work vehicle 10.

[0107] According to Figure 4 the selection screen P1 shown, the operator can grasp the current state of each monitored work vehicle 10.

[0108] Here, the display processing unit 212 is in the selection screen P1 (refer to Figure 3 andFigure 4 ) causes the work vehicle 10 to be displayed in a selectable manner. Specifically, the display processing unit 212 causes the icon images A1 and B1 to be displayed in a selectable manner on the map of the selection screen P1. The acceptance processing unit 213 accepts the selection operation of the icon image from the operator on the selection screen P1. For example Figure 5 As shown, when the operator touches the icon image A1 of the work vehicle 10a displayed on the selection screen P1 with a finger, the acceptance processing unit 213 accepts this touch operation. For example, if the operator observes the selection screen P1 and knows that the work vehicle 10a has stopped automatic driving, in order to confirm the reason for the stop, the operator touches the icon image A1 of the work vehicle 10a.

[0109] The display processing unit 212 causes the specified information related to the work vehicle 10 selected by the operator to be displayed on the operation display unit 23. Specifically, the display processing unit 212 causes the above-mentioned specified information to be displayed on the information display screen P2 (the second screen of the present invention). The above-mentioned specified information includes at least any one of information related to the current surrounding environment of the work vehicle 10, information related to the current driving status of the work vehicle 10, and information related to the current work status of the work vehicle 10. For example, when the operator selects the icon image A1 of the work vehicle 10a (refer to Figure 5 ), as Figure 6A shown, the display processing unit 212 causes the captured image around the work vehicle 10a captured by the camera 53 mounted on the work vehicle 10a to be displayed on the information display screen P2. Here, the display processing unit 212 obtains the front image, rear image, left image, and right image of the work vehicle 10a from the camera 53 and displays them on the information display screen P2. In addition, as Figure 6A shown, the display processing unit 212 causes the front image to be displayed on the upper side of the information display screen P2, and causes the rear image, left image, and right image to be displayed on the lower side of the information display screen P2. As another embodiment, as Figure 6B shown, the display processing unit 212 may also cause the front image to be displayed on the lower side of the information display screen P2, and cause the rear image, left image, and right image to be displayed on the upper side of the information display screen P2. In addition, the configuration, size, display / non-display of each image can also be set (customized) by the operator.

[0110] In addition, when the work vehicle 10a detects an obstacle, the display processing unit 212 may also cause the captured image including the obstacle among the multiple captured images of the camera 53 to be displayed on the information display screen P2 in a recognizable manner. For example, when the obstacle sensor 54 of the work vehicle 10a detects an obstacle in front of the work vehicle 10a, as Figure 6AAs shown, the display processing unit 212 emphasizes and displays the front image on the information display screen P2.

[0111] In this way, the display processing unit 212 displays the icon images of the multiple pre-registered work vehicles 10 in the map of the selection screen P1 in a display form corresponding to the current state of the work vehicle 10 (see Figure 5 ), and displays the captured image around the work vehicle 10 captured by the camera 53 mounted on the work vehicle 10 corresponding to the icon image selected by the operator in the selection screen P1 among the multiple icon images on the information display screen P2 (see Figure 6A ).

[0112] In addition, the display processing unit 212 displays switching buttons ("Next" button, "Return" button) for switching display images on the information display screen P2. If the operator presses the "Next" button once, the display processing unit 212 enlarges and displays the front image (see Figure 6C ). Then, if the operator presses the "Next" button once, the display processing unit 212 enlarges and displays the right image. Then, if the operator presses the "Next" button once, the display processing unit 212 enlarges and displays the rear image. Then, if the operator presses the "Next" button once, the display processing unit 212 enlarges and displays the left image. Each time the operator presses the "Next" button, the display processing unit 212 sequentially switches the front image, the right image, the rear image, and the left image. In addition, each time the operator presses the "Return" button, the display processing unit 212 sequentially switches the front image, the left image, the rear image, and the right image. In addition, if the operator presses Figure 6C the "4 Cameras" button on the information display screen P2 shown, the display processing unit 212 displays the captured images in four directions (front image, rear image, left image, and right image) on the information display screen P2 (see Figure 6A ).

[0113] In addition, the display processing unit 212 can also change (zoom) the display magnification (field of view size) of the captured image on the information display screen P2 shown in Figures 6A - 6C . For example, on the information display screen P2 shown in Figure 6C , if the operator performs a zoom-in operation (enlarge), the display processing unit 212 enlarges and displays the front image (move closer), and if the operator performs a zoom-out operation (reduce), the display processing unit 212 reduces and displays the front image (move farther away). In addition, the display processing unit 212 can also change the shooting range (shooting direction) of the camera 53 according to the operator's operation.

[0114] In addition, the display processing unit 212 can also emphasize and display the detected obstacle by surrounding it with a frame image in the captured image, or track the obstacle. In addition, the display processing unit 212 can also display the distance information to the detected obstacle and the information on the type of the obstacle (such as a person, a vehicle, a structure, a material, etc.) on the information display screen P2. In addition, the camera 53 can also be configured to include a part of the vehicle body of the work vehicle 10 so as to easily grasp the sense of distance to the detected obstacle.

[0115] In addition, the display processing unit 212 displays a "start" button and a "stop" button on the information display screen P2. When the operator presses the "start" button, the reception processing unit 213 receives this operation and outputs an instruction to the work vehicle 10 to start or restart the automatic driving. For example, when the work vehicle 10a detects an obstacle and stops the automatic driving, if the operator selects the work vehicle 10a on the selection screen P1 (refer to Figure 5 ), and confirms the safety of the work vehicle 10a on the information display screen P2, then presses the "start" button. Thereby, the reception processing unit 213 outputs an instruction to the work vehicle 10a to restart the automatic driving, and the work vehicle 10a restarts the automatic driving according to the above instruction. As another embodiment, the reception processing unit 213 can also allow the pressing operation of the "start" button on the condition that no obstacle is detected any more.

[0116] In addition, for example, during the automatic driving of the work vehicle 10b, if the operator selects the work vehicle 10b on the selection screen P1 and presses the "stop" button on the information display screen P2, the reception processing unit 213 outputs an instruction to the work vehicle 10b to stop the automatic driving. Thereby, the work vehicle 10b stops the automatic driving according to the above instruction.

[0117] In addition, if the operator presses the "map" button on the information display screen P2, the display processing unit 212 switches to the selection screen P1 (refer to Figure 4 ). In this way, the operator can switch between the selection screen P1 and the information display screen P2.

[0118] As another embodiment, the display processing unit 212 can also arrange and display the selection screen P1 and the information display screen P2 in the left - right direction or the up - down direction. For example Figure 7 as shown, the display processing unit 212 displays the selection screen P1 in the display area on the left side of the operation display unit 23, and displays the information display screen P2 corresponding to the work vehicle 10a selected on the selection screen P1 in the display area on the right side. In addition, if the operator presses the "map" button on the information display screen P2 shown in Figure 7 , the display processing unit 212 enlarges and displays the selection screen P1 (refer to Figure 5), if the operator presses the "Camera" button, the display processing unit 212 enlarges and displays the information display screen P2 (see Figure 6A ). In addition, the display processing unit 212 may also display the front image on the lower side of the information display screen P2, and display the rear image, the left image, and the right image on the upper side of the information display screen P2. Additionally, the configuration, size, display / non-display of each image may be set (customized) by the operator.

[0119] In addition, the operation control unit 21 gives an automatic driving instruction to the work vehicle 10. For example, if the operator selects a field and presses a start button (not shown) when starting the automatic driving of the work vehicle 10, the operation control unit 21 outputs path data of the target path to the work vehicle 10. When the path data generated in the operation terminal 20 is transferred to the work vehicle 10, the work vehicle 10 stores it in the storage unit 12. The work vehicle 10 performs automatic driving processing based on the above path data while detecting the current position of the work vehicle 10 through the positioning antenna 174. In addition, the current position of the work vehicle 10 generally coincides with the position of the positioning antenna 174.

[0120] In addition, the work vehicle 10 is configured to be able to perform automatic driving when the current position coincides with the driving start position in the field or is within a specified range (a situation where the start condition for automatic driving is satisfied). If the work vehicle 10 satisfies the above start condition and the start button is pressed by the operator on the operation screen (not shown) to give a driving start instruction, the automatic driving of the target path starts through the driving processing unit 111 of the work vehicle 10.

[0121] In addition, the setting process of the above setting processing unit 211 and the process of starting automatic driving by the operation control unit 21 may also be executed by a device different from the operation terminal 20. For example, on a tablet terminal different from the operation terminal 20, the operator performs the setting operations of the above various setting information, the generation operation of the target path, the automatic driving instruction, etc. In this case, the operation terminal 20 may only have a monitoring function for monitoring the work vehicle 10, and the tablet terminal performs the setting process of the above various setting information, the generation process of the target path, the automatic driving process, etc. Additionally, the operation terminal 20 according to the present embodiment may be implemented by adding the above monitoring function to the above tablet terminal. As another embodiment, the operation control unit 21 may also display the selection screen P1 and the information display screen P2 on different devices respectively. For example, the operation control unit 21 may display the selection screen P1 on the operation terminal 20 and display the information display screen P2 on a device different from the operation terminal 20 (the above tablet terminal, the mobile terminal of the operator or worker, etc.).

[0122] In addition, the operation terminal 20 may also be able to access the website (agricultural support website) of the agricultural support service provided by a server (not shown) via the communication network N1. In this case, the operation terminal 20 can function as an operation terminal for the above server by executing a browser program using the operation control unit 21. Moreover, the above server includes the above respective processing units and executes various processes.

[0123] [Display control processing]

[0124] Hereinafter, with reference to Figure 8 An example of the display control processing executed by the automatic driving system 1 will be described.

[0125] In addition, the present invention can be understood as an invention of a display control method (an example of the display control method of the present invention) that executes one or more steps included in the above display control processing. In addition, one or more steps included in the above display control processing described here may be appropriately omitted. In addition, for each step in the above display control processing, the execution order may be different within the range where the same effect is produced. And here, the case where the operation control unit 21 executes each step in the above display control processing is taken as an example for description, but as another embodiment, it may also be a display control method in which one or more processors separately execute each step in the above display control processing.

[0126] In step S1, the operation control unit 21 determines whether the position information of the work vehicle 10 has been acquired. If the operation control unit 21 acquires the position information of the current position from each registered work vehicle 10 that is the object to be monitored (S1: Yes), the process proceeds to step S2. The operation control unit 21 stands by until the position information of each work vehicle 10 is acquired (S1: No).

[0127] In step S2, the operation control unit 21 displays a map of a specified area including the current position of the work vehicle 10 on the selection screen P1 of the operation display unit 23, and in the map, displays an icon image (vehicle icon) of the work vehicle 10 at a position corresponding to the current position of the work vehicle 10. For example, in the case where the work vehicle 10a and the work vehicle 10b are registered, as Figure 3 shown, the operation control unit 21 displays a map of a specified area including the current positions of the work vehicle 10a and the work vehicle 10b on the selection screen P1, and in the map, displays the icon image A1 of the work vehicle 10a at a position corresponding to the current position of the work vehicle 10a, and displays the icon image B1 of the work vehicle 10b at a position corresponding to the current position of the work vehicle 10b.

[0128] Next, in step S3, the operation control unit 21 determines the current state of the work vehicle 10. Specifically, the operation control unit 21 determines whether the work vehicle 10 is performing autonomous driving or has stopped autonomous driving. As another embodiment, the operation control unit 21 may also determine whether the work vehicle 10 has detected an obstacle, whether the operation has ended, whether the remaining amount of the spreading material or fuel is less than the specified amount, whether the harvested amount of the harvested product is equal to or more than the specified amount, whether a DPF regeneration operation is required, whether a urea water replenishment operation is required, etc.

[0129] Next, in step S4, the operation control unit 21 causes the identification information to be displayed in a display form corresponding to the current state of the work vehicle 10. Specifically, the operation control unit 21 causes the icon image to be displayed on the above map in a manner that enables identification of whether the work vehicle 10 is in autonomous driving or has stopped autonomous driving. For example Figure 4 As shown, when the work vehicle 10a has stopped autonomous driving and the work vehicle 10b is in autonomous driving, the operation control unit 21 surrounds the icon image A1 with a frame image A2 in the first display form (e.g., a red frame image), and surrounds the icon image B1 with a frame image B2 in the second display form (e.g., a blue frame image). In addition, the operation control unit 21 may also cause the frame image not to be displayed when the work vehicle 10 is performing autonomous driving normally.

[0130] As another embodiment, the operation control unit 21 may also cause the display form of the frame image to be different according to the reason for the stop of the work vehicle 10. For example, when the work vehicle 10a has stopped autonomous driving due to detecting an obstacle, the operation control unit 21 causes the frame image A2 to be lit in red. In contrast, when the work vehicle 10a has stopped autonomous driving due to the end of the operation, the operation control unit 21 causes the frame image A2 to flash in red.

[0131] Next, in step S5, the operation control unit 21 determines whether a selection operation of the work vehicle 10 is received from the operator on the selection screen P1. Specifically, in Figure 4 As shown in the selection screen P1, the operation control unit 21 receives an operation of selecting the icon image of the work vehicle 10 from the operator. For example Figure 5 As shown, when the operator touches the icon image A1 displayed on the selection screen P1 with a finger, the operation control unit 21 receives the touch operation. If the operation control unit 21 receives a selection operation of the work vehicle 10 from the operator (S5: Yes), the process proceeds to step S6. On the other hand, when the operation control unit 21 does not receive a selection operation of the work vehicle 10 from the operator (S5: No), the process proceeds to step S7.

[0132] Next, in step S6, the operation control unit 21 causes specified information related to the work vehicle 10 selected by the operator to be displayed on the operation display unit 23. Specifically, the operation control unit 21 causes information related to the current surrounding environment of the work vehicle 10, information related to the current driving status of the work vehicle 10, information related to the current work status of the work vehicle 10, etc. to be displayed on the information display screen P2. For example, when the operator selects the icon image A1 of the work vehicle 10a (refer to Figure 5 ), as shown in Figure 6A , the operation control unit 21 causes captured images (front image, rear image, left image, and right image) of the surroundings of the work vehicle 10a captured by the camera 53 mounted on the work vehicle 10a to be displayed on the information display screen P2.

[0133] The operation control unit 21 can cause the information display screen P2 to be displayed on a page different from the selection screen P1 (refer to Figure 5 ), or can cause the information display screen P2 and the selection screen P1 to be displayed in a split-screen manner on the same page (refer to Figure 6A ). Figure 7 ).

[0134] In addition, the operation control unit 21 receives from the operator an operation to switch the captured images (front image, rear image, left image, and right image) (refer to Figures 6A - 6C ), an operation to start, restart, or stop the automatic driving, an operation to switch to the selection screen P1, etc. in the information display screen P2, and switches the display screen according to this operation.

[0135] In step S7, the operation control unit 21 determines whether an end operation has been received from the operator. For example, when the operator performs an end operation (such as a logout operation) to end the monitoring of the work vehicle 10, the operation control unit 21 receives the end operation (S7: Yes) and ends the above display control process. When the operation control unit 21 has not received an end operation from the operator (S7: No), it returns the process to step S1 and executes the above process. The operation control unit 21 repeats the processes of steps S1 to S6 until an end operation is received from the operator. In addition, during the period when the operation control unit 21 has not received a selection operation of the work vehicle 10 in the selection screen P1 (S5: No), the operation control unit 21 updates the position of the icon image in real time according to the current position of the work vehicle 10 in the selection screen P1. In this way, the operation control unit 21 executes the above display control process.

[0136] As described above, the operation terminal 20 according to the present embodiment controls the display information of the operation terminal 20. Specifically, the operation terminal 20 causes the pre-registered work vehicle 10 to be displayed on the selection screen P1 (first screen) of the operation terminal 20 in a manner selectable by the user (operator), and causes the specified information related to the work vehicle 10 selected by the user to be displayed on the information display screen P2 (second screen) of the operation terminal 20. For example, the operation terminal 20 causes at least any one of the information related to the current surrounding environment of the work vehicle 10, the information related to the current driving condition of the work vehicle 10, and the information related to the current working condition of the work vehicle 10 to be displayed on the information display screen P2.

[0137] According to the above structure, the operator can grasp the registered work vehicle 10 on the selection screen P1. In addition, the operator can confirm the information related to the selected work vehicle 10 on the selection screen P1. Thus, the operator can remotely monitor the work vehicle 10. Therefore, the convenience of the operation terminal 20 that displays the information related to the work vehicle 10 capable of autonomous driving can be improved.

[0138] In addition, the operation terminal 20 may display the operation unit capable of setting operations on the information display screen P2, and not only display the information related to the current driving condition (vehicle speed, engine speed, etc.) of the work vehicle 10. For example, the operation terminal 20 may display the above operation unit together with the camera image on the information display screen P2, or may display the above operation unit on another screen (a setting screen dedicated to the operation unit) switched from the camera image. In addition, in the structure where the above operation unit is displayed together with the camera image, it may be possible for the operator to set the displayed operation unit such as only displaying the vehicle speed operation unit or only displaying the engine speed operation unit. According to the structure in which the operator can set the operation unit, for example, when the operation terminal 20 is implemented by a mobile terminal (such as a smart phone), it is possible to prevent the deterioration of operability and visibility due to the increase in the operation unit.

[0139] In addition, the operation terminal 20 can also display the type of operation (type of the work machine 14) and the information of the target ground height of the work machine 14 on the information display screen P2, and display the operation unit for setting the target ground height. For example, the operation terminal 20 can also display the above operation unit together with the camera image on the information display screen P2, or can display the above operation unit on another screen (a dedicated setting screen for the operation unit) switched from the camera image. In addition, in the structure where the above operation unit is displayed together with the camera image, it is also possible for the operator to set the displayed operation unit including the operation unit in the driving state. According to the structure in which the operator can set the operation unit, for example, when the operation terminal 20 is implemented by a mobile terminal (such as a smart phone), it is possible to prevent the decrease in operability and visibility accompanying the increase in the operation unit.

[0140] [Other Embodiments]

[0141] The present invention is not limited to the above embodiments, and may also be the following embodiments.

[0142] For example, when the operator selects the work vehicle 10 on the selection screen P1, the operation control unit 21 can also display the information related to the selected work vehicle 10 within the selection screen P1. For example Figure 9 As shown, when the operator selects the icon image A1 of the work vehicle 10a on the selection screen P1, the operation control unit 21 causes the information display screen P2 including the current state of the work vehicle 10a, such as stopping the automatic driving due to detecting an obstacle, the name of the work vehicle 10a, the work content, the progress of the work, etc., to be displayed (pop-up display) near the icon image A1. For example, the operation control unit 21 can also be configured to confirm the progress by displaying a screen including the set path and the icon image (vehicle icon) on the path in the information display screen P2. In addition, the operation control unit 21 can also display a screen including the camera image, the above set path, and the icon image on the path, or can switch the above camera image to display a screen including the above set path and the icon image on the path.

[0143] In addition, for example Figure 10 As shown, when the operator selects the icon image B1 of the work vehicle 10b on the selection screen P1, the operation control unit 21 causes the information display screen P2 including the current state of the work vehicle 10b, such as being in the automatic driving state, the name of the work vehicle 10b, the work content, the progress of the work, etc., to be displayed (pop-up display) near the icon image B1.

[0144] In addition, as Figure 9 and Figure 10As shown, the operation control unit 21 can also cause the "Camera Image" button to be displayed on the information display screen P2. When the operator presses the "Camera Image" button, the operation control unit 21 switches to the information display screen P2 that displays the captured image of the camera 53 (refer to Figure 6A ).

[0145] Alternatively, when the operator briefly touches (short presses) the icon image of the work vehicle 10 once on the selection screen P1, the operation control unit 21 causes the information display screen P2 to pop up and be displayed on the selection screen P1 (refer to Figure 9 and Figure 10 ). When the operator touches (long presses) the icon image of the work vehicle 10 once for a long time on the selection screen P1, the operation control unit 21 switches from the selection screen P1 to the information display screen P2 (refer to Figure 6A ).

[0146] In the above-described embodiment, when the operator selects the work vehicle 10 on the selection screen P1 (refer to Figure 5 ), the operation control unit 21 causes the specified information related to the selected work vehicle 10 to be displayed on the information display screen P2. As another embodiment, the operation control unit 21 can also cause the specified information related to the work vehicle 10 to be displayed on the information display screen P2 when the registered work vehicle 10 moves to a specified state. Specifically, when the work vehicle 10 in autonomous driving detects an obstacle, or when the work vehicle 10 performs autonomous driving in a specific area, the operation control unit 21 causes the specified information related to the work vehicle 10 to be displayed on the information display screen P2. The above-mentioned specific area is, for example, the path between fields on the road.

[0147] For example, when the work vehicle 10a performs autonomous driving on the path between fields, the operator needs to monitor the driving status. Therefore, when the work vehicle 10a starts autonomous driving on the path between fields, even if the operator does not perform an operation of selecting the work vehicle 10a on the selection screen P1, the operation control unit 21 automatically switches the selection screen P1 to the information display screen P2 (refer to Figure 6A ).

[0148] In addition, when the work vehicle 10 detects an obstacle and stops, it is necessary to promptly notify the operator. Therefore, when the work vehicle 10a detects an obstacle within the stop control range, even if the operator has not selected the work vehicle 10a on the selection screen P1, the operation control unit 21 automatically switches the selection screen P1 to the information display screen P2. Further, in this configuration, the operation control unit 21 may also switch to the information display screen P2 on the condition that the operator has selected the work vehicle 10a on the selection screen P1 when the work vehicle 10a detects an obstacle within the reporting control range or the deceleration control range.

[0149] Furthermore, the configuration for automatically displaying the specified information related to the work vehicle 10 on the information display screen P2 is not limited to the case where the work vehicle 10 in automatic travel detects an obstacle or the case where the work vehicle 10 performs automatic travel in a specific area. For example, the operation control unit 21 may also automatically display the specified information related to the work vehicle 10 on the information display screen P2 in the case of specific operations such as replenishment of spreading material or fuel, discharge of harvested products, regeneration of the DPF, and replenishment of urea water.

[0150] In addition, the vehicle control device 11 may also permit the automatic travel of the field path of the work vehicle 10 when specified conditions are satisfied. For example, the vehicle control device 11 may permit the automatic travel of the field path only during the period when the operator directs his / her line of sight to the camera image on the information display screen P2. As another example, the vehicle control device 11 may permit the automatic travel of the field path on the condition that the battery remaining amount of the operation terminal 20 is equal to or more than a specified amount before starting the automatic travel of the field path. Thereby, it is possible to prevent the power supply of the operation terminal 20 from being cut off during the automatic travel of the work vehicle 10 along the field path and the operator being unable to remotely monitor the work vehicle 10.

[0151] As another example, the vehicle control device 11 may permit the automatic travel of the field path on the condition that a plurality of operation terminals 20 are connected. Thereby, it is possible to prevent the operator from being unable to remotely monitor the work vehicle 10 due to the power supply of one operation terminal 20 being turned off or a communication abnormality occurring during the automatic travel of the work vehicle 10 along the field path. Further, the vehicle control device 11 may also permit the automatic travel of the field path even when one operation terminal 20 is connected, but prompt connection to another operation terminal 20 when the connection of this operation terminal 20 is interrupted. In this case, the vehicle control device 11 stops the automatic travel of the field path until another operation terminal 20 is connected.

[0152] In addition, the vehicle control device 11 may also stop the automatic driving on the field path when the communication with the operation terminal 20 is interrupted, or when the application programs for displaying the selection screen P1 and the information display screen P2 in the operation terminal 20 are started or stopped, or enter the background state. In addition, the vehicle control device 11 may also automatically move the work vehicle 10 close to the road shoulder when stopping the automatic driving on the field path.

[0153] Here, for example, when the work vehicle 10 is automatically driving on the field path set on a road (such as a farm road), in order to ensure safety, an operation of always displaying the image around the work vehicle 10 on the operation terminal 20 can be considered. In this case, it is preferably configured that, for example, during the period when the work vehicle 10a is automatically driving on the field path, the operation control unit 21 always displays the captured image of the camera 53 of the work vehicle 10a on the information display screen P2 of the operation terminal 20 (refer to Figure 6A ). However, if this structure is adopted, for example, when the work vehicle 10a is automatically driving on the field path and another work vehicle 10 (such as the work vehicle 10b) stops after detecting an obstacle, the information related to the work vehicle 10b is not displayed on the operation terminal 20, so there may be a problem that the operator cannot grasp the stop state of the work vehicle 10b.

[0154] Therefore, as Figure 11 shown, the operation control unit 21 displays (pops up) the information (message M1) indicating that the work vehicle 10b has stopped after detecting an obstacle on the information display screen P2 that displays the captured image corresponding to the work vehicle 10a that is automatically driving on the field path. In addition, the operation control unit 21 displays a selection button for selecting whether to switch to the information display screen P2 that displays the information of the work vehicle 10b in the message M1, and accepts the selection operation from the operator. If the operator presses the "Yes" button of the message M1, the operation control unit 21 switches to the information display screen P2 that displays the information of the work vehicle 10b. In this case, since the captured image related to the work vehicle 10a that is automatically driving on the field path is no longer displayed, the operation control unit 21 outputs a stop instruction for the automatic driving to the work vehicle 10a. If the work vehicle 10a receives the stop instruction, it can temporarily stop on the spot, or drive to a specified place (road shoulder, shelter, etc.) and temporarily stop. In addition, the operator can also remotely operate the work vehicle 10a through the operation terminal 20 to move it to a specified place.

[0155] Thus, it is also possible that the operation control unit 21 causes the pre-registered work vehicle 10 to be displayed on the selection screen P1 (first screen) of the operation terminal 20, and when an obstacle is detected by the work vehicle 10 during autonomous driving, or when the work vehicle 10 performs autonomous driving in a specific area, the specified information related to the work vehicle 10 is displayed on the information display screen P2 (second screen) of the operation terminal 20.

[0156] As another embodiment of the present invention, the operation control unit 21 may also display on the selection screen P1 the identification information of the operator pre-registered for the work vehicle 10 in correspondence with the identification information of the work vehicle 10. For example, in the case where an operator X who manages and assists the operation of the work vehicle 10a and an operator Y who manages and assists the operation of the work vehicle 10b are registered, as Figure 12 shown, the operation control unit 21 displays on the map of the selection screen P1 the icon image C1 of the operator X in correspondence with the icon image A1 of the work vehicle 10a, and displays the icon image C2 of the operator Y in correspondence with the icon image B1 of the work vehicle 10b. The operation control unit 21 may also display the icon images C1 and C2 in different display forms from each other.

[0157] Thereby, the operator monitoring the work vehicle 10 can easily grasp the information that operators are registered for each work vehicle 10. In addition, the operation control unit 21 may also acquire the position information of the operator and display the icon image of the operator at the current position of the operator. For example, the operation control unit 21 may acquire the current positions of the operator X and the operator Y based on the position information of the operation terminals 20 held by the operator X and the operator Y, and may also acquire the current position of the operator X based on the position information of the remote controller for performing close-range operation on the work vehicle 10a, and acquire the current position of the operator Y based on the position information of the remote controller for performing close-range operation on the work vehicle 10b. In addition, the operation control unit 21 may be configured to display the position of the remote controller instead of the position of the operator.

[0158] In addition, it is also possible that when an operator (such as operators X and Y) different from the operator who remotely monitors the work vehicle 10 starts an application program that displays the selection screen P1 and the information display screen P2 on their own operation terminal 20 different from the operation terminal 20 of the operator, the operator can request the operator to transfer the operator's authority to their own operation terminal 20.

[0159] In addition, in Figure 12In the selection screen P1 shown, the operation control unit 21 can also accept operations on the icon images of the selecting operators. For example, if an operator selects the icon image C1 of operator X in the selection screen P1, then as Figure 13 shown, the operation control unit 21 displays the instruction screen P3 used to output a specified instruction to operator X. For example, if an operator presses the "Notify Operator" button in the instruction screen P3, the operation control unit 21 notifies the mobile terminal held by operator X of an obstacle confirmation instruction, an instruction to remove an obstacle, etc. As another embodiment, when an operator selects the icon image C1 of operator X in the selection screen P1, the operation control unit 21 can also start an application for sending and receiving messages with the mobile terminal of operator X and notify the mobile terminal of operator X of a message.

[0160] In addition, as another embodiment, when an operator rides on the work vehicle 10 and manually travels in the field or an operator rides on the work vehicle 10 and automatically travels based on the path in the field, or when an operator rides on the work vehicle 10 and manually travels to the storage location when finishing the day's work and organizing the work vehicle 10 to the storage location or an operator rides on the work vehicle 10 and automatically travels on the travel path to the storage location, based on the arrival time of the work end position of the work vehicle 10 and the position of the operator or the operator, corresponding to the arrival time of the work end position of the work vehicle 10, the operation terminal 20 is notified at the timing when the operator or the operator can reach the work end position.

[0161] In addition, as another embodiment, the operation control unit 21 can also display a list of operators near the work vehicle 10a and output a specified instruction to the operator selected by the operator from the list. In addition, as another embodiment, an operator can be registered for the work vehicle 10, and an operator can also be registered for the work plan of the day.

[0162] In the above embodiment, the operation control unit 21 displays the pre-registered work vehicle 10 in a selectable manner on the map of the selection screen P1 (refer to Figure 3 and Figure 4)。As other embodiments, the operation control unit 21 may also display a list of the pre-registered work vehicles 10 (registered vehicle list) on the selection screen P1. In addition, the operation control unit 21 may also have a filtering function for determining the work vehicles 10 to be displayed on the above map or the above list. For example, the items of the above filtering function include vehicles displayed on the map (within a specified scale), running vehicles, vehicles moving between fields, stopped vehicles, vehicles with problems, non-monitored vehicles not participating in this operation, vehicles stopped outside the field due to problems or detection of obstacles, vehicles that immediately need or will soon need to replenish materials (chemicals, fertilizers, seedlings, etc.) for the work machine, etc. The operation control unit 21 can display the vehicles of the set items on the above map or the above list, or display the vehicles other than the vehicles of the set items on the above map or the above list.

[0163] In addition, as other embodiments, the operation control unit 21 may also display the work vehicles 10 on the map in a selectable manner and display a list of selectable work vehicles 10 on the same page of the selection screen P1. In addition, as other embodiments, the operation control unit 21 may also display the work vehicles 10 on the map in a selectable manner and display the work vehicles 10 in a selectable manner in the list on the same page of the selection screen P1. When a work vehicle 10 is selected in the list, the map of the position of the selected work vehicle 10 is displayed.

[0164] In addition, the operation control unit 21 may also display the identification information in a display form corresponding to the current state of the work vehicle 10 in the above list. For example, when the work vehicle 10a stops autonomous driving, the operation control unit 21 highlights the name of the work vehicle 10a in the above list.

[0165] As described above, in the above embodiment, the operation terminal 20 corresponds to the display control system according to the present invention. However, the display control system according to the present invention may be constituted by the operation terminal 20 alone, or may be configured to include the work vehicle 10 and the operation terminal 20. For example, the display control system according to the present invention may also be the autonomous driving system 1.

[0166] [Remote operation system]

[0167] The autonomous driving system 1 may also construct a remote operation system 100. As Figure 14As shown in the figure, the remote operation system 100 is configured to include a work vehicle 10, an operation terminal 20, and a remote operation device 30. The work vehicle 10, the operation terminal 20, and the remote operation device 30 may each be one or multiple. In the present embodiment, an example in which the remote operation system 100 is composed of multiple work vehicles 10, one operation terminal 20, and one remote operation device 30 will be described.

[0168] The remote operation system 100 is a system in which a user (operator, monitor) operates (remotely drives) the remote operation device 30 at a remote location far from the field where the work vehicle 10 performs operations, causing the work vehicle 10 to travel. Figure 15 The remote operation device 30 and the situation of remote operation provided at a remote location (such as a monitoring room) are schematically shown. The operator can remotely drive the work vehicle 10 while observing the captured image taken by the camera 53 of the work vehicle 10 to be remotely operated in the monitoring room.

[0169] As Figure 14 shown, the remote operation device 30 is an information processing device having an operation control unit 31, a storage unit 32, a display unit 33, an operation unit 34, a communication unit 35, etc. For example, the remote operation device 30 may also be a fixed (desktop) personal computer.

[0170] The communication unit 35 is a communication interface for connecting the remote operation device 30 to the communication network N1 by wire or wirelessly and performing data communication compliant with a specified communication protocol with external devices such as the work vehicle 10 and the operation terminal 20 via the communication network N1.

[0171] The display unit 33 is a monitor such as a liquid crystal display or an organic EL display that displays various information. In addition, the display unit 33 may also be composed of a touch panel that accepts touch operations.

[0172] The operation unit 34 accepts various operations by the operator. Specifically, as Figure 16 shown, the operation unit 34 includes: a steering wheel 341 for steering the work vehicle 10; an accelerator pedal 342 for increasing the vehicle speed of the work vehicle 10; a brake pedal 343 for decelerating and stopping the work vehicle 10; a neutral pedal 344 for switching the gear position to the neutral position. In addition, a plurality of operation buttons (such as a mode switching button BT1) are arranged on the steering wheel 341. As another embodiment, the above-mentioned plurality of operation buttons may also be provided on a device other than the steering wheel 341. For example, an operation device (not shown) provided with the above-mentioned plurality of operation buttons may be arranged in the monitoring room. The above-mentioned operation device is included in the operation unit 34.

[0173] The storage unit 32 is a non-volatile storage unit such as an HDD or an SSD that stores various types of information. A control program for causing the operation control unit 31 to execute various processes is stored in the storage unit 32. For example, the above control program is non-temporarily recorded on a computer-readable recording medium such as a CD or a DVD, read by a prescribed reading device (not shown), and stored in the storage unit 32. Further, the above control program may be downloaded from a server (not shown) to the remote operation device 30 via the communication network N1 and stored in the storage unit 32.

[0174] The operation control unit 31 includes control devices such as a CPU, a ROM, and a RAM. The CPU is a processor that executes various arithmetic processes. The ROM is a non-volatile storage unit that pre-stores control programs such as a BIOS and an OS for causing the CPU to execute various arithmetic processes. The RAM is a volatile or non-volatile storage unit that stores various types of information and is used as a temporary storage memory (work area) for various processes executed by the CPU. Moreover, the operation control unit 31 controls the remote operation device 30 by causing the CPU to execute various control programs pre-stored in the ROM or the storage unit 32.

[0175] Specifically, the operation control unit 31 executes a process of switching to a remote operation mode in which an operator can remotely operate the work vehicle 10 via the remote operation device 30. For example, in a state where a camera image of the work vehicle 10 that can be remotely operated is displayed on the display unit 33, if the operator presses the mode switching button BT1 disposed on the steering wheel 341 (refer to Figure 16 ), the operation control unit 31 switches the traveling mode of the work vehicle 10 from an automatic traveling mode in which it can automatically travel without depending on the operation of the operator to a remote operation mode. In this way, the operation control unit 31 displays the captured image of the camera 53 mounted on the work vehicle 10 that is the remote operation target on the display unit 33, and allows the switching to the remote operation mode on the condition that the operator can confirm the situation around the work vehicle 10.

[0176] In addition, the operation control unit 31 may also cause a camera image to be displayed on the display unit 33 according to an instruction from the operation terminal 20. For example, the operator selects the work vehicle 10 to be remotely operated on a selection screen P1 (refer to Figure 3 ) displayed on the operation terminal 20. If the operator selects the work vehicle 10, then as Figure 17A shown, the operation terminal 20 causes an information display screen P2 to be displayed, and causes the camera image of the work vehicle 10 and a request button BT2 ("mode switching request" button) for requesting the switching of the traveling mode to the remote operation mode to be displayed on the information display screen P2. If the operator presses the request button BT2, then as Figure 17BAs shown, the operation terminal 20 causes the message M2 for reminding to press the mode switching button BT1 of the steering wheel 341 of the remote operation device 30 (refer to Figure 16 ) to be displayed (popped up) on the information display screen P2. If the operator presses the OK button of the message M2, the operation terminal 20 outputs mode switching information to the remote operation device 30. The above mode switching information includes the identification information of the work vehicle 10 of the remote operation object selected by the operator, etc.

[0177] If the operation control unit 31 of the remote operation device 30 obtains the above mode switching information from the operation terminal 20, then as Figure 18 shown, it causes the camera image of the work vehicle 10 of the remote operation object to be displayed on the display unit 33. Specifically, the operation control unit 31 causes the front image Pa, the rear image Pb, the left image Pc, and the right image Pd of the work vehicle 10 to be displayed. In addition, the camera image displayed on the remote operation device 30 may be the same as the camera image displayed on the operation terminal 20. If the operation control unit 31 causes the camera image of the work vehicle 10 to be displayed on the display unit 33, it permits the switching to the remote operation mode. Thus, for example, if the operator presses the mode switching button BT1 of the steering wheel 341 (refer to Figure 16 ), the operation control unit 31 accepts this operation and switches the driving mode from the automatic driving mode to the remote operation mode. In addition, when switching to the remote operation mode, the driving processing unit 111 of the work vehicle 10 may also cause the work vehicle 10 to temporarily stop.

[0178] If the driving mode is switched to the remote operation mode, the operation control unit 31 accepts the driving operation of the operator and outputs a driving instruction corresponding to this operation to the work vehicle 10. The driving processing unit 111 of the work vehicle 10 causes the work vehicle 10 to travel according to the above driving instruction. For example, if the operator turns the steering wheel 341 to the right, the driving processing unit 111 steers the work vehicle 10 to the right, and if the operator turns the steering wheel 341 to the left, the driving processing unit 111 steers the work vehicle 10 to the left. In addition, if the operator steps on the accelerator pedal 342, the driving processing unit 111 accelerates the work vehicle 10, and if the operator steps on the brake pedal 343, the driving processing unit 111 decelerates the work vehicle 10. Thus, the operator can remotely operate (remote drive) the work vehicle 10 from a remote location. In addition, for example, even in the case where the work vehicles 10 stop by recognizing each other as obstacles, the operator can make the work vehicle 10 perform an avoidance maneuver through remote operation. In addition, even during the remote monitoring of the work vehicle 10 by the operator, the operator can switch the driving mode to the remote operation mode from a remote location to perform remote operation.

[0179] In addition, in the remote operation mode, if the operator depresses the accelerator pedal 342, the travel processing unit 111 increases the vehicle speed according to the depression amount. Further, the travel processing unit 111 shifts the operation vehicle 10 according to the above depression amount within the set vehicle speed (maximum vehicle speed) set for the operation vehicle 10. For example, when the maximum vehicle speed during field work is set to 10 km, if the operator depresses the accelerator pedal 342 to the maximum, the travel processing unit 111 increases the vehicle speed to 10 km. When the maximum vehicle speed during travel between fields is set to 30 km, if the operator depresses the accelerator pedal 342 to the maximum, the travel processing unit 111 increases the vehicle speed to 30 km. Thus, according to the situation of traveling in the field and the situation of traveling outside the field, the setting of the maximum vehicle speed is different. However, since the vehicle speed is controlled by the accelerator opening (depression amount) relative to the maximum vehicle speed, it is easy for the operator to perform driving operations.

[0180] In addition, the operation control unit 31 may also display vehicle information such as the state of the operation vehicle 10, the engine speed, and the vehicle speed on the display unit 33 during the period when the remote operation mode is set. In addition, the operation control unit 31 may also output the traveling sound of the operation vehicle 10 from the remote operation device 30 during the period when the remote operation mode is set. For example, the operation control unit 31 may also acquire the sound collected by the microphone mounted on the operation vehicle 10 and output it from the speaker provided in the remote operation device 30 or the monitoring room. Generally, it is difficult to grasp the sense of speed and the state of the operation load in a silent state. According to the above structure, the operator can perform remote driving with a feeling close to actual driving by playing the operation sound.

[0181] In addition, the operation control unit 31 may also output an operation instruction corresponding to the operation of the operation button (refer to Figure 16 ) arranged on the steering wheel 341 to the operation vehicle 10 in the remote operation mode. For example, the above operation buttons include a button for switching the traveling direction of the operation vehicle 10, buttons for raising and lowering the working machine 14 (raising button, lowering button), a button for temporarily stopping the operation vehicle 10, and the like. For example, if the operator presses the lowering button, the vehicle control device 11 acquires the operation instruction from the remote operation device 30 and lowers the working machine 14.

[0182] The operation control unit 31 may also be able to set the functions assigned to the above respective operation buttons. For example, the operation control unit 31 causes a function setting screen (not shown) to be displayed on the display unit 33 and accepts an operation for the operator to select the functions assigned to the respective operation buttons. The operation control unit 31 assigns the above functions to the respective operation buttons according to the selection operation of the operator.

[0183] Here, when the operation control unit 31 displays the camera image on the display unit 33, it can also be displayed in such a way that a part of the vehicle body is included in each image. For example, the operation control unit 31 displays the front image Pa in such a way that a part of the hood of the work vehicle 10 and a part of each of the left and right wheels are included, displays the rear image Pb in such a way that a part or all of the work implement 14 and the work marks are included, displays the left image Pc in such a way that a part of the left vehicle body of the work vehicle 10 and a part of the left mirror are included, and displays the right image Pd in such a way that a part of the right vehicle body of the work vehicle 10 and a part of the right mirror are included. Thereby, the operator can easily grasp the sense of distance from the vehicle body of the work vehicle 10 to the object (obstacle, boundary of the field shape, etc.), and can also easily confirm the turning direction. In addition, the operator can easily perform the alignment of the work vehicle 10 during operation. In addition, the operator can easily confirm the operation status (such as the presence or absence of missed tillage).

[0184] In addition, the operation control unit 31 can also switch between the normal image and the virtual image (mirror image) for the rear image Pb. For example, the operation control unit 31 displays the rear image in the normal image or the virtual image according to the setting operation of the operator. In addition, the operation control unit 31 can also display the virtual rear image when the work vehicle 10 lowers the work implement 14 and is in operation, and display the normal rear image when the work vehicle 10 raises the work implement 14 and is in the operation stop state.

[0185] In addition, the operation control unit 31 can also move the shooting range of the front image according to the steering operation of the steering wheel 341. For example, if the operator turns the steering wheel 341 to the left, the operation control unit 31 displays the range on the left side of the front as the front image Pa according to the movement amount of the steering wheel 341. In addition, the vehicle control device 11 can also move (rotate) the camera 53 according to the movement amount of the steering wheel 341 to change the shooting area. In addition, the operation control unit 31 can also change the magnification (zoom) of the camera image according to the operation of the operator on the operation button.

[0186] In addition, the operation control unit 31 can also display the front image Pa and the side images (left image Pc, right image Pd) in a combined manner. For example, when the operator turns the steering wheel 341 to the right, the operation control unit 31 combines and displays the front image Pa obtained by shooting the range on the right side of the front and the right image Pd, and when the operator turns the steering wheel 341 to the left, the operation control unit 31 combines and displays the front image Pa obtained by shooting the range on the left side of the front and the left image Pc. Thereby, the operator can, for example, confirm the image of the turning direction, so that the remote operation can be performed with the feeling of actual vehicle driving. In addition, it is possible to confirm the image of the front tilt direction in which the steering wheel 341 is steered without adding a camera.

[0187] In addition, the operation control unit 31 can also display information such as the boundary line of the field shape, path information, and the boundary line of the uncultivated land on the camera image. For example, the operation control unit 31 can also synthesize and display the object images corresponding to the above-mentioned various information on each camera image. Thus, the operator can easily grasp the driving position during operation.

[0188] In addition, as Figure 19 shown, the operation control unit 31 can also synthesize and display the grid lines G1 to G4 (marking lines) corresponding to the distance from the front end of the work vehicle 10 on the front image Pa. For example, the grid line G1 is displayed at the position (0 m) of the front end of the work vehicle 10, the grid line G2 is displayed at the position 1 m away from the front end of the work vehicle 10, the grid line G3 is displayed at the position 2 m away from the front end of the work vehicle 10, and the grid line G4 is displayed at the position 3 m away from the front end of the work vehicle 10. Thus, the operator can accurately grasp the distance to the boundary of the field shape, obstacles and other objects. In addition, the operation control unit 31 can always display the grid lines G1 to G4, or can display them at the moment when the distance to the object is less than the threshold value (for example, less than 5 m). In addition, the operation control unit 31 can also be such that the faster the vehicle speed of the work vehicle 10, the wider the interval of the grid lines G1 to G4 is set, and the slower the vehicle speed of the work vehicle 10, the narrower the interval of the grid lines G1 to G4 is set.

[0189] In addition, for example, when the work vehicle 10 is in the remote operation state in the remote operation mode, the information indicating that the work vehicle 10 is in the remote operation state can also be displayed in a recognizable manner on the selection screen P1 in the operation terminal 20. For example, the operation terminal 20 can display the icon image of the work vehicle 10 in the remote operation state in a specified color, or can pop up and display the information (text information, marks, etc.) indicating that it is in the remote operation state on the icon image of the work vehicle 10.

[0190] In the above embodiment, the remote operation device 30 is configured to be able to remotely operate the work vehicle 10 selected by the operator among multiple work vehicles 10. However, as another embodiment, the remote operation device 30 can also be provided corresponding to a specific one work vehicle 10. In this case, the operation of the operator selecting the work vehicle 10 in the operation terminal 20 can also be omitted.

[0191] In addition, as another embodiment of the remote operation system 100, the operation of the operator requesting to switch the driving mode to the remote operation mode in the operation terminal 20 can also be omitted (refer to Figure 17A and Figure 17B ). In this case, for example, it can also be that the operator presses the mode switch button BT1 in the remote operation device 30 in the state where the camera image of the work vehicle 10 is displayed (refer toFigure 16 ) and can be switched to the remote operation mode.

[0192] In addition, as another embodiment of the remote operation system 100, the remote operation device 30 may also allow switching to the remote operation mode on the condition that the work vehicle 10 has detected an obstacle and stopped.

[0193] In addition, as another embodiment of the remote operation system 100, the remote operation device 30 may also cancel the remote operation mode and stop the work vehicle 10 when the camera image of the work vehicle 10 being remotely operated is no longer displayed on the display unit 33. Thereby, it is possible to prevent the work vehicle 10 from being remotely operated in a state where the operator cannot perform remote monitoring.

[0194] Furthermore, in the remote operation system 100, the remote operation device 30 may also be the operation terminal 20. In addition, the display unit 33 of the remote operation device 30 may also be the operation terminal 20. The remote operation device 30 is an example of the operation terminal of the present invention.

[0195] As another embodiment of the present invention, the display processing unit 212 may also display each work vehicle 10 in a recognizable manner on the selection screen P1 (refer to Figure 3 ). Specifically, the display processing unit 212 displays the icon image A1 of the work vehicle 10a in a first display form, and displays the icon image B1 of the work vehicle 10b in a second display form. For example, the display processing unit 212 displays the icon image A1 in light blue and the icon image B1 in red. Additionally, for example, the display processing unit 212 displays the icon image A1 as an illustration image of the vehicle and the icon image B1 as a captured image (photo) of the vehicle. Additionally, for example, the display processing unit 212 respectively displays (dialog box display) the identification information (vehicle name, model, serial number, etc.) of the work vehicle 10 on the icon image A1 and the icon image B1. Furthermore, it may be possible for the operator to set and change the display form of each icon image.

[0196] [Notes on the Invention]

[0197] Hereinafter, notes are made on the invention summary extracted from the embodiments. In addition, each structure and each processing function described in the following notes can be selected and combined arbitrarily.

[0198] <Note 1>

[0199] A display control method, which is a display control method for controlling the display information of an operation terminal, and performs:

[0200] displaying the pre-registered work vehicles on a first screen of the operation terminal in a manner that can be selected by the user; and

[0201] Cause the regulation information related to the above work vehicle selected by the above user to be displayed on the second screen of the above operation terminal.

[0202] <Note 2>

[0203] According to the display control method described in Note 1, wherein,

[0204] The above regulation information includes at least any one of information related to the current surrounding environment of the above work vehicle, information related to the current driving condition of the above work vehicle, and information related to the current working condition of the above work vehicle.

[0205] <Note 3>

[0206] According to the display control method described in Note 1 or 2, wherein,

[0207] Cause a map to be displayed on the above first screen,

[0208] In the above map, cause the identification information of the above work vehicle to be displayed at a position corresponding to the current position of the above work vehicle.

[0209] <Note 4>

[0210] According to the display control method described in Note 3, wherein,

[0211] In the above map, display the above identification information in a display form corresponding to the current state of the above work vehicle.

[0212] <Note 5>

[0213] According to the display control method described in Note 4, wherein,

[0214] In the above map, make the display form when the above work vehicle is in autonomous driving different from the display form when the above work vehicle has stopped autonomous driving.

[0215] <Note 6>

[0216] According to the display control method described in any one of Notes 1 to 5, wherein,

[0217] Cause the captured image of the surrounding of the above work vehicle captured by the camera mounted on the above work vehicle to be displayed on the above second screen.

[0218] <Note 7>

[0219] According to the display control method described in Note 6, wherein,

[0220] When the above-mentioned work vehicle detects an obstacle, a captured image including the obstacle among the multiple captured images of the above-mentioned camera is displayed in a recognizable manner on the above-mentioned second screen.

[0221] <Note 8>

[0222] According to the display control method described in any one of Notes 1 to 7, wherein

[0223] In the above-mentioned map, icon images of multiple pre-registered above-mentioned work vehicles are displayed in a display form corresponding to the current state of the above-mentioned work vehicle.

[0224] A captured image of the periphery of the above-mentioned work vehicle captured by the camera mounted on the above-mentioned work vehicle selected by the above-mentioned user among multiple above-mentioned work vehicles is displayed on the above-mentioned second screen.

[0225] <Note 9>

[0226] According to the display control method described in any one of Notes 1 to 8, wherein

[0227] In the above-mentioned first screen, the identification information of the operator pre-registered for the above-mentioned work vehicle is displayed in correspondence with the identification information of the above-mentioned work vehicle.

[0228] <Note 10>

[0229] According to the display control method described in Note 9, wherein

[0230] When the above-mentioned operator displayed on the above-mentioned first screen is selected, an instruction screen for outputting a specified instruction to the operator is displayed.

[0231] <Note 11>

[0232] According to the display control method described in Note 6, wherein

[0233] The above-mentioned captured image is displayed in such a way that a part of the vehicle body of the above-mentioned work vehicle is included in the above-mentioned captured image.

[0234] <Note 12>

[0235] According to the display control method described in any one of Notes 1 to 11, wherein

[0236] When a request to switch to the remote operation mode is received from the user at the above-mentioned operation terminal, and under the state where a captured image of the periphery of the above-mentioned work vehicle captured by the camera mounted on the above-mentioned work vehicle is displayed on the remote operation device capable of remote operation and a specified operation is received from the user, the above-mentioned remote operation mode is switched.

[0237] <Footnote 13>

[0238] A display control method is a display control method for controlling the display information of an operation terminal. Among them, the following operations are performed:

[0239] Display a pre-registered work vehicle on a first screen of the operation terminal; and

[0240] When the work vehicle in autonomous driving detects an obstacle, or when the work vehicle performs autonomous driving in a specific area, display specified information related to the work vehicle on a second screen of the operation terminal.

[0241] <Footnote 14>

[0242] A display control program is a display control program for controlling the display information of an operation terminal. Among them, it is used to cause one or more processors to perform the following operations:

[0243] Display a pre-registered work vehicle on a first screen of the operation terminal in a manner that can be selected by a user; and

[0244] Display specified information related to the work vehicle selected by the user on a second screen of the operation terminal.

[0245] <Footnote 15>

[0246] A display control system is a display control system for controlling the display information of an operation terminal. Among them, it includes:

[0247] A first display processing unit that displays a pre-registered work vehicle on a first screen of the operation terminal in a manner that can be selected by a user; and

[0248] A second display processing unit that displays specified information related to the work vehicle selected by the user on a second screen of the operation terminal.

Claims

1. A display control method is a display control method for controlling display information of an operation terminal, characterized in that: implement: Displaying pre-registered work vehicles on a first screen of the operation terminal in a user-selectable manner; and Prescribed information related to the work vehicle selected by the user is displayed on a second screen of the operation terminal.

2. The display control method according to claim 1, characterized in that: The prescribed information includes at least any one of information related to a current surrounding environment of the work vehicle, information related to a current travel condition of the work vehicle, and information related to a current working condition of the work vehicle.

3. The display control method according to claim 1, characterized in that: Displaying the map on the first screen, The identification information of the work vehicle is displayed at a position corresponding to the current position of the work vehicle on the map.

4. The display control method according to claim 3, characterized in that: The identification information is displayed in the map in a display mode corresponding to the current state of the work vehicle.

5. The display control method according to claim 4, characterized in that: In the map, the display form when the work vehicle is performing automatic driving is different from the display form when the work vehicle stops automatic driving.

6. The display control method according to claim 1, characterized in that: An image of the periphery of the work vehicle captured by a camera mounted on the work vehicle is displayed on the second screen.

7. The display control method according to claim 6, characterized in that: When the work vehicle detects an obstacle, an image including the obstacle among the plurality of images captured by the camera is displayed on the second screen in a recognizable manner.

8. The display control method according to claim 3, characterized in that: In the map, icon images of each of the plurality of work vehicles registered in advance are displayed in a display mode corresponding to the current state of the work vehicle. An image of the periphery of the work vehicle captured by a camera mounted on the work vehicle corresponding to the icon image selected by the user from among the plurality of icon images on the first screen is displayed on the second screen.

9. The display control method according to claim 3, characterized in that: On the first screen, identification information of a worker pre-registered for the work vehicle and identification information of the work vehicle are displayed in correspondence with each other.

10. The display control method according to claim 9, characterized in that: When the operator displayed on the first screen is selected, an instruction screen for outputting a predetermined instruction to the operator is displayed.

11. The display control method according to claim 6, characterized in that: The captured image is displayed in a manner such that a portion of a vehicle body of the work vehicle is reflected in the captured image.

12. The display control method according to claim 1, characterized in that: When the operation terminal receives a request to switch to the remote operation mode from the user, and when the remote operation device capable of remote operation displays a captured image of the surrounding area of ​​the work vehicle taken by a camera mounted on the work vehicle and receives a prescribed operation from the user, it switches to the remote operation mode.

13. A display control method is a display control method for controlling display information of an operation terminal, characterized in that: implement: causing a pre-registered work vehicle to be displayed on a first screen of the operation terminal; and When the work vehicle detects an obstacle while being driven automatically, or when the work vehicle is driven automatically in a specific area, predetermined information related to the work vehicle is displayed on a second screen of the operation terminal.

14. A display control program, which is a display control program for controlling display information of an operation terminal, characterized in that: Used to cause one or more processors to execute: Displaying pre-registered work vehicles on a first screen of the operation terminal in a user-selectable manner; and Prescribed information related to the work vehicle selected by the user is displayed on a second screen of the operation terminal.

15. A display control system is a display control system for controlling the display information of an operation terminal, characterized in that: have: A first display processing unit that displays pre-registered work vehicles on a first screen of the operation terminal in a manner that can be selected by a user; and The second display processing unit displays predetermined information related to the work vehicle selected by the user on a second screen of the operation terminal.

Images