Display control method, display control program, and display control system

Abstract

To provide a display control method which enables improvement of convenience of an operational terminal which displays information related to a work vehicle which enables automatic traveling, and to provide a display control program and a display control system.SOLUTION: A display processing unit 212 of an operation terminal 20 causes a pre-registered work vehicle 10 to be displayed on a first screen of the operation terminal 20 in a manner that a user can select it and causes predetermined information related to the work vehicle 10 selected by the user to be displayed on a second screen of the operation terminal 20.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a technique for displaying information on a work vehicle capable of autonomous driving.

Background Art

[0002] Conventionally, there has been known a technique for automatically driving a work vehicle in a work area according to a preset target route. There is also known a technique for remotely monitoring a work vehicle during autonomous driving (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the conventional technology, for example, when there are a plurality of work vehicles to be monitored, it becomes difficult for an operator to acquire necessary information about each work vehicle, and there arises a problem that the convenience of an operation terminal for displaying information about the work vehicle decreases.

[0005] An object of the present invention is to provide a display control method, a display control program, and a display control system capable of improving the convenience of an operation terminal for displaying information on a work vehicle capable of autonomous driving.

Means for Solving the Problems

[0006] The display control method according to the present invention is a method for controlling display information of an operation terminal, and includes causing a first screen of the operation terminal to display work vehicles registered in advance so that a user can select them, and causing a second screen of the operation terminal to display predetermined information about the work vehicle selected by the user.

[0007] In addition, the display control method according to the present invention is a method for controlling display information of an operation terminal, and includes displaying a pre-registered work vehicle on a first screen of the operation terminal, and when the work vehicle during automatic driving detects an obstacle, or when the work vehicle automatically drives in a specific area, displaying predetermined information about the work vehicle on a second screen of the operation terminal.

[0008] In addition, the display control program according to the present invention is a program for controlling display information of an operation terminal, and includes causing one or more processors to display a pre-registered work vehicle on a first screen of the operation terminal so that the user can select it, and displaying predetermined information about the work vehicle selected by the user on a second screen of the operation terminal.

[0009] In addition, the display control system according to the present invention is a system for controlling display information of an operation terminal. The display control system includes a first display processing unit that displays a pre-registered work vehicle on a first screen of the operation terminal so that the user can select it, and a second display processing unit that displays predetermined information about the work vehicle selected by the user on a second screen of the operation terminal.

Advantages of the Invention

[0010] According to the present invention, it is possible to provide a display control method, a display control program, and a display control system that can improve the convenience of an operation terminal for displaying information about an automatically drivable work vehicle.

Brief Description of the Drawings

[0011]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6A

Figure 6B

Figure 6C

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17A

Figure 17B

Figure 18

Figure 19

Embodiments for Carrying Out the Invention

[0012] The following embodiments are an example of embodying the present invention and do not limit the technical scope of the present invention.

[0013] As shown in FIG. 1, an automatic driving system 1 according to an embodiment of the present invention includes a work vehicle 10 and an operation terminal 20. The number of work vehicles 10 may be one or a plurality. In the present embodiment, a case where the automatic driving system 1 includes a plurality of work vehicles 10 (two work vehicles 10a and 10b in FIG. 1) will be described as an example. Hereinafter, when the work vehicle 10a and the work vehicle 10b are not distinguished, they are referred to as "work vehicle 10".

[0014] The work vehicle 10 and the operation terminal 20 can communicate via a communication network N1. For example, the work vehicle 10 and the operation terminal 20 can communicate via a mobile phone line network, a packet line network, or a wireless LAN.

[0015] In this embodiment, the case where the work vehicle 10 is a tractor will be described as an example. As another embodiment, the work vehicle 10 may be a combine harvester, a rice transplanter, a sprayer, a construction machine, a snowplow, or the like. Further, the work vehicles 10a and 10b may be vehicles of different types or vehicles of the same type. The work vehicle 10 is configured to be able to automatically travel (autonomously travel) along a preset target path within the work area of the farm field. Further, the work vehicle 10 is capable of performing a predetermined operation while automatically traveling within the work area. Furthermore, the work vehicle 10 is configured to be able to automatically travel along a preset inter-field path on a road (connection road) connecting a plurality of farm fields.

[0016] Based on the position information of the current position of the work vehicle 10 calculated by the positioning unit 17, the work vehicle 10 can automatically travel along a preset target path and an inter-field path within the farm field and outside the farm field (road). Note that the work vehicle 10 may perform automatic driving with an operator on board.

[0017] For example, when a target path including a work path, a travel start position, and a travel end position are set for a farm field, the work vehicle 10 performs a predetermined operation while automatically traveling from the travel start position to the travel end position along the target path within the farm field.

[0018] In addition, when the work of one farm field is completed, the work vehicle 10 can move to another farm field to perform work. For example, when the work of the first farm field is completed, the work vehicle 10 automatically travels along a preset inter-field path on the road and moves to the second farm field. When the work vehicle 10 arrives at the second farm field, it performs a predetermined operation while automatically traveling along a preset target path in the second farm field.

[0019] The target path within the field is appropriately set according to the work content. Also, the inter-field path on the road is preset according to the operation (teaching operation) by the operator. The inter-field path may be a road dedicated to work vehicles such as farm roads, forest roads, public roads, private roads, and motorways, or it may be a road passable by general vehicles (such as passenger cars).

[0020] The operation terminal 20 is an information processing device that displays various information regarding the work vehicle 10. For example, the operation terminal 20 displays the position information of the current position, the running status, the work status, etc. for the pre-registered work vehicle 10. The user (operator, monitor) of the operation terminal 20 can remotely monitor the work vehicle 10 by checking the said information.

[0021] [Work vehicle 10] As shown in FIGS. 1 and 2, the work vehicle 10 includes a vehicle control device 11, a storage unit 12, a traveling device 13, a work implement 14, an obstacle detection device 15, a communication unit 16, a positioning unit 17, etc. The vehicle control device 11 is electrically connected to the storage unit 12, the traveling device 13, the work implement 14, the obstacle detection device 15, the positioning unit 17, etc. Note that the vehicle control device 11 and the obstacle detection device 15 may be capable of wireless communication, and the vehicle control device 11 and the positioning unit 17 may be capable of wireless communication.

[0022] The communication unit 16 is a communication interface for connecting the work vehicle 10 to the communication network N1 by wire or wirelessly and performing data communication according to a predetermined communication protocol with external devices (such as the operation terminal 20) via the communication network N1.

[0023] The storage unit 12 is a non-volatile storage unit such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive) that stores various types of information. The storage unit 12 stores control programs such as an automatic driving program for causing the vehicle control device 11 to execute automatic driving processing. For example, the automatic driving program is non-temporarily recorded on a computer-readable recording medium such as a CD or a DVD, read by a predetermined reading device (not shown), and stored in the storage unit 12. Note that the automatic driving program may be downloaded from a server (not shown) to the work vehicle 10 via the communication network N1 and stored in the storage unit 12. Further, the storage unit 12 stores data such as target route data.

[0024] The traveling device 13 is a driving unit that causes the work vehicle 10 to travel. As shown in FIG. 2, the traveling device 13 includes an engine 131, front wheels 132, rear wheels 133, a transmission 134, a front axle 135, a rear axle 136, a steering wheel 137, and the like. Note that the front wheels 132 and the rear wheels 133 are respectively provided on the left and right sides of the work vehicle 10. Further, the traveling device 13 is not limited to a wheel type including the front wheels 132 and the rear wheels 133, and may be a crawler type including crawlers provided on the left and right sides of the work vehicle 10.

[0025] The engine 131 is a drive source such as a diesel engine or a gasoline engine that is driven using fuel supplied from a fuel tank (not shown). The traveling device 13 may include an electric motor as a drive source together with the engine 131 or instead of the engine 131. Note that a generator (not shown) is connected to the engine 131, and electric power is supplied from the generator to electric components such as the vehicle control device 11, the obstacle detection device 15, and the positioning unit 17 provided in the work vehicle 10, and a battery and the like. Note that the battery is charged by the electric power supplied from the generator. Then, electric components such as the vehicle control device 11, the obstacle detection device 15, and the positioning unit 17 provided in the work vehicle 10 can be driven by the electric power supplied from the battery even after the engine 131 stops.

[0026] The driving force of the engine 131 is transmitted to the front wheels 132 via the transmission 134 and the front axle 135, and is transmitted to the rear wheels 133 via the transmission 134 and the rear axle 136. Also, the driving force of the engine 131 is transmitted to the work implement 14 via a PTO shaft (not shown). When the work vehicle 10 performs automatic driving, the traveling device 13 performs a traveling operation according to an instruction from the vehicle control device 11. Also, the traveling device 13 decelerates or stops the work vehicle 10 according to an instruction from the vehicle control device 11.

[0027] The work implement 14 is, for example, a tiller, a lawn mower, a plow, a fertilizer applicator, a sprayer (chemical sprayer), a rotary cultivator, or a seeder, etc., and is detachable from the work vehicle 10. Thereby, the work vehicle 10 can perform various operations using each of the work implements 14. FIG. 2 shows a case where the work implement 14 is a tiller.

[0028] The steering wheel 137 is an operation unit operated by an operator or the vehicle control device 11. For example, in the traveling device 13, according to the operation of the steering wheel 137 by the vehicle control device 11, the angle of the front wheels 132 is changed by a hydraulic power steering mechanism (not shown) or the like, and the traveling direction of the work vehicle 10 is changed.

[0029] Also, in addition to the steering wheel 137, the traveling device 13 includes a shift lever, an accelerator, a brake, etc. (not shown) operated by the vehicle control device 11. In the traveling device 13, according to the operation of the shift lever by the vehicle control device 11, the gear of the transmission 134 is switched to a forward gear or a reverse gear, etc., and the traveling mode of the work vehicle 10 is switched to forward or reverse, etc. Also, the vehicle control device 11 controls the rotational speed of the engine 131 by operating the accelerator. Also, the vehicle control device 11 operates the brake to brake the rotation of the front wheels 132 and the rear wheels 133 using an electromagnetic brake.

[0030] The positioning unit 17 is a communication device including a positioning control unit 171, a storage unit 172, a communication unit 173, a positioning antenna 174, and the like. For example, as shown in FIG. 2, the positioning unit 17 is provided above the cabin 138 on which the operator rides. Further, the installation location of the positioning unit 17 is not limited to the cabin 138. Furthermore, the positioning control unit 171, the storage unit 172, the communication unit 173, and the positioning antenna 174 of the positioning unit 17 may be distributed and arranged at different positions in the work vehicle 10. As described above, the battery is connected to the positioning unit 17, and the positioning unit 17 can operate even when the engine 131 is stopped. Further, as the positioning unit 17, for example, a mobile phone terminal, a smartphone, a tablet terminal, or a quantum compass may be substituted.

[0031] The positioning control unit 171 is a computer system including one or more processors and a storage memory such as a non-volatile memory and a RAM. The storage unit 172 is a non-volatile memory or the like that stores data such as a program for causing the positioning control unit 171 to execute positioning processing, positioning information, and movement information. For example, the program is non-temporarily recorded on a computer-readable recording medium such as a CD or a DVD, and is read by a predetermined reading device (not shown) and stored in the storage unit 172. Note that the program may be downloaded from a server (not shown) to the positioning unit 17 via the communication network N1 and stored in the storage unit 172.

[0032] The communication unit 173 is a communication interface for connecting the positioning unit 17 to the communication network N1 by wire or wirelessly and performing data communication according to a predetermined communication protocol with an external device such as a base station server via the communication network N1.

[0033] The positioning antenna 174 is an antenna that receives radio waves (GNSS signals) transmitted from satellites.

[0034] The positioning control unit 171 calculates the current position of the work vehicle 10 based on the GNSS signals received by the positioning antenna 174 from satellites. For example, when the work vehicle 10 automatically travels within the farm field, when the positioning antenna 174 receives radio waves (transmission time, orbital information, etc.) transmitted from each of a plurality of satellites, the positioning control unit 171 calculates the distances between the positioning antenna 174 and each satellite, and calculates the current position (latitude and longitude) of the work vehicle 10 based on the calculated distances. Further, the positioning control unit 171 may perform positioning by a real-time kinematic method (RTK-GNSS positioning method (RTK method)) of calculating the current position of the work vehicle 10 using correction information corresponding to a base station (reference station) close to the work vehicle 10. Thus, the work vehicle 10 performs automatic driving using the positioning information by the RTK method. Note that the current position of the work vehicle 10 may be the same position as the positioning position (for example, the position of the positioning antenna 174), or may be a position deviated from the positioning position. Note that the positioning control unit 171 may calculate (position) the current position of the work vehicle 10 using a quantum compass.

[0035] When the obstacle detection device 15 detects a detection target while the work vehicle 10 is automatically traveling, it outputs detection information (measurement information) to the vehicle control device 11. Specifically, the obstacle detection device 15 includes a detection control unit 51, a storage unit 52, a camera 53, an obstacle sensor 54, a communication unit 55, and the like. The obstacle detection device 15 may be configured as a single unit and mounted on the work vehicle 10, or a plurality of components may be distributed and arranged on the work vehicle 10.

[0036] The communication unit 55 is a communication interface for connecting the obstacle detection device 15 to the communication network N1 by wire or wirelessly and performing data communication according to a predetermined communication protocol with an external device (such as the operation terminal 20) via the communication network N1.

[0037] The storage unit 52 is a non-volatile storage unit such as an HDD or an SSD that stores various types of information. The storage unit 52 stores control programs such as an obstacle detection program for causing the obstacle detection device 15 to execute obstacle detection processing. For example, the obstacle detection program is non-temporarily recorded on a computer-readable recording medium such as a CD or a DVD, read by a predetermined reading device (not shown), and stored in the storage unit 52. Note that the obstacle detection program may be downloaded from a server (not shown) to the obstacle detection device 15 via the communication network N1 and stored in the storage unit 52.

[0038] The camera 53 is a digital camera that captures an image of a subject included in a predetermined imaging range and outputs it as digital image data. The camera 53 continuously captures images of the subject at a predetermined frame rate, generates frame images (captured images) of a predetermined resolution, and sequentially transmits them to the detection control unit 51. Further, the camera 53 transmits the image data of the captured image to the operation terminal 20 via the communication unit 55. The operation terminal 20 can display the captured image on the operation screen of the operation display unit 23 (see FIG. 6A etc.).

[0039] The camera 53 includes a front camera 53a capable of imaging the front imaging range as viewed from the work vehicle 10, a rear camera 53b capable of imaging the rear imaging range as viewed from the work vehicle 10, a left camera 53c capable of imaging the left imaging range as viewed from the work vehicle 10, and a right camera 53d (not shown) capable of imaging the right imaging range as viewed from the work vehicle 10. As shown in FIG. 2, the front camera 53a is disposed on the upper front side of the cabin 138, the rear camera 53b is disposed on the upper rear side of the cabin 138, the left camera 53c is disposed on the upper left side of the cabin 138, and the right camera 53d is disposed on the upper right side of the cabin 138. Note that the camera 53 may be configured by only the front camera 53a and the rear camera 53b. Further, the camera 53 may be a single camera that is an omnidirectional camera capable of photographing all directions around the work vehicle 10.

[0040] The obstacle sensor 54 is a sensor that detects detection targets within a predetermined detection range using infrared rays, ultrasonic waves, etc. For example, the obstacle sensor 54 may be a lidar sensor (distance sensor) capable of measuring the distance to a detection target in three dimensions using a laser, or a sonar sensor having a plurality of sonars capable of measuring the distance to a detection target using ultrasonic waves. The obstacle sensor 54 is installed at the front center, rear center, etc. of the body of the work vehicle 10, monitors the surroundings of the work vehicle 10, and detects obstacles. In the present embodiment, an example will be described in which the obstacle sensor 54 includes a front obstacle sensor 54a capable of detecting detection targets within the detection ranges in the front and side (left and right) directions as viewed from the work vehicle 10, and a rear obstacle sensor 54b capable of detecting detection targets within the detection ranges in the rear and side (left and right) directions as viewed from the work vehicle 10. As shown in FIG. 2, the front obstacle sensor 54a is disposed on the upper front side of the cabin 138, and the rear obstacle sensor 54b is disposed on the upper rear side of the cabin 138. The front obstacle sensor 54a and the rear obstacle sensor 54b can detect detection targets within a 360-degree detection range around the work vehicle 10. The front camera 53a and the front obstacle sensor 54a may be configured as one unit, and the rear camera 53b and the rear obstacle sensor 54b may be configured as one unit. Note that the obstacle sensor 54 may include a left obstacle sensor capable of detecting detection targets within the detection range on the left side as viewed from the work vehicle 10, and a right obstacle sensor capable of detecting detection targets within the detection range on the right side as viewed from the work vehicle 10.

[0041] The obstacle sensor 54 measures, for example, the distance to each ranging point (measurement target) existing in the measurement range using a laser (such as near-infrared laser light), and generates a distance image based on the measurement information. The obstacle sensor 54 includes an electronic control unit in which a microcontroller or the like is integrated, and a processing unit constructed by various control programs and the like, and is connected to the detection control unit 51, the vehicle control device 11, etc. via CAN so as to be able to communicate with each other.

[0042] The obstacle sensor 54 measures the distance to the ranging points within the measurement range from the obstacle sensor 54 by the TOF (Time Of Flight) method that measures the distance to the ranging points based on the round-trip time until the irradiated laser reaches the ranging points and returns. The obstacle sensor 54 performs three-dimensional measurement within the measurement range by scanning the laser horizontally and vertically at high speed over the entire measurement range and sequentially measuring the distances to the ranging points for each scanning angle (coordinates). The obstacle sensor 54 sequentially measures the intensity of the reflected light (reflection intensity) from each ranging point obtained when scanning the laser horizontally and vertically at high speed over the entire measurement range. The obstacle sensor 54 repeatedly measures in real time the distances to each ranging point within the measurement range, the reflection intensities, etc.

[0043] The obstacle sensor 54 generates a distance image from the measurement information such as the distances to the measured ranging points and the scanning angles (coordinates) for each ranging point, extracts the group of ranging points estimated as obstacles, and transmits the measurement information regarding the extracted group of ranging points to the detection control unit 51 as the measurement information regarding the obstacles.

[0044] In addition, a control range corresponding to the distance from the work vehicle 10 is set in the measurement range (detection range) of the obstacle sensor 54. For example, in the measurement range, the range where the distance from the work vehicle 10 is L1 is set as the stop control range, the range from the distance L1 to the distance L2 is set as the deceleration control range, and the range from the distance L2 to the distance L3 is set as the notification control range (however, L1 < L2 < L3). Each control range can be set according to the type, model, work content, vehicle speed, etc. of the work vehicle 10. Also, the control range may be switchable according to the traveling direction of the work vehicle 10. For example, the detection control unit 51 sets the control range on the front side of the vehicle body when the work vehicle 10 is traveling forward, and sets the control range on the rear side of the vehicle body when the work vehicle 10 is traveling backward.

[0045] The detection control unit 51 outputs the measurement information obtained from the obstacle sensor 54 to the vehicle control device 11. While the work vehicle 10 is automatically traveling, each time the detection control unit 51 obtains measurement information from the obstacle sensor 54, it sequentially outputs the measurement information to the vehicle control device 11. As another embodiment, the detection control unit 51 may determine the type (person, vehicle, structure, material, etc.) of the detection target (measurement target) based on the captured image obtained from the camera 53 and the measurement information obtained from the obstacle sensor 54, and output the determination result to the vehicle control device 11.

[0046] The vehicle control device 11 includes control devices such as a CPU, a ROM, and a RAM. The CPU is a processor that executes various arithmetic processes. The ROM is a non-volatile storage unit in which control programs such as BIOS and OS for causing the CPU to execute various arithmetic processes are stored in advance. The RAM is a volatile or non-volatile storage unit that stores various information and is used as a temporary storage memory for various processes executed by the CPU. Then, the vehicle control device 11 controls the work vehicle 10 by causing the CPU to execute various control programs stored in advance in the ROM or the storage unit 12.

[0047] Specifically, as shown in FIG. 1, the vehicle control device 11 includes various processing units such as a travel processing unit 111. Note that the vehicle control device 11 functions as the various processing units by causing the CPU to execute various processes according to the automatic driving program. Also, some or all of the processing units may be configured by electronic circuits. Note that the automatic driving program may be a program for causing a plurality of processors to function as the processing units.

[0048] The traveling processing unit 111 controls the traveling of the work vehicle 10. For example, when the traveling mode of the work vehicle 10 is automatic traveling (automatic traveling mode), the traveling processing unit 111 automatically travels the work vehicle 10 based on the position information (positioning information) indicating the current position of the work vehicle 10 measured by the positioning unit 17. For example, when the work vehicle 10 satisfies the start condition of automatic traveling and the traveling start instruction is obtained from the operator, the traveling processing unit 111 starts the automatic traveling of the work vehicle 10 based on the positioning information. For example, the traveling processing unit 111 automatically travels the work vehicle 10 from the traveling start position to the traveling end position according to the target route previously generated and set on the operation terminal 20.

[0049] Note that when the traveling mode of the work vehicle 10 is manual traveling (manual traveling mode), it is possible to manually travel the work vehicle 10 based on the operation of the operator (manual steering). For example, the traveling processing unit 111 acquires operation information corresponding to driving operations such as a steering wheel operation, a shift operation, a traveling direction switching operation, and a brake operation by the operator, and causes the traveling device 13 to execute a traveling operation based on the operation information.

[0050] In addition, when the traveling processing unit 111 detects an obstacle while the work vehicle 10 is automatically traveling, the traveling processing unit 111 causes the work vehicle 10 to execute a predetermined countermeasure process. Specifically, when the traveling processing unit 111 detects an obstacle within the notification control range, the traveling processing unit 111 causes the work vehicle 10 to emit a warning sound. Also, when the traveling processing unit 111 detects an obstacle within the deceleration control range, the traveling processing unit 111 decelerates the work vehicle 10 traveling automatically from a preset vehicle speed. Further, when the traveling processing unit 111 detects an obstacle within the stop control range, the traveling processing unit 111 temporarily stops (halts) the work vehicle 10 traveling automatically. Note that the traveling processing unit 111 may cause the work vehicle 10 to execute a countermeasure process according to the type of the detection target detected by the obstacle detection device 15.

[0051] In addition, the travel control unit 111 controls travel according to an instruction acquired from the operation terminal 20. For example, when the travel control unit 111 acquires a travel start instruction from the operation terminal 20, it starts the automatic travel of the work vehicle 10, and when it acquires a travel stop instruction from the operation terminal 20, it stops the automatic travel of the work vehicle 10.

[0052] [Operation terminal 20] As shown in FIG. 1, the operation terminal 20 is an information processing device including an operation control unit 21, a storage unit 22, an operation display unit 23, a communication unit 24, and the like. For example, the operation terminal 20 is composed of a portable terminal such as a smartphone (see FIG. 2 etc.), a tablet terminal, etc. Note that the operation terminal 20 may be a stationary (desktop type) personal computer.

[0053] The communication unit 24 is a communication interface for connecting the operation terminal 20 to the communication network N1 by wire or wirelessly and performing data communication according to a predetermined communication protocol with external devices such as one or more work vehicles 10 via the communication network N1.

[0054] The operation display unit 23 is a user interface including a display unit such as a liquid crystal display or an organic EL display for displaying various information, and an operation unit such as a touch panel, a mouse, or a keyboard for receiving operations. A map of a predetermined area including the field to be worked, predetermined information about the work vehicle 10, etc. are displayed on the display unit. The predetermined information includes at least any one of information about the current surrounding environment of the work vehicle 10, information about the current travel status of the work vehicle 10, and information about the current work status of the work vehicle 10. The operator can grasp the travel state of the work vehicle 10 that automatically travels on the field, the detection result of obstacles, etc. based on the predetermined information displayed on the operation terminal 20 at a location away from the work vehicle 10. That is, the operator can remotely monitor the work vehicle 10 using the operation terminal 20.

[0055] In addition, an operation screen for giving instructions such as a travel start instruction, a travel stop instruction, and a travel restart instruction to the work vehicle 10 is displayed on the display unit. Also, a registration screen for registering work vehicle information, field information, work information, etc. is displayed on the display unit.

[0056] The storage unit 22 is a non-volatile storage unit such as an HDD or an SSD that stores various types of information. In the storage unit 22, control programs such as a display control program for causing the operation control unit 21 to execute display control processing (see FIG. 8) described later are stored. For example, the control program is non-temporarily recorded on a computer-readable recording medium such as a CD or a DVD, and is read by a predetermined reading device (not shown) and stored in the storage unit 22. Note that the control program may be downloaded from a server (not shown) to the operation terminal 20 via the communication network N1 and stored in the storage unit 22.

[0057] Also, data such as work vehicle information, which is information about the work vehicle 10, and target route information, which is information about the target route, are stored in the storage unit 22. The work vehicle information includes information such as a vehicle number and a model for each work vehicle 10. The vehicle number is identification information of the work vehicle 10. The model is the model of the work vehicle 10. Note that the storage unit 22 may store the work vehicle information regarding one work vehicle 10, or may store the work vehicle information regarding a plurality of work vehicles 10. In the present embodiment, the work vehicle information regarding each of the two work vehicles 10a and 10b is stored in the storage unit 22.

[0058] The target route information includes information such as a route name, a field name, an address, a field area, and a work time for each target route. The route name is the route name of the target route generated on the operation terminal 20. The field name is the name of the work target field where the target route is set. The address is the address of the field, and the field area is the area of the field. The work time is the time required for the work vehicle 10 to work on the field.

[0059] When the target route corresponds to a route for a road (a route between fields), the target route information includes information such as a route name, an address, a driving distance, and a driving time. The route name is the name of the road, and the address is the address of the road. The driving distance is the distance that the work vehicle 10 travels on the road, for example, the distance from the first field to the second field. The driving time is the time that the work vehicle 10 travels on the road, for example, the time required for the movement from the first field to the second field.

[0060] Note that the storage unit 22 may store the target route information regarding one target route, or may store the target route information regarding a plurality of target routes. For example, when an operator generates a plurality of target routes for one or a plurality of fields under his / her management, the target route information regarding each target route is stored in the storage unit 22. Note that for one field, one target route may be set, or a plurality of target routes may be set. Also, for a set of fields, one route between fields may be set, or a plurality of routes between fields may be set. In the present embodiment, the storage unit 22 stores the target route information corresponding to the target route on which the work vehicle 10a travels on the first field and the target route information corresponding to the target route on which the work vehicle 10b travels on the second field.

[0061] As another embodiment, part or all of the information such as the work vehicle information and the target route information may be stored in a server accessible from the operation terminal 20. The operator may perform an operation of registering the work vehicle information and the target route information in the server (for example, a personal computer, a cloud server, etc.). In this case, the operation control unit 21 may acquire the information from the server and execute each process.

[0062] The operation control unit 21 includes control devices such as a CPU, a ROM, and a RAM. The CPU is a processor that executes various arithmetic processes. The ROM is a non-volatile storage unit in which control programs such as BIOS and OS for causing the CPU to execute various arithmetic processes are stored in advance. The RAM is a volatile or non-volatile storage unit that stores various information and is used as a temporary storage memory (working area) for various processes executed by the CPU. Then, the operation control unit 21 controls the operation terminal 20 by causing the CPU to execute various control programs stored in advance in the ROM or the storage unit 22.

[0063] As shown in FIG. 1, the operation control unit 21 includes various processing units such as a setting processing unit 211, a display processing unit 212, and a reception processing unit 213. Note that the operation control unit 21 functions as the various processing units by causing the CPU to execute various processes according to the control program. Also, some or all of the processing units may be configured by electronic circuits. Note that the control program may be a program for causing a plurality of processors to function as the processing units.

[0064] The setting processing unit 211 sets information regarding the work vehicle 10 (hereinafter referred to as work vehicle information), information regarding the farm field (hereinafter referred to as farm field information), and information regarding how to specifically perform the work (hereinafter referred to as work information).

[0065] Specifically, the setting processing unit 211 sets the information by having the operator perform an operation of registering on the operation terminal 20 regarding information such as the model of the work vehicle 10, the position where the positioning antenna 174 is attached to the work vehicle 10, the type of the work implement 14, the size and shape of the work implement 14, the position of the work implement 14 with respect to the work vehicle 10, the vehicle speed and engine speed during operation of the work vehicle 10, and the vehicle speed and engine speed during turning of the work vehicle 10.

[0066] In addition, the setting processing unit 211 sets the information such as the position and shape of the field, the work start position (travel start position) where the work starts and the work end position (travel end position) where the work ends, and the work direction by performing an operation of registering on the operation terminal 20.

[0067] The information on the position and shape of the field can be automatically acquired, for example, by the operator boarding the work vehicle 10 and driving it to go around once along the outer periphery of the field and recording the change in the position information of the positioning antenna 174 at that time. Also, the position and shape of the field can be acquired based on a polygon obtained by the operator operating the operation terminal 20 to specify a plurality of points on the map while the map is displayed on the operation terminal 20.

[0068] In addition, the setting processing unit 211 is configured to be able to set, as work information, the presence or absence of cooperative work between the work vehicle 10 (unmanned tractor) and the manned work vehicle 10, the skip number which is the number of work routes to be skipped when the work vehicle 10 turns on the headland, the width of the headland, and the width of the non-cultivated land.

[0069] In addition, the setting processing unit 211 generates a target route for automatically driving the work vehicle 10 in the field based on the respective setting information. Specifically, the setting processing unit 211 generates a target route in the field based on the travel start position and the travel end position registered in the field setting. For example, the setting processing unit 211 generates a target route including a travel start position, a travel end position, a straight travel route, and a turning route based on the setting operation of the operator. The setting processing unit 211 registers the generated target route in association with the field.

[0070] The display processing unit 212 causes various information to be displayed on the operation display unit 23. Specifically, the display processing unit 212 causes the work vehicle 10 registered in advance to be displayed on the selection screen P1 (the first screen of the present invention) so that the operator can select it. For example, when the operator registers the work vehicles 10a and 10b to be monitored (managed), the display processing unit 212 causes the work vehicles 10a and 10b to be displayed on the selection screen P1 so that they can be selected. The work vehicle 10 displayed on the selection screen P1 may be a work vehicle 10 for which information is registered in the operation terminal 20, or may be a work vehicle 10 selected by the operator to start work from among the plurality of registered work vehicles 10.

[0071] FIG. 3 shows an example of the selection screen P1. As shown in FIG. 3, the display processing unit 212 causes a map of a predetermined area including the current positions of the work vehicles 10a and 10b registered in advance to be displayed on the selection screen P1. Further, the display processing unit 212 causes the identification information of the work vehicle 10 to be displayed at a position corresponding to the current position of the work vehicle 10 on the map. Specifically, as shown in FIG. 3, the display processing unit 212 causes the icon image A1 of the work vehicle 10a to be displayed at a position corresponding to the current position of the work vehicle 10a on the map, and causes the icon image B1 of the work vehicle 10b to be displayed at a position corresponding to the current position of the work vehicle 10b. The display processing unit 212 acquires position information (positioning information) from the work vehicle 10 and displays the icon image of the work vehicle 10 on the map. The display processing unit 212 updates the position of the icon image on the map in real time. As another embodiment, the display processing unit 212 may display the name (text information) of the work vehicle 10a on the map instead of the icon image.

[0072] In addition, the display processing unit 212 may display the icon image A1 of the work vehicle 10a and the icon image B1 of the work vehicle 10b in a distinguishable manner. For example, the display processing unit 212 may display the name of the work vehicle 10a near the icon image A1 and the name of the work vehicle 10b near the icon image B1. Further, when the operator touches or mouses over the icon image A1, the display processing unit 212 may display the identification information (such as the name) of the work vehicle 10a, and when the operator touches or mouses over the icon image B1, the display processing unit 212 may display the identification information (such as the name) of the work vehicle 10b.

[0073] In addition, the display processing unit 212 may display the map in an enlarged or reduced manner in response to an operation of enlarging or reducing the map on the selection screen P1. Further, the display processing unit 212 may automatically adjust the display magnification according to the position of the work vehicle 10 to be monitored. For example, the display processing unit 212 determines the display magnification of the map so that the icon images of all the work vehicles 10 to be monitored fit on one screen (the same page).

[0074] According to the selection screen P1 shown in FIG. 3, the operator can grasp the current positions of all the work vehicles 10 to be monitored. Note that the display processing unit 212 may be configured to display on the selection screen P1 also the work vehicles 10 stored in the storage location (such as a warehouse) during non-operation, or may be configured not to display on the selection screen P1 the work vehicles 10 stored in the storage location during non-operation. Further, it may be possible for the operator to set whether or not to display on the selection screen P1 the work vehicles 10 stored in the storage location during non-operation.

[0075] Further, the display processing unit 212 causes the identification information to be displayed in a display mode corresponding to the current state of the work vehicle 10 on the map. Specifically, the display processing unit 212 causes an icon image to be displayed on the map so as to be able to identify whether the work vehicle 10 is in the process of automatic driving or stopped. That is, the display processing unit 212 makes the display mode when the work vehicle 10 is in the process of automatic driving different from the display mode when the work vehicle 10 is stopped on the map. In the example shown in FIG. 4, when the work vehicle 10a is stopped, the display processing unit 212 causes the icon image A1 to be displayed in a mode indicating that it is stopped. Further, when the work vehicle 10b is in the process of automatic driving, the display processing unit 212 causes the icon image B1 to be displayed in a mode indicating that it is in the process of automatic driving. Note that the display mode may be, for example, the type, color, and thickness of the line of the frame image surrounding the icon image, or the color, size, and display method (lighting, blinking) of the icon image. For example, as shown in FIG. 4, the display processing unit 212 causes the frame image A2 of the icon image A1 and the frame image B2 of the icon image B1 to be displayed in different display modes. Note that the operator may be able to set and change the display mode of each icon image. For example, on the setting screen, the operator may set the frame image of the icon image representing the automatic driving in the first display mode and set the frame image of the icon image representing the stopped state in the second display mode.

[0076] As another embodiment, the display processing unit 212 may make the frame image non-displayed when the work vehicle 10 is normally in the process of automatic driving and cause the frame image to be displayed when the work vehicle 10 has stopped the automatic driving.

[0077] Further, the display processing unit 212 may display a message (text information) indicating that it is stopped in the vicinity of the icon image A1 and display a message indicating that it is in the process of automatic driving in the vicinity of the icon image B1. Further, the display processing unit 212 may display a message indicating that it is stopped when the operator touches or mouses over the icon image A1, and display a message indicating that it is in the process of automatic driving when the operator touches or mouses over the icon image B1.

[0078] In addition, when the work vehicle 10 detects an obstacle, finishes the work, the remaining amount of the spraying material or fuel becomes less than the specified amount, the yield of the harvested product becomes more than the specified amount, the DPF (Diesel Particulate Filter) regeneration work is required, the urea water replenishment work is required, etc., the automatic driving is stopped.

[0079] As another embodiment, the display processing unit 212 may display identification information that can identify the detection of an obstacle when the obstacle is detected on the selection screen P1. For example, when the work vehicle 10a detects an obstacle, the display processing unit 212 lights or blinks the icon image A1, or displays the frame image A2 of the icon image A1 in a predetermined color. Further, the display processing unit 212 may display the icon image A1 or the frame image A2 in a display mode corresponding to the detection position (control range) when the work vehicle 10a detects an obstacle. For example, when the work vehicle 10a detects an obstacle within the notification control range, the display processing unit 212 displays the frame image A2 in green, and when the work vehicle 10a detects an obstacle within the deceleration control range, the display processing unit 212 displays the frame image A2 in yellow, and when the work vehicle 10a detects an obstacle within the stop control range, the display processing unit 212 displays the frame image A2 in red.

[0080] In addition, the display processing unit 212 may display identification information that can identify the working status on the selection screen P1. For example, when the work vehicle 10a finishes working in the field to be worked on, the display processing unit 212 lights or blinks the icon image A1, or displays the frame image A2 of the icon image A1 in a predetermined color. Also, for example, when the work vehicle 10a is a vehicle (spraying machine) that sprays a spraying material (chemical solution, water, etc.), the display processing unit 212 lights or blinks the icon image A1, or displays the frame image A2 of the icon image A1 in a predetermined color when the remaining amount of the spraying material in the work vehicle 10a becomes less than the specified amount, or when the replenishment timing of the spraying material arrives. Also, for example, when the work vehicle 10a is a vehicle (combine, etc.) that harvests the harvested product, the display processing unit 212 lights or blinks the icon image A1, or displays the frame image A2 of the icon image A1 in a predetermined color when the yield of the harvested product becomes equal to or more than the specified amount, or when the discharge timing of the harvested product arrives.

[0081] In this way, when the work vehicle 10 stops automatic driving, the display processing unit 212 may display the stop state of the work vehicle 10 in an identifiable manner on the selection screen P1, or when the work vehicle 10 stops automatic driving, the display processing unit 212 may display identification information that can identify the cause of the stop of the work vehicle 10 on the selection screen P1.

[0082] According to the selection screen P1 shown in FIG. 4, the operator can grasp the current state for each work vehicle 10 to be monitored.

[0083] Here, the display processing unit 212 displays the work vehicle 10 on the selection screen P1 (see FIGS. 3 and 4) so that it can be selected. Specifically, the display processing unit 212 displays the icon images A1 and B1 on the map of the selection screen P1 so that they can be selected. The reception processing unit 213 receives an operation of selecting an icon image from the operator on the selection screen P1. For example, as shown in FIG. 5, when the operator touches the icon image A1 of the work vehicle 10a displayed on the selection screen P1 with a finger, the reception processing unit 213 receives the touch operation. For example, when the operator sees the selection screen P1 and grasps that the work vehicle 10a has stopped automatic driving, the operator touches the icon image A1 of the work vehicle 10a to confirm the cause of the stop.

[0084] The display processing unit 212 causes the operation display unit 23 to display predetermined information regarding the work vehicle 10 selected by the operator. Specifically, the display processing unit 212 causes the information display screen P2 (the second screen of the present invention) to display the predetermined information. The predetermined information includes at least any one of information regarding the current surrounding environment of the work vehicle 10, information regarding the current driving status of the work vehicle 10, and information regarding the current work status of the work vehicle 10. For example, when the operator selects the icon image A1 of the work vehicle 10a (see FIG. 5), the display processing unit 212 causes the information display screen P2 to display a captured image of the surroundings of the work vehicle 10a captured by the camera 53 mounted on the work vehicle 10a as shown in FIG. 6A. Here, the display processing unit 212 acquires a front image, a rear image, a left image, and a right image of the work vehicle 10a from the camera 53 and displays them on the information display screen P2. Further, as shown in FIG. 6A, the display processing unit 212 displays the front image on the upper side of the information display screen P2 and displays the rear image, the left image, and the right image on the lower side of the information display screen P2. As another embodiment, as shown in FIG. 6B, the display processing unit 212 may display the front image on the lower side of the information display screen P2 and display the rear image, the left image, and the right image on the upper side of the information display screen P2. Also, the arrangement, size, display / non-display of each image may be set (customized) by the operator.

[0085] In addition, when the work vehicle 10a detects an obstacle, the display processing unit 212 may display, on the information display screen P2, an imaging image including the obstacle among a plurality of imaging images captured by the camera 53 in an identifiable manner. For example, when the obstacle sensor 54 of the work vehicle 10a detects an obstacle in front of the work vehicle 10a, the display processing unit 212 highlights and displays the front image on the information display screen P2 as shown in FIG. 6A.

[0086] In this way, the display processing unit 212 displays, on the map of the selection screen P1, the respective icon images of the plurality of pre-registered work vehicles 10 in a display mode corresponding to the current state of the work vehicle 10 (see FIG. 5), and displays, on the information display screen P2, an imaging image of the periphery of the work vehicle 10 captured by the camera 53 mounted on the work vehicle 10 corresponding to the icon image selected by the operator among the plurality of icon images on the selection screen P1 (see FIG. 6A).

[0087] In addition, the display processing unit 212 displays, on the information display screen P2, switching buttons ("NEXT" button, "BACK" button) for switching the display image. When the operator presses the "NEXT" button once, the display processing unit 212 enlarges and displays the front image (see FIG. 6C). Subsequently, when the operator presses the "NEXT" button once, the display processing unit 212 enlarges and displays the right-side image. Subsequently, when the operator presses the "NEXT" button once, the display processing unit 212 enlarges and displays the rear image. Subsequently, when the operator presses the "NEXT" button once, the display processing unit 212 enlarges and displays the left-side image. The display processing unit 212 switches the front image, the right-side image, the rear image, and the left-side image in this order each time the operator presses the "NEXT" button. In addition, the display processing unit 212 switches the front image, the left-side image, the rear image, and the right-side image in this order each time the operator presses the "BACK" button. In addition, when the operator presses the "4CAM" button on the information display screen P2 shown in FIG. 6C, the display processing unit 212 displays the imaging images in four directions (front image, rear image, left-side image, and right-side image) on the information display screen P2 (see FIG. 6A).

[0088] In addition, the display processing unit 212 may be able to change (zoom) the display magnification (field of view size) of the captured image on the information display screen P2 shown in FIGS. 6A to 6C. For example, on the information display screen P2 shown in FIG. 6C, when the operator performs an enlargement operation (pinch out), the display processing unit 212 enlarges and displays (zooms in) the front image, and when the operator performs a reduction operation (pinch in), the display processing unit 212 reduces and displays (zooms out) the front image. Further, the display processing unit 212 may change the imaging range (imaging direction) of the camera 53 according to the operation of the operator.

[0089] In addition, the display processing unit 212 may surround the detected obstacle with a frame image and highlight it, or track the obstacle in the captured image. Further, the display processing unit 212 may display the distance information to the detected obstacle and the information on the type of the obstacle (person, vehicle, structure, material, etc.) on the information display screen P2. Note that the camera 53 may be arranged so that a part of the vehicle body of the work vehicle 10 is reflected therein so as to easily grasp the distance to the detected obstacle.

[0090] In addition, the display processing unit 212 displays a "START" button and a "STOP" button on the information display screen P2. When the operator presses the "START" button, the reception processing unit 213 receives the operation and outputs an instruction to start or resume the automatic driving to the work vehicle 10. For example, when the work vehicle 10a detects an obstacle and stops the automatic driving, if the operator selects the work vehicle 10a on the selection screen P1 (see FIG. 5) and confirms the safety of the work vehicle 10a on the information display screen P2, the "START" button is pressed. As a result, the reception processing unit 213 outputs an instruction to resume the automatic driving to the work vehicle 10a, and the work vehicle 10a resumes the automatic driving according to the instruction. As another embodiment, the reception processing unit 213 may permit the pressing operation of the "START" button on the condition that no obstacle is detected.

[0091] Also, for example, when the operator selects the work vehicle 10b on the selection screen P1 and presses the "STOP" button on the information display screen P2 while the work vehicle 10b is in automatic driving, the reception processing unit 213 outputs an instruction to stop the automatic driving of the work vehicle 10b. Thereby, the work vehicle 10b stops the automatic driving according to the instruction.

[0092] In addition, when the operator presses the "MAP" button on the information display screen P2, the display processing unit 212 switches to the selection screen P1 (see FIG. 4). In this way, the operator can switch between the selection screen P1 and the information display screen P2.

[0093] As another embodiment, the display processing unit 212 may display the selection screen P1 and the information display screen P2 side by side in the horizontal or vertical direction. For example, as shown in FIG. 7, the display processing unit 212 displays the selection screen P1 in the left display area of the operation display unit 23 and displays the information display screen P2 corresponding to the work vehicle 10a selected on the selection screen P1 in the right display area. Also, when the operator presses the "MAP" button on the information display screen P2 shown in FIG. 7, the display processing unit 212 enlarges and displays the selection screen P1 (see FIG. 5), and when the operator presses the "CAMERA" button, the display processing unit 212 enlarges and displays the information display screen P2 (see FIG. 6A). In addition, the display processing unit 212 may display the front image below the information display screen P2 and display the rear image, the left image, and the right image above the information display screen P2. Also, the arrangement, size, display / non-display of each image may be set (customized) by the operator.

[0094] In addition, the operation control unit 21 gives an automatic driving instruction to the work vehicle 10. For example, when the operator selects a field and presses a start button (not shown) to start the automatic driving of the work vehicle 10, the operation control unit 21 outputs the route data of the target route to the work vehicle 10. When the route data generated on the operation terminal 20 is transferred to the work vehicle 10, the work vehicle 10 stores it in the storage unit 12. The work vehicle 10 executes an automatic driving process based on the route data while detecting the current position of the work vehicle 10 by the positioning antenna 174. Note that the current position of the work vehicle 10 usually coincides with the position of the positioning antenna 174.

[0095] Further, the work vehicle 10 is configured to be able to perform automatic driving when the current position coincides with or is within a predetermined range of the driving start position within the field (when the start condition for automatic driving is satisfied). When the work vehicle 10 satisfies the start condition and the start button is pressed on the operation screen (not shown) by the operator to give a driving start instruction, the driving process unit 111 of the work vehicle 10 starts the automatic driving of the target route.

[0096] Note that the setting process in the above-described setting process unit 211 and the process of starting automatic driving in the operation control unit 21 may be executed by a device different from the operation terminal 20. For example, on a tablet terminal different from the operation terminal 20, the operator performs the setting operation of each setting information, the generation operation of the target route, the automatic driving instruction, and the like. In this case, the operation terminal 20 may be provided only with a monitoring function for monitoring the work vehicle 10, and the tablet terminal may perform the setting process of each setting information, the generation process of the target route, the automatic driving process, and the like. Further, the operation terminal 20 according to the present embodiment may be realized by adding the monitoring function to the tablet terminal. Further, as another embodiment, the operation control unit 21 may display the selection screen P1 and the information display screen P2 on different devices, respectively. For example, the operation control unit 21 may display the selection screen P1 on the operation terminal 20 and display the information display screen P2 on a device different from the operation terminal 20 (such as the tablet terminal, the operator's or worker's mobile terminal).

[0097] Note that the operation terminal 20 may be able to access a website (agricultural support site) of an agricultural support service provided by a server (not shown) via the communication network N1. In this case, the operation terminal 20 can function as an operation terminal of the server when a browser program is executed by the operation control unit 21. And the server includes each of the above-described processing units and executes each process.

[0098] [Display control process] Hereinafter, an example of the display control process executed by the automatic driving system 1 will be described with reference to FIG. 8.

[0099] Note that the present invention can be regarded as an invention of a display control method (an example of the display control method of the present invention) that executes one or more steps included in the display control process. Also, one or more steps included in the display control process described here may be omitted as appropriate. Note that the execution order of each step in the display control process may be different within a range that produces the same operational effects. Further, here, the case where the operation control unit 21 executes each step in the display control process is taken as an example for description, but a display control method in which one or more processors execute each step in the display control process in a distributed manner is also conceivable as another embodiment.

[0100] In step S1, the operation control unit 21 determines whether it has acquired the position information of the work vehicle 10. When the operation control unit 21 acquires the position information of the current position from each registered work vehicle 10 that is a monitoring target (S1: Yes), the process proceeds to step S2. The operation control unit 21 waits until the position information of each work vehicle 10 is acquired (S1: No).

[0101] In step S2, the operation control unit 21 causes the operation display unit 23 to display a map of a predetermined area including the current position of the work vehicle 10 on the selection screen P1, and on the map, displays an icon image (vehicle icon) of the work vehicle 10 at a position corresponding to the current position of the work vehicle 10. For example, when the work vehicles 10a and 10b are registered, as shown in FIG. 3, the operation control unit 21 causes the selection screen P1 to display a map of a predetermined area including the respective current positions of the work vehicles 10a and 10b, and on the map, displays an icon image A1 of the work vehicle 10a at a position corresponding to the current position of the work vehicle 10a, and displays an icon image B1 of the work vehicle 10b at a position corresponding to the current position of the work vehicle 10b.

[0102] Next, in step S3, the operation control unit 21 determines the current state of the work vehicle 10. Specifically, the operation control unit 21 determines whether the work vehicle 10 is automatically traveling or has stopped automatic traveling. As another embodiment, the operation control unit 21 may determine whether the work vehicle 10 has detected an obstacle, whether the work has been completed, whether the remaining amount of the sprayed material or fuel has become less than the specified amount, whether the yield of the harvested product has become equal to or greater than the specified amount, whether a DPF regeneration operation is required, whether a urea water replenishment operation is required, and the like.

[0103] Next, in step S4, the operation control unit 21 displays the identification information in a display mode corresponding to the current state of the work vehicle 10. Specifically, the operation control unit 21 displays an icon image on the map so as to be able to identify whether the work vehicle 10 is automatically traveling or has stopped automatic traveling. For example, as shown in FIG. 4, when the work vehicle 10a has stopped automatic traveling and the work vehicle 10b is automatically traveling, the operation control unit 21 surrounds the icon image A1 with a frame image A2 in the first display mode (for example, a red frame image), and surrounds the icon image B1 with a frame image B2 in the second display mode (for example, a blue frame image). Note that the operation control unit 21 may make the frame image non-displayed when the work vehicle 10 is automatically traveling normally.

[0104] As another embodiment, the operation control unit 21 may vary the display mode of the frame image according to the cause of the stoppage of the work vehicle 10. For example, when the work vehicle 10a stops automatic driving by detecting an obstacle, the operation control unit 21 causes the frame image A2 to be lit in red. On the other hand, when the work vehicle 10a stops automatic driving by finishing the work, the operation control unit 21 causes the frame image A2 to blink in red.

[0105] Next, in step S5, the operation control unit 21 determines whether or not it has received a selection operation of the work vehicle 10 from the operator on the selection screen P1. Specifically, on the selection screen P1 shown in FIG. 4, the operation control unit 21 receives an operation of selecting the icon image of the work vehicle 10 from the operator. For example, as shown in FIG. 5, when the operator touches the icon image A1 displayed on the selection screen P1 with a finger, the operation control unit 21 receives the touch operation. When the operation control unit 21 receives a selection operation of the work vehicle 10 from the operator (S5: Yes), the process proceeds to step S6. On the other hand, when the operation control unit 21 does not receive a selection operation of the work vehicle 10 from the operator (S5: No), the process proceeds to step S7.

[0106] Next, in step S6, the operation control unit 21 causes the operation display unit 23 to display predetermined information regarding the work vehicle 10 selected by the operator. Specifically, the operation control unit 21 causes information such as information regarding the current surrounding environment of the work vehicle 10, information regarding the current traveling state of the work vehicle 10, and information regarding the current working state of the work vehicle 10 to be displayed on the information display screen P2. For example, when the operator selects the icon image A1 of the work vehicle 10a (see FIG. 5), the operation control unit 21 causes the imaging images (front image, rear image, left image, and right image) of the surroundings of the work vehicle 10a captured by the camera 53 mounted on the work vehicle 10a to be displayed on the information display screen P2 as shown in FIG. 6A.

[0107] The operation control unit 21 may display the information display screen P2 on a page different from the selection screen P1 (see FIG. 5) (see FIG. 6A), or may display the information display screen P2 and the selection screen P1 side by side on the same page (see FIG. 7).

[0108] Further, in the information display screen P2, the operation control unit 21 accepts operations such as switching the captured images (front image, rear image, left image, and right image) from the operator (see FIGS. 6A to 6C), starting, resuming, or stopping the automatic driving, and switching to the selection screen P1, and switches the display screen according to the operation.

[0109] In step S7, the operation control unit 21 determines whether or not it has received an end operation from the operator. For example, when the operator performs an end operation (for example, a logoff operation) to end the monitoring of the work vehicle 10, the operation control unit 21 receives the end operation (S7: Yes) and ends the display control process. When the operation control unit 21 does not receive an end operation from the operator (S7: No), the process returns to step S1 and the above-described process is executed. The operation control unit 21 repeatedly executes the processes of steps S1 to S6 until it receives an end operation from the operator. Further, while the operation control unit 21 does not receive a selection operation of the work vehicle 10 from the operator on the selection screen P1 (S5: No), on the selection screen P1, the position of the icon image is updated in real time according to the current position of the work vehicle 10. As described above, the operation control unit 21 executes the display control process.

[0110] As described above, the operation terminal 20 according to the present embodiment controls the display information of the operation terminal 20. Specifically, the operation terminal 20 causes the selection screen P1 (first screen) of the operation terminal 20 to display the work vehicle 10 registered in advance so that the user (operator) can select it, and causes the information display screen P2 (second screen) of the operation terminal 20 to display predetermined information regarding the work vehicle 10 selected by the user. For example, the operation terminal 20 causes the information display screen P2 to display at least any one of information regarding the current surrounding environment of the work vehicle 10, information regarding the current traveling status of the work vehicle 10, and information regarding the current work status of the work vehicle 10.

[0111] According to the above configuration, the operator can grasp the registered work vehicle 10 on the selection screen P1. Further, the operator can confirm information regarding the work vehicle 10 selected on the selection screen P1. Thereby, the operator can remotely monitor the work vehicle 10. Therefore, it is possible to improve the convenience of the operation terminal 20 that displays information regarding the work vehicle 10 capable of autonomous driving.

[0112] In addition, the operation terminal 20 may display not only information regarding the current traveling status (such as vehicle speed and engine speed) of the work vehicle 10 on the information display screen P2, but also an operation unit enabling setting operations. For example, the operation terminal 20 may display the operation unit together with the camera image on the information display screen P2, or may display the operation unit on another screen (a setting screen dedicated to the operation unit) switched from the camera image. Further, in the configuration of displaying the operation unit together with the camera image, the operator may be able to set the operation unit to be displayed, such as only the vehicle speed operation unit or only the engine speed operation unit. According to the configuration in which the operator can set the operation unit, for example, when the operation terminal 20 is realized by a mobile terminal (such as a smartphone), it is possible to prevent a decrease in operability and visibility due to an increase in the operation unit.

[0113] Further, the operation terminal 20 may display information on the type of work (type of the work machine 14) and the target ground height of the work machine 14 on the information display screen P2, and may also display an operation unit for setting the target ground height. For example, the operation terminal 20 may display the operation unit together with the camera image on the information display screen P2, or may display the operation unit on another screen (a dedicated setting screen for the operation unit) switched from the camera image. Further, in the configuration of displaying the operation unit together with the camera image, the operator may be able to set the operation units to be displayed, including the operation units for the traveling status. According to the configuration in which the operator can set the operation unit, for example, when the operation terminal 20 is realized by a mobile terminal (such as a smartphone), it is possible to prevent a decrease in operability and visibility due to an increase in the operation unit.

[0114] [Other Embodiments] The present invention is not limited to the above-described embodiments, and may be the following embodiments.

[0115] For example, when the operator selects the work vehicle 10 on the selection screen P1, the operation control unit 21 may display information on the selected work vehicle 10 within the selection screen P1. For example, as shown in FIG. 9, when the operator selects the icon image A1 of the work vehicle 10a on the selection screen P1, the operation control unit 21 displays an information display screen P2 including the current state of the work vehicle 10a, for example, that the automatic traveling has been stopped due to detection of an obstacle, the name of the work vehicle 10a, the work content, the progress status of the work, etc., in the vicinity of the icon image A1 (pop-up display). For example, the operation control unit 21 may be configured such that the progress status can be confirmed by displaying a screen including the route set on the information display screen P2 and the icon images (vehicle icons) on the route. Further, the operation control unit 21 may display a screen including the camera image, the set route, and the icon images on the route, or may switch the camera image and display a screen including the set route and the icon images on the route.

[0116] Also, for example, as shown in FIG. 10, when the operator selects the icon image B1 of the work vehicle 10b on the selection screen P1, the operation control unit 21 displays (pops up) an information display screen P2 including information such as the current state of the work vehicle 10b, for example, being in automatic driving, the name of the work vehicle 10b, the work content, and the progress of the work, in the vicinity of the icon image B1.

[0117] Also, as shown in FIGS. 9 and 10, the operation control unit 21 may display a "camera image" button on the information display screen P2. When the operator presses the "camera image" button, the operation control unit 21 switches to the information display screen P2 (see FIG. 6A) that displays the captured image of the camera 53.

[0118] Also, when the operator touches (short presses) the icon image of the work vehicle 10 on the selection screen P1 once in a short time, the operation control unit 21 pops up the information display screen P2 on the selection screen P1 (see FIGS. 9 and 10). When the operator touches (long presses) the icon image of the work vehicle 10 on the selection screen P1 once in a long time, the operation control unit 21 may switch from the selection screen P1 to the information display screen P2 (see FIG. 6A).

[0119] In the above-described embodiment, when the operator selects the work vehicle 10 on the selection screen P1 (see FIG. 5), the operation control unit 21 displays predetermined information regarding the selected work vehicle 10 on the information display screen P2. As another embodiment, when the registered work vehicle 10 transitions to a predetermined state, the operation control unit 21 may display predetermined information regarding the work vehicle 10 on the information display screen P2. Specifically, when the work vehicle 10 in automatic driving detects an obstacle, or when the work vehicle 10 automatically drives in a specific area, the operation control unit 21 displays predetermined information regarding the work vehicle 10 on the information display screen P2. The specific area is, for example, a route between fields on a road.

[0120] For example, when the work vehicle 10a automatically travels along the route between fields, the operator needs to monitor the driving situation. Therefore, when the work vehicle 10a starts automatic driving along the route between fields, even if the operation control unit 21 does not receive an operation by which the operator selects the work vehicle 10a on the selection screen P1, the operation control unit 21 automatically switches the selection screen P1 to the information display screen P2 (see FIG. 6A).

[0121] Also, when the work vehicle 10 detects an obstacle and stops, it is necessary to promptly notify the operator. Therefore, when the work vehicle 10a detects an obstacle within the stop control range, even if the operation control unit 21 does not receive an operation by which the operator selects the work vehicle 10a on the selection screen P1, the operation control unit 21 automatically switches the selection screen P1 to the information display screen P2. In this configuration, the operation control unit 21 may switch to the information display screen P2 on the condition that the operator has selected the work vehicle 10a on the selection screen P1 when the work vehicle 10a detects an obstacle within the notification control range or the deceleration control range.

[0122] Note that the configuration for automatically displaying predetermined information regarding the work vehicle 10 on the information display screen P2 is not limited to the case where the work vehicle 10 in automatic driving detects an obstacle or the case where the work vehicle 10 automatically travels in a specific area. For example, the operation control unit 21 may automatically display predetermined information regarding the work vehicle 10 on the information display screen P2 when specific operations such as replenishment of spraying materials or fuel, discharge of harvested products, regeneration of DPF, and replenishment of aqueous urea solution are required.

[0123] In addition, when the vehicle control device 11 satisfies a predetermined condition, it may permit the automatic driving of the inter-field route of the work vehicle 10. For example, the vehicle control device 11 may permit the automatic driving of the inter-field route only while the operator is looking at the camera image on the information display screen P2. Also, for example, the vehicle control device 11 may permit the automatic driving of the inter-field route on the condition that the remaining battery level of the operation terminal 20 is equal to or more than a predetermined amount before starting the automatic driving of the inter-field route. Thereby, it is possible to prevent the power of the operation terminal 20 from being cut off while the work vehicle 10 is automatically driving on the inter-field route, making it impossible for the operator to remotely monitor the work vehicle 10.

[0124] Also, for example, the vehicle control device 11 may permit the automatic driving of the inter-field route on the condition that a plurality of operation terminals 20 are connected. Thereby, it is possible to prevent the operator from being unable to remotely monitor the work vehicle 10 due to the power being cut off or a communication abnormality occurring in one operation terminal 20 while the work vehicle 10 is automatically driving on the inter-field route. Note that the vehicle control device 11 permits the automatic driving of the inter-field route even with the connection of one operation terminal 20, but may prompt to connect to another operation terminal 20 when the connection of the said operation terminal 20 is interrupted. In this case, the vehicle control device 11 stops the automatic driving of the inter-field route until another operation terminal 20 is connected.

[0125] In addition, when the communication with the operation terminal 20 is cut off, or the startup of the application for displaying the selection screen P1 and the information display screen P2 on the operation terminal 20 stops, or it enters the background state, the vehicle control device 11 may stop the automatic driving of the inter-field route. Also, when stopping the automatic driving of the inter-field route, the vehicle control device 11 may automatically move the work vehicle 10 closer to the road shoulder.

[0126] Here, for example, when the work vehicle 10 automatically travels on a field-to-field path set on a road (e.g., a rural road), in order to ensure safety, it may be considered to adopt an operation of constantly displaying an image around the work vehicle 10 on the operation terminal 20. In this case, for example, while the work vehicle 10a is automatically traveling on the field-to-field path, it is desirable that the operation control unit 21 is configured to constantly display a captured image of the camera 53 of the work vehicle 10a on the information display screen P2 of the operation terminal 20 (see FIG. 6A). However, if this configuration is adopted, for example, when the work vehicle 10a is automatically traveling on the field-to-field path and another work vehicle 10 (e.g., the work vehicle 10b) detects an obstacle and stops, information regarding the work vehicle 10b may not be displayed on the operation terminal 20, resulting in a problem that the operator cannot grasp the stopped state of the work vehicle 10b.

[0127] Therefore, as shown in FIG. 11, the operation control unit 21 causes information (message M1) indicating that the work vehicle 10b has detected an obstacle and stopped to be displayed (popped up) on the information display screen P2 that displays the captured image corresponding to the work vehicle 10a that is automatically traveling on the field-to-field path. Further, the operation control unit 21 displays a selection button for selecting whether to switch to the information display screen P2 that displays the information of the work vehicle 10b in the message M1 and accepts a selection operation from the operator. When the operator presses the "Yes" button in the message M1, the operation control unit 21 switches to the information display screen P2 that displays the information of the work vehicle 10b. In this case, since the captured image regarding the work vehicle 10a that is automatically traveling on the field-to-field path is no longer displayed, the operation control unit 21 outputs a stop instruction for automatic driving to the work vehicle 10a. When the work vehicle 10a acquires the stop instruction, it may temporarily stop on the spot or may travel to a predetermined location (such as a road shoulder or a shelter) and then temporarily stop. Further, the operator may remotely operate the work vehicle 10a by the operation terminal 20 to move it to a predetermined location.

[0128] In this way, the operation control unit 21 causes the work vehicle 10 registered in advance to be displayed on the selection screen P1 (first screen) of the operation terminal 20, and when the work vehicle 10 in automatic travel detects an obstacle, or when the work vehicle 10 automatically travels in a specific area, predetermined information regarding the work vehicle 10 may be displayed on the information display screen P2 (second screen) of the operation terminal 20.

[0129] As another embodiment of the present invention, the operation control unit 21 may display, on the selection screen P1, the identification information of the worker registered in advance for the work vehicle 10 in association with the identification information of the work vehicle 10. For example, when a worker X who manages and assists the work by the work vehicle 10a and a worker Y who manages and assists the work by the work vehicle 10b are registered, the operation control unit 21, as shown in FIG. 12, on the map of the selection screen P1, displays the icon image C1 of the worker X in association with the icon image A1 of the work vehicle 10a, and displays the icon image C2 of the worker Y in association with the icon image B1 of the work vehicle 10b. The operation control unit 21 may display the icon image C1 and the icon image C2 in different display modes from each other.

[0130] Thereby, an operator who monitors the work vehicle 10 can easily grasp that workers are registered for each work vehicle 10. Note that the operation control unit 21 may acquire the position information of the worker and display the icon image of the worker at the current position of the worker. For example, the operation control unit 21 may acquire the current positions of the worker X and the worker Y based on the position information of the operation terminals 20 held by the worker X and the worker Y, or may acquire the current position of the worker X based on the position information of a remote controller for operating the work vehicle 10a at a short distance, and acquire the current position of the worker Y based on the position information of a remote controller for operating the work vehicle 10b at a short distance. Further, the operation control unit 21 may be configured to display the position of the remote controller instead of the configuration for displaying the position of the worker.

[0131] In addition, when an operator (such as operator X or Y) different from the operator who remotely monitors the work vehicle 10 starts an application that displays a selection screen P1 and an information display screen P2 on his or her own operation terminal 20 different from the operator's operation terminal 20, the operator may request the operator to delegate the operator's authority to his or her own operation terminal 20.

[0132] Also, on the selection screen P1 shown in FIG. 12, the operation control unit 21 may receive an operation of selecting the operator's icon image. For example, when the operator selects the icon image C1 of operator X on the selection screen P1, the operation control unit 21 displays an instruction screen P3 for outputting a predetermined instruction to operator X as shown in FIG. 13. For example, when the operator presses the "Notify the operator" button on the instruction screen P3, the operation control unit 21 notifies the mobile terminal held by operator X of an instruction to confirm an obstacle, an instruction to remove an obstacle, etc. As another embodiment, when the operator selects the icon image C1 of operator X on the selection screen P1, the operation control unit 21 may start an application for transmitting and receiving messages with the mobile terminal of operator X and notify the mobile terminal of operator X of a message.

[0133] Also, as another embodiment, when the operator rides on the work vehicle 10 and manually travels between fields or when the operator rides on the work vehicle 10 and automatically travels based on the route between fields, or when the operator rides on the work vehicle 10 and manually travels to the storage location or the operator rides on the work vehicle 10 and automatically travels the travel route to the storage location when storing the work vehicle 10 at the storage location after the work of the day is completed, based on the arrival time of the work end position of the work vehicle 10 and the position of the operator or the operator, in accordance with the arrival time of the work end position of the work vehicle 10, the operator or the operator may be notified to the operation terminal 20 at the timing when they can arrive at the work end position.

[0134] As another embodiment, the operation control unit 21 may display a list of workers near the work vehicle 10a and output a predetermined instruction to the worker selected by the operator from the list. As another embodiment, the worker may be registered with the work vehicle 10 or with the work plan for that day.

[0135] In the above-described embodiment, the operation control unit 21 displays the pre-registered work vehicle 10 on the map of the selection screen P1 so that it can be selected (see FIGS. 3 and 4). As another embodiment, the operation control unit 21 may display a list (registered vehicle list) of the pre-registered work vehicles 10 on the selection screen P1. Further, the operation control unit 21 may be provided with a filter function for determining the work vehicle 10 to be displayed on the map or the list. For example, the items of the filter function include vehicles displayed on the map (within a predetermined scale), operating vehicles, vehicles moving between fields, stopped vehicles, vehicles with problems, unmonitored vehicles not participating in the current work, vehicles stopped outside the field due to a problem or detection of an obstacle, and vehicles that immediately need or will soon need replenishment of work implement materials (such as chemicals, fertilizers, seedlings, etc.). The operation control unit 21 can display the vehicles of the set items on the map or the list, or display vehicles other than the vehicles of the set items on the map or the list.

[0136] As another embodiment, the operation control unit 21 may display the work vehicle 10 on the map so that it can be selected and display a list of work vehicles 10 that can be selected on the same page as the selection screen P1. As another embodiment, the operation control unit 21 may display the work vehicle 10 on the map so that it can be selected and display the work vehicle 10 on the list so that it can be selected on the same page as the selection screen P1. When the work vehicle 10 is selected in the list, a map of the position of the selected work vehicle 10 may be displayed.

[0137] Further, the operation control unit 21 may display the identification information in a display mode corresponding to the current state of the work vehicle 10 in the list. For example, when the work vehicle 10a has stopped automatic driving, the operation control unit 21 causes the name of the work vehicle 10a to be highlighted in the list.

[0138] As described above, in the above-described embodiment, the operation terminal 20 corresponds to the display control system according to the present invention. However, the display control system according to the present invention may be configured by the operation terminal 20 alone, or may be configured to include the work vehicle 10 and the operation terminal 20. For example, the display control system according to the present invention may be the automatic driving system 1.

[0139] [Remote operation system] The automatic driving system 1 may construct a remote operation system 100. The remote operation system 100 is configured to include the work vehicle 10, the operation terminal 20, and the remote operation device 30, as shown in FIG. 14. The work vehicle 10, the operation terminal 20, and the remote operation device 30 may each be one or a plurality of units. In the present embodiment, an example in which the remote operation system 100 is configured by a plurality of work vehicles 10, one operation terminal 20, and one remote operation device 30 will be described.

[0140] The remote operation system 100 is a system in which a user (operator, monitor) operates (remote drives) the remote operation device 30 at a remote location away from the field where the work vehicle 10 performs work, and causes the work vehicle 10 to travel. FIG. 15 schematically shows the remote operation device 30 installed at a remote location (for example, a monitoring room) and the state of remote operation. The operator can remotely drive the work vehicle 10 while viewing the captured image captured by the camera 53 of the work vehicle 10 to be remotely operated in the monitoring room.

[0141] As shown in FIG. 14, the remote operation device 30 is an information processing device including an operation control unit 31, a storage unit 32, a display unit 33, an operation unit 34, a communication unit 35, and the like. For example, the remote operation device 30 may be a stationary (desktop) personal computer.

[0142] The communication unit 35 is a communication interface for connecting the remote control device 30 to the communication network N1 by wire or wirelessly and performing data communication according to a predetermined communication protocol with external devices such as the work vehicle 10 and the operation terminal 20 via the communication network N1.

[0143] The display unit 33 is a monitor such as a liquid crystal display or an organic EL display that displays various types of information. Note that the display unit 33 may be configured as a touch panel that accepts touch operations.

[0144] The operation unit 34 accepts various operations of the operator. Specifically, as shown in FIG. 16, the operation unit 34 includes a steering wheel 341 for steering the work vehicle 10, an accelerator pedal 342 for increasing the vehicle speed of the work vehicle 10, a brake pedal 343 for decelerating and stopping the work vehicle 10, and a neutral pedal 344 for switching the gear position to the neutral position. In addition, a plurality of operation buttons (for example, a mode switch button BT1) are arranged on the steering wheel 341. As another embodiment, the plurality of operation buttons may be provided in a device different from the steering wheel 341. For example, an operation device (not shown) provided with the plurality of operation buttons may be arranged in the monitoring room. The operation device is included in the operation unit 34.

[0145] The storage unit 32 is a non-volatile storage unit such as an HDD or an SSD that stores various types of information. A control program for the operation control unit 31 to execute various processes is stored in the storage unit 32. For example, the control program is non-temporarily recorded on a computer-readable recording medium such as a CD or a DVD, and is read by a predetermined reading device (not shown) and stored in the storage unit 32. Note that the control program may be downloaded from a server (not shown) to the remote control device 30 via the communication network N1 and stored in the storage unit 32.

[0146] The operation control unit 31 includes control devices such as a CPU, a ROM, and a RAM. The CPU is a processor that executes various arithmetic processes. The ROM is a non-volatile storage unit in which control programs such as BIOS and OS for causing the CPU to execute various arithmetic processes are stored in advance. The RAM is a volatile or non-volatile storage unit that stores various information and is used as a temporary storage memory (working area) for various processes executed by the CPU. Then, the operation control unit 31 controls the remote operation device 30 by causing the CPU to execute various control programs stored in advance in the ROM or the storage unit 32.

[0147] Specifically, the operation control unit 31 executes a process of switching the work vehicle 10 to a remote operation mode in which an operator can remotely operate the work vehicle 10 by the remote operation device 30. For example, when the operator presses a mode switching button BT1 (see FIG. 16) disposed on the steering wheel 341 in a state where a camera image of the work vehicle 10 capable of remote operation is displayed on the display unit 33, the operation control unit 31 switches the traveling mode of the work vehicle 10 from an automatic traveling mode capable of automatic traveling regardless of the operator's operation to a remote operation mode. In this way, the operation control unit 31 permits the switching to the remote operation mode on the condition that the captured image of the camera 53 mounted on the work vehicle 10 to be remotely operated is displayed on the display unit 33 and the operator can confirm the situation around the work vehicle 10.

[0148] In addition, the operation control unit 31 may cause the display unit 33 to display the camera image according to an instruction from the operation terminal 20. For example, the operator selects the work vehicle 10 for remote operation on the selection screen P1 (see FIG. 3) displayed on the operation terminal 20. When the operator selects the work vehicle 10, the operation terminal 20 displays the information display screen P2 as shown in FIG. 17A, and on the information display screen P2, a camera image of the work vehicle 10 and a request button BT2 ("mode switch request" button) for requesting to switch the driving mode to the remote operation mode are displayed. When the operator presses the request button BT2, the operation terminal 20 displays (pops up) a message M2 on the information display screen P2 to prompt the operator to press the mode switch button BT1 (see FIG. 16) of the handle 341 of the remote operation device 30. When the operator presses the OK button of the message M2, the operation terminal 20 outputs mode switch information to the remote operation device 30. The mode switch information includes identification information of the work vehicle 10 to be remotely operated selected by the operator and the like.

[0149] When the operation control unit 31 of the remote operation device 30 acquires the mode switch information from the operation terminal 20, as shown in FIG. 18, it causes the display unit 33 to display a camera image of the work vehicle 10 to be remotely operated. Specifically, the operation control unit 31 displays the front image Pa, the rear image Pb, the left image Pc, and the right image Pd of the work vehicle 10. Note that the camera image displayed on the remote operation device 30 may be the same as the camera image displayed on the operation terminal 20. When the operation control unit 31 causes the display unit 33 to display the camera image of the work vehicle 10, it permits the switching to the remote operation mode. Thereby, for example, when the operator presses the mode switch button BT1 (see FIG. 16) of the handle 341, the operation control unit 31 accepts the operation and switches the driving mode from the automatic driving mode to the remote operation mode. Note that when switching to the remote operation mode, the travel processing unit 111 of the work vehicle 10 may temporarily stop the work vehicle 10.

[0150] When the traveling mode is switched to the remote operation mode, the operation control unit 31 receives the driving operation of the operator and outputs a traveling instruction corresponding to the operation to the work vehicle 10. The traveling processing unit 111 of the work vehicle 10 drives the work vehicle 10 according to the traveling instruction. For example, when the operator turns the steering wheel 341 to the right, the traveling processing unit 111 steers the work vehicle 10 to the right, and when the operator turns the steering wheel 341 to the left, the traveling processing unit 111 steers the work vehicle 10 to the left. Also, when the operator depresses the accelerator pedal 342, the traveling processing unit 111 accelerates the work vehicle 10, and when the operator depresses the brake pedal 343, the traveling processing unit 111 decelerates the work vehicle 10. Thereby, the operator can remotely operate (remote drive) the work vehicle 10 at a remote location. Further, for example, even when the work vehicles 10 recognize each other as obstacles and stop, the operator can cause the work vehicle 10 to perform a retreating travel by remote operation. Also, the operator can switch the traveling mode to the remote operation mode and remotely operate at a remote location even while remotely monitoring the work vehicle 10.

[0151] Also, in the remote operation mode, when the operator depresses the accelerator pedal 342, the traveling processing unit 111 increases the vehicle speed according to the depression amount. Note that the traveling processing unit 111 shifts gears according to the depression amount within the range of the set vehicle speed (maximum vehicle speed) set for the work vehicle 10. For example, when the maximum vehicle speed when working in the field is set to 10 km, when the operator depresses the accelerator pedal 342 to the maximum, the traveling processing unit 111 increases the vehicle speed to 10 km, and when the maximum vehicle speed when moving between fields is set to 30 km, when the operator depresses the accelerator pedal 342 to the maximum, the traveling processing unit 111 increases the vehicle speed to 30 km. In this way, although the setting of the maximum vehicle speed is different between when traveling in the field and when traveling outside the field, since the vehicle speed is controlled by the accelerator opening (depression amount) with respect to the maximum vehicle speed, it becomes easier for the operator to perform the driving operation.

[0152] Further, while the operation control unit 31 is set to the remote operation mode, it may cause the display unit 33 to display vehicle information such as the status of the work vehicle 10, engine speed, and vehicle speed. Further, while the operation control unit 31 is set to the remote operation mode, it may cause the running sound of the work vehicle 10 to be output from the remote operation device 30. For example, the operation control unit 31 may acquire the sound collected by a microphone mounted on the work vehicle 10 and output it from a speaker installed in the remote operation device 30 or the monitoring room. Generally, it is difficult to grasp the speed feeling and the working load state in a silent state. However, according to the above configuration, the operator can remotely operate with a feeling close to actual driving by playing the working sound.

[0153] Further, in the remote operation mode, the operation control unit 31 may output an operation instruction corresponding to an operation on an operation button (see FIG. 16) arranged on the steering wheel 341 to the work vehicle 10. For example, the operation buttons include a button for switching the traveling direction of the work vehicle 10, buttons for raising and lowering the work implement 14 (raising button, lowering button), a button for temporarily stopping the work vehicle 10, and the like. For example, when the operator presses the lowering button, the vehicle control device 11 acquires an operation instruction from the remote operation device 30 and lowers the work implement 14.

[0154] The operation control unit 31 may be able to set the functions assigned to the respective operation buttons. For example, the operation control unit 31 causes a function setting screen (not shown) to be displayed on the display unit 33 and accepts an operation for the operator to select the functions assigned to the respective operation buttons. The operation control unit 31 assigns the functions to the respective operation buttons according to the selection operation of the operator.

[0155] Here, when the operation control unit 31 displays the camera image on the display unit 33, it may be displayed such that a part of the vehicle body is reflected in each image. For example, the operation control unit 31 displays the front image Pa so that a part of the bonnet of the work vehicle 10 and a part of each of the left and right wheels are reflected, and displays the rear image Pb so that a part or all of the working machine 14 and the working track are reflected. The left-side image Pc is displayed so that a part of the left-side vehicle body of the work vehicle 10 and a part of the left-side mirror are reflected, and the right-side image Pd is displayed so that a part of the right-side vehicle body of the work vehicle 10 and a part of the right-side mirror are reflected. Thereby, the operator can easily grasp the sense of distance from the vehicle body of the work vehicle 10 to the object (obstacle, boundary of the field shape, etc.), and it becomes easier to confirm the turning direction. Also, the operator can easily align the position of the work vehicle 10 during work. Further, the operator can easily confirm the working situation (such as the presence or absence of remaining tillage).

[0156] Also, the operation control unit 31 may be able to switch between a normal image and a virtual image (mirror image) for the rear image Pb. For example, the operation control unit 31 displays a normal rear image or a virtual rear image according to the setting operation of the operator. Also, when the work vehicle 10 is lowering the working machine 14 and working, the operation control unit 31 may display a virtual rear image, and when the work vehicle 10 is raising the working machine 14 and is in a work stop state, it may display a normal rear image.

[0157] Also, the operation control unit 31 may move the shooting range of the front image in accordance with the steering operation of the steering wheel 341. For example, when the operator turns the steering wheel 341 to the left, the operation control unit 31 displays the range on the front left side as the front image Pa according to the movement amount of the steering wheel 341. The vehicle control device 11 may move (rotate) the camera 53 according to the movement amount of the steering wheel 341 to change the shooting area. Also, the operation control unit 31 may change the magnification (zoom) of the camera image according to the operation of the operation button by the operator.

[0158] Further, the operation control unit 31 may synthesize and display the front image Pa and the side images (left image Pc and right image Pd). For example, when the operator turns the steering wheel 341 to the right, the operation control unit 31 synthesizes and displays the front image Pa that captures the front right range and the right image Pd. When the operator turns the steering wheel 341 to the left, the operation control unit 31 synthesizes and displays the front image Pa that captures the front left range and the left image Pc. Thereby, the operator can, for example, view the video in the turning direction, and thus can remotely operate with the feeling of driving a real vehicle. Also, without adding a camera, the operator can view the video in the diagonally forward direction when steering the steering wheel 341.

[0159] Further, the operation control unit 31 may display information such as the boundary line of the field outline, the path information, and the boundary line of the uncultivated land on the camera image. For example, the operation control unit 31 may synthesize and display an object image corresponding to each piece of the above information on each camera image. Thereby, the operator can easily grasp the traveling position when working.

[0160] Further, as shown in FIG. 19, the operation control unit 31 may synthesize and display grid lines G1 to G4 (marking lines) corresponding to the distance from the front end of the work vehicle 10 on the front image Pa. For example, the grid line G1 is displayed at the position (0 m) of the front end of the work vehicle 10, the grid line G2 is displayed at the position 1 m from the front end of the work vehicle 10, the grid line G3 is displayed at the position 2 m from the front end of the work vehicle 10, and the grid line G4 is displayed at the position 3 m from the front end of the work vehicle 10. Thereby, the operator can accurately grasp the distance to objects such as the boundary of the field outline and obstacles. Note that the operation control unit 31 may always display each of the grid lines G1 to G4, or may display them when the distance to the object becomes less than a threshold value (for example, less than 5 m). Also, the operation control unit 31 may set the intervals of the grid lines G1 to G4 wider as the vehicle speed of the work vehicle 10 is higher, and set the intervals of the grid lines G1 to G4 narrower as the vehicle speed of the work vehicle 10 is lower.

[0161] Further, for example, when the work vehicle 10 is being remotely operated in the remote operation mode, on the operation terminal 20, the selection screen P1 may be displayed so as to be able to identify that the work vehicle 10 is being remotely operated. For example, the operation terminal 20 may display the icon image of the work vehicle 10 being remotely operated in a predetermined color, or may pop up and display information (text information, mark, etc.) indicating that the work vehicle 10 is being remotely operated on the icon image of the work vehicle 10.

[0162] In the above-described embodiment, the remote operation device 30 is configured to enable remote operation of the work vehicle 10 selected by the operator among the plurality of work vehicles 10. However, as another embodiment, the remote operation device 30 may be installed corresponding to a specific one work vehicle 10. In this case, the operation by which the operator selects the work vehicle 10 on the operation terminal 20 may be omitted.

[0163] Further, as another embodiment of the remote operation system 100, the operation by which the operator requests to switch the traveling mode to the remote operation mode on the operation terminal 20 (see FIGS. 17A and 17B) may be omitted. In this case, for example, the operator may be able to switch to the remote operation mode by pressing the mode switch button BT1 (see FIG. 16) while the camera image of the work vehicle 10 is being displayed on the remote operation device 30.

[0164] Further, as another embodiment of the remote operation system 100, the remote operation device 30 may permit switching to the remote operation mode on the condition that the work vehicle 10 has detected an obstacle and stopped.

[0165] Further, as another embodiment of the remote operation system 100, when the camera image of the work vehicle 10 being remotely operated is no longer displayed on the display unit 33, the remote operation device 30 may release the remote operation mode and stop the work vehicle 10. Thereby, it is possible to prevent the work vehicle 10 from being remotely operated in a state where the operator cannot remotely monitor it.

[0166] In the remote operation system 100, the remote operation device 30 may be the operation terminal 20. Also, the display unit 33 of the remote operation device 30 may be the operation terminal 20. The remote operation device 30 is an example of the operation terminal of the present invention.

[0167] As another embodiment of the present invention, on the selection screen P1 (see FIG. 3), the display processing unit 212 may display each work vehicle 10 in a distinguishable manner. Specifically, the display processing unit 212 displays the icon image A1 of the work vehicle 10a in the first display mode and the icon image B1 of the work vehicle 10b in the second display mode. For example, the display processing unit 212 displays the icon image A1 in light blue and the icon image B1 in red. Also, for example, the display processing unit 212 displays the icon image A1 as an illustration image of a vehicle and the icon image B1 as a photographed image (photo) of a vehicle. Also, for example, the display processing unit 212 displays (in a balloon display) the identification information (vehicle name, model, serial number, etc.) of the work vehicle 10 on each of the icon image A1 and the icon image B1. Note that the display mode of each icon image may be set and changed by the operator.

[0168] [Supplementary Note of the Invention] Hereinafter, the outline of the invention extracted from the embodiments will be appended. Note that each configuration and each processing function described in the following supplementary notes can be arbitrarily combined by making selections.

[0169] <Supplementary Note 1> A display control method for controlling the display information of an operation terminal, causing a work vehicle registered in advance to be displayed on a first screen of the operation terminal so that a user can select it, causing predetermined information regarding the work vehicle selected by the user to be displayed on a second screen of the operation terminal, and a display control method for executing the above.

[0170] <Supplementary Note 2> The predetermined information includes at least any one of information regarding the current surrounding environment of the work vehicle, information regarding the current driving situation of the work vehicle, and information regarding the current working situation of the work vehicle. The display control method according to Supplementary Note 1.

[0171] <Supplementary Note 3> Cause a map to be displayed on the first screen. On the map, display the identification information of the work vehicle at a position corresponding to the current position of the work vehicle. The display control method according to Supplementary Note 1 or 2.

[0172] <Supplementary Note 4> On the map, display the identification information in a display mode corresponding to the current state of the work vehicle. The display control method according to Supplementary Note 3.

[0173] <Supplementary Note 5> On the map, make the display mode when the work vehicle is automatically driving different from the display mode when the work vehicle has stopped automatic driving. The display control method according to Supplementary Note 4.

[0174] <Supplementary Note 6> On the second screen, display a captured image of the surroundings of the work vehicle captured by a camera mounted on the work vehicle. The display control method according to any one of Supplementary Notes 1 to 5.

[0175] <Supplementary Note 7> When the work vehicle detects an obstacle, cause the captured image including the obstacle among a plurality of captured images of the camera to be visibly displayed on the second screen. The display control method according to Supplementary Note 6.

[0176] <Supplementary Note 8> On the map, display the respective icon images of a plurality of pre-registered work vehicles in a display mode corresponding to the current state of the work vehicle. On the first screen, causing an imaging image of the periphery of the work vehicle imaged by a camera mounted on the work vehicle selected by the user among the plurality of work vehicles to be displayed on the second screen. The display control method according to any one of Appendices 1 to 7.

[0177] <Appendix 9> On the first screen, causing identification information of an operator registered in advance for the work vehicle to be displayed in association with the identification information of the work vehicle. The display control method according to any one of Appendices 1 to 8.

[0178] <Appendix 10> When the operator displayed on the first screen is selected, causing an instruction screen for outputting a predetermined instruction to the operator to be displayed. The display control method according to Appendix 9.

[0179] <Appendix 11> Causing the imaging image to be displayed so that a part of the vehicle body of the work vehicle is reflected in the imaging image. The display control method according to Appendix 6.

[0180] <Appendix 12> When a request to switch to a remote operation mode is received from a user on the operation terminal, and when a predetermined operation is received from the user in a state where an imaging image of the periphery of the work vehicle imaged by a camera mounted on the work vehicle is displayed in a remotely operable remote operation device, switching to the remote operation mode. The display control method according to any one of Appendices 1 to 11.

[0181] <Appendix 13> A display control method for controlling display information of an operation terminal, Causing a pre-registered work vehicle to be displayed on a first screen of the operation terminal; When the work vehicle in automatic driving detects an obstacle, or when the work vehicle automatically drives in a specific area, causing predetermined information about the work vehicle to be displayed on a second screen of the operation terminal. A display control method for execution.

[0182] <Appendix 14> A display control program for controlling the display information of an operation terminal, causing a first screen of the operation terminal to display a work vehicle registered in advance so that a user can select it; causing a second screen of the operation terminal to display predetermined information regarding the work vehicle selected by the user; A display control program for causing one or more processors to execute.

[0183] <Appendix 15> A display control system for controlling the display information of an operation terminal, a first display processing unit for causing a first screen of the operation terminal to display a work vehicle registered in advance so that a user can select it; a second display processing unit for causing a second screen of the operation terminal to display predetermined information regarding the work vehicle selected by the user; A display control system comprising the same.

Explanation of Signs

[0184] 1: Automatic driving system 10: Work vehicle 11: Vehicle control device 12: Storage unit 15: Obstacle detection device 20: Operation terminal 21: Operation control unit 23: Operation display unit 53: Camera 54: Obstacle sensor 30: Remote operation device (operation terminal) 33: Display unit 34: Operation unit 111: Travel processing unit 211: Setting processing unit 212: Display processing unit (first display processing unit, second display processing unit) 213: Reception processing unit A1: Icon image (identification information of the work vehicle 10a) A2: Frame image (identification information of the work vehicle 10a) B1: Icon image (identification information of the work vehicle 10b) B2: Frame image (identification information of the work vehicle 10b) C1: Icon image (identification information of the operator X) C2: Icon image (identification information of the operator Y) M1: Message (predetermined information) P1: Selection screen (first screen) P2: Information display screen (second screen) P3: Instruction screen

Claims

1. A display control method for controlling display information of an operation terminal, comprising: causing a work vehicle registered in advance to be selectable and displayed on a first screen of the operation terminal by a user; causing predetermined information regarding the work vehicle selected by the user to be displayed on a second screen of the operation terminal; A display control method for executing the above.

2. The predetermined information includes at least any one of information regarding the current surrounding environment of the work vehicle, information regarding the current driving status of the work vehicle, and information regarding the current working status of the work vehicle. The display control method according to Claim 1.

3. Causing a map to be displayed on the first screen; In the map, causing identification information of the work vehicle to be displayed at a position corresponding to the current position of the work vehicle. The display control method according to Claim 1.

4. In the map, causing the identification information to be displayed in a display mode corresponding to the current state of the work vehicle. The display control method according to Claim 3.

5. In the map, making the display mode when the work vehicle is automatically driving different from the display mode when the work vehicle has stopped automatic driving. The display control method according to Claim 4.

6. Causing a captured image of the surroundings of the work vehicle captured by a camera mounted on the work vehicle to be displayed on the second screen. The display control method according to Claim 1.

7. When the work vehicle detects an obstacle, causing the captured image including the obstacle among a plurality of captured images of the camera to be displayed on the second screen in an identifiable manner. The display control method according to Claim 6.

8. In the map, causing icon images of a plurality of the work vehicles registered in advance to be displayed in a display mode corresponding to the current state of the work vehicle; On the first screen, causing a captured image of the surroundings of the work vehicle captured by a camera mounted on the work vehicle corresponding to the icon image selected by the user among the plurality of icon images to be displayed on the second screen. The display control method according to Claim 3.

9. On the first screen, causing identification information of an operator registered in advance for the work vehicle to be displayed in association with the identification information of the work vehicle. The display control method according to Claim 3.

10. When the operator displayed on the first screen is selected, causing an instruction screen for outputting a predetermined instruction to the operator to be displayed. The display control method according to Claim 9.

11. The captured image is displayed so that a part of the vehicle body of the work vehicle is reflected in the captured image. The display control method according to claim 6.

12. When a switching request to a remote operation mode is received from a user on the operation terminal, and when a predetermined operation is received from the user in a state where a captured image of the periphery of the work vehicle captured by a camera mounted on the work vehicle is displayed in a remotely operable remote operation device, switch to the remote operation mode. The display control method according to claim 1.

13. A display control method for controlling display information of an operation terminal, displaying a pre-registered work vehicle on a first screen of the operation terminal; when the work vehicle in automatic driving detects an obstacle or when the work vehicle automatically drives in a specific area, displaying predetermined information about the work vehicle on a second screen of the operation terminal; A display control method for executing the above.

14. A display control program for controlling display information of an operation terminal, displaying a pre-registered work vehicle on a first screen of the operation terminal so that the user can select it; displaying predetermined information about the work vehicle selected by the user on a second screen of the operation terminal; A display control program for causing one or more processors to execute the above.

15. A display control system for controlling display information of an operation terminal, a first display processing unit that displays a pre-registered work vehicle on a first screen of the operation terminal so that the user can select it; a second display processing unit that displays predetermined information about the work vehicle selected by the user on a second screen of the operation terminal; A display control system comprising the above.

Images