Driving control system, work vehicle, driving control method, and computer program

Abstract

The present invention provides a driving control system, a work vehicle, and a driving control method that enable work vehicles to perform repetitive actions efficiently. [Solution] The driving control system comprises a positioning device that outputs position data of the work vehicle and a control device that controls the operation of the work vehicle. In recording mode, the control device records position data acquired when the work vehicle is in motion in a storage device, and in playback mode, controls the speed and steering of the work vehicle based on the position data recorded in the storage device, thereby driving the work vehicle automatically. In playback mode, the control device compares first sensor data regarding the state of the work vehicle and / or second sensor data regarding the state of the surroundings of the work vehicle acquired in playback mode with first and / or second sensor data acquired in recording mode, thereby detecting whether the state of the work vehicle or the state of the surroundings of the work vehicle is a state that should be notified to the user.

Description

Technical Field

[0001] The present invention relates to a driving control system, a work vehicle, a driving control method, and a computer program.

Background Art

[0002] As next-generation agriculture, research and development of smart agriculture utilizing ICT (Information and Communication Technology) and IoT (Internet of Things) are underway. Research and development are also underway for the automation and unmanned operation of work vehicles such as tractors used in fields. For example, work vehicles that can travel automatically by steering using a positioning system such as GNSS (Global Navigation Satellite System) capable of precise positioning have been put into practical use.

[0003] Patent Document 1 discloses a work vehicle that can autonomously move between multiple tree rows by using SLAM (Simultaneous Localization and Mapping) technology that simultaneously performs position estimation and map creation in an orchard such as a vineyard. Patent Document 1 describes that in an orchard, while a work vehicle travels between multiple tree rows, operations such as mowing and control are performed using a work implement (agricultural implement) connected to the work vehicle.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Work vehicles sometimes repeatedly perform the same tasks while traveling along the same route in the same way within a field (e.g., an orchard). In such cases, if autonomous driving using, for example, SLAM technology is performed each time, the processing load for autonomous driving increases unnecessarily.

[0006] Efficiently performing repetitive actions (including driving) of work vehicles is required not only for agricultural machinery, but also for work vehicles used for non-agricultural purposes, such as construction vehicles or snowplows. Furthermore, even for driving of work vehicles that does not involve work (for example, driving outside the field), efficient execution of repeated driving along the same route is required.

[0007] The present invention aims to provide a driving control system, a work vehicle, and a driving control method that can efficiently perform repetitive actions (including driving and other actions) of a work vehicle. [Means for solving the problem]

[0008] According to embodiments of the present invention, the following solutions are provided.

[0009] [Item 1] A vehicle driving control system for work vehicles, A positioning device that outputs positional data relating to the position of the aforementioned work vehicle, One or more internal sensors that output first sensor data relating to the state of the work vehicle, and / or one or more external sensors that output second sensor data relating to the state of the surroundings of the work vehicle, A control device that controls the operation of the aforementioned work vehicle and Equipped with, The control device is It can operate in both recording and playback modes. In the recording mode described above, the position data acquired when the work vehicle is in motion is recorded in the storage device. In the playback mode, the speed and steering of the work vehicle are controlled based on the position data recorded in the storage device, thereby driving the work vehicle automatically. In the recording mode and the playback mode, the first sensor data and / or the second sensor data are acquired. A driving control system that, in the playback mode, detects whether the state of the work vehicle or the state of the surroundings of the work vehicle is a state that should be notified to the user by comparing the first sensor data and / or the second sensor data acquired in the playback mode with the first sensor data and / or the second sensor data recorded in the storage device.

[0010] [Item 2] The control device is In the recording mode and the playback mode, while the work vehicle is in motion, the first sensor data is acquired. The driving control system according to item 1, wherein in the playback mode, if the difference between the first sensor data acquired in the playback mode and the first sensor data acquired in the recording mode exceeds a predetermined value, it is determined that the status of the work vehicle should be notified to the user.

[0011] [Item 3] The control device is In the recording mode, the first sensor data is recorded in the storage device. The driving control system according to item 2, wherein in the playback mode, the system interpolates the first sensor data recorded in the recording mode and compares it with the first sensor data acquired in the playback mode.

[0012] [Item 4] The control device is In the recording mode and the playback mode, while the work vehicle is in motion, the first sensor data is acquired. In the playback mode, when the value of the first sensor data obtained in the playback mode exceeds a predetermined value, it is determined that the state of the work vehicle is a state to be notified to the user. The travel control system according to any one of items 1 to 3.

[0013] [Item 5] The first sensor data includes Data on the roll angle or pitch angle of the work vehicle. The travel control system according to any one of items 1 to 4.

[0014] [Item 6] The position data includes data on the altitude of the work vehicle, The control device In the playback mode, by comparing the altitude data obtained in the playback mode with the altitude data recorded in the storage device, it is detected that the state of the work vehicle or the state around the work vehicle is a state to be notified to the user. The travel control system according to any one of items 1 to 5.

[0015] [Item 7] The control device In the recording mode and the playback mode, while the work vehicle is running, the second sensor data is acquired, The second sensor data obtained in the recording mode is recorded in the storage device in association with the position data, In the playback mode, based on the comparison between the second sensor data obtained in the playback mode and the second sensor data obtained in the recording mode, it is detected that the state of the work vehicle or the state around the work vehicle is a state to be notified to the user. The travel control system according to any one of items 1 to 6.

[0016] [Item 8] The control device The driving control system according to item 7, which records information on the presence or absence of an obstacle and the location of the obstacle in the storage device based on the second sensor data acquired in the recording mode.

[0017] [Item 9] The control device is The driving control system according to item 8, which records information on the type of obstacle in the storage device based on the second sensor data acquired in the recording mode.

[0018] [Item 10] The control device is A driving control system according to item 7 or 9, which records information about the road traveled by the work vehicle in the storage device based on the second sensor data acquired in the recording mode.

[0019] [Item 11] The control device is A driving control system according to any one of items 1 to 10, wherein, in the playback mode, if it is determined that the state of the work vehicle is in a state that should be notified to the user, the system notifies the user, slows down the work vehicle, and stops the work vehicle from moving.

[0020] [Item 12] The control device is A driving control system according to any one of items 1 to 11, which, in the playback mode, determines that the status of the work vehicle is such that it should be notified to the user, and stops the driving of the work vehicle.

[0021] [Item 13] The control device is A driving control system according to any one of items 1 to 12, which updates the first sensor data and / or second sensor data recorded in the storage device based on the first sensor data and / or second sensor data acquired in the playback mode.

[0022] [Item 14] A driving control system described in any one of items 1 to 13, Running gear including the steering wheels, A drive unit that drives the aforementioned traveling device and Equipped with, A work vehicle that, in the playback mode, controls the drive unit based on the position data recorded in the recording mode, thereby driving the work vehicle automatically.

[0023] [Item 15] A control device for controlling the operation of a work vehicle, which is executed by a control device capable of operating in recording mode and playback mode, and a method for controlling the movement of a work vehicle, In the recording mode, the position data relating to the position of the work vehicle acquired when the work vehicle was in motion is recorded in the storage device, In the playback mode, the speed and steering of the work vehicle are controlled based on the position data recorded in the storage device, thereby driving the work vehicle automatically. In the recording mode and the playback mode, first sensor data relating to the state of the work vehicle and / or second sensor data relating to the state of the surroundings of the work vehicle are acquired. In the playback mode, the system detects whether the state of the work vehicle or the state of the surroundings of the work vehicle is in a state that should be notified to the user by comparing the first sensor data and / or the second sensor data acquired in the playback mode with the first sensor data and / or the second sensor data recorded in the storage device. A driving control method including the above.

[0024] [Item 16] A computer program executed by a processor in a control device that controls the operation of a work vehicle and is capable of operating in recording mode and playback mode, The aforementioned processor, In the recording mode, the position data relating to the position of the work vehicle acquired when the work vehicle was in motion is recorded in the storage device, In the playback mode, the speed and steering of the work vehicle are controlled based on the position data recorded in the storage device, thereby driving the work vehicle automatically. In the recording mode and the playback mode, first sensor data relating to the state of the work vehicle and / or second sensor data relating to the state of the surroundings of the work vehicle are acquired. In the playback mode, the system detects whether the state of the work vehicle or the state of the surroundings of the work vehicle is in a state that should be notified to the user by comparing the first sensor data and / or the second sensor data acquired in the playback mode with the first sensor data and / or the second sensor data recorded in the storage device. A computer program that executes something.

[0025] [Item 17] A control device that performs the driving control method described in item 15.

[0026] [Item 18] A computer program executed by a computer that controls the operation of a work vehicle, A computer program that causes the computer to execute the steps of the driving control method described in item 15.

[0027] [Item 19] A computer program medium executed by a computer that controls the operation of a work vehicle, A computer program medium that causes the computer to execute the driving control method described in item 15.

[0028] [Item 20] A vehicle driving control system for work vehicles, A positioning device that outputs positional data relating to the position of the aforementioned work vehicle, The control device described in item 17 and A driving control system having the following features.

[0029] [Item 21] A control device for controlling the operation of a work vehicle, which is capable of operating in recording mode and playback mode, In the recording mode, means for recording position data relating to the position of the work vehicle acquired when the work vehicle was in motion into a storage device, In the playback mode, means for driving the work vehicle by automatic driving by controlling the speed and steering of the work vehicle based on the position data recorded in the storage device, In the recording mode and the playback mode, means for acquiring first sensor data relating to the state of the work vehicle and / or second sensor data relating to the state of the surroundings of the work vehicle, In the playback mode, means for detecting whether the state of the work vehicle or the state of the surroundings of the work vehicle is a state that should be notified to the user, by comparing the first sensor data and / or the second sensor data acquired in the playback mode with the first sensor data and / or the second sensor data recorded in the storage device. A control device, including a control device.

[0030] [Item 22] A vehicle driving control system for work vehicles, A positioning device that outputs positional data relating to the position of the aforementioned work vehicle, The control device described in item 21 and A driving control system having the following features.

[0031] [Item 23] One or more processors, The above-mentioned one or more processors have one or more memories that store a computer program that causes the steps of the driving control method described in item 15 to be executed, and A control device having

[0032] [Item 24] The control device described in item 23, A first drive unit that drives the running gear of the aforementioned work vehicle and Equipped with, The control device is a driving control system that, in the playback mode, controls the first drive unit based on the position data recorded in the storage device to drive the work vehicle automatically.

[0033] Comprehensive or specific embodiments of the present invention may be realized by apparatus, systems, methods, integrated circuits, computer programs, or computer-readable non-temporary storage media, or any combination thereof. Computer-readable storage media may include volatile storage media or non-volatile storage media. An apparatus may consist of multiple devices. If an apparatus consists of two or more devices, these two or more devices may be located in a single device or in two or more separate devices. [Effects of the Invention]

[0034] According to embodiments of the present invention, a driving control system, a work vehicle, and a driving control method are provided that can efficiently perform repetitive operations (including driving and other operations) of a work vehicle. [Brief explanation of the drawing]

[0035] [Figure 1] This is a schematic side view showing an example of a work vehicle in an embodiment of the present invention. [Figure 2] This is a schematic block diagram showing an example of the configuration of a work vehicle and work machine in an embodiment of the present invention. [Figure 3A] This block diagram shows a schematic configuration example of a driving control system according to an embodiment of the present invention. [Figure 3B] This is a block diagram showing an example of the configuration of a control device in a driving control system according to an embodiment of the present invention. [Figure 4]This is a schematic diagram showing an example of the configuration of a driving control system according to an embodiment of the present invention. [Figure 5A] This figure schematically shows an example of a route traveled by a work vehicle according to an embodiment of the present invention in recording mode. [Figure 5B] This figure schematically shows an example of a route traveled by a work vehicle according to an embodiment of the present invention in regeneration mode. [Figure 6] This figure shows an example of waypoint data recorded in a storage device. [Figure 7] This flowchart shows an example of processing performed by the control unit in recording mode. [Figure 8A] This is an example of a display screen shown on a user-operated terminal. [Figure 8B] This is an example of a display screen shown on a user-operated terminal. [Figure 8C] This is an example of a display screen shown on a user-operated terminal. [Figure 8D] This is an example of a display screen shown on a user-operated terminal. [Figure 9A] This is an example of a display screen shown on a user-operated terminal. [Figure 9B] This is an example of a display screen shown on a user-operated terminal. [Figure 9C] This is an example of a display screen shown on a user-operated terminal. [Figure 9D] This is an example of a display screen shown on a user-operated terminal. [Figure 10A] This is an example of a display screen shown on a user-operated terminal. [Figure 10B] This is an example of a display screen shown on a user-operated terminal. [Figure 10C] This is an example of a display screen shown on a user-operated terminal. [Figure 11] This flowchart shows an example of processing performed by the control unit in playback mode. [Figure 12]This is a schematic diagram illustrating an example of processing performed by a control device. [Figure 13] This flowchart shows an example of processing performed by the control unit in recording mode. [Figure 14] This flowchart shows an example of processing performed by the control unit in playback mode. [Figure 15] This flowchart shows an example of processing performed by the control unit in playback mode. [Figure 16] This is a schematic diagram illustrating the process performed in step S138. [Figure 17] This flowchart shows an example of processing performed by the control unit in recording mode. [Figure 18] This flowchart shows an example of processing performed by the control unit in playback mode. [Modes for carrying out the invention]

[0036] (Definition of terms) In this specification, “work vehicle” means a vehicle used to perform work in a work area. “Work area” is any place where work can be performed, such as a field, forest, or construction site. “Field” is any place where agricultural work can be performed, such as an orchard, farm, rice paddy, grain farm, or pasture. A work vehicle may be agricultural machinery such as a tractor, rice transplanter, combine harvester, riding cultivator, or riding mower, or a vehicle used for non-agricultural purposes, such as a construction vehicle or snowplow. A work vehicle may be configured to be equipped with work implements (also called “working devices” or “implements”) on at least one of its front and rear ends, depending on the work being performed. In particular, work implements attached to agricultural tractors are sometimes called “agricultural implements.” The act of a work vehicle driving while performing work with work implements may be referred to as “work driving.” The “operation” of a work vehicle includes not only the driving of the work vehicle but also other operations.

[0037] "Automated driving" means that the vehicle's movement is controlled by a control device, without manual operation by the driver. During automated driving, not only the vehicle's movement but also the operation of work (e.g., the operation of work equipment) may be controlled automatically. The movement of the vehicle under automated driving conditions is referred to as "automated driving." The control device can control at least one of the following necessary for the vehicle's movement: steering, adjustment of driving speed, starting and stopping the vehicle. When controlling a work vehicle equipped with work equipment, the control device may also control operations such as raising and lowering the work equipment and starting and stopping the operation of the work equipment. Driving under automated driving conditions may include not only driving the vehicle along a predetermined route toward a destination but also driving while following a target. In addition to automated driving mode, a vehicle performing automated driving may also operate in manual driving mode, where it is driven by the driver's manual operation. Driving under the driver's manual operation is referred to as "manual driving." "Driver's manual operation" includes not only manual operation by the driver on the vehicle but also remote operation by an operator outside the vehicle. A vehicle performing automated driving conditions may be driven partially based on the driver's manual operation. "Automatic steering" refers to the steering of a vehicle by a control device, without manual operation by the driver. Part or all of the control device may be located outside the vehicle. Communication, such as control signals, commands, or data, may take place between the external control device and the vehicle. A vehicle capable of autonomous driving may operate autonomously, sensing its surroundings without human intervention in controlling its movement. A vehicle capable of autonomous driving can operate unmanned. Obstacle detection and obstacle avoidance may occur during autonomous driving.

[0038] A "crop row" refers to a row of crops, trees, or other plants growing in a field such as an orchard or farm, or in a forest. In this specification, the term "crop row" includes the concept of "tree row."

[0039] (Embodiment) Embodiments of the present invention will be described below. However, unnecessarily detailed explanations will be omitted. Yes. For example, detailed explanations of already well-known matters and redundant explanations of substantially identical components may be omitted. This is to avoid the following explanation becoming unnecessarily verbose and to facilitate understanding by those skilled in the art. The inventors provide the accompanying drawings and the following explanation so that those skilled in the art can fully understand the invention, and not to limit the subject matter described in the claims. In the following explanation, components having the same or similar function are denoted by the same reference numerals.

[0040] The following embodiments are illustrative, and the technology of the present invention is not limited to these embodiments. For example, the numerical values, shapes, materials, steps, and order of steps shown in the following embodiments are merely examples, and various modifications are possible as long as they do not create a technical inconsistency. Furthermore, it is possible to combine one embodiment with another.

[0041] The following describes an embodiment in which the work vehicle is a tractor used for agricultural work in fields such as orchards. The technology of the present invention is not limited to tractors, but can also be applied to other types of agricultural machinery such as rice transplanters, combine harvesters, riding cultivators, and riding lawnmowers. The technology of the present invention can also be applied to work vehicles used for purposes other than agriculture, such as construction vehicles or snowplows. The technology of the present invention can also be applied to the driving of work vehicles outside of work areas, and to the driving of work vehicles without performing work.

[0042] [Outline of the work vehicle configuration] Figure 1 is a schematic side view showing an example of a work vehicle 100 and a work machine 300 connected to the work vehicle 100. Figure 2 is a schematic block diagram showing an example configuration of the work vehicle 100 and the work machine 300.

[0043] As shown in Figures 1 and 2, the work vehicle 100 includes a positioning device 110 (e.g., a GNSS unit) that outputs positional data relating to the position of the work vehicle 100, and a control device 180 that controls the operation of the work vehicle 100.

[0044] The work vehicle 100 may further include an internal sensor group 150 (sometimes referred to as "sensor group 150") that outputs sensor data related to the state of the work vehicle 100. The internal sensor group 150 includes one or more internal sensors. The "internal sensors" include various sensors that detect the state of the work vehicle 100.

[0045] The work vehicle 100 may further be equipped with a plurality of external sensors (a group of external sensors 121) that sense the surroundings of the work vehicle 100. An "external sensor" is a sensor that senses the external conditions of the work vehicle. In the example in Figure 1, the group of external sensors 121 includes a plurality of LiDAR sensors 140, a plurality of cameras 120, and a plurality of obstacle sensors 130.

[0046] In the example shown in Figure 2, the work vehicle 100 includes a positioning device 110, an external sensor group 121 (including a camera 120, an obstacle sensor 130, and a LiDAR sensor 140), an internal sensor group 150, a storage device 170, a control device 180, and an operating terminal 200, as well as a communication device 190, an operating switch group 210, and a drive device 240 (sometimes referred to as the "first drive device"). These components are connected to each other via a bus so as to be able to communicate with one another.

[0047] As shown in Figure 1, the work vehicle 100 comprises a body 101, a prime mover (engine) 102, and a transmission 103. The body 101 is provided with a running gear including wheels with tires 104 and a cabin 105. The running gear includes four wheels 104, axles that rotate the four wheels, and brakes that brake each axle. The wheels 104 include a pair of front wheels 104F and a pair of rear wheels 104R. Inside the cabin 105 are a driver's seat 107, a steering gear 106, an operating terminal 200, and a group of switches for operation. One or both of the front wheels 104F and the rear wheels 104R may be replaced with multiple wheels fitted with tracks (crawlers) instead of wheels with tires.

[0048] The prime mover 102 may be, for example, a diesel engine. An electric motor may be used instead of a diesel engine. The transmission 103 can change the propulsion force and travel speed of the work vehicle 100 by shifting gears. The transmission 103 can also switch the work vehicle 100 between forward and reverse.

[0049] The steering system 106 includes a steering wheel, a steering shaft connected to the steering wheel, and a power steering system that assists steering by the steering wheel. The front wheels 104F are steering wheels, and the direction of travel of the work vehicle 100 can be changed by changing their steering angle (also referred to as the "steering angle"). The steering angle of the front wheels 104F can be changed by operating the steering wheel. The power steering system includes a hydraulic system or electric motor that supplies auxiliary force to change the steering angle of the front wheels 104F. When automatic steering is performed, the steering angle is automatically adjusted by the force of the hydraulic system or electric motor under control from a control device located inside the work vehicle 100.

[0050] A coupling device 108 is provided at the rear of the vehicle body 101. The coupling device 108 includes, for example, a three-point support device (also called a "three-point hitch" or "three-point link"), a PTO (Power Take Off) shaft, a universal joint, and a communication cable. The coupling device 108 allows the work implement 300 to be attached to and detached from the work vehicle 100. The coupling device 108 can change the position or orientation of the work implement 300 by raising and lowering the three-point hitch, for example, by a hydraulic system. Power can also be supplied from the work vehicle 100 to the work implement 300 via the universal joint. The work vehicle 100 can pull the work implement 300 and have the work implement 300 perform a predetermined task. The coupling device may be provided at the front of the vehicle body 101. In that case, the work implement can be connected to the front of the work vehicle 100.

[0051] The implement 300 shown in Figure 1 is a sprayer for spraying chemicals onto crops, but the implement 300 is not limited to a sprayer. For example, any implement such as a mower, seeder, spreader, rake, baler, harvester, plow, harrow, or rotary tiller can be connected to the work vehicle 100 and used.

[0052] The positioning device 110 receives satellite signals (also referred to as GNSS signals) transmitted from multiple GNSS satellites and performs positioning based on these satellite signals. GNSS is a general term for satellite positioning systems such as GPS (Global Positioning System), QZSS (Quasi-Zenith Satellite System, e.g., Michibiki), GLONASS, Galileo, and BeiDou. In this embodiment, the positioning device 110 is located on top of the cabin 105, but it may be located in other positions.

[0053] As shown in Figure 2, the positioning device 110 comprises a GNSS receiver 111, an RTK receiver 112, and a processing circuit 116. The positioning device 110 may further include an inertial measurement unit (IMU) 115.

[0054] The GNSS receiver 111 includes an antenna for receiving signals from GNSS satellites and a processing circuit for determining the position of the work vehicle 100 based on the signals received by the antenna. The GNSS receiver 111 receives satellite signals transmitted from multiple GNSS satellites and generates GNSS data based on the satellite signals. The GNSS data is generated in a predetermined format, such as NMEA-0183 format. The GNSS data may include, for example, the identification number, elevation angle, azimuth angle, and received signal strength of each satellite from which the satellite signal was received.

[0055] The positioning device 110 may perform positioning of the work vehicle 100 using RTK (Real Time Kinematic)-GNSS. In RTK-GNSS positioning, in addition to satellite signals transmitted from multiple GNSS satellites, correction signals transmitted from a base station are used. The base station may be installed near the work site where the work vehicle 100 will be driving (for example, within 10 km of the work vehicle 100). Based on the satellite signals received from multiple GNSS satellites, the base station generates a correction signal, for example, in RTCM format and transmits it to the positioning device 110. The RTK receiver 112 includes an antenna and a modem and receives the correction signal transmitted from the base station. The processing circuit 116 of the positioning device 110 corrects the positioning result from the GNSS receiver 111 based on the correction signal. By using RTK-GNSS, it is possible to perform positioning with an accuracy of, for example, an error of a few centimeters. Position information, including latitude, longitude, and altitude information, is acquired by high-precision positioning using RTK-GNSS. The positioning device 110 calculates the position of the work vehicle 100 at a frequency of, for example, 1 to 10 times per second. The positioning method is not limited to RTK-GNSS; any positioning method that can obtain the necessary accuracy of positional information (such as interferometric positioning or relative positioning) can be used. For example, positioning may be performed using VRS (Virtual Reference Station) or DGPS (Differential Global Positioning System).

[0056] The positioning device 110 in this embodiment further includes an IMU 115. By including the IMU 115, the positioning device 110 can supplement position data using signals from the IMU 115. By supplementing position data based on satellite signals using data acquired by the IMU 115, the positioning performance can be improved.

[0057] The IMU115 may be equipped with a 3-axis accelerometer and a 3-axis gyroscope. The IMU115 may also be equipped with an orientation sensor, such as a 3-axis geomagnetic sensor. The IMU115 functions as a motion sensor and can output signals indicating various quantities such as acceleration, velocity, displacement, and attitude of the work vehicle 100. The processing circuit 116 can estimate the position and orientation of the work vehicle 100 with higher accuracy based on the signals output from the IMU115 in addition to the satellite signals and correction signals. The signals output from the IMU115 can be used to correct or complement the position calculated based on the satellite signals and correction signals. The IMU115 outputs signals at a higher frequency than the GNSS receiver 111. For example, the IMU115 outputs signals at a frequency of several tens to several thousand times per second. Using these high-frequency signals, the processing circuit 116 can measure the position and orientation of the work vehicle 100 at a higher frequency (e.g., 10 Hz or higher). Instead of the IMU115, a 3-axis accelerometer and a 3-axis gyroscope may be provided separately. The IMU 115 may be provided as a separate device from the positioning device 110.

[0058] The internal sensor group 150 may include various sensors (i.e., internal sensors) that detect the state of the work vehicle 100 or work machine 300. For example, the internal sensor group 150 may include a steering wheel sensor 152, a steering angle sensor 154, and an axle sensor 156.

[0059] The steering wheel sensor 152 measures the rotation angle of the steering wheel of the work vehicle 100. The steering angle sensor 154 measures the steering angle of the front wheels 104F, which are the steering wheels. The values ​​measured by the steering wheel sensor 152 and the steering angle sensor 154 can be used for steering control by the control device 180.

[0060] The axle sensor 156 measures the rotational speed of the axle connected to the wheel 104, i.e., the number of rotations per unit time. The axle sensor 156 may be a sensor that utilizes, for example, a magnetoresistive element (MR), a Hall element, or an electromagnetic pickup. The axle sensor 156 outputs a numerical value indicating, for example, the number of rotations of the axle per minute (unit: rpm). The axle sensor 156 is used to measure the speed of the work vehicle 100. The value measured by the axle sensor 156 can be used for speed control by the control device 180.

[0061] The storage device 170 includes one or more storage media such as flash memory or magnetic disks. The storage device 170 stores various data generated by the positioning device 110, the external sensor group 121 (including the camera 120, obstacle sensor 130, and LiDAR sensor 140), the internal sensor group 150, and the control device 180. The data stored in the storage device 170 may include an environmental map of the environment in which the work vehicle 100 travels, an obstacle map that is generated sequentially during travel, and route data for autonomous driving. The storage device 170 also stores computer programs that cause each ECU in the control device 180 to perform various operations described later. Such computer programs may be provided to the work vehicle 100 via a storage medium (e.g., semiconductor memory or optical disk) or a telecommunications line (e.g., the Internet). Such computer programs may be sold as commercial software.

[0062] The control device 180 includes a plurality of ECUs. These plurality of ECUs include, for example, an ECU 181 for speed control, an ECU 182 for steering control, an ECU 183 for work equipment control, and an ECU 184 for automatic driving control.

[0063] The ECU 181 controls the speed of the work vehicle 100 by controlling the prime mover 102, the transmission 103, and the brakes, which are included in the drive unit 240.

[0064] The ECU 182 controls the steering of the work vehicle 100 by controlling the hydraulic system or electric motor included in the steering device 106 based on the measurements of the steering wheel sensor 152.

[0065] The ECU 183 controls the operation of the three-point hitch and PTO shaft, etc., included in the coupling device 108, in order to make the work implement 300 perform the desired operation. The ECU 183 also generates signals to control the operation of the work implement 300 and transmits these signals from the communication device 190 to the work implement 300.

[0066] The ECU 184 performs calculations and controls to achieve autonomous driving based on data output from the positioning device 110, the external sensor group 121 (including the camera 120, obstacle sensor 130, and LiDAR sensor 140), and the internal sensor group 150. For example, the ECU 184 estimates the position of the work vehicle 100 based on data output from at least one of the positioning device 110, the camera 120, and the LiDAR sensor 140. In situations where the reception strength of satellite signals from GNSS satellites is sufficiently high, the ECU 184 may determine the position of the work vehicle 100 based only on the data output from the positioning device 110. On the other hand, in environments such as orchards where there are obstacles such as trees that obstruct the reception of satellite signals around the work vehicle 100, the ECU 184 estimates the position of the work vehicle 100 using data output from the LiDAR sensor 140 or the camera 120. During autonomous driving, ECU 184 performs calculations necessary for the work vehicle 100 to travel along the target path based on the estimated position of the work vehicle 100. ECU 184 sends a command to ECU 181 to change speed and a command to ECU 182 to change steering angle. ECU 181 changes the speed of the work vehicle 100 by controlling the engine 102, transmission 103, or brakes in response to the command to change speed. ECU 182 changes the steering angle by controlling the steering device 106 in response to the command to change steering angle.

[0067] Through the operation of these ECUs, the control unit 180 enables autonomous driving. During autonomous driving, the control unit 180 controls the drive unit 240 based on the measured or estimated position of the work vehicle 100 and the sequentially generated target path. This allows the control unit 180 to drive the work vehicle 100 along the target path.

[0068] Multiple ECUs included in the control unit 180 can communicate with each other according to a vehicle bus standard such as CAN (Controller Area Network). Instead of CAN, a faster communication method such as Automotive Ethernet (registered trademark) may be used. In Figure 2, each of the ECUs 181 to 184 is shown as a separate block, but each of their functions may be implemented by multiple ECUs. An on-board computer integrating at least some of the functions of ECUs 181 to 184 may be provided. The control unit 180 may also include ECUs other than ECUs 181 to 184, and any number of ECUs can be provided depending on their function. Each ECU includes a processing circuit containing one or more processors.

[0069] Cameras 120 may be installed, for example, on the front, rear, left, and right sides of the work vehicle 100. Cameras 120 capture images of the environment around the work vehicle 100 and generate image data. The images acquired by cameras 120 may be transmitted, for example, to a terminal device for remote monitoring. These images may be used to monitor the work vehicle 100 during unmanned operation. Cameras 120 may be installed as needed, and their number is arbitrary.

[0070] The LiDAR sensor 140 is an example of an external sensor that outputs sensor data showing the distribution of features around the work vehicle 100. In the example in Figure 1, two LiDAR sensors 140 are located at the front and rear of the cabin 105. The LiDAR sensors 140 may be located in other places (for example, at the lower front of the vehicle body 101). Each LiDAR sensor 140 repeatedly outputs sensor data showing the distance and direction to each measurement point of an object in the surrounding environment, or the two-dimensional or three-dimensional coordinate values ​​of each measurement point, while the work vehicle 100 is in motion. The number of LiDAR sensors 140 is not limited to two; it may be one or three or more.

[0071] The LiDAR sensor 140 may be configured to output two-dimensional or three-dimensional point cloud data as sensor data. In this specification, “point cloud data” broadly means data showing the distribution of multiple reflection points observed by the LiDAR sensor 140. The point cloud data may include, for example, the coordinate values ​​of each reflection point in two-dimensional or three-dimensional space, or information indicating the distance and direction of each reflection point. The point cloud data may also include brightness information for each reflection point. The LiDAR sensor 140 may be configured to repeatedly output the point cloud data, for example, at a preset period. Thus, the ambient sensor may include one or more LiDAR sensors 140 that output point cloud data as sensor data.

[0072] Sensor data output from the LiDAR sensor 140 is processed by a control device that controls the automatic driving of the work vehicle 100. While the work vehicle 100 is driving, the control device can sequentially generate an obstacle map showing the distribution of objects around the work vehicle 100 based on the sensor data output from the LiDAR sensor 140. The control device can also generate an environmental map by stitching together the obstacle maps during automatic driving, for example, using an algorithm such as SLAM. The control device can also estimate the position and orientation of the work vehicle 100 (i.e., self-localization) by matching the sensor data with the environmental map.

[0073] The multiple obstacle sensors 130 shown in Figure 1 are located at the front and rear of the cabin 105. Obstacle sensors 130 may also be located in other areas. For example, one or more obstacle sensors 130 may be provided at any location on the sides, front, and rear of the vehicle body 101. Obstacle sensors 130 may include, for example, laser scanners or ultrasonic sonar. Obstacle sensors 130 are used to detect surrounding obstacles during autonomous driving and to stop or bypass the work vehicle 100.

[0074] The control device of the work vehicle 100 may use sensing data acquired by a sensing device such as a camera 120 or a LiDAR sensor 140 for positioning, in addition to the positioning results from the positioning device 110. If there are features that function as characteristic points in the environment in which the work vehicle 100 travels, such as farm roads, forest roads, public roads, or orchards, the position and orientation of the work vehicle 100 can be estimated with high accuracy based on the data acquired by the camera 120 or LiDAR sensor 140 and an environmental map stored in a storage device in advance. By correcting or supplementing the position data based on satellite signals using the data acquired by the camera 120 or LiDAR sensor 140, the position of the work vehicle 100 can be determined with even higher accuracy.

[0075] The work vehicle 100 and the work machine 300 can communicate with each other via a communication cable included in the coupling device 108. The work vehicle 100 can also communicate with a terminal device 400 for remote monitoring via the network 60. The terminal device 400 is any computer, such as a personal computer (PC), laptop computer, tablet computer, or smartphone.

[0076] The work machine 300 includes a drive unit 340 (sometimes referred to as the "second drive unit"), a control device 380, and a communication device 390. Figure 2 shows the components that are relatively highly relevant to the operation of the work vehicle 100's automatic driving, and other components are not shown.

[0077] Camera 120 is an imaging device that captures the environment around the work vehicle 100. Camera 120 includes an image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal Xide Semiconductor). Camera 120 may also include an optical system including one or more lenses and a signal processing circuit. While the work vehicle 100 is in motion, Camera 120 captures the environment around the work vehicle 100 and generates image (e.g., video) data. Camera 120 can capture video at a frame rate of, for example, 3 frames per second (fps) or higher. The images generated by Camera 120 can be used, for example, when a remote observer uses a terminal device 400 to check the environment around the work vehicle 100. The images generated by Camera 120 may be used for positioning or obstacle detection. As shown in Figure 1, multiple cameras 120 may be installed at different locations on the work vehicle 100, or a single camera may be installed. A visible light camera that generates visible light images and an infrared camera that generates infrared images may be provided separately. Both the visible light camera and the infrared camera may be provided as cameras that generate surveillance images. The infrared camera can also be used for detecting obstacles at night.

[0078] The obstacle sensor 130 detects objects present around the work vehicle 100. The obstacle sensor 130 may include, for example, a laser scanner or an ultrasonic sonar. The obstacle sensor 130 outputs a signal indicating the presence of an obstacle when an object is closer than a predetermined distance from the obstacle sensor 130. Multiple obstacle sensors 130 may be installed at different locations on the work vehicle 100. For example, multiple laser scanners and multiple ultrasonic sonars may be placed at different locations on the work vehicle 100. By providing many such obstacle sensors 130, blind spots in monitoring obstacles around the work vehicle 100 can be reduced.

[0079] The drive system 240 includes various devices necessary for the movement of the work vehicle 100 and the driving of the work equipment 300, such as the prime mover 102, the transmission 103, the steering system 106, and the coupling device 108. The prime mover 102 may be an internal combustion engine, such as a diesel engine. The drive system 240 may also be equipped with an electric motor for traction, either in place of or in conjunction with the internal combustion engine.

[0080] The communication device 190 is a device that includes a circuit for communicating with the work machine 300 and the terminal device 400. The communication device 190 includes a circuit for transmitting and receiving signals compliant with the ISOBUS standard, such as ISOBUS-TIM, to and from the communication device 390 of the work machine 300. This makes it possible to make the work machine 300 perform desired operations or to obtain information from the work machine 300. The communication device 190 may further include an antenna and a communication circuit for transmitting and receiving signals via the network 60 to and from the terminal device 400. The network 60 may include, for example, a cellular mobile communication network such as 3G, 4G, or 5G and the internet. The communication device 190 may also have a function for communicating with a mobile terminal used by a supervisor near the work vehicle 100. Communication with such a mobile terminal may be conducted in accordance with any wireless communication standard, such as Wi-Fi®, cellular mobile communication such as 3G, 4G, or 5G, or Bluetooth®.

[0081] The operation terminal 200 is a terminal for the user to perform operations related to the movement of the work vehicle 100 and the operation of the work machine 300, and is also called a virtual terminal (VT). The operation terminal 200 may be equipped with a display device such as a touch screen and / or one or more buttons. The display device may be a display such as a liquid crystal or organic light-emitting diode (OLED). By operating the operation terminal 200, the user can perform various operations such as switching the automatic driving mode on / off, switching the recording (teaching) mode and playback mode (described later) on / off, and switching the work machine 300 on / off. At least some of these operations can also be achieved by operating the operation switch group 210. The operation terminal 200 may be configured to be detachable from the work vehicle 100. A user located away from the work vehicle 100 may control the operation of the work vehicle 100 by operating the detached operation terminal 200. The operation terminal 200 may be equipped with a storage device. The storage device in the operating terminal 200 may store various data necessary for the operation of the work vehicle 100 instead of the storage device 170.

[0082] The drive unit 340 in the work machine 300 shown in Figure 2 performs the operations necessary for the work machine 300 to perform a predetermined operation. The drive unit 340 includes devices such as a hydraulic system, an electric motor, or a pump, depending on the application of the work machine 300. The control device 380 controls the operation of the drive unit 340. The control device 380 causes the drive unit 340 to perform various operations in response to signals transmitted from the work vehicle 100 via the communication device 390. It can also transmit signals corresponding to the status of the work machine 300 from the communication device 390 to the work vehicle 100.

[0083] [Driving control system] A driving control system according to an embodiment of the present invention will now be described. The driving control system according to an embodiment of the present invention is applied, for example, to the work vehicle 100 described above. In the example shown in Figures 1 and 2, a work machine 300 is connected to the work vehicle 100, but it is not essential that the work machine 300 is connected to the work vehicle 100. That is, the driving control system according to an embodiment of the present invention can also be applied to a work vehicle 100 that is not connected to a work machine 300.

[0084] Figure 3A is a block diagram showing a schematic configuration example of a driving control system 1000 according to an embodiment of the present invention. As shown in Figure 3A, the driving control system 1000 according to this embodiment includes a positioning device 110 that detects the position of a work vehicle 100 and outputs position data, and a control device 180 that controls the operation of the work vehicle 100. In this embodiment, as shown in Figure 2, the positioning device 110 and the control device 180 are installed on the work vehicle 100. The control device 180 works in cooperation with the positioning device 110 to function as the driving control system 1000 of the work vehicle 100. The control device 180 and the positioning device 110 can be connected to communicate with each other via a bus 810.

[0085] Figure 3A shows together an internal sensor group 150 that outputs sensor data (sometimes referred to as "first sensor data") related to the state of the work vehicle 100, and an external sensor group 121 that outputs sensor data (sometimes referred to as "second sensor data") related to the state of the surroundings of the work vehicle 100. Some or all of the internal sensors included in the internal sensor group 150 may be included in the driving control system 1000, or the internal sensor group 150 may be an external element of the driving control system 1000. In this embodiment, the internal sensor group 150 is provided on the work vehicle 100 as shown in Figure 2. The internal sensor group 150 may be connected to the control device 180 and the positioning device 110 so as to be able to communicate with each other via the bus 810. Some or all of the external sensors included in the external sensor group 121 may be included in the driving control system 1000, or the external sensor group 121 may be an external element of the driving control system 1000. The external sensor group 121 may be installed on the work vehicle 100, as shown in Figure 2, or some or all of the external sensors included in the external sensor group 121 may be installed separately from the work vehicle 100. The external sensor group 121 is not limited to the LiDAR sensor 140, camera 120, and obstacle sensor 130 described above, and may include sensors that output sensor data regarding the conditions around the work vehicle 100.

[0086] Figure 3A also shows a storage device 870 in which information acquired by the control device 180 is recorded. The storage device 870 may be included in the travel control system 1000 or may be an external component of the travel control system 1000. The storage device 870 may be mounted on the work vehicle 100 or on the work machine 300. The storage device 870 may be connected to the control device 180 so as to be able to communicate with each other via the bus 810. For example, the storage device 870 may be the storage device 170 shown in Figure 2 or a storage device provided in the operation terminal 200. The operation terminal 200 may be included in the travel control system 1000. The storage device 870 may be located outside the work vehicle 100 and the work machine 300. A storage device 870 located outside the work vehicle 100 and the work machine 300 may be connected to the control device 180 via a communication network. The storage device 870 may be included in a server computer connected to the control device 180 via a communication network.

[0087] In the example shown in Figure 1, the positioning device 110 is mounted on the work vehicle 100, but the positioning device 110 may also be mounted on a work machine 300 connected to the work vehicle 100. In addition to, or instead of, the positioning device mounted on the work vehicle 100, a positioning device (e.g., a GNSS unit) mounted on the work machine 300 may function as the positioning device 110 of the driving control system 1000. The position measured by the positioning device mounted on the work vehicle 100 or the work machine 300 is strictly speaking the position of the point where the positioning device is located, but in this specification, that position is referred to as the "position of the work vehicle".

[0088] The internal sensor group 150 is not limited to the steering wheel sensor 152, steering angle sensor 154, and axle sensor 156 described above, but may include various sensors mounted on the work vehicle 100. For example, the internal sensor group 150 may include one or more sensors selected from a temperature sensor, illuminance sensor, fuel sensor, water temperature sensor, oil level gauge, engine speed sensor, vehicle speed sensor, battery voltage sensor, shuttle sensor, hand accelerator sensor, accelerator pedal sensor, main transmission lever sensor, sub-transmission lever sensor, seat belt sensor, PM sensor, acceleration sensor, angular velocity sensor, IMU (Inertial Measurement Unit), and geomagnetic sensor. The internal sensor group 150 may also include a PTO sensor that detects the on / off state of rotation of the PTO shaft, and / or a 3P position sensor that detects the height position of the 3-point hitch (hereinafter also simply referred to as "height"). Furthermore, in addition to one or more sensors mounted on the work vehicle 100, or in place of one or more sensors mounted on the work machine 300, one or more sensors mounted on the work machine 300 may be included in the internal sensor group 150 of the travel control system 1000.

[0089] In the example shown in Figure 3A, the control unit 180 includes multiple ECUs. These ECUs may include, for example, ECUs 181 to 184 shown in Figure 2. However, the control unit 180 may be a single ECU or other computing device. Figure 3B is a block diagram showing an example configuration of such a control unit 180. In the example in Figure 3B, the control unit 180 comprises a processor 281, a ROM (Read Only Memory) 283, a RAM (Random Access Memory) 285, a communication device 287, and a storage device 289. These components may be interconnected via a bus 290.

[0090] The processor 281 is a semiconductor integrated circuit, also referred to as a central processing unit (CPU) or microprocessor. The processor 281 may include an image processing unit (GPU). The processor 281 sequentially executes a computer program describing a predetermined set of instructions stored in the ROM 283, thereby realizing the processing performed by the driving control system according to an embodiment of the present invention. The control device 180 may comprise a plurality of processors 281. The processing performed by the driving control system according to an embodiment of the present invention may be performed collaboratively by the plurality of processors 281. Part or all of the processor 281 may be an FPGA (Field Programmable Gate Array), ASIC (Application Specific Integrated Circuit), or ASSP (Application Specific Standard Product) equipped with a CPU.

[0091] The communication device 287 is an interface for data communication between the control device 180 and an external computing device. The communication device 287 can perform wired communication such as CAN (Controller Area Network), or wireless communication compliant with the Bluetooth® standard and / or Wi-Fi® standard.

[0092] The storage device 289 can store location data acquired from the positioning device 110, first sensor data acquired from the internal sensor group 150, second sensor data acquired from the external sensor group 121, location data and / or sensor data during processing, first information acquired from location data and second information acquired from sensor data, etc. The storage device 289 includes, for example, a hard disk drive or a non-volatile semiconductor memory. In this example, the storage device 289 may also function as the storage device 870 in the example of Figure 3A.

[0093] The hardware configuration of the control device 180 is not limited to the example above. It is not necessary for part or all of the control device 180 to be mounted on the work vehicle 100. By utilizing the communication device 287, one or more computing devices located outside the work vehicle 100 can function as part or all of the control device 180. For example, one or more server computers and / or computing devices included in a terminal device connected to a network can function as part or all of the control device 180. Alternatively, one or more computing devices mounted on the work vehicle 100 may perform all the functions required of the control device 180.

[0094] Figure 4 is a schematic diagram showing another configuration example of a driving control system according to an embodiment of the present invention. The system shown in Figure 4 includes a work vehicle 100, other work vehicles 700, a server computer 500, and a plurality of terminal devices 400. The terminal devices 400 may be portable or fixed terminal devices. Some or all of the functions of the control device 180 shown in Figure 3B may be implemented by one or more computing devices connected to the communication device 287 of the control device 180 in the work vehicle 100 via a communication network 800. Such computing devices may be the server computer 500 or the terminal devices 400. Other work vehicles (e.g., agricultural machinery) 700 may be connected to such a communication network 800. Communication may take place between the control device 180 in the work vehicle 100 and the other work vehicles 700. Some of the data used for processing by the control device 180 in the work vehicle 100 may be provided to the control device 180 from the other work vehicles 700 via the communication network 800. For example, waypoint data defining a route and a series of operations generated by a control device in another work vehicle 700 may be transmitted from the other work vehicle 700 to the control device 180 of work vehicle 100. Based on this waypoint data, the control device 180 can perform a playback operation in the playback mode described later.

[0095] As shown in Figure 3B, one example of a "control device" in an embodiment of the present invention is a computing device comprising at least one processor and at least one memory that stores a computer program (code) that defines a control process executed by the processor. The "control device" may also be a computing device comprising a hardware accelerator such as an FPGA (Field-Programmable Gate Array), ASSP (Application Specific Standard Product), or ASIC (Application-Specific Integrated Circuit) configured to execute the control process.

[0096] In embodiments of the present invention, "processor" refers to hardware electronic circuits such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), DSP (Digital Signal Processor), ISP (Image Signal Processor), or NPU (Neural Network Processing Unit). "Memory" refers to hardware electronic circuits such as ROM (Read Only Memory) or RAM (Random Access Memory). Part of the memory may be a storage medium connected to the processor by wiring or a network. These hardware electronic circuits may be implemented by one or more integrated circuits (ICs) or large-scale integrated circuits (LSIs). Each functional unit or block and associated component within the electronic circuit may be manufactured individually as separate integrated circuit chips, or some or all of these functional units or blocks may be combined and manufactured as a single integrated circuit chip.

[0097] A program defining the operation of the processor is designed to cause the processor to perform one or more functions, operations, steps, or processes in embodiments of the present invention.

[0098] [Recording mode and playback mode] As described below, the travel control system 1000 can control the operation of the work vehicle 100 using the so-called teaching-playback method used in the field of robot control. The control device 180 in the travel control system 1000 can operate in recording mode and playback mode. Recording mode is a mode in which a plurality of positions (hereinafter sometimes referred to as "waypoints") that define the path traveled by the work vehicle 100 are recorded. In recording mode, the operation of the work vehicle 100 at each waypoint may be further recorded. Playback mode is a mode in which the travel path of the work vehicle 100 is reproduced based on the position data of the work vehicle 100 recorded in recording mode. If the operation of the work vehicle 100 at each waypoint is recorded in recording mode, the operation of the work vehicle 100 at each waypoint may also be reproduced in playback mode. The operations in recording mode and playback mode correspond to the teaching operation and playback operation in the teaching-playback method, respectively. The operation of the control device 180 in recording mode and playback mode may be referred to as "teaching" and "playback," respectively. Recording mode may also be referred to as "teaching mode," and playback mode as "playback mode."

[0099] The operation of the control device 180 in the travel control system 1000 in recording mode and playback mode will be explained with reference to Figures 5A and 5B. Figure 5A is a schematic diagram showing an example of a route 31T on which the work vehicle 100 travels in recording mode. Figure 5B is a schematic diagram showing an example of a route 31P on which the work vehicle 100 travels in playback mode. In this example, routes 31T and 31P are routes connecting the first field 70a and the second field 70b.

[0100] (Recording mode) As shown in Figure 5A, in recording mode, the work vehicle 100 travels along route 31T from position R1 in the first field 70a to position R2 in the second field 70b. In recording mode, the control device 180 records the position data output from the positioning device 110, acquired when the work vehicle 100 travels along route 31T, into the storage device 870.

[0101] Figure 5A shows the state in which the work vehicle 100 is located at the starting point (position R1) of the route 31T and the state in which the work vehicle 100 is located at the ending point (position R2). In this example, the route 31T starts from position R1 in the first field 70a, exits the first field 70a through the entrance / exit 73a of the first field 70a, travels outside the field, enters the second field 70b through the entrance / exit 73b of the second field 70b, and ends at position R2 in the second field 70b. Position R1 in the first field 70a is, for example, located within a predetermined area 71a in the first field 70a. Position R2 in the second field 70b is, for example, located within a predetermined area 71b in the second field 70b.

[0102] In recording mode, the control device 180 records multiple waypoint data in the storage device 870 based on position data output from the positioning device 110, for example, while the work vehicle 100 is traveling along route 31T. Each of the multiple waypoint data includes information about the position of the work vehicle 100 (sometimes referred to as "first information"). For example, as shown in Figure 5A, position data indicating each of the multiple positions (waypoints) Pr on the traveled route 31T is recorded in the storage device 870 as waypoint data. The multiple waypoint data can be recorded in the storage device 870 as "route data" indicating route 31T, associated with information about route 31T. Specific examples will be described later with reference to Figure 6.

[0103] In recording mode, the control device 180 may further record sensor data relating to the state of the work vehicle 100, output from the internal sensor group 150, in the storage device 870. In such a case, for example, each of the multiple waypoint data further includes information relating to the state of the work vehicle 100 (sometimes referred to as "second information"). The second information included in each of the multiple waypoint data may be recorded in association with the corresponding first information. By recording the second information in association with the corresponding first information, information on the state of the work vehicle 100 at each position on the route 31T traveled by the work vehicle 100 is recorded.

[0104] Figure 6 is a schematic diagram showing an example of waypoint data recorded in the storage device 870. Each waypoint data 89 shown in Figure 6 includes a waypoint number (No.) 90, first information 91 indicating the position of the work vehicle 100, and second information 92 indicating the status of the work vehicle 100. The first information 91 indicates the position coordinates of the waypoint. The position coordinates may, for example, show latitude and longitude in a geographic coordinate system, or they may show position coordinates in a coordinate system different from the geographic coordinate system. In addition to latitude and longitude, the position coordinates may also include altitude information. In the example in Figure 6, the second information 92 includes information indicating the vehicle speed, heading, steering angle, and attitude of the work vehicle 100. The second information 92 may include only a part of this information. Alternatively, the second information 92 may include other information not shown in Figure 6. As mentioned above, the waypoint data does not have to include the second information 92. Route data 88, which includes a collection of waypoint data 89 (i.e., multiple waypoint data), may be recorded in a storage device 870, associated with an identifier (e.g., route ID) 80 that identifies the route. The route identifier 80 is associated with information 81 about the route. In the illustrated example, the information 81 about the route includes, for example, information about the date and time the route data was created, information about the starting point of the route (e.g., the field before movement), information about the ending point of the route (e.g., the field after movement), information about the latitude and longitude of the reference points of the route (e.g., the start or end point of the route), and information about the size of the implement attached to the work vehicle when the route was traveled. The information 81 about the route may include only some of this information, or it may include other information not shown in Figure 6.

[0105] The second information broadly includes information about the state of the work vehicle 100 other than its position. The second information includes, for example, information about the operation of the work vehicle 100, such as its driving state. The driving state of the work vehicle 100 is determined by the speed of the work vehicle 100, acceleration (i.e., rate of change of speed per unit time), direction of travel (orientation), etc. Information about the driving state of the work vehicle 100 includes, for example, one or more of the following: information about the speed of the work vehicle 100, information about the engine speed of the work vehicle 100, information about the acceleration of the work vehicle 100, information about the orientation of the work vehicle 100, information about the steering angle of the steering wheels of the work vehicle 100, information about the gear ratio of the transmission 103 of the work vehicle 100, etc. The second information may also include information about the attitude of the work vehicle 100. Information about the orientation of the work vehicle 100 may include, for example, information about the angle between the horizontal component of the direction of travel of the work vehicle 100 and the reference direction (e.g., north). The orientation information of the work vehicle 100 may include, for example, information on the roll angle and pitch angle of the work vehicle 100. The orientation information of the work vehicle 100 may also include, for example, information on the orientation of the work vehicle 100. The second information is not limited to information on the operation of the work vehicle 100, but may also include, for example, information on the temperature of the work vehicle 100 (e.g., engine coolant temperature), information on whether or not there is a malfunction in the work vehicle 100 (e.g., diagnostic trouble code: DTC), etc. Specific examples of methods for acquiring the second information will be described later.

[0106] The second information may include information regarding the state of the coupling device 108 for connecting the work implement 300. The coupling device 108 may include, for example, a PTO shaft that supplies power to the work implement 300 and a three-point hitch for adjusting the height of the work implement 300. The information regarding the state of the coupling device 108 may include, for example, one or more of the following: information on whether the rotation of the PTO shaft is on or off, and information on the height of the three-point hitch.

[0107] The second information may include, in addition to information regarding the status of the work vehicle 100, information regarding the status of the work machine 300 if the work machine 300 is connected to the work vehicle 100. For example, if a positioning device is attached to the work machine 300, information regarding the position or orientation (e.g., angle relative to a reference orientation) of the work machine 300 may be included in the second information. Alternatively, if a sensor for detecting the movement of a movable part of the work machine 300 is provided on the work machine 300, information detected by that sensor may be included in the second information.

[0108] In the example shown in Figure 5A, in recording mode, the work vehicle 100 is driven manually by the driver 9 on the work vehicle 100. However, the example is not limited to this, and in recording mode, the work vehicle 100 may also be driven automatically. When the work vehicle 100 is driven automatically in recording mode, it may drive autonomously without the intervention of the driver's manual operation, or it may drive automatically while partially based on the driver's manual operation. For example, automatic steering control may be performed during driving in recording mode, where the driver controls the driving speed of the work vehicle 100 and the steering is controlled automatically. Alternatively, during driving in recording mode, the work vehicle 100 may be driven automatically while the work machine 300 is operated by the driver's manual operation. The driver's manual operation includes not only the driver's manual operation on the work vehicle 100, but also remote operation by an operator outside the work vehicle 100. Such remote operation can be performed using, for example, the terminal device 400 shown in Figure 4, or other remote control devices.

[0109] (Playback mode) In the playback mode shown in Figure 5B, the work vehicle 100 drives automatically. In playback mode, the control device 180 drives the work vehicle 100 automatically by controlling its speed and steering based on the position data recorded in the storage device 870. In playback mode, the control device 180 drives the work vehicle 100 along a target path 31P defined by, for example, first information contained in multiple waypoint data recorded in the storage device 870. For example, the control device 180 controls the steering of the work vehicle 100 to minimize the deviation of the work vehicle 100's position and orientation (azimuth) from the target path 31P. This allows the work vehicle 100 to drive along the target path 31P. In playback mode, the work vehicle 100 can automatically reproduce the path of the work vehicle 100 recorded in recording mode.

[0110] In playback mode, as shown in Figure 5B, the automatic driving of the work vehicle 100 may be controlled by a driver (user) 9 located away from the work vehicle 100 operating a terminal device 400. The driver operating the terminal device or control terminal may be on the work vehicle 100, and this is not limited to the illustrated example. The terminal device 400 operated by the driver 9 may be a portable control terminal or a fixed control terminal. A fixed control terminal may be attached to the work vehicle 100 or located away from the work vehicle 100. The terminal device 400 may be equipped with a display device such as a touchscreen. The display device may be a display such as a liquid crystal or organic light-emitting diode (OLED). The terminal device 400 may further be equipped with one or more buttons. The terminal device 400 may be equipped with a storage device.

[0111] According to the driving control system of this embodiment, in playback mode, the operation (e.g., driving) of the work vehicle 100 can be automatically reproduced based on first information regarding the position of the work vehicle 100 recorded in the storage device 870, so that repetitive operations of the work vehicle 100 can be performed efficiently. Therefore, automation and unmanned operation of the work vehicle 100 are promoted.

[0112] In recording mode, if second information relating to the state of the work vehicle 100 other than its position is further recorded in association with first information relating to the position of the work vehicle 100, the automation and unmanned operation of the work vehicle 100 can be further promoted.

[0113] As shown in the examples in Figures 5A and 5B, when the work machine 300 is connected to the work vehicle 100, the control device 180 can control the operation of the work vehicle 100 and the work machine 300 while the work vehicle 100 is automatically driven, based on the first information (or the first and second information) contained in the multiple waypoint data recorded in recording mode. In other words, in playback mode, the work vehicle 100 can automatically reproduce not only the operation of the work vehicle 100 recorded in recording mode, but also the operation of the work machine 300.

[0114] When a work implement 300 is attached to a work vehicle 100, in playback mode, the operation of the work vehicle 100 with the work implement 300 attached can be reproduced based on the first information recorded in recording mode, thus enabling efficient repetitive operations of the work vehicle 100 with the work implement 300 attached. For example, in recording mode, by recording second information regarding the state of the work implement 300 in association with first information regarding the position of the work vehicle 100, automation and unmanned operation of work by the work implement 300 can be promoted. In other words, the work vehicle 100 can automatically reproduce not only the operation of the work vehicle 100 recorded in recording mode, but also the operation of the work implement 300, thus enabling efficient repetitive operations performed by the work implement 300.

[0115] The control device 180 may, in recording mode and / or playback mode, acquire sensor data (sometimes referred to as "second sensor data") related to the conditions around the work vehicle 100, output from the external sensor group 121, while the work vehicle 100 is in motion. More details will be described later.

[0116] (Example of processing in recording mode) Figure 7 is a flowchart showing an example of processing performed by the control device 180 in recording mode.

[0117] The timing of the start of the recording mode is specified, for example, by the user. For example, the control device 180 may start the recording mode when a signal containing an instruction to start the recording mode is sent to the control device 180 by the driver's operation. For example, the driver on the work vehicle 100 can send a signal containing an instruction to start the recording mode to the control device 180 by operating an input device such as a predetermined operation switch or operation terminal 200 provided inside the work vehicle 100. The recording mode may be started while the work vehicle 100 is in motion, or it may be started when the work vehicle 100 is stopped.

[0118] When recording mode is started, in step S101, the control device 180 acquires position data output from the positioning device 110. The control device 180 may acquire position data at regular intervals, or it may acquire position data each time the work vehicle 100 travels a certain distance.

[0119] In step S102, the control device 180 determines whether there has been a change of a predetermined value or more in the driving state of the work vehicle 100. The driving state of the work vehicle 100 is defined by the speed, acceleration, direction of travel (orientation), etc., of the work vehicle 100, as described above. For example, if any change in the speed, acceleration, or direction of travel (orientation) of the work vehicle 100 exceeds a predetermined value, it is determined that there has been a change of a predetermined value or more in the driving state of the work vehicle 100 ("Yes" in step S102). For example, if the work vehicle 100 stops from a moving state, or if the work vehicle 100 starts moving from a stopped state, it is determined that there has been a change of a predetermined value or more in the driving state of the work vehicle 100. If "Yes" is answered in step S102, the process proceeds to step S103. If "No" is answered in step S102, the process proceeds to step S104.

[0120] In step S103, the control device 180 records the position data acquired in step S101. Recording the position data in step S103 includes not only recording the position data output from the positioning device 110 in the storage device 870, but also temporarily storing it in a storage device different from the storage device 870. The storage device different from the storage device 870 may be, for example, a memory of the control device 180 such as the RAM 285 shown in Figure 3B, or a storage device included in a server computer connected to the control device 180 via a communication network.

[0121] In step S104, the control device 180 determines whether the distance traveled from the position where the previous position data was recorded exceeds a threshold. The distance traveled threshold can be set to a value of, for example, several tens of centimeters (cm) to several meters (m). Alternatively, in step S104, the control device 180 may determine whether the time elapsed since the previous position data was recorded exceeds a threshold. The time threshold can be set to a value within the range of, for example, 1 second to 10 seconds.

[0122] If the answer in step S104 is "Yes", proceed to step S103. If the answer in step S104 is "No", no location data is recorded, and the process returns to step S101.

[0123] The control device 180 repeats the processes of steps S101, S102, S103, and S104 until it receives a signal that includes an instruction to terminate the recording of position data (step S105).

[0124] (Example of a display screen related to recording mode) Figures 8A to 8D, 9A to 9D, and 10A to 10C show examples of display screens shown on an operating terminal operated by a user (e.g., the driver of the work vehicle 100). These display screens include a graphical user interface (GUI) for the user to configure settings related to the recording mode. These display screens are shown, for example, on a display device on the operating terminal 200 of the work vehicle 100. Here, we show an example of recording position data when the work vehicle 100 travels along a route connecting fields in recording mode.

[0125] Figures 8A, 8B, 8C, and 8D are examples of display screens for setting the recording mode before the work vehicle 100 starts running in recording mode. Figures 9A, 9B, 9C, 9D, 10A, 10B, and 10C are examples of display screens displayed in recording mode. Figures 10A, 10B, and 10C are examples of display screens displayed after the work vehicle 100 has finished running in recording mode.

[0126] As shown in Figure 8A, a display screen is shown that allows the user to select whether to create or delete route data. On the display screen in Figure 8A, the user can select either the "Create" button 51a or the "Delete" button 51b depending on the operation they are about to perform. To create new route data, select the "Create" button 51a, as shown in the example. If the "Create" button 51a is selected on the display screen in Figure 9A, the user will be redirected to the display screen in Figure 8B or Figure 8C.

[0127] The display screen in Figure 8B includes a GUI that allows the user to set the size of the implement 300. In the illustrated example, the user can enter the width and length of the implement 300 in boxes 66a and 66b, respectively. The GUI may be configured to allow the user to enter other information about the work in addition to the width and length of the implement 300. For example, the GUI may be configured to allow the user to enter at least one of the type of work, the type of implement (e.g., model), or the maximum height of the implement when raised.

[0128] The display screen in Figure 8C includes a GUI that allows the user to set the field before relocation. In the illustrated example, the user selects the field before relocation from the displayed field images 52a, 52b, 52c, and 52d (sometimes referred to simply as "fields 52a-52d"). The display screen also shows an image 53 representing the work vehicle 100. Image 53 representing the work vehicle 100 may be shown at a position that reflects the work vehicle 100's current location. This allows the user to confirm the positional relationship between the work vehicle 100 and the field from the display screen. In the example in Figure 8C, field 52a is selected as the field before relocation. After selecting the field before relocation, the display screen in Figure 8D is accessed.

[0129] The display screen in Figure 8D includes a GUI that allows the user to set the field to be moved. In the illustrated example, the user selects the field to be moved from among fields 52b, 52c, and 52d, which are different from the field 52a before the move. In the illustrated example, it is shown that route data for the route 54 connecting the selected field 52a and field 52d before the move has already been recorded. In the example in Figure 8D, field 52b is selected as the field to be moved.

[0130] The display screen shown in Figure 9A may be displayed until the work vehicle 100 moves to the field 52a before moving. After the work vehicle 100 moves to the field 52a before moving, the display screen shown in Figure 9B will be displayed.

[0131] The display screen in Figure 9B includes a GUI for the user to perform operations to start recording location data. On the display screen in Figure 9B, the user performs the operation to start recording location data by selecting the "Start Recording" button 64a. Once the operation to start recording location data is performed, the display screen in Figure 9C is displayed.

[0132] The display screen in Figure 9C may be displayed while location data is being recorded. The display screen in Figure 9C includes an image 53 of the work vehicle 100 and a trajectory 63 of the work vehicle 100.

[0133] The display screen in Figure 9D may be displayed, for example, after the work vehicle 100 has reached the field after moving. The display screen in Figure 9D includes a GUI for the user to perform an operation to end the recording of location data. On the display screen in Figure 9D, the user performs an operation to end the recording of location data by selecting the "End Recording" button 64b. If the operation to end the recording of location data is performed, the display screens in Figures 10A, 10B, and 10C may be displayed.

[0134] The display screen in Figure 10A includes a GUI for the user to perform operations to set whether or not to record the acquired location data as route data in the storage device 870. The user can set whether or not to record the data as route data in the storage device 870 by selecting the "Yes" button 59a or the "No" button 59b. In this example, while the work vehicle 100 is in motion, the location data acquired from the positioning device 110 is temporarily stored in the storage device 870 or a different storage device (for example, a memory such as the RAM 285 shown in Figure 3B), and when the user selects the "Yes" button 59a on the display screen in Figure 10A, the data is associated with a route identifier and route information and recorded in the storage device 870 as route data.

[0135] As shown in the display screen of Figure 10B, if route data for the path connecting the field 52a before relocation and the field 52b after relocation has already been recorded in the storage device 870, a screen may be displayed to set whether or not to overwrite the route data. The user can set whether or not to overwrite (replace) the route data by selecting the "Yes" button 61a or the "No" button 61b. Multiple route data sets with the same combination of field before relocation and field after relocation may be recorded in the storage device 870. In the display screen of Figure 10B, route 64 and information 62a related to route 64 that have already been recorded in the storage device 870 may also be displayed.

[0136] When route data is recorded in the storage device 870, the display screen shown in Figure 10C may be displayed. The display screen shown in Figure 10C includes a notification 65a that indicates that route data has been recorded in the storage device 870.

[0137] (Example of processing in playback mode) Figure 11 is a flowchart showing an example of processing performed by the control device 180 in playback mode.

[0138] In playback mode, the control device 180 automatically drives the work vehicle 100 based on pre-recorded waypoint data. The control device 180 acquires position data indicating the position of the work vehicle 100 output from the positioning device 110 (step S121). Next, the control device 180 calculates the deviation between the position of the work vehicle 100 and the target path (step S122). The target path is defined by the position information (first information) of multiple waypoints recorded in recording mode. The deviation represents the distance between the position of the work vehicle 100 at that time and the target path. The control device 180 determines whether the calculated position deviation exceeds a preset threshold (step S123). If the deviation exceeds the threshold (if "Yes" is answered in step S123), the control device 180 changes the steering angle by changing the control parameters of the steering device 106 included in the drive device 240 so that the deviation becomes smaller (step S124). If the deviation does not exceed the threshold in step S123 (i.e., "No" in step S123), the process in step S124 is not performed. The control device 180 repeats the operations from steps S121 to S124 until it receives a signal including an instruction to end the playback mode (step S125).

[0139] In playback mode, the control device 180 automatically drives the work vehicle 100 along the target path by executing, for example, the process shown in Figure 11. The control device 180 may further control the operation of the work vehicle 100 based on state information (second information) corresponding to each of the multiple waypoints that define the target path. For example, if the second information includes information on the steering angle of the steering wheels of the work vehicle 100, in addition to the process shown in Figure 11, steering control of the work vehicle 100 based on the steering angle included in the second information may be performed. If the second information includes information on the speed of the work vehicle 100, the speed of the work vehicle 100 is controlled based on the speed information included in the second information.

[0140] Control technologies such as PID control or MPC control (model predictive control) can be applied to the steering and speed control of the work vehicle 100. By applying these control technologies, the control of the work vehicle 100 to approach the target path and target speed can be made smoother.

[0141] (If the second piece of information includes information about the driving status of the work vehicle) Referring to Figure 12, an example of processing performed by the control device 180 when the second information includes information regarding the driving state of the work vehicle 100 will be explained. Figure 12 is a schematic diagram illustrating an example of processing performed by the control device 180 in the driving control system 1000. In addition to the driving control system 1000, Figure 12 also shows the drive unit 240 and the operation switch group 210. For simplicity, some components are omitted from the illustration in Figure 12.

[0142] (Control of the speed of work vehicles) The control device 180 controls the speed of the work vehicle 100 by controlling the prime mover 102, the braking device (brake) 293, and the transmission 103, which are included in the drive unit 240. The braking device 293 brakes the axle that rotates the wheels 104 of the work vehicle 100. Specifically, the speed of the work vehicle 100 can be controlled by controlling the engine speed of the prime mover (engine) 102 and / or the gear ratio of the transmission 103. For example, the transmission 103 has multiple gears, and the control device 180 controls the gear ratio of the transmission 103 by switching the gears of the transmission 103. The multiple gears of the transmission 103 may be composed of a combination of multiple main gears and multiple sub-gears. When the work vehicle 100 is being driven manually, the control device 180 controls the speed of the work vehicle 100 by controlling the engine 102, the braking system (brakes) 293, and the transmission 103 in response to the driver's operation of the accelerator control device 215 (e.g., accelerator lever or accelerator pedal), the braking control device 216 (e.g., brake pedal), and / or the gear shift control switch 218 (e.g., shift lever). The gear shift control switch 218 is a switch for selecting the gear of the transmission 103. The control device 180 may further switch between two-wheel drive mode and four-wheel drive mode in response to the driver's operation.

[0143] In recording mode, the control device 180 sequentially acquires sensor data output from vehicle speed sensors such as the axle sensor 156, the engine speed sensor 158, and the gear ratio sensor 159, which detects the gear ratio information of the transmission 103. Based on this sensor data, the control device 180 generates and records information on the speed of the work vehicle 100, the engine speed of the work vehicle 100, and the gear ratio information of the transmission 103 as second information, associated with the position information (first information) of each waypoint. In this case, in playback mode, the control device 180 controls the speed of the work vehicle 100 by controlling the prime mover 102, the transmission 103, and the braking system 293 included in the drive unit 240, based on the second information recorded in recording mode. The gear ratio sensor 159 is mounted on the rotating shaft within the transmission 103 and may be a sensor that detects the gear ratio, or it may be a shift position sensor that identifies the selected gear by detecting the position of the shift lever (gear position operation switch 218) for selecting a gear. The gear ratio information of the transmission 103 is not limited to information indicating the gear ratio itself, but may also be information that identifies the selected gear among the multiple gears of the transmission 103. Since one gear corresponds to one gear ratio, if the gear is identified, the gear ratio can be identified.

[0144] The work vehicle 100 may be equipped with a double-speed turn mode (front wheel speed increase function). Double-speed turn is an operation that increases the speed of the front wheels when the steering angle of the front wheels exceeds a threshold due to the driver turning the steering wheel a large amount. Performing a double-speed turn reduces the turning radius and enables smoother turning. The work vehicle 100 may be equipped with a solenoid (referred to as the "double-speed solenoid") for driving a clutch to switch the double-speed turn mode on and off. The control device 180 can switch the double-speed solenoid on and off via a hydraulic circuit. When the double-speed solenoid is on, the rotational speed of the front wheels is approximately twice as fast as when the double-speed solenoid is off.

[0145] The second information may further include information regarding the driving mode of the work vehicle 100. For example, the information regarding the driving mode of the work vehicle 100 may include information on whether it is moving forward or backward. The information regarding the driving mode may also include information on whether the driving mode of the work vehicle 100 is in four-wheel drive mode or two-wheel drive mode. The information regarding the driving mode may also include information on whether the double-speed turn mode is on or off. The information regarding the driving mode may further include information on whether the automatic single-sided brake mode is on or off. When the automatic single-sided brake mode is on, it is a mode in which the inner rear wheel is lightly braked if the steering angle of the front wheel 104F, which is the steering wheel, exceeds a predetermined value while driving. In playback mode, the control device 180 controls the driving mode of the work vehicle 100 by controlling the prime mover 102, the transmission 103, and the braking device 293 included in the drive unit 240 based on the second information recorded in recording mode.

[0146] (Steering control of work vehicles) The control device 180 changes the steering angle of the front wheels 104F, which are the steering wheels of the work vehicle 100, by controlling the steering device 106, and changes the direction of the work vehicle 100 by changing the steering angle of the steering wheels. When the work vehicle 100 is being driven manually, the control device 180 changes the steering angle of the steering wheels and the direction of the work vehicle 100 by controlling the steering device 106 in response to the driver's operation of the steering wheel 217.

[0147] In recording mode, the control device 180 acquires information on the steering angle of the steering wheels of the work vehicle 100 as second information, based on sensor data (measured values) output from the steering wheel sensor 152 and / or steering angle sensor 154. In this case, in playback mode, the control device 180 controls the steering of the work vehicle 100 by controlling the hydraulic system or electric motor included in the steering system 106, based on the second information recorded in recording mode.

[0148] The second piece of information may further include information regarding the posture of the work vehicle 100. The posture of the work vehicle 100 may include, for example, the roll angle θ. R , pitch angle θ P , and yaw angle θ Y It is represented by the roll angle θ. R This represents the amount of rotation of the work vehicle 100 around its longitudinal axis. Pitch angle θ P This represents the amount of rotation of the work vehicle 100 around its left-right axis. Yaw angle θ Y This represents the amount of rotation of the work vehicle 100 around its vertical axis. The attitude can also be defined by other angles, such as Euler angles, or by quaternions. The control device 180 obtains information about the attitude of the work vehicle 100 based, for example, on data output from the IMU 115. [Comparison of sensor data acquired in recording mode and sensor data acquired in playback mode] Figure 13 is a flowchart showing an example of processing performed by the control device 180 in recording mode, and Figure 14 is a flowchart showing an example of processing performed by the control device 180 in playback mode. In this example, in recording mode and playback mode, the control device 180 acquires first sensor data output from the internal sensor group 150 when the work vehicle 100 is in motion, and in playback mode, it compares the first sensor data acquired in playback mode with the first sensor data recorded in recording mode.

[0149] This section will primarily explain the differences between the flowchart in Figure 13 and the flowchart in Figure 7.

[0150] When recording mode is started, as shown in Figure 13, in step S101a, the control device 180 acquires position data output from the positioning device 110 and first sensor data output from the internal sensor group 150. The control device 180 may acquire position data and first sensor data at regular intervals, or it may acquire position data and first sensor data each time the work vehicle 100 travels a certain distance.

[0151] The process in step S102a may be carried out in the same way as the process in step S102 in the example in Figure 7. The determination in step S102a as to whether or not there has been a change of a predetermined amount or more in the driving state of the work vehicle 100 may be made based on the first sensor data acquired in step S101a. That is, the control device 180 determines whether or not there has been a change of a predetermined amount or more in any of the speed, acceleration, or direction of the work vehicle 100 based on the first sensor data acquired in step S101a. If the result in step S102a is "Yes", proceed to step S103a. If the result in step S102a is "No", proceed to step S104a.

[0152] In step S103a, the control device 180 records the position data and first sensor data acquired in step S101a. The position data and first sensor data are recorded in association with each other. The recording of the position data and first sensor data in step S103a includes not only recording these data in the storage device 870, but also temporarily storing them in a storage device different from the storage device 870. The storage device different from the storage device 870 may be, for example, a memory of the control device 180 such as the RAM 285 shown in Figure 3B, or it may be a storage device included in a server computer connected to the control device 180 via a communication network.

[0153] The processes in steps S104a and S105a can be carried out in the same way as the processes in steps S104 and S105 in the example in Figure 7.

[0154] When playback mode is started, as shown in Figure 14, in step S131, the control device 180 acquires position data output from the positioning device 110 and first sensor data output from the internal sensor group 150 while the work vehicle 100 is automatically driven. The control device 180 may acquire position data and first sensor data at regular intervals, or it may acquire position data and first sensor data each time the work vehicle 100 travels a certain distance. Here, "regular interval" or "regular distance" may be the same as or different from the time interval or travel distance interval for acquiring position data and first sensor data in step S101a of the recording mode.

[0155] In step S132, the control device 180 compares the first sensor data acquired in step S131 with the first sensor data recorded in recording mode (for example, the first sensor data included in a plurality of waypoint data recorded as route data in the storage device 870) to determine whether or not the state should be notified to the user. For example, if the difference between the first sensor data acquired in playback mode and the first sensor data recorded in recording mode exceeds a predetermined value, the control device 180 determines that the state of the work vehicle 100 should be notified to the user. The first sensor data includes, for example, data on the roll angle or pitch angle of the work vehicle 100. The predetermined value (i.e., the threshold for determination) can be set for each sensor data.

[0156] In step S132, the control device 180 may compare the altitude data of the work vehicle 100 included in the position data. That is, the control device 180 may determine whether or not the status should be notified to the user by comparing the altitude data included in the position data acquired in step S131 with the altitude data included in the position data recorded in recording mode (for example, position sensor data included in multiple waypoint data recorded as route data in the storage device 870). For example, if the difference between the altitude data acquired in playback mode and the altitude data recorded in recording mode exceeds a predetermined value, the control device 180 determines that the status of the work vehicle 100 should be notified to the user. The predetermined value (i.e., the threshold for determination) can be set for each sensor data.

[0157] The state of the work vehicle 100 or the state of the work vehicle 100's surroundings is considered a "state that should be notified to the user" when the work vehicle 100 is operating automatically. This means that the state of the work vehicle 100 or the state of the work vehicle 100's surroundings should be notified to the user (e.g., a warning). A "state that should be notified to the user" is not limited to a state where there is a problem with the work vehicle 100's automatic operation, but also includes a state where problems may occur if the work vehicle 100 continues to operate automatically. By the control device 180 detecting a state that should be notified to the user based on the first sensor data, the user can ensure that the work vehicle 100 operates automatically smoothly. For example, when the control device 180 detects a state that should be notified to the user, it can cause the work vehicle 100 to perform an "action to deal with a state that should be notified to the user," as described later. Therefore, the work vehicle 100 can operate automatically efficiently in regeneration mode.

[0158] For example, if the difference between the inclination angle (roll angle and / or pitch angle) of the work vehicle 100 and the recorded data is too large, it is determined that the user should be notified. For example, between the time the position data and first sensor data acquired in recording mode are recorded and the time the work vehicle 100 is driven automatically using this data in playback mode, the road conditions along the driving route may change, for example, the slope or unevenness of the ground may change. Examples of cases in which the slope or unevenness of the ground changes include, for example, the road (e.g., farm road) or entrance / exit to a field along the driving route being maintained, ruts being formed on the road along the driving route, the depth of the ruts changing, the road surface along the driving route collapsing at least partially, or an object appearing on the driving route, which can cause the slope or unevenness of the ground to change. In such cases, the difference between the inclination angle (roll angle and / or pitch angle) of the work vehicle 100 and the recorded data may become large, and if the difference exceeds a predetermined value, it is determined that the user should be notified. The predetermined value (i.e., the threshold for judgment) can be set in advance by the user, for example.

[0159] If a condition requiring notification to the user is detected (if the answer is "Yes" in step S133), in step S134, the control device 180 causes the work vehicle 100 to perform an action to address the condition requiring notification to the user. The "action to address the condition requiring notification to the user" is an action to avoid problems that may occur if the work vehicle 100 continues to drive automatically. The "action to address the condition requiring notification to the user" includes, for example, at least one of the following: outputting a notification from a notification device to the user that a condition requiring notification to the user has been detected; slowing down the work vehicle 100; or stopping the work vehicle 100 from driving. The notification device may include, for example, a display device and / or an audio output device such as a buzzer or speaker. The notification device is provided, for example, on an operating terminal operated by the user (for example, an operating terminal 200 on the work vehicle 100). The notification device may be mounted on the work vehicle 100, or is not limited to this example as long as it is in a form that can output a notification to the user. Notifications may be issued in a way that stimulates the user's five senses, such as through images, light, sound, or vibration. A combination of one or more of these may also be used to make a notification.

[0160] The control device 180 may update the first sensor data recorded in the storage device 870 based on the first sensor data acquired in the regeneration mode. By reflecting changes in the road conditions along the travel route in the waypoint data, the automatic driving of the work vehicle 100 can be made smoother when the waypoint data is used in subsequent regeneration modes.

[0161] If no condition requiring notification to the user is detected (i.e., "No" in step S133), the control device 180 repeats the processes of steps S131 to S134 until the automatic driving of the work vehicle 100 in regeneration mode is completed (step S135).

[0162] Figure 15 is a flowchart showing another example of processing performed by the control device 180 in playback mode.

[0163] The process in step S131 can be carried out in the same manner as in the example in Figure 14.

[0164] In step S136, the control device 180 determines whether the first sensor data acquired in step S131 exceeds a predetermined value. If it exceeds the predetermined value (i.e., "Yes" in step S136), it determines that the user should be notified. The first sensor data includes, for example, data on the roll angle or pitch angle of the work vehicle 100. For example, if the tilt angle (roll angle and / or pitch angle) of the work vehicle 100 is too large, it is undesirable to continue the automatic driving of the work vehicle 100. By determining that the user should be notified, the user can ensure that the work vehicle 100 operates automatically smoothly. The predetermined value (i.e., the threshold for determination) can be set in advance by the user, for example. If "Yes" is obtained in step S136, the process proceeds to step S137, in which the control device 180 causes the work vehicle 100 to perform an action to address the condition that should be notified to the user. The processing in step S137 may be performed in the same manner as in step S134 in Figure 14. If the answer in step S136 is "No", proceed to step S138.

[0165] In step S138, the control device 180 identifies waypoint data to be used for comparison with the first sensor data acquired in step S131. The process performed in step S138 will be explained with reference to Figure 16. Figure 16 is a schematic diagram illustrating the process performed in step S138.

[0166] For example, as explained with reference to Figure 6, the route data recorded in the storage device 870 includes multiple waypoint data, and each of the multiple waypoint data includes position data and first sensor data. For example, in the example shown in Figure 16, each waypoint data, if the waypoint No. is "n", includes position data r(n) and two types of first sensor data X1(n) and X2(n). The two types of first sensor data X1(n) and X2(n) may be, for example, roll angle and pitch angle data. As in the example in Figure 16, suppose that in step S131, position data r(rm) and first sensor data X1(rm) and X2(rm) are acquired at point rm. Point rm is located between the position data r(n-1) included in the waypoint data No. (n-1) and the position data r(n) included in the waypoint data No. n. The distance between point rm and position data r(n-1) is smaller than the distance between point rm and position data r(n). In such a case, the control device 180 may identify waypoint data No. (n-1) as waypoint data containing the position data closest to point rm, and use the values ​​X1(n-1) and X2(n-1) of the first sensor data included in that waypoint data to compare with the values ​​X1(rm) and X2(rm) of the first sensor data acquired in step S131.

[0167] Alternatively, the control device 180 may interpolate the first sensor data recorded in the storage device 870 in recording mode. The control device 180 identifies waypoint data No. (n-1) and waypoint data No. n, which include position data located on both sides of point rm, as waypoint data to be used to interpolate the first sensor data. The control device 180 calculates the first sensor data at point rm by interpolating (e.g., linear interpolation) the position data and the first sensor data, respectively, using the identified waypoint data No. (n-1) and waypoint data No. n, and can compare this with the first sensor data X1(rm) and X2(rm) acquired in step S131.

[0168] In step S139, the control device 180 uses the waypoint data identified in step S138 to compare the first sensor data acquired in step S131 with the first sensor data recorded in recording mode. If the difference between the first sensor data acquired in step S131 and the first sensor data recorded in recording mode exceeds a predetermined value (if the result is "Yes" in step S140), the control device 180 determines that the state of the work vehicle 100 is one that should be notified to the user, and proceeds to step S137. If the result is "No" in step S140, that is, if the difference between the first sensor data acquired in step S131 and the first sensor data recorded in recording mode does not exceed a predetermined value, the control device 180 repeats the processes of steps S131, S136 to S140 until the automatic driving of the work vehicle 100 in playback mode is completed (step S141).

[0169] Referring to Figures 17 and 18, other examples of processing performed by the control device 180 will be described. Figure 17 is a flowchart showing another example of processing performed by the control device 180 in recording mode, and Figure 18 is a flowchart showing another example of processing performed by the control device 180 in playback mode.

[0170] In the example described with reference to Figures 13 to 16, the control device 180 acquires first sensor data when the work vehicle 100 is in motion in both recording and playback modes, and in playback mode compares the acquired first sensor data with the first sensor data acquired in recording mode. In contrast, in the examples of Figures 17 and 18, the control device 180 acquires second sensor data output from the external sensor group 121 when the work vehicle 100 is in motion in both recording and playback modes, and in playback mode compares the second sensor data acquired in playback mode with the second sensor data recorded in recording mode.

[0171] The processing shown in the examples in Figures 17 and 18 may be combined with any of the examples described above. That is, both the first sensor data and the second sensor data may be acquired and compared in recording mode and playback mode.

[0172] This section will primarily explain the differences between the flowchart in Figure 17 and the flowchart in Figure 7.

[0173] As shown in Figure 17, when the recording mode is started, in step S101b, the control device 180 acquires the position data output from the positioning device 110 and the second sensor data output from the external sensor group 121. The control device 180 may acquire the position data at regular intervals, or it may acquire the position data each time the work vehicle 100 travels a certain distance. The acquisition of the second sensor data may be performed at the same time as the acquisition of the position data, or at a different time.

[0174] The process in step S102b can be carried out in the same way as the process in step S102 in the example in Figure 7. If the answer in step S102b is "Yes", proceed to step S103b. If the answer in step S102b is "No", proceed to step S104b.

[0175] In step S103b, the control device 180 records the position data and second sensor data acquired in step S101b. The position data and second sensor data may be recorded in association with each other. For example, each of the multiple waypoint data may be recorded to include second sensor data, or the second sensor data may be recorded in a different format from the multiple waypoint data and associated with the route identifier. The second sensor data may include, for example, image data (still images or videos) acquired by the camera 120, point cloud data acquired by the LiDAR sensor 140, etc. Based on the second sensor data, the control device 180 may acquire information about the state of the surroundings of the work vehicle 100 and record the acquired information. Information about the state of the surroundings of the work vehicle 100 may include, for example, information about obstacles (e.g., information about the presence or absence of obstacles, information about the location of detected obstacles, information about the type of detected obstacles, etc.), information about the road that the work vehicle 100 traveled on (e.g., information about the surface condition of the road, information about the width of the road, etc.). For example, even if an object is detected around the work vehicle 100, an object that does not pose a problem for the movement of the work vehicle 100 may be recorded as a non-obstacle object.

[0176] The recording of position data and second sensor data in step S103b includes not only recording these data in the storage device 870, but also temporarily storing them in a storage device different from the storage device 870. The storage device different from the storage device 870 may be, for example, a memory of the control device 180 such as the RAM 285 shown in Figure 3B, or a storage device included in a server computer connected to the control device 180 via a communication network.

[0177] The processes in steps S104b and S105b can be carried out in the same way as the processes in steps S104 and S105 in the example in Figure 7.

[0178] When the playback mode is started, as shown in Figure 18, in step S151, the control device 180 automatically drives the work vehicle 100 while acquiring position data output from the positioning device 110 and second sensor data output from the external sensor group 121.

[0179] In step S152, the control device 180 compares the second sensor data acquired in step S151 with the second sensor data recorded in recording mode to determine whether the state of the work vehicle 100 or the state of the work vehicle 100's surroundings is a state that should be notified to the user. For example, the control device 180 may also make a comparison with information about the state of the work vehicle 100's surroundings that has been acquired and recorded based on the second sensor data. For example, if information about detected obstacles has been recorded, the control device 180 may detect whether there has been a change in the position of the obstacles, etc. For example, if information about the road the work vehicle 100 traveled has been recorded, the control device 180 may detect whether there has been a change in the road conditions. Based on the detected changes, the control device 180 determines whether the state should be notified to the user. For example, the control device 180 may determine whether the state should be notified to the user by determining whether the detected changes could cause problems if the work vehicle 100 continues to drive automatically.

[0180] If a condition requiring notification to the user is detected (if the answer is "Yes" in step S153), in step S154, the control device 180 causes the work vehicle 100 to perform an action to address the condition requiring notification to the user. The processing in step S154 may be performed in the same manner as the processing in step S134 in the example in Figure 14. The control device 180 may update the second sensor data (or information recorded based on the second sensor data) recorded in the storage device 870 based on the second sensor data acquired in the regeneration mode.

[0181] If no condition requiring notification to the user is detected (i.e., "No" in step S153), the control device 180 repeats the processes of steps S151 to S154 until the automatic driving of the work vehicle 100 in regeneration mode is completed (step S155).

[0182] The travel control system according to embodiments of the present invention is not limited to those illustrated. For example, the above example shows the creation of route data for a route connecting fields, but the invention is not limited to this example. For example, route data may be created for a route taken by a work vehicle within a field. Route data may also be created for a route taken by a work vehicle within a field while performing work using a work implement.

[0183] The driving control systems in the above embodiments can also be retrofitted to work vehicles that do not possess those functions. Such control systems can be manufactured and sold independently of the work vehicles. Computer programs used in such control systems can also be manufactured and sold independently of the work vehicles. Computer programs can be provided, for example, stored in a computer-readable non-temporary storage medium. Computer programs can also be provided by download via telecommunications lines (e.g., the Internet). [Industrial applicability]

[0184] The driving control system according to the embodiment of the present invention is widely applicable to various types of work vehicles used in smart agriculture. [Explanation of Symbols]

[0185] 100...Work vehicle, 110...Positioning device (GNSS unit), 150...Sensor group (internal sensor group), 180...Control device, 210...Operation switch group, 300...Work machine, 1000...Driving control system

Claims

1. A vehicle driving control system for work vehicles, A positioning device that outputs positional data relating to the position of the aforementioned work vehicle, One or more internal sensors that output first sensor data relating to the state of the work vehicle, and / or one or more external sensors that output second sensor data relating to the state of the surroundings of the work vehicle, A control device that controls the operation of the aforementioned work vehicle and Equipped with, The control device is It can operate in both recording and playback modes. In the recording mode described above, the position data acquired when the work vehicle is in motion is recorded in the storage device. In the playback mode, the speed and steering of the work vehicle are controlled based on the position data recorded in the storage device, thereby driving the work vehicle automatically. In the recording mode and the playback mode, the first sensor data and / or the second sensor data are acquired. A driving control system that, in the playback mode, detects whether the state of the work vehicle or the state of the surroundings of the work vehicle is a state that should be notified to the user by comparing the first sensor data and / or the second sensor data acquired in the playback mode with the first sensor data and / or the second sensor data recorded in the storage device.

2. The control device is In the recording mode and the playback mode, while the work vehicle is in motion, the first sensor data is acquired. The driving control system according to claim 1, wherein in the playback mode, if the difference between the first sensor data acquired in the playback mode and the first sensor data acquired in the recording mode exceeds a predetermined value, it is determined that the state of the work vehicle is in a state that should be notified to the user.

3. The control device is In the recording mode, the first sensor data is recorded in the storage device. The driving control system according to claim 2, wherein in the playback mode, the first sensor data recorded in the recording mode is interpolated and compared with the first sensor data acquired in the playback mode.

4. The control device is In the recording mode and the playback mode, while the work vehicle is in motion, the first sensor data is acquired. The driving control system according to any one of claims 1 to 3, wherein in the playback mode, if the value of the first sensor data acquired in the playback mode exceeds a predetermined value, it is determined that the state of the work vehicle is in a state that should be notified to the user.

5. The first sensor data is A driving control system according to any one of claims 1 to 3, including data on the roll angle or pitch angle of the aforementioned work vehicle.

6. The position data includes the altitude data of the work vehicle, The control device is A driving control system according to any one of claims 1 to 3, wherein in the playback mode, the system detects whether the state of the work vehicle or the state of the surroundings of the work vehicle is in a state that should be notified to the user by comparing the altitude data acquired in the playback mode with the altitude data recorded in the storage device.

7. The control device is In the recording mode and the playback mode, while the work vehicle is in motion, the second sensor data is acquired. The second sensor data acquired in the recording mode is recorded in the storage device in association with the position data. A driving control system according to any one of claims 1 to 3, wherein in the playback mode, the system detects whether the state of the work vehicle or the state of the surroundings of the work vehicle is in a state that should be notified to the user, based on a comparison between the second sensor data acquired in the playback mode and the second sensor data acquired in the recording mode.

8. The control device is The driving control system according to claim 7, wherein information on the presence or absence of an obstacle and the location of the obstacle is recorded in the storage device based on the second sensor data acquired in the recording mode.

9. The control device is The driving control system according to claim 8, wherein information on the type of obstacle is recorded in the storage device based on the second sensor data acquired in the recording mode.

10. The control device is The driving control system according to claim 8, wherein information about the road traveled by the work vehicle is recorded in the storage device based on the second sensor data acquired in the recording mode.

11. The control device is In the playback mode, if it is determined that the state of the work vehicle is in a state that should be notified to the user, the driving control system according to any one of the following: notifying the user, slowing down the work vehicle, and stopping the movement of the work vehicle.

12. The control device is A driving control system according to any one of claims 1 to 3, wherein in the playback mode, if it is determined that the state of the work vehicle is in a state that should be notified to the user, the driving of the work vehicle is stopped.

13. The control device is A driving control system according to any one of claims 1 to 3, wherein the first sensor data and / or second sensor data recorded in the storage device are updated based on the first sensor data and / or second sensor data acquired in the playback mode.

14. A driving control system according to any one of claims 1 to 3, Running gear including the steering wheels, A drive unit that drives the aforementioned traveling device and Equipped with, A work vehicle that, in the playback mode, controls the drive unit based on the position data recorded in the recording mode, thereby driving the work vehicle automatically.

15. A control device for controlling the operation of a work vehicle, which is executed by a control device capable of operating in recording mode and playback mode, and a method for controlling the movement of a work vehicle, In the recording mode, the position data relating to the position of the work vehicle acquired when the work vehicle was in motion is recorded in the storage device, In the playback mode, the speed and steering of the work vehicle are controlled based on the position data recorded in the storage device, thereby driving the work vehicle automatically. In the recording mode and the playback mode, first sensor data relating to the state of the work vehicle and / or second sensor data relating to the state of the surroundings of the work vehicle are acquired. In the playback mode, the system detects whether the state of the work vehicle or the state of the surroundings of the work vehicle is in a state that should be notified to the user by comparing the first sensor data and / or the second sensor data acquired in the playback mode with the first sensor data and / or the second sensor data recorded in the storage device. A driving control method including the above.

16. A computer program executed by a processor in a control device that controls the operation of a work vehicle and is capable of operating in recording mode and playback mode, The aforementioned processor, In the recording mode, the position data relating to the position of the work vehicle acquired when the work vehicle was in motion is recorded in the storage device, In the playback mode, the speed and steering of the work vehicle are controlled based on the position data recorded in the storage device, thereby driving the work vehicle automatically. In the recording mode and the playback mode, first sensor data relating to the state of the work vehicle and / or second sensor data relating to the state of the surroundings of the work vehicle are acquired. In the playback mode, the system detects whether the state of the work vehicle or the state of the surroundings of the work vehicle is in a state that should be notified to the user by comparing the first sensor data and / or the second sensor data acquired in the playback mode with the first sensor data and / or the second sensor data recorded in the storage device. A computer program that executes something.

Images