Vehicle control method, vehicle control program, and vehicle control system

Abstract

To provide a vehicle control method, a vehicle control program, and a vehicle control system, which are capable of causing a work vehicle to execute appropriate handling processing for an obstacle while preventing a decrease in the working efficiency of the work vehicle.SOLUTION: An acquisition processing unit 511 acquires: distance information of a distance to a detection target detected by an obstacle sensor 54 when a work vehicle 10 travels automatically along a target route; and image information on a captured image of the detection target captured by a camera 53. A determination processing unit 512 determines the type of the detection target based on the image information. A handling processing unit 112 causes the work vehicle 10 to perform handling processing according to the type of the detection target and the distance information.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a technique for controlling the operation of a work vehicle when a detection target such as an obstacle is detected. [Background technology]

[0002] Conventionally, there are known work vehicles equipped with obstacle sensors such as infrared sensors and ultrasonic sensors, and capable of detecting a detection target within a detection range during autonomous driving (see, for example, Patent Document 1). When the detection target is an obstacle, the work vehicle executes avoidance control to avoid a collision with the obstacle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-65115 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in conventional technology, the presence or absence of an obstacle is determined based on the distance to the detection target, so even if the detection target is an object for which the work vehicle does not need to take avoidance action (a work target such as a harvesting target or a harvest target), the work vehicle will be forced to take avoidance action when the distance to the detection target falls below a threshold value. This causes a problem of reduced work efficiency of the work vehicle.

[0005] An object of the present invention is to provide a vehicle control method, a vehicle control program, and a vehicle control system that are capable of causing a work vehicle to execute appropriate countermeasures against an obstacle while preventing a decrease in the work efficiency of the work vehicle. [Means for solving the problem]

[0006] The vehicle control method of the present invention is a method that, when a work vehicle is automatically traveling along a target route, acquires distance information of the distance to a detection target detected by a detection unit, acquires image information of an image of the detection target captured by an imaging unit, determines the type of the detection target based on the image information, and causes the work vehicle to execute response processing according to the type of the detection target and the distance information.

[0007] The vehicle control program of the present invention is a program for causing one or more processors to execute the following operations when a work vehicle is automatically traveling along a target route: acquiring distance information of the distance to a detection target detected by a detection unit; acquiring image information of an image of the detection target captured by an imaging unit; determining the type of the detection target based on the image information; and causing the work vehicle to execute response processing according to the type of the detection target and the distance information.

[0008] The vehicle control system according to the present invention includes an acquisition processing unit, a determination processing unit, and a countermeasure processing unit. When a work vehicle automatically travels along a target route, the acquisition processing unit acquires distance information of a distance to a detection target detected by a detection unit and image information of an image of the detection target captured by an imaging unit. The determination processing unit determines a type of the detection target based on the image information. The countermeasure processing unit causes the work vehicle to execute a countermeasure process according to the type of the detection target and the distance information. Effect of the Invention

[0009] According to the present invention, it is possible to provide a vehicle control method, a vehicle control program, and a vehicle control system that are capable of causing a work vehicle to execute appropriate countermeasures against an obstacle while preventing a decrease in the work efficiency of the work vehicle. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing a configuration of an automatic driving system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is an external view showing an example of a work vehicle according to an embodiment of the present invention. [Diagram 3] FIG. 3 is a diagram showing an example of a travel route of the work vehicle according to the embodiment of the present invention. [Figure 4] FIG. 4 is a diagram showing an example of the detection range of an obstacle sensor provided in the work vehicle according to the embodiment of the present invention. [Diagram 5] FIG. 5 is a diagram showing an example of a likelihood threshold value used in the automated driving system according to the embodiment of the present invention. [Figure 6] FIG. 6 is a diagram showing an example of the response process executed in the work vehicle according to the embodiment of the present invention. [Figure 7] FIG. 7 is a diagram showing an example of a menu screen displayed on the operation terminal according to the embodiment of the present invention. [Figure 8] FIG. 8 is a diagram showing an example of a driving screen displayed on the operation terminal according to the embodiment of the present invention. [Figure 9] FIG. 9 is a diagram showing an example of a sensitivity setting screen displayed on the operation terminal according to the embodiment of the present invention. [Figure 10A] FIG. 10A is a diagram showing an example of likelihood thresholds according to sensitivity used in the automated driving system according to the embodiment of the present invention. [Figure 10B] FIG. 10B is a diagram showing an example of likelihood thresholds according to sensitivity used in the automated driving system according to the embodiment of the present invention. [Figure 11] FIG. 11 is a diagram showing an example of a control target setting screen displayed on the operation terminal according to the embodiment of the present invention. [Figure 12] FIG. 12 is a flowchart showing an example of a procedure of an automatic driving process executed by the automatic driving system according to the embodiment of the present invention. [Figure 13A] FIG. 13A is a diagram showing another example of the likelihood threshold value used in the automated driving system according to the embodiment of the present invention. [Figure 13B] FIG. 13B is a diagram showing another example of the likelihood threshold value used in the automated driving system according to the embodiment of the present invention. [Figure 13C] FIG. 13C is a diagram showing another example of the likelihood threshold value used in the automated driving system according to the embodiment of the present invention. [Figure 14] FIG. 14 is a diagram showing an example of a monitoring mode setting screen displayed on the operation terminal according to the embodiment of the present invention. [Figure 15] FIG. 15 is a diagram showing an example of a work mode setting screen displayed on the operation terminal according to the embodiment of the present invention. [Figure 16] FIG. 16 is a diagram showing an example of a work mode setting screen displayed on the operation terminal according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The following embodiment is an example of the present invention, and is not intended to limit the technical scope of the present invention.

[0012] 1, an automated driving system 1 according to an embodiment of the present invention includes a work vehicle 10 and an operation terminal 20. The work vehicle 10 and the operation terminal 20 can communicate with each other via a communication network N1. For example, the work vehicle 10 and the operation terminal 20 can communicate with each other via a mobile phone network, a packet network, or a wireless LAN.

[0013] In this embodiment, the work vehicle 10 will be described as a tractor. In other embodiments, the work vehicle 10 may be a combine harvester, a rice transplanter, a construction machine, a snowplow, or the like. The work vehicle 10 is a so-called robot tractor that is configured to be able to automatically travel in a farm field F (see FIG. 3 ) according to a preset target route R. For example, the work vehicle 10 can automatically travel according to a pre-generated target route R based on position information of the current position of the work vehicle 10 calculated by the positioning unit 17.

[0014] For example, the work vehicle 10 travels back and forth in parallel from a travel start position S to a travel end position G in a work area of ​​a field F shown in Fig. 3. The outer periphery of the field F is, for example, a headland area, and the work vehicle 10 turns in the headland area. The target route R is not limited to the route shown in Fig. 3, and is set appropriately depending on the work content.

[0015] The operation terminal 20 is a portable terminal capable of remotely operating the work vehicle 10, and is configured, for example, as a tablet terminal, a notebook personal computer, a smartphone, or the like. A worker (operator) can perform setting operations for various setting items on the operation terminal 20. In addition, the operation terminal 20 displays information such as the work status and driving status of the work vehicle 10 during automatic driving. The worker can grasp the work status and driving status on the operation terminal 20.

[0016] [Work vehicle 10] 1 and 2, the work vehicle 10 includes a vehicle control device 11, a memory unit 12, a traveling device 13, a work implement 14, an obstacle detection device 15, a communication unit 16, a positioning unit 17, etc. The vehicle control device 11 is electrically connected to the memory unit 12, the traveling device 13, the work implement 14, the obstacle detection device 15, the positioning unit 17, etc. Note that the vehicle control device 11 and the obstacle detection device 15 may be capable of wireless communication, and the vehicle control device 11 and the positioning unit 17 may be capable of wireless communication.

[0017] The communication unit 16 is a communication interface that connects the work vehicle 10 to the communication network N1 either wired or wirelessly, and executes data communication with external devices (such as the operation terminal 20) via the communication network N1 in accordance with a predetermined communication protocol.

[0018] The storage unit 12 is a non-volatile storage unit such as an HDD (Hard Disk Drive) or SSD (Solid State Drive) that stores various information. The storage unit 12 stores control programs such as an automatic driving program for causing the vehicle control device 11 to execute an automatic driving process (see FIG. 12) described later. For example, the automatic driving program is non-temporarily recorded on a computer-readable recording medium such as a CD or DVD, and is read by a predetermined reading device (not shown) and stored in the storage unit 12. The automatic driving program may be downloaded from a server (not shown) to the work vehicle 10 via a communication network N1 and stored in the storage unit 12. The storage unit 12 also stores data of a target route R generated in the operation terminal 20, etc.

[0019] The traveling device 13 is a drive unit that causes the work vehicle 10 to travel. As shown in Fig. 2, the traveling device 13 includes an engine 131, front wheels 132, rear wheels 133, a transmission 134, a front axle 135, a rear axle 136, a handlebar 137, and the like. The front wheels 132 and the rear wheels 133 are provided on the left and right sides of the work vehicle 10, respectively. The traveling device 13 is not limited to a wheel type that includes the front wheels 132 and the rear wheels 133, but may be a crawler type that includes crawlers provided on the left and right sides of the work vehicle 10.

[0020] The engine 131 is a drive source such as a diesel engine or a gasoline engine that is driven by fuel supplied to a fuel tank (not shown). The traveling device 13 may be equipped with an electric motor as a drive source together with or instead of the engine 131. A generator (not shown) is connected to the engine 131, and power is supplied from the generator to the vehicle control device 11, the obstacle detection device 15, the positioning unit 17, and other electrical components provided in the work vehicle 10, as well as to a battery. The battery is charged by the power supplied from the generator. The vehicle control device 11, the obstacle detection device 15, the positioning unit 17, and other electrical components provided in the work vehicle 10 can be driven by the power supplied from the battery even after the engine 131 is stopped.

[0021] The driving force of the engine 131 is transmitted to front wheels 132 via a transmission 134 and a front axle 135, and to rear wheels 133 via a transmission 134 and a rear axle 136. The driving force of the engine 131 is also transmitted to the work equipment 14 via a PTO shaft (not shown). When the work vehicle 10 performs autonomous traveling, the traveling device 13 performs a traveling operation in accordance with commands from the vehicle control device 11. The traveling device 13 also decelerates or stops the work vehicle 10 in accordance with commands from the vehicle control device 11.

[0022] The working implement 14 is, for example, a cultivator, a grass cutter, a plow, a fertilizer applicator, a sprayer (chemical applicator), a tiller, or a seed sower, and can be attached to and detached from the work vehicle 10. This allows the work vehicle 10 to perform various types of work using each of the working implements 14. FIG. 2 shows a case in which the working implement 14 is a cultivator. For example, the working implement 14 is attached to the rear of the work vehicle 10. The work vehicle 10 performs cultivating work by traveling through a field F with the working implement 14 attached to the rear.

[0023] The handle 137 is an operating section that is operated by an operator or the vehicle control device 11. For example, in the traveling device 13, the angle of the front wheels 132 is changed by a hydraulic power steering mechanism (not shown) or the like in response to the operation of the handle 137 by the vehicle control device 11, and the traveling direction of the work vehicle 10 is changed.

[0024] In addition to the handlebars 137, the traveling device 13 is equipped with a shift lever, accelerator, brake, etc. (not shown) that are operated by the vehicle control device 11. In the traveling device 13, the gear of the transmission 134 is switched to a forward gear, a reverse gear, etc. in response to operation of the shift lever by the vehicle control device 11, and the traveling mode of the work vehicle 10 is switched to forward, reverse, etc. The vehicle control device 11 also operates the accelerator to control the rotation speed of the engine 131. The vehicle control device 11 also operates the brake to brake the rotation of the front wheels 132 and the rear wheels 133 using an electromagnetic brake.

[0025] The positioning unit 17 is a communication device including a positioning control unit 171, a memory unit 172, a communication unit 173, and a positioning antenna 174. For example, as shown in FIG. 2, the positioning unit 17 is provided on the upper part of a cabin 138 in which a worker boards. The installation location of the positioning unit 17 is not limited to the cabin 138. The positioning control unit 171, the memory unit 172, the communication unit 173, and the positioning antenna 174 of the positioning unit 17 may be disposed in different positions in the work vehicle 10. As described above, the battery is connected to the positioning unit 17, and the positioning unit 17 can operate even when the engine 131 is stopped. The positioning unit 17 may be substituted with, for example, a mobile phone terminal, a smartphone, a tablet terminal, or a quantum compass.

[0026] The positioning control unit 171 is a computer system including one or more processors and storage memories such as a non-volatile memory and a RAM. The storage unit 172 is a non-volatile memory that stores a program for causing the positioning control unit 171 to execute a positioning process, and data such as positioning information and movement information. For example, the program is non-temporarily recorded on a computer-readable recording medium such as a CD or a DVD, and is read by a predetermined reading device (not shown) and stored in the storage unit 172. The program may be downloaded from a server (not shown) to the positioning unit 17 via the communication network N1 and stored in the storage unit 172.

[0027] The communication unit 173 is a communication interface for connecting the positioning unit 17 to the communication network N1 by wire or wirelessly, and for executing data communication in accordance with a predetermined communication protocol with an external device such as a base station server via the communication network N1.

[0028] The positioning antenna 174 is an antenna that receives radio waves (GNSS signals) transmitted from satellites.

[0029] The positioning control unit 171 calculates the current position of the work vehicle 10 based on the GNSS signal received by the positioning antenna 174 from a satellite. For example, when the work vehicle 10 automatically travels in the field F, when the positioning antenna 174 receives radio waves (such as transmission time and orbit information) transmitted from each of a plurality of satellites, the positioning control unit 171 calculates the distance between the positioning antenna 174 and each satellite, and calculates the current position (latitude and longitude) of the work vehicle 10 based on the calculated distance. The positioning control unit 171 may also perform positioning using a real-time kinematic method (RTK-GNSS positioning method (RTK method)) that calculates the current position of the work vehicle 10 using correction information corresponding to a base station (reference station) close to the work vehicle 10. In this way, the work vehicle 10 automatically travels using positioning information by the RTK method. The current position of the work vehicle 10 may be the same as the positioning position (for example, the position of the positioning antenna 174), or may be a position shifted from the positioning position.

[0030] When the work vehicle 10 detects a detection target while the work vehicle 10 is traveling autonomously, the obstacle detection device 15 determines the type of the detection target (person, vehicle, other (structure, material, etc.)) and outputs the determination result to the vehicle control device 11. Specifically, the obstacle detection device 15 includes a detection control unit 51, a memory unit 52, a camera 53, an obstacle sensor 54, a communication unit 55, etc. The obstacle detection device 15 may be configured as a single unit and mounted on the work vehicle 10, or multiple components may be distributed and arranged on the work vehicle 10.

[0031] The communication unit 55 is a communication interface that connects the obstacle detection device 15 to the communication network N1 via a wired or wireless manner, and performs data communication with an external device (such as the operation terminal 20) via the communication network N1 in accordance with a predetermined communication protocol.

[0032] The storage unit 52 is a non-volatile storage unit such as an HDD or SSD that stores various information. The storage unit 52 stores control programs such as an obstacle detection program for causing the obstacle detection device 15 to execute an obstacle detection process. For example, the obstacle detection program is non-temporarily recorded on a computer-readable recording medium such as a CD or DVD, and is read by a predetermined reading device (not shown) and stored in the storage unit 52. The obstacle detection program may be downloaded from a server (not shown) to the obstacle detection device 15 via the communication network N1 and stored in the storage unit 52.

[0033] Camera 53 is a digital camera that captures an image of a subject included in a predetermined imaging range and outputs the image as digital image data. Camera 53 continuously captures images of the subject at a predetermined frame rate, generates frame images (captured images) of a predetermined resolution, and transmits them sequentially to detection control unit 51. Camera 53 also transmits image data of the captured images to operation terminal 20 via communication unit 55. Operation terminal 20 is capable of displaying the captured images on the operation screen of operation display unit 23 (see FIG. 8).

[0034] The camera 53 also includes a front camera 53f capable of capturing an image of a range in front of the work vehicle 10, and a rear camera 53r capable of capturing an image of a range in the rear of the work vehicle 10. As shown in Fig. 2, the front camera 53f is disposed on the upper front side of the cabin 138, and the rear camera 53r is disposed on the upper rear side of the cabin 138.

[0035] The obstacle sensor 54 is a sensor that detects a detection target in a predetermined detection range using infrared rays, ultrasonic waves, etc. For example, the obstacle sensor 54 may be a lidar sensor (distance sensor) that can measure the distance to the detection target in three dimensions using a laser, or may be a sonar sensor having multiple sonars that can measure the distance to the detection target using ultrasonic waves. The obstacle sensor 54 is installed in the center front, center rear, right side, left side, etc. of the body of the work vehicle 10, and monitors the surroundings of the work vehicle 10 to detect obstacles. In this embodiment, an example will be described in which the obstacle sensor 54 includes a front obstacle sensor 54f that can detect a detection target in a detection range in front of the work vehicle 10, and a rear obstacle sensor 54r that can detect a detection target in a detection range behind the work vehicle 10. As shown in FIG. 2, the front obstacle sensor 54f is arranged on the upper front side of the cabin 138, and the rear obstacle sensor 54r is arranged on the upper rear side of the cabin 138. The front camera 53f and the front obstacle sensor 54f may be configured as one unit, and the rear camera 53r and the rear obstacle sensor 54r may be configured as one unit. The respective positions and numbers of the camera 53 and the obstacle sensor 54 are not limited and may be determined according to the application.

[0036] 4 shows an example of the measurement range (detection range) of each of the front obstacle sensor 54f and the rear obstacle sensor 54r. In the following, the front obstacle sensor 54f and the rear obstacle sensor 54r will be referred to as the "obstacle sensor 54" in a common description.

[0037] The obstacle sensor 54 measures the distance to each ranging point (measurement target) present within the measurement range using, for example, a laser (near-infrared laser light, etc.), and generates a distance image based on the measurement information. The obstacle sensor 54 includes an electronic control unit integrating a microcontroller, etc., and a processing unit constructed by various control programs, etc., and is connected to the detection control unit 51, the vehicle control device 11, etc. via a CAN so as to be able to communicate with each other.

[0038] As shown in Figures 2 and 4, the forward obstacle sensor 54f has a measurement range set to a first measurement range Rm1 forward from the cabin 138, and the rear obstacle sensor 54r has a measurement range set to a second measurement range Rm2 rearward from the cabin 138.

[0039] As shown in FIG. 4, the front obstacle sensor 54f and the rear obstacle sensor 54r are disposed on the left-right center line of the work vehicle 10, similarly to the front camera 53f and the rear camera 53r. The front obstacle sensor 54f is disposed in a left-right center position at the upper part of the front end side of the cabin 138, in a forward-downward posture looking down on the front side of the work vehicle 10 from an obliquely upward side. As a result, for the front obstacle sensor 54f, a predetermined range on the front side of the vehicle body with the left-right center line of the work vehicle 10 as the axis of symmetry is set as the first measurement range Rm1. The rear obstacle sensor 54r is disposed in a left-right center position at the upper part of the rear end side of the cabin 138, in a rear-downward posture looking down on the rear side of the work vehicle 10 from an obliquely upward side. As a result, for the rear obstacle sensor 54r, a predetermined range on the rear side of the vehicle body with the left-right center line of the work vehicle 10 as the axis of symmetry is set as the second measurement range Rm2.

[0040] The front obstacle sensor 54f and the rear obstacle sensor 54r measure the distance from each sensor 54f, 54r to each distance measurement point in the first measurement range Rm1 or the second measurement range Rm2 by a TOF (Time Of Flight) method that measures the distance to the distance measurement point based on the round trip time until the emitted laser reaches the distance measurement point and returns. The front obstacle sensor 54f and the rear obstacle sensor 54r perform three-dimensional measurement in the first measurement range Rm1 or the second measurement range Rm2 by scanning the laser vertically and horizontally at high speed over the entire first measurement range Rm1 or the second measurement range Rm2 and sequentially measuring the distance to the distance measurement point for each scanning angle (coordinate). The front obstacle sensor 54f and the rear obstacle sensor 54r sequentially measure the intensity of reflected light (reflection intensity) from each distance measurement point obtained when the laser is scanned vertically and horizontally at high speed over the entire first measurement range Rm1 or the second measurement range Rm2. The front obstacle sensor 54f and the rear obstacle sensor 54r repeatedly measure the distance to each distance measurement point in the first measurement range Rm1 or the second measurement range Rm2, the reflection intensity, and the like in real time.

[0041] The forward obstacle sensor 54f and the rear obstacle sensor 54r generate a distance image from measurement information such as the distance to each measured ranging point and the scanning angle (coordinates) for each ranging point, and extract a group of ranging points that are estimated to be obstacles, and transmit the measurement information regarding the extracted group of ranging points to the detection control unit 51 as measurement information regarding the obstacles.

[0042] In addition, the front obstacle sensor 54f and the rear obstacle sensor 54r perform cutting and masking processes based on vehicle body information, etc. on the first measurement range Rm1 or the second measurement range Rm2, thereby limiting the obstacle detection ranges of the front obstacle sensor 54f and the rear obstacle sensor 54r to a first detection range Rd1 set on the forward side of the work vehicle 10 and a second detection range Rd2 set on the reverse side of the work vehicle 10 (see Figure 4).

[0043] In the cutting process, the front obstacle sensor 54f and the rear obstacle sensor 54r acquire the maximum left-right width of the vehicle body including the work implement 14 (the left-right width of the tiller in this embodiment) through communication with the vehicle control device 11, and set the detection target width W1 of the obstacle by adding a predetermined safety zone to the maximum left-right width of the vehicle body. Then, in the first measurement range Rm1 and the second measurement range Rm2, the left-right ranges outside the detection target width W1 are set as first non-detection ranges Rn by the cutting process and excluded from each detection range Rd1, Rd2.

[0044] In the masking process, the forward obstacle sensor 54f and the rear obstacle sensor 54r set the range in which the front end of the work vehicle 10 enters into the first measurement range Rm1 and the range in which the rear end of the work implement 14 enters into the second measurement range Rm2 plus a predetermined safety band as a second non-detection range Rs by masking process, and exclude them from each detection range Rd1, Rd2.

[0045] By limiting the obstacle detection range to the first detection range Rd1 and the second detection range Rd2 in this manner, an increase in the detection load caused by the front obstacle sensor 54f and the rear obstacle sensor 54r detecting an obstacle that is outside the detection target width W1 and is not likely to collide with the work vehicle 10, and the risk of erroneously detecting the front end of the work vehicle 10 or the rear end of the work implement 14 that enters the first measurement range Rm1 or the second measurement range Rm2 as an obstacle are avoided.

[0046] Information relating to the first detection range Rd1, the second detection range Rd2, the first non-detection range Rn, and the second non-detection range Rs is included in the distance image described above, and is transmitted to the detection control unit 51 together with the distance image.

[0047] As shown in FIG. 4, the detection ranges Rd1, Rd2 of the front obstacle sensor 54f and the rear obstacle sensor 54r are divided into a stop control range Ra, a deceleration control range Rb, and a notification control range Rc based on a collision determination process in which the collision prediction time is a set time (e.g., 3 seconds). Specifically, the stop control range Ra is set to a range from the front obstacle sensor 54f or the rear obstacle sensor 54r to a determination reference position of the collision determination process. The deceleration control range Rb is set to a range from the determination reference position to the deceleration start position. The notification control range Rc is set to a range from the deceleration start position to the measurement limit position of the front obstacle sensor 54f or the rear obstacle sensor 54r. The determination reference position is set to a position a certain distance L1 (e.g., 2 m) away from the front end or rear end of the vehicle body including the work machine 14 in the vehicle front-rear direction. In this embodiment, the range of distance L1 from the work vehicle 10 is set as the stop control range Ra, the range from distance L1 to distance L2 (e.g., 10 m or less) is set as the deceleration control range Rb, and the range from distance L2 to distance L3 (e.g., 20 m or less) is set as the notification control range Rc.

[0048] The control ranges Ra, Rb, Rc in the first detection range Rd1 of the front obstacle sensor 54f and the second detection range Rd2 of the rear obstacle sensor 54r can be set according to the type, model, work content, vehicle speed, etc. of the work vehicle 10. Furthermore, the first measurement range Rm1 of the front obstacle sensor 54f and the second measurement range Rm2 of the rear obstacle sensor 54r may not be subjected to cutting processing.

[0049] The detection control unit 51 determines the type of the detection target (measurement target) based on the captured image acquired from the camera 53 and the measurement information acquired from the obstacle sensor 54, and outputs the determination result to the vehicle control device 11. Note that the detection control unit 51 may determine the type of the detection target (measurement target) based on the captured image acquired from the camera 53. In the following, an example will be described in which a detection target in front is detected when the work vehicle 10 travels forward, and an example in which a detection target in the rear is detected when the work vehicle 10 travels backward will be omitted.

[0050] The detection control unit 51 has control devices such as a CPU, a ROM, and a RAM. The CPU is a processor that executes various arithmetic processing. The ROM is a non-volatile storage unit in which control programs such as a BIOS and an OS for causing the CPU to execute various arithmetic processing are stored in advance. The RAM is a volatile or non-volatile storage unit that stores various information, and is used as a temporary storage memory (work area) for various processes executed by the CPU. The detection control unit 51 controls the obstacle detection device 15 by executing various control programs stored in advance in the ROM or the storage unit 52 by the CPU.

[0051] Specifically, as shown in Fig. 1, the detection control unit 51 includes various processing units such as an acquisition processing unit 511, a judgment processing unit 512, and an output processing unit 513. The obstacle detection device 15 functions as the various processing units by executing various processes according to the obstacle detection program in the CPU. Some or all of the processing units may be configured with electronic circuits. The obstacle detection program may be a program for causing a plurality of processors to function as the processing units.

[0052] The acquisition processing unit 511 acquires captured images from the camera 53 and acquires measurement information from the obstacle sensor 54. Specifically, when the work vehicle 10 starts automatic traveling, the acquisition processing unit 511 sequentially acquires captured images of the imaging range ahead in the traveling direction from the front camera 53f during the automatic traveling. The acquisition processing unit 511 stores the acquired captured images in the storage unit 52. In addition, when the work vehicle 10 starts automatic traveling, the acquisition processing unit 511 sequentially acquires measurement information (such as a range-finding point cloud, a distance image, and information relating to the first detection range Rd1) of the first detection range Rd1 (see FIG. 4) ahead in the traveling direction from the front obstacle sensor 54f during the automatic traveling. The acquisition processing unit 511 stores the acquired measurement information in the storage unit 52.

[0053] The determination processing unit 512 determines the type of the detection target based on the captured image and the measurement information acquired by the acquisition processing unit 511. Specifically, the determination processing unit 512 performs image analysis on the captured image to determine the type of the detection target (person, vehicle, structure, material, work target (harvested object, harvested object, etc.)). The determination processing unit 512 may estimate the type of the detection target by using a well-known estimation model (trained model).

[0054] Furthermore, the determination processing unit 512 calculates the likelihood (certainty) of the determination result when the type of the detection target is determined. Specifically, the determination processing unit 512 calculates the likelihood (probability) based on the distance information included in the measurement information, i.e., the distance from the work vehicle 10 (here, the front end of the work vehicle 10) to the detection target. The determination processing unit 512 calculates the likelihood so that the shorter the distance, the higher the likelihood.

[0055] For example, when the determination processing unit 512 determines that the detection target is a person based on the captured image, if the distance to the detection target is farther than a predetermined distance, the determination processing unit 512 calculates the likelihood of the person as "50%" and if the distance to the detection target is closer than a predetermined distance, the determination processing unit 512 calculates the likelihood of the person as "90%." Similarly, when the determination processing unit 512 determines that the detection target is a vehicle based on the captured image, the determination processing unit 512 calculates the likelihood of the vehicle according to the distance to the detection target.

[0056] The determination processing unit 512 may also calculate the likelihood based on the distance information and the captured image. For example, when the captured image includes only a part of the detection target and it is difficult to grasp the entire detection target, the determination processing unit 512 calculates a lower likelihood of the detection target compared to when the captured image includes the entire detection target. For this reason, for example, the determination processing unit 512 determines the detection target to be a person based on the captured image, and when the distance to the detection target is closer than a predetermined distance, calculates a likelihood (for example, "70%)" lower than the likelihood ("90%) when the entire person is included in the captured image. In this way, the determination processing unit 512 may adjust the likelihood according to the imaging state of the detection target included in the captured image.

[0057] The determination processing unit 512 may calculate the likelihood based only on the captured image. For example, the larger the size of the detection target reflected in the captured image, the easier it is to identify the type of the detection target, and the smaller the size of the detection target reflected in the captured image, the more difficult it is to identify the type of the detection target. For this reason, the determination processing unit 512 calculates the likelihood of the detection target to be high when the size of the detection target reflected in the captured image is large, and calculates the likelihood of the detection target to be low when the size of the detection target reflected in the captured image is small. For example, the determination processing unit 512 determines the detection target to be a person from the captured image, calculates the likelihood to be "90%" when the reflection of the detection target is large, and calculates the likelihood to be "70%" when the reflection of the detection target is small. In this way, the determination processing unit 512 may calculate the likelihood according to the size (occupancy area, etc.) of the detection target included in the captured image.

[0058] The output processing unit 513 outputs the determination result of the determination processing unit 512 to the vehicle control device 11. Specifically, the output processing unit 513 outputs to the vehicle control device 11 the determination result (type of detection target), the likelihood of the determination result according to the distance to the detection target, and a likelihood threshold value (see FIG. 5) corresponding to the distance to the detection target.

[0059] The likelihood threshold is a set parameter for determining whether to cause the work vehicle 10 to execute a predetermined countermeasure process (for example, a stop process, a deceleration process, a notification process, etc.), and is a threshold for the calculated likelihood. For example, when the calculated likelihood exceeds the likelihood threshold, the work vehicle 10 is caused to execute a countermeasure process.

[0060] Here, the likelihood threshold is set to a smaller value as the distance to the detection target is shorter. For example, as shown in FIG. 5, when the distance from the work vehicle 10 to the detection target is less than L1 (for example, 2 m), the detection control unit 51 sets the likelihood threshold to Th1 (for example, 50%), and when the distance from the work vehicle 10 to the detection target is L2 or more (however, L1 < L2), the detection control unit 51 sets the likelihood threshold to Th2 (for example, 90%) (however, Th1 < Th2). Further, when the distance from the work vehicle 10 to the detection target is equal to or more than L1 and less than L2, the detection control unit 51 sets the likelihood threshold corresponding to the distance between Th1 and Th2.

[0061] Also, the likelihood threshold is set to different values according to the content of the countermeasure process. For example, as shown in FIG. 5, the detection control unit 51 sets different likelihood thresholds for each of a stop control range Ra in which a stop process is executed, a deceleration control range Rb in which a deceleration process is executed, and a notification control range Rc in which a notification process is executed. Specifically, in the stop control range Ra where the distance from the work vehicle 10 is short, since there is no margin until approaching or contacting the detection target, the detection control unit 51 sets the likelihood threshold low so that a countermeasure process is easily executed even for a detection target with a low likelihood. On the other hand, in the notification control range Rc where the distance from the work vehicle 10 is far, since there is a margin until approaching or contacting the detection target, the detection control unit 51 sets the likelihood threshold high so that only detection targets with a high likelihood are selected and a countermeasure process is executed. Further, in the deceleration control range Rb, the detection control unit 51 sets the likelihood threshold lower as the distance from the work vehicle 10 is shorter so that a countermeasure process is more easily executed as the distance to the detection target is shorter.

[0062] The detection control unit 51 prestores information on the likelihood threshold (see FIG. 5) corresponding to the distance from the work vehicle 10 to the detection target in the storage unit 52.

[0063] The output processing unit 513 acquires a likelihood threshold corresponding to the distance from the work vehicle 10 to the detection target by referring to the setting information of the likelihood threshold in Fig. 5. For example, when the determination processing unit 512 determines that the detection target located at a distance Lx from the work vehicle 10 is a person and calculates the likelihood of the person to be X1%, the output processing unit 513 outputs to the vehicle control device 11 a determination result including the type of the detection target "person", the likelihood "X1%", and the likelihood threshold "Thx" corresponding to the distance Lx.

[0064] In this way, while the work vehicle 10 is traveling automatically, the detection control unit 51 determines the type of detection target (measurement target) each time it acquires an image from the camera 53 and acquires measurement information from the obstacle sensor 54, and sequentially outputs the determination results to the vehicle control device 11.

[0065] When the vehicle control device 11 acquires the determination result output from the obstacle detection device 15 (detection control unit 51), it causes the work vehicle 10 to execute a countermeasure process according to the determination result.

[0066] The vehicle control device 11 has control devices such as a CPU, a ROM, and a RAM. The CPU is a processor that executes various types of arithmetic processing. The ROM is a non-volatile storage unit in which control programs such as a BIOS and an OS for causing the CPU to execute various types of arithmetic processing are stored in advance. The RAM is a volatile or non-volatile storage unit that stores various types of information, and is used as a temporary storage memory for various types of processing executed by the CPU. The vehicle control device 11 controls the work vehicle 10 by executing various control programs stored in advance in the ROM or the storage unit 12 with the CPU.

[0067] Specifically, as shown in Fig. 1, the vehicle control device 11 includes various processing units such as a driving processing unit 111 and a countermeasure processing unit 112. The vehicle control device 11 functions as the various processing units by executing various processes according to the automatic driving program with the CPU. Also, a part or all of the processing units may be configured with electronic circuits. The automatic driving program may be a program for causing a plurality of processors to function as the processing units. Also, the automatic driving program includes the vehicle control program of the present invention.

[0068] The driving processing unit 111 controls the driving of the work vehicle 10. For example, when the driving mode of the work vehicle 10 is automatic driving (automatic driving mode), the driving processing unit 111 automatically drives the work vehicle 10 based on position information (positioning information) indicating the current position of the work vehicle 10 measured by the positioning unit 17. For example, when the work vehicle 10 satisfies the start condition for automatic driving and acquires a driving start instruction from the worker, the driving processing unit 111 starts automatic driving of the work vehicle 10 based on the positioning information. In addition, the driving processing unit 111 automatically drives the work vehicle 10 from the driving start position S to the driving end position G according to a target route R (see FIG. 3) that is generated and set in advance in the operation terminal 20.

[0069] When the driving mode of the work vehicle 10 is manual driving (manual driving mode), the work vehicle 10 can be manually driven based on the operation (manual steering) of the worker. For example, the driving processing unit 111 acquires operation information corresponding to driving operations such as steering, gear shifting, driving direction switching, and braking by the worker, and causes the traveling device 13 to perform a driving operation based on the operation information.

[0070] The response processing unit 112 causes the work vehicle 10 to execute a predetermined response process when an obstacle is detected while the work vehicle 10 is autonomously traveling. Specifically, the response processing unit 112 causes the work vehicle 10 to execute a response process according to the type of detection target detected by the obstacle detection device 15. In addition, the response processing unit 112 causes the work vehicle 10 to execute a response process according to the type of detection target detected by the obstacle detection device 15 and the distance to the detection target detected by the obstacle detection device 15.

[0071] For example, when the obstacle detection device 15 detects a detection target while the work vehicle 10 is traveling autonomously, it outputs a determination result including the type of the detection target, the likelihood of the type, and a likelihood threshold value (see FIG. 5) corresponding to the distance to the detection target, and the countermeasure processing unit 112 acquires the determination result. The countermeasure processing unit 112 determines whether or not it is necessary to have the work vehicle 10 execute a countermeasure process based on the determination result, and when it determines that it is necessary to have the work vehicle 10 execute a countermeasure process, it causes the work vehicle 10 to execute the countermeasure process. In addition, the countermeasure processing unit 112 causes the work vehicle 10 to execute a countermeasure process based on the likelihood calculated by the obstacle detection device 15. A specific example of the countermeasure process will be described below.

[0072] For example, as shown in FIG. 6, when the obstacle detection device 15 detects a detection target at a position P1 that is a distance Lc (where L2 < Lc < L3) away from the work vehicle 10, determines that the detection target is a "person", and calculates the likelihood of the person as "A1"%, if the likelihood "A1"% exceeds the likelihood threshold Th2 corresponding to the distance Lc, the countermeasure processing unit 112 determines that an obstacle has been detected within the notification control range Rc (see FIG. 4) and causes the work vehicle 10 to execute a notification process. For example, the countermeasure processing unit 112 causes the work vehicle 10 to emit a warning sound. As another embodiment, the countermeasure processing unit 112 may cause the work vehicle 10 to emit a warning sound and display a warning message on the operation terminal 20. Further, the countermeasure processing unit 112 may change the warning sound (such as increasing the volume or shortening the sounding period) as the distance Lc decreases (the work vehicle 10 approaches the obstacle). Thereby, it is possible to prompt a person in front of the work vehicle 10 to pay attention early. Therefore, for example, by having the person move, it is possible to avoid interruption of the work by the work vehicle 10 and continue the work.

[0073] Note that when the likelihood "A1"% is less than or equal to the likelihood threshold Th2, the countermeasure processing unit 112 does not execute the notification process. Thereby, excessive notification processing can be suppressed. In this way, in the notification control range Rc, when the distance from the work vehicle 10 to the detection target is far, there is a margin until approaching or contacting the detection target. In this case, since only the obstacle that is surely the target of the countermeasure process needs to be specified, excessive notification processing can be suppressed by setting the likelihood threshold high.

[0074] Also, for example, as shown in FIG. 6, when the obstacle detection device 15 detects a detection target at a position P2 that is a distance Lb (where L1 < Lb < L2) away from the work vehicle 10, determines the detection target as a "person", and calculates the likelihood of the person as "A2"%, if the likelihood "A2"% exceeds the likelihood threshold Th3 corresponding to the distance Lb, the countermeasure processing unit 112 determines that an obstacle has been detected within the deceleration control range Rb (see FIG. 4) and causes the work vehicle 10 to execute a deceleration process. For example, the countermeasure processing unit 112 decelerates the work vehicle 10 that is traveling automatically from a preset vehicle speed. As another embodiment, the countermeasure processing unit 112 may execute both a deceleration process and a notification process. Further, the countermeasure processing unit 112 may output a warning sound corresponding to the vehicle speed (such as increasing the volume as the vehicle speed increases). Thereby, safety can be ensured for the person in front of the work vehicle 10.

[0075] Note that when the likelihood "A2"% is less than or equal to the likelihood threshold, the countermeasure processing unit 112 does not execute the deceleration process. In this way, in the deceleration control range Rb, the likelihood threshold is changed according to the distance. Thereby, when the distance from the work vehicle 10 is far, since there is a margin until approaching or contacting the detection target, the likelihood threshold is set high to suppress excessive deceleration processing. When the distance from the work vehicle 10 is close, since there is no margin until approaching or contacting the detection target, the likelihood threshold is set low to make it easier to execute the deceleration process and ensure safety.

[0076] Also, for example, as shown in FIG. 6, when the obstacle detection device 15 detects a detection target at a position P3 that is a distance La (where 0 < La < L1) away from the work vehicle 10, determines the detection target as a "person", and calculates the likelihood of the person as "A3"%, if the likelihood "A3"% exceeds the likelihood threshold Th1 corresponding to the distance La, the countermeasure processing unit 112 determines that an obstacle has been detected within the stop control range Ra (see FIG. 4) and causes the work vehicle 10 to execute a stop process. For example, the countermeasure processing unit 112 temporarily stops the work vehicle 10 that is automatically traveling. As another embodiment, the countermeasure processing unit 112 may execute both a stop process and a notification process. Thereby, safety can be ensured for a person in front of the work vehicle 10.

[0077] Note that when the likelihood "A3"% is less than or equal to the likelihood threshold, the countermeasure processing unit 112 does not execute the stop process. In this way, in the stop control range Ra, when the distance from the work vehicle 10 is short, there is no margin until approaching or contacting the detection target, so by setting the likelihood threshold low, safety can be ensured by executing the stop process even for a detection target with a low possibility of being an obstacle.

[0078] In this way, when the likelihood of the detection target detected while the work vehicle 10 is automatically traveling exceeds the likelihood threshold corresponding to the distance to the detection target, the countermeasure processing unit 112 causes the work vehicle 10 to execute a countermeasure process.

[0079] Here, the countermeasure processing unit 112 omits the countermeasure processing when the detection target is a detection target that does not require the work vehicle 10 to execute a countermeasure processing. For example, when the detection target is a work target of the work vehicle 10 (such as an object to be harvested or harvested), the countermeasure processing unit 112 omits the countermeasure processing since it is not necessary to cause the work vehicle 10 to execute a countermeasure processing. Note that the obstacle detection device 15 may be configured not to output the determination result to the vehicle control device 11 when it is determined that the detection target is a detection target that does not require the work vehicle 10 to execute a countermeasure processing. The obstacle detection device 15 can determine whether the detection target is a detection target that requires the work vehicle 10 to execute a countermeasure processing based on the captured image. The obstacle detection device 15 may also be configured to determine whether the calculated likelihood exceeds a likelihood threshold, output the determination result to the vehicle control device 11 when the likelihood exceeds the likelihood threshold, and not output the determination result to the vehicle control device 11 when the likelihood is equal to or less than the likelihood threshold.

[0080] The countermeasure processing may include avoidance driving (avoidance processing) in which the work vehicle 10 avoids an obstacle. For example, if the detection target is still detected even after the notification processing and the deceleration processing are executed, or if the likelihood exceeds a likelihood threshold, and the distance from the work vehicle 10 to the detection target becomes equal to or shorter than a predetermined distance, the countermeasure processing unit 112 generates an avoidance route for avoiding the detection target and causes the work vehicle 10 to travel along the avoidance route. The countermeasure processing unit 112 may also determine whether to cause the work vehicle 10 to execute a stop process or an avoidance process depending on the type of the detection target. For example, the countermeasure processing unit 112 causes the stop process to be executed when the detection target is a moving object (such as a person or a vehicle), and causes the work vehicle 10 to execute an avoidance process when the detection target is a fixed object.

[0081] In the above embodiment, the obstacle detection device 15 is configured as a device separate from the vehicle control device 11, but in another embodiment, the obstacle detection device 15 and the vehicle control device 11 may be configured as an integrated device. Also, the detection control unit 51 may be included in the vehicle control device 11.

[0082] [Operation terminal 20] 1, the operation terminal 20 is an information processing device including an operation control unit 21, a storage unit 22, an operation display unit 23, and a communication unit 24. The operation terminal 20 may be configured as a mobile terminal such as a tablet terminal or a smartphone.

[0083] The communication unit 24 is a communication interface for connecting the operation terminal 20 to the communication network N1 via a wired or wireless connection, and for performing data communication in accordance with a predetermined communication protocol with external devices such as one or more work vehicles 10 via the communication network N1.

[0084] The operation display unit 23 is a user interface that includes a display unit such as a liquid crystal display or an organic EL display that displays various information, and an operation unit such as a touch panel, a mouse, or a keyboard that accepts operations. The worker can operate the operation unit to register various information (such as work vehicle information, field information, and work information described below) on the operation screen displayed on the display unit. The worker can also operate the operation unit to give instructions to the work vehicle 10 to start work and stop traveling. The worker can also grasp the traveling state of the work vehicle 10 that automatically travels in the field F according to the target route R from the traveling trajectory displayed on the operation terminal 20 and the image captured by the camera 53 at a location away from the work vehicle 10 (see FIG. 8). The worker can also grasp the obstacle detection results, the execution status of the countermeasure processing, and the like displayed on the operation terminal 20.

[0085] The storage unit 22 is a non-volatile storage unit such as an HDD or SSD that stores various types of information. A control program for causing the operation control unit 21 to execute various control processes is stored in the storage unit 22. For example, the control program is non-temporarily recorded on a computer-readable recording medium such as a CD or DVD, and is read by a predetermined reading device (not shown) and stored in the storage unit 22. The control program may be downloaded from a server (not shown) to the operation terminal 20 via the communication network N1 and stored in the storage unit 22.

[0086] The operation control unit 21 has control devices such as a CPU, a ROM, and a RAM. The CPU is a processor that executes various arithmetic processes. The ROM is a non-volatile storage unit in which control programs such as a BIOS and an OS for causing the CPU to execute various arithmetic processes are stored in advance. The RAM is a volatile or non-volatile storage unit that stores various information, and is used as a temporary storage memory (work area) for various processes executed by the CPU. The operation control unit 21 controls the operation terminal 20 by executing various control programs stored in advance in the ROM or the storage unit 22 by the CPU.

[0087] As shown in Fig. 1, the operation control unit 21 includes various processing units such as a setting processing unit 211 and an output processing unit 212. The operation control unit 21 functions as the various processing units by executing various processes according to the control program with the CPU. Some or all of the processing units may be configured with electronic circuits. The control program may be a program for causing a plurality of processors to function as the processing units.

[0088] The setting processing unit 211 sets various setting information for causing the work vehicle 10 to perform automatic traveling. Specifically, the setting processing unit 211 sets information relating to the work vehicle 10 (hereinafter referred to as work vehicle information). The setting processing unit 211 sets information such as the type (model) of the work vehicle 10, the position where the positioning antenna 174 is attached on the work vehicle 10, the type of the work implement 14, the size and shape of the work implement 14, the position of the work implement 14 relative to the work vehicle 10, the vehicle speed and engine speed of the work vehicle 10 during work, and the vehicle speed and engine speed of the work vehicle 10 while turning, by having the worker perform an operation to register the information on the operation terminal 20.

[0089] For example, the setting processing unit 211 causes the operation display unit 23 to display the menu screen D1 shown in Fig. 7. The worker registers work machine information relating to the work machine 14 by, for example, selecting "Work Machine Registration" on the menu screen D1.

[0090] The setting processing unit 211 also sets information about the field F (hereinafter referred to as field information). The setting processing unit 211 sets information such as the position and shape of the field F, the travel start position S where work begins and the travel end position G where work ends (see FIG. 3), the work direction, etc., by performing an operation to register this information on the operation terminal 20. Note that the work direction means the direction in which the work vehicle 10 travels while working with the work implement 14 in the work area excluding the headland and non-work area from the field F. For example, the worker registers the field information by selecting "Field registration" on the menu screen D1.

[0091] Information on the position and shape of the field F can be automatically acquired, for example, by an operator getting on board the work vehicle 10 and driving it around the perimeter of the field F, and recording the transition of the position information of the positioning antenna 174 at that time. The position and shape of the field F can also be acquired based on a polygon obtained by an operator operating the operation terminal 20 while a map is displayed on the map and specifying multiple points on the map. The area specified by the acquired position and shape of the field F is the area in which the work vehicle 10 can be driven (travel area).

[0092] Furthermore, the setting processing unit 211 sets information relating to how the work is to be performed specifically (hereinafter referred to as work information). The setting processing unit 211 is configured to be able to set, as work information, the presence or absence of cooperative work between an unmanned work vehicle 10 and a manned work vehicle 10, the skip number which is the number of work routes to skip when the work vehicle 10 turns on the headland, the width of the headland, and the width of the non-cultivated land, etc. For example, the worker registers work information by selecting "Work registration" on the menu screen D1.

[0093] Further, the setting processing unit 211 generates a target route R, which is a route along which the work vehicle 10 will automatically travel, based on each of the setting information. The target route R is, for example, a work route from a travel start position S to a travel end position G (see FIG. 3). The target route R shown in FIG. 3 includes a work route that is a straight route along which the work vehicle 10 travels back and forth in parallel in the field F, and a turning route that connects the work routes. The setting processing unit 211 generates and stores the target route R for the work vehicle 10 based on each of the setting information. For example, the worker selects "Route Creation" on the menu screen D1 to instruct the generation of the target route R. The setting processing unit 211 is capable of generating and storing multiple target routes R according to the work content for one field F.

[0094] The output processing unit 212 outputs route data of the target route R to the work vehicle 10. For example, when a worker selects a desired target route R on the operation screen and issues a work start instruction, the output processing unit 212 outputs the route data of the selected target route R to the work vehicle 10.

[0095] The work vehicle 10 is configured such that route data of the target route R generated in the operation terminal 20 is transferred to the work vehicle 10 and stored in the memory unit 12, and the work vehicle 10 is capable of autonomously traveling along the target route R while detecting the current position of the work vehicle 10 by the positioning antenna 174. The current position of the work vehicle 10 normally coincides with the position of the positioning antenna 174.

[0096] When the current position of the work vehicle 10 coincides with the travel start position S, and the worker presses the work start button on the operation screen to give a work start instruction, the travel processing unit 111 starts automatic travel and the work equipment 14 (see FIG. 2) starts work. That is, the operation control unit 21 permits automatic travel of the work vehicle 10 on the condition that the current position coincides with the travel start position S. Note that the conditions for permitting automatic travel of the work vehicle 10 are not limited to the above conditions.

[0097] The driving processing unit 111 of the work vehicle 10 automatically drives the work vehicle 10 from the driving start position S to the driving end position G according to the target route R acquired from the operation terminal 20.

[0098] Furthermore, when the operation control unit 21 acquires a detection result indicating that an obstacle has been detected from the work vehicle 10, it causes the detected position of the obstacle to be displayed in the captured image captured by the camera 53 on the driving screen D2 (see FIG. 8) of the operation terminal 20. This allows the worker to confirm on the operation terminal 20 that the work vehicle 10 has detected an obstacle, that the work vehicle 10 has decelerated or stopped, the state of the obstacle, and the like.

[0099] Here, the operator may be able to set the sensitivity, which is setting information for making the countermeasure processing easier or harder to execute, in the operation terminal 20. In other words, the sensitivity is a setting parameter for making it easier or harder to recognize a detection target as an obstacle. FIG. 9 shows an example of a sensitivity setting screen D3 for accepting a setting operation of the sensitivity. For example, when the sensitivity is set to "high", the likelihood threshold is lowered. As a result, even if the detection target has a low likelihood, the likelihood is more likely to exceed the likelihood threshold, so that the countermeasure processing (stop processing, deceleration processing, notification processing) is more likely to be performed, improving safety. On the other hand, the countermeasure processing is more likely to be performed excessively, reducing work efficiency. On the other hand, when the sensitivity is set to "low", the likelihood threshold is higher. As a result, even if the detection target has a high likelihood, the likelihood is less likely to exceed the likelihood threshold, so that the countermeasure processing (stop processing, deceleration processing, notification processing) is less likely to be performed, and excessive countermeasure processing is suppressed, improving work efficiency. On the other hand, the countermeasure processing is less likely to be performed, reducing safety.

[0100] The worker can set the sensitivity according to the application of the work vehicle 10. The obstacle detection device 15 sets the likelihood threshold according to the sensitivity setting operation performed by the worker. Specifically, as shown in FIG. 10A, information on multiple (here, three) likelihood thresholds corresponding to sensitivities ("high", "medium", and "low") may be registered in advance in the storage unit 52. The obstacle detection device 15 acquires from the storage unit 52 the likelihood threshold corresponding to the sensitivity selected by the worker on the sensitivity setting screen D3.

[0101] As shown in Fig. 10A, for a plurality of likelihood thresholds according to sensitivity, the likelihood thresholds may be equal in the stop control range Ra and the notification control range Rc, and may differ for each sensitivity in the deceleration control range Rb. As another embodiment, as shown in Fig. 10B, for a plurality of likelihood thresholds according to sensitivity, the likelihood thresholds may differ for each sensitivity in the stop control range Ra, the deceleration control range Rb, and the notification control range Rc. The obstacle detection device 15 may set the likelihood threshold according to a setting operation of the likelihood threshold by an operator (an input operation of "X" %).

[0102] In another embodiment, the obstacle detection device 15 may automatically set the sensitivity based on the state of the field F and the work content. That is, the detection control unit 51 may automatically set the likelihood threshold based on at least one of the state of the field F and the work content by the work vehicle 10. For example, when the work vehicle 10 is performing culm harvesting work and a detection target such as a person is hidden by the culm, the detection control unit 51 sets the sensitivity high (the likelihood threshold low) to increase safety. Also, for example, when the work vehicle 10 is performing tilling work and visibility is good over the entire field F, the detection control unit 51 sets the sensitivity low (the likelihood threshold high) to increase work efficiency.

[0103] Furthermore, the detection control unit 51 may set the likelihood threshold and the sensitivity based on the type of work machine 14, the work content, etc., set by the worker on the operation terminal 20, or may set the likelihood threshold and the sensitivity based on the state of the field F captured by the camera 53. Furthermore, the detection control unit 51 may change the sensitivity based on a sensitivity changing operation by the worker after the work vehicle 10 starts automatic traveling. Furthermore, the detection control unit 51 may detect a change in the state of the field F from the captured image and change the likelihood threshold and the sensitivity based on the change in the state of the field F after the work vehicle 10 starts automatic traveling.

[0104] Furthermore, the worker may be able to set the type of detection target (control target) for which a countermeasure should be taken on the operation terminal 20. That is, the detection control unit 51 may be able to set whether or not to have the work vehicle 10 take action on each type of detection target. FIG. 11 shows an example of a control target setting screen D4 that accepts the setting operation of the control target. For example, the control target setting screen D4 displays "person", "vehicle", and "other" each selectable. "Other" includes structures, materials, and the like.

[0105] The worker selects one or more types (control targets) for which a countermeasure should be executed from among "person," "vehicle," and "other." For example, if the worker selects all of "person," "vehicle," and "other," the countermeasure will be easier to apply to various detection targets, improving safety but decreasing work efficiency. For example, if the worker selects only "person," the countermeasure will be harder to apply to detection targets other than people, decreasing safety but increasing work efficiency.

[0106] The detection control unit 51 executes obstacle detection processing (such as likelihood calculation processing) for the control target set by the operator on the operation terminal 20. Furthermore, the countermeasure processing unit 112 executes countermeasure processing for the control target set by the operator on the operation terminal 20. Note that in the initial setting, all of "Person", "Vehicle", and "Other" may be selected with priority given to safety.

[0107] As another embodiment, the detection control unit 51 may be able to set the likelihood threshold and the sensitivity for each control target. For example, the detection control unit 51 sets the likelihood threshold low (high sensitivity) for "people" by prioritizing safety, and sets the likelihood threshold high (low sensitivity) for "vehicles" and "others" by prioritizing work efficiency. For example, the detection control unit 51 may set the likelihood threshold low (for example, 60%) for "people" and "vehicles" and set the likelihood threshold high (for example, 80%) for "others" such as animals. As another embodiment, when the worker selects the "safety priority mode", the detection control unit 51 may set the sensitivity to "high", and when the worker selects the "operability priority mode", the detection control unit 51 may set the sensitivity to "low".

[0108] The operation terminal 20 may be able to access a website (agricultural support site) of an agricultural support service provided by a server (not shown) via the communication network N1. In this case, the operation terminal 20 can function as an operation terminal for the server by executing a browser program by the operation control unit 21. The server includes the above-mentioned processing units and executes each process.

[0109] [Automatic driving processing] An example of the automatic driving process executed by the vehicle control device 11 and the obstacle detection device 15 will be described below with reference to Fig. 12. For example, the automatic driving process is started by the vehicle control device 11 and the obstacle detection device 15 when the work vehicle 10 starts automatic driving. The automatic driving process includes an obstacle detection process executed by the obstacle detection device 15.

[0110] The present invention may be understood as an invention of an automatic driving method in which the vehicle control device 11 and the obstacle detection device 15 execute a part or all of the automatic driving process, or an invention of an automatic driving program for causing the vehicle control device 11 and the obstacle detection device 15 to execute a part or all of the automatic driving method. Furthermore, one or more processors may execute the automatic driving process. The automatic driving program includes an obstacle detection program.

[0111] In step S1, the vehicle control device 11 determines whether the work vehicle 10 is in a state capable of autonomous driving. When the work vehicle 10 satisfies the conditions for starting autonomous driving (S1: Yes), the vehicle control device 11 shifts the process to step S2. The vehicle control device 11 waits until the work vehicle 10 satisfies the conditions for starting autonomous driving (S1: No).

[0112] In step S2, the vehicle control device 11 causes the work vehicle 10 to start automatic traveling. For example, when the worker issues a traveling start instruction on the operation screen of the operation terminal 20, the operation control unit 21 outputs a traveling start instruction to the work vehicle 10. When the vehicle control device 11 acquires the traveling start instruction from the operation terminal 20, it causes the work vehicle 10 to start automatic traveling. As a result, the work vehicle 10 starts automatic traveling in the field F according to the target route R (see FIG. 3), and starts work by the work implement 14.

[0113] In step S3, the obstacle detection device 15 determines whether or not the detection target has been detected. Specifically, the obstacle detection device 15 determines the type of the detection target (measurement target) based on the captured image acquired from the camera 53 and the measurement information acquired from the obstacle sensor 54. If the determined type of the detection target is included in the preset control targets (see FIG. 11), the obstacle detection device 15 determines that the detection target has been detected (S3: Yes) and shifts the process to step S4. If the obstacle detection device 15 does not detect the detection target or if the determined type of the detection target is not included in the preset control targets (S3: No), the obstacle detection device 15 shifts the process to step S9.

[0114] In step S4, the obstacle detection device 15 calculates the likelihood of the determined detection target. For example, when the obstacle detection device 15 determines that the detection target is a person based on the captured image, it calculates the likelihood (probability) indicating the possibility (certainty) that the detection target is a person based on the distance to the detection target. For example, when the obstacle detection device 15 determines that the detection target is a vehicle based on the captured image, it calculates the likelihood indicating the possibility that the detection target is a vehicle based on the distance to the detection target. After calculating the likelihood, the obstacle detection device 15 outputs the determination result (type of detection target), the likelihood for the determination result, and a likelihood threshold value (see FIG. 5) corresponding to the distance to the detection target to the vehicle control device 11. Note that the obstacle detection device 15 may determine the type of detection target (person, vehicle, etc.) based on the captured image, and calculate the likelihood of the type of detection target based on the size (occupancy area, etc.) of the detection target reflected in the captured image.

[0115] Next, in step S5, the vehicle control device 11 determines whether the likelihood of the detection target calculated by the obstacle detection device 15 exceeds the likelihood threshold. If the vehicle control device 11 determines that the likelihood of the detection target exceeds the likelihood threshold (S5: Yes), it shifts the process to step S6. On the other hand, if the vehicle control device 11 determines that the likelihood of the detection target is equal to or less than the likelihood threshold (S5: No), it shifts the process to step S9.

[0116] In step S6, the vehicle control device 11 causes the work vehicle 10 to execute a countermeasure process. Specifically, the vehicle control device 11 causes the work vehicle 10 to execute a countermeasure process (stopping process, deceleration process, notification process) according to the position of the detection target. For example, if the position of the detection target is within the notification control range Rc (see FIG. 4), the vehicle control device 11 causes the work vehicle 10 to issue a warning sound. For another example, if the position of the detection target is within the deceleration control range Rb, the vehicle control device 11 decelerates the vehicle speed of the work vehicle 10. For another example, if the position of the detection target is within the stop control range Ra, the vehicle control device 11 stops the work vehicle 10.

[0117] Next, in step S7, the vehicle control device 11 determines whether the detection target has moved. For example, if the detection target has moved outside the first detection range Rd1 (see FIG. 4), the vehicle control device 11 determines that the detection target has moved (S7: Yes) and transitions the process to step S8. While the detection target is located within the first detection range Rd1, the vehicle control device 11 repeatedly executes the processes of steps S4 to S6 (S7: No).

[0118] In step S8, the vehicle control device 11 ends the handling process. For example, when a detection target located within the notification control range Rc moves out of the first detection range Rd1, the vehicle control device 11 stops the notification process. Also, for example, when a detection target located within the deceleration control range Rb moves out of the first detection range Rd1, the vehicle control device 11 returns the vehicle speed of the work vehicle 10 to the original vehicle speed. Also, for example, when a detection target located within the stop control range Ra moves out of the first detection range Rd1, the vehicle control device 11 releases the temporary stop of the work vehicle 10 and resumes automatic driving.

[0119] Next, in step S9, the vehicle control device 11 determines whether or not the work vehicle 10 has finished work. For example, when the work vehicle 10 reaches the travel end position G (see FIG. 3), the vehicle control device 11 determines that the work vehicle 10 has finished work (S9: Yes) and ends the automatic driving process. The vehicle control device 11 repeatedly executes the above-mentioned process until the work vehicle 10 reaches the travel end position G (S9: No). In this manner, the vehicle control device 11 and the obstacle detection device 15 execute the automatic driving process.

[0120] As described above, the automatic driving system 1 according to this embodiment acquires distance information of the distance to the detection target detected by the obstacle sensor 54, and acquires image information of the image of the detection target captured by the camera 53, when the work vehicle 10 automatically travels along the target route R. The automatic driving system 1 also determines the type of the detection target based on the image information, and causes the work vehicle 10 to execute a response process according to the type of the detection target. The automatic driving system 1 also causes the work vehicle 10 to execute a response process according to the type and distance information of the detection target.

[0121] Furthermore, the autonomous driving system 1 calculates the likelihood of the type of the detection target based on the image information, and causes the work vehicle 10 to execute a countermeasure process when the calculated likelihood exceeds a likelihood threshold (see FIG. 5). Note that the autonomous driving system 1 may determine the type of the detection target based on the image information and the distance information. Also, the autonomous driving system 1 may calculate the likelihood of the type of the detection target based on the image information and the distance information.

[0122] Furthermore, in the automated driving system 1, the likelihood threshold may be set to a smaller value as the distance from the work vehicle 10 to the detection target becomes shorter.

[0123] According to the above configuration, for example, when the detection target is a detection target (such as a work object) that does not require the work vehicle 10 to execute a countermeasure process, the countermeasure process can be omitted, and when the detection target is a detection target (such as a person or vehicle) that requires the work vehicle 10 to execute a countermeasure process, the countermeasure process can be executed. This makes it possible to prevent a decrease in the work efficiency of the work vehicle 10 while causing the work vehicle 10 to execute an appropriate countermeasure process for the obstacle.

[0124] Also, according to the above configuration, for example, in the notification control range Rc where the distance from the work vehicle 10 is far, giving priority to work efficiency, when the likelihood of the detection target is high, a response process (notification process) is executed for the work vehicle 10 (see FIGS. 5 and 6). Also, for example, in the stop control range Ra where the distance from the work vehicle 10 is close, giving priority to safety, a response process (stop process) is executed for the work vehicle 10 even when the likelihood of the detection target is low. Also, for example, in the deceleration control range Rb, it is possible to make it easier to execute a response process (deceleration process) as the distance gets closer, and more difficult to execute a response process (deceleration process) as the distance gets farther.

[0125] The present invention is not limited to the above-described embodiments, and the following embodiments may also be applicable. In the above-described embodiments, as shown in FIG. 6, the likelihood threshold is set to linearly change in the deceleration control range Rb (distance L1 to L2). As another embodiment, as shown in FIG. 13A, the likelihood threshold may be set to a constant value (Th1) in the stop control range Ra (distance 0 to L1), and set to a constant value (Th2) in the notification control range Rc (distance L2 to L3). Also, the likelihood threshold at the distance L1 in the stop control range Ra may be set to Th1, the likelihood threshold at the distance L2 in the notification control range Rc may be set to Th2, and the likelihood threshold may be set to linearly change between Th1 and Th2 in the deceleration control range Rb (distance L1 to L2).

[0126] Also, as shown in FIG. 13B, the likelihood threshold may be set in three steps. For example, the likelihood threshold may be set to Th1 in the stop control range Ra (distance 0 to L1), set to Th4 (where Th1 < Th4 < Th2) in the deceleration control range Rb (distance L1 to L2), and set to Th2 in the notification control range Rc (distance L2 to L3).

[0127] Also, the likelihood threshold may be set to linearly change over the entire range (distance 0 to L3) of the stop control range Ra, the deceleration control range Rb, and the notification control range Rc.

[0128] Furthermore, in the above-described embodiment, the likelihood threshold is set according to the distance from the work vehicle 10 to the detection target. In another embodiment, as shown in Fig. 13C, the likelihood threshold may be set to a constant value (e.g., Th1) in the entire range (distance 0 to L3) of the stop control range Ra, the deceleration control range Rb, and the notification control range Rc. For example, the detection control unit 51 sets the likelihood threshold low when safety is prioritized, and sets the likelihood threshold high when work efficiency is prioritized. Furthermore, the detection control unit 51 may set a constant likelihood threshold value for each state of the field F, work content, and control target.

[0129] In addition, as another embodiment, the automatic driving system 1 may be a form that does not use the likelihood information described above. For example, in the automatic driving system 1, an operator may be able to set the contents of the countermeasure processing according to the distance (range) for each type of detection target. For example, the operator sets the countermeasure processing (e.g., stop processing, deceleration processing, notification processing, etc.) according to the distance to the detection target for each type of detection target (control target) (person, vehicle, etc.) on the setting screen. This makes it possible to vary the countermeasure processing according to the type of detection target and the distance, such as stopping the work vehicle 10 if the detection target is a person when the distance to the detection target is L0, and decelerating the work vehicle 10 if the detection target is a vehicle. In this case, the operator may be able to set the threshold value of the distance (range of each countermeasure processing) for each type of detection target.

[0130] In another embodiment, the automated driving system 1 may execute a countermeasure process corresponding to a detection target that is close to the work vehicle 10. Specifically, the automated driving system 1 executes the countermeasure process when it detects a detection target that is less than a predetermined distance from the work vehicle 10, and does not execute the countermeasure process when it detects a detection target that is the predetermined distance or more from the work vehicle 10.

[0131] In another embodiment, the automatic driving system 1 may set a priority according to the type of detection target. For example, the automatic driving system 1 sets the priority of a person higher than the priority of a vehicle. In this case, when the automatic driving system 1 detects a person and a vehicle at the same distance, it executes a countermeasure process corresponding to the detection target with a higher priority. Here, the automatic driving system 1 executes the countermeasure process set for a person. In this case, an operator may be able to set a priority for each type of detection target.

[0132] [Other ways to set the likelihood threshold] Specific examples of other methods for setting the likelihood threshold (first to third setting methods) will be described below.

[0133] [First setting method] In the first setting method, the detection control unit 51 sets the likelihood threshold value according to the setting contents of the monitoring mode in the automatic driving mode. Specifically, the work vehicle 10 includes a short-distance mode (short-distance monitoring mode) in which the operation of the work vehicle 10 can be controlled within a predetermined range from the work vehicle 10, and a long-distance mode (long-distance monitoring mode) in which the operation of the work vehicle 10 can be controlled (operated via cloud connection) via a communication network N1 (such as the Internet), and the vehicle control device 11 can set and switch between the short-distance mode and the long-distance mode. Note that the short-distance mode is a monitoring mode in which the operation of the work vehicle 10 can be controlled by short-distance communication, and the long-distance mode may be a monitoring mode in which the operation of the work vehicle 10 can be controlled by long-distance communication. In addition, the short-distance mode may be a remote control mode, and the long-distance mode may be a smartphone mode. In addition, the vehicle control device 11 is capable of controlling the operation of the work vehicle 10 according to the set monitoring mode. For example, the worker can select the short distance mode or the long distance mode on an operation device (on-board monitor) fixed inside the cabin 138 of the work vehicle 10, or on a setting screen D5 of an operation terminal 20 that the worker has brought into the cabin 138. An example of the setting screen D5 is shown in Fig. 14. The vehicle control device 11 sets the monitoring mode selected by the worker, and executes processing according to the set monitoring mode.

[0134] For example, when the vehicle control device 11 is set to the short distance mode, the worker can use a mobile terminal such as a smartphone, tablet terminal, or remote control to start / stop the automatic driving of the work vehicle 10, set / change the vehicle speed and engine RPM of the work vehicle 10, raise / lower the work equipment 14, etc. On the other hand, when the vehicle control device 11 is set to the long distance mode, the worker can use an operation terminal such as a smartphone, tablet terminal (operation terminal 20), or personal computer to check camera images during automatic driving, stop the automatic driving of the work vehicle 10, set / change the vehicle speed and engine RPM of the work vehicle 10, raise / lower the work equipment 14, etc.

[0135] The detection control unit 51 sets the likelihood threshold depending on whether the monitoring mode is set to the short distance mode or the long distance mode. For example, when the monitoring mode is set to the short distance mode, the detection control unit 51 sets the likelihood threshold to a high value to make it difficult to recognize the detection target as an obstacle (lowering the sensitivity). In this case, when the detection control unit 51 has determined that the detection target is definitely an obstacle (person, vehicle, etc.), that is, when a detection target with a high likelihood is detected, the vehicle control device 11 executes a countermeasure process (stopping process, deceleration process, notification process). This makes it possible to improve work efficiency while ensuring safety in the short distance mode.

[0136] On the other hand, when the monitoring mode is set to the long distance mode, the detection control unit 51 sets the likelihood threshold to a low value to make it easier to identify the detection target as an obstacle (increasing the sensitivity). In this case, when the detection control unit 51 determines that the detection target is likely to be an obstacle (person, vehicle, etc.), that is, when a detection target with low likelihood is detected, the vehicle control device 11 executes a countermeasure process (stopping process, deceleration process, notification process). As a result, in the long distance mode, automatic driving can be performed with increased safety.

[0137] [Second setting method] In the second setting method, the detection control unit 51 sets the likelihood threshold according to the position of the detection target. Specifically, the detection control unit 51 sets different likelihood thresholds for when the detection target is located inside the field and when it is located outside the field. For example, the detection control unit 51 sets (registers) in advance a first likelihood threshold to be applied when the detection target is detected inside the field and a second likelihood threshold to be applied when the detection target is detected outside the field. In addition, the detection control unit 51 sets the second likelihood threshold to a value higher than the first likelihood threshold. That is, the detection control unit 51 sets the first likelihood threshold to a low value (high sensitivity) so that the detection target is easily recognized as an obstacle, and sets the second likelihood threshold to a high value (low sensitivity) so that the detection target is less likely to be recognized as an obstacle.

[0138] When the work vehicle 10 detects a detection target during automatic travel, the detection control unit 51 determines whether the detection target is located inside or outside the field F based on the distance from the work vehicle 10 to the detection target and the external shape data (map information) of the field F. When the detection control unit 51 determines that the detection target is located inside the field F, it determines whether the detection target is an obstacle based on a first likelihood threshold. Specifically, the detection control unit 51 determines that the detection target is an obstacle when the likelihood of the detection target exceeds the first likelihood threshold. In contrast, when the detection target is located outside the field F, the detection control unit 51 determines whether the detection target is an obstacle based on a second likelihood threshold. Specifically, the detection control unit 51 determines that the detection target is an obstacle when the likelihood of the detection target exceeds the second likelihood threshold.

[0139] The vehicle control device 11 executes a countermeasure process (stopping process, deceleration process, notification process) when the detection control unit 51 determines that the detection target is an obstacle. According to the above configuration, even if a detection target located outside the field is detected, it is difficult to execute a countermeasure process. Therefore, for example, for a detection target located outside the field F that is unlikely to come into contact with the work vehicle 10, the number of notifications can be reduced and unnecessary driving control (deceleration, stopping) can be avoided. This can reduce the worker's confirmation work effort and improve work efficiency. Note that, when the likelihood of a detection target detected outside the field exceeds the second likelihood threshold, that is, when an obstacle located outside the field is detected, the vehicle control device 11 may execute only the notification process (such as issuing a warning sound from the work vehicle 10 to the surroundings and notifying the operation terminal 20 of a warning message) without executing the stop process and the deceleration process. In addition, in the notification process, the vehicle control device 11 may change the notification manner (increasing the volume, shortening the notification sound cycle, increasing the frequency of warning message notifications, highlighting the warning message, etc.) as the distance to the obstacle decreases (as the work vehicle 10 approaches the obstacle).

[0140] [Third setting method] In the third setting method, the detection control unit 51 sets the likelihood threshold value according to the work mode (solo work mode, cooperative work mode). Specifically, the worker can select whether to have one work vehicle 10 perform work in one field F (solo work mode) or to have multiple work vehicles 10 perform work (cooperative work mode). In the solo work mode, one work vehicle 10 performs automatic travel and work in the field F, and in the cooperative work mode, multiple work vehicles 10 perform automatic travel and work cooperatively in the field F. For example, the worker can select the solo work mode or the cooperative work mode on the setting screen D6 of the operation device (vehicle monitor) or the operation terminal 20. FIG. 15 shows an example of the setting screen D6. The vehicle control device 11 sets the work mode selected by the worker and executes processing according to the set work mode.

[0141] The detection control unit 51 sets the likelihood threshold depending on whether the work mode is set to the solo work mode or the collaborative work mode. For example, when the solo work mode is set, the detection control unit 51 sets the likelihood threshold to a high value to make it difficult to recognize the detection target as an obstacle (lowering the sensitivity). In this case, when the detection control unit 51 has determined that the detection target is definitely an obstacle (person, vehicle, etc.), that is, when a detection target with a high likelihood is detected, the vehicle control device 11 executes a countermeasure process (stopping process, deceleration process, notification process). This makes it possible to improve work efficiency while ensuring safety in the solo work mode.

[0142] On the other hand, when the work mode is set to the cooperative work mode, the detection control unit 51 sets the likelihood threshold to a low value to make it easier to identify the detection target as an obstacle (increasing the sensitivity). In this case, when the detection control unit 51 determines that the detection target is likely to be an obstacle (person, vehicle, etc.), that is, when a detection target with low likelihood is detected, the vehicle control device 11 executes a countermeasure process (stopping process, deceleration process, notification process). As a result, in the cooperative work mode, it becomes easier to recognize other work vehicles 10, so that automatic driving can be performed with increased safety.

[0143] As another embodiment of the third setting method, the detection control unit 51 may set the likelihood threshold to a low value (set the sensitivity high) when the independent work mode is set, and set the likelihood threshold to a high value (set the sensitivity low) when the cooperative work mode is set. This makes it difficult for a countermeasure process to be executed in the cooperative work mode, for example, and can improve the efficiency of the cooperative work. In the cooperative work mode, the travel route (target route) is set so that multiple work vehicles 10 do not come into contact with each other, so safety can be ensured even if the sensitivity is set low.

[0144] As another embodiment of the third setting method, the detection control unit 51 may accept a selection operation of a priority mode for collaborative work on the setting screen D6 shown in Fig. 16 when the worker selects the collaborative work mode. For example, in the collaborative work mode, the worker can select a "safety priority mode" that prioritizes safety, or a "workability priority mode" that prioritizes work efficiency. The detection control unit 51 sets the sensitivity high when the worker selects the "safety priority mode," and sets the sensitivity low when the worker selects the "workability priority mode."

[0145] The detection control unit 51 may set the likelihood threshold for the workability priority mode in the collaborative work mode to a lower value (higher sensitivity) than the likelihood threshold for the solo work mode. That is, the detection control unit 51 may set the magnitude of the likelihood threshold for the work mode to satisfy the relational expression "likelihood threshold for collaborative work mode / safety priority mode" < "likelihood threshold for collaborative work mode / workability priority mode" < "likelihood threshold for solo work mode".

[0146] Furthermore, the detection control unit 51 may set the likelihood threshold for the workability priority mode in the collaborative work mode to a higher value (set the sensitivity lower) than the likelihood threshold for the solo work mode. That is, the detection control unit 51 may set the magnitude of the likelihood threshold for the work mode to satisfy the relational expression "likelihood threshold for collaborative work mode / safety priority mode" < "likelihood threshold for solo work mode" < "likelihood threshold for collaborative work mode / workability priority mode".

[0147] As another example of a method for setting the likelihood threshold, the detection control unit 51 may set the likelihood threshold according to the position of the worker relative to the work vehicle 10. Specifically, when the worker is located within a predetermined distance from the work vehicle 10, the detection control unit 51 sets the likelihood threshold to a high value (sets the sensitivity low) because the worker can visually determine an obstacle. In contrast, when the worker is located outside the predetermined distance from the work vehicle 10, the detection control unit 51 sets the likelihood threshold to a low value (sets the sensitivity high) because it is difficult for the worker to visually determine an obstacle. Note that the detection control unit 51 may set the likelihood threshold according to the distance between the work vehicle 10 and the worker, for example, by setting the likelihood threshold to a lower value as the distance between the work vehicle 10 and the worker increases.

[0148] [Notes on the invention] The following will provide an overview of the invention extracted from each of the above-described embodiments. Note that the configurations and processing functions described in the following supplementary notes can be selected and combined as desired.

[0149] <Appendix 1> When the work vehicle is automatically traveling along the target route, distance information of a distance to a detection target detected by a detection unit is acquired; acquiring image information of an image of the detection target captured by an imaging unit; determining a type of the detection target based on the image information; and causing the work vehicle to execute a countermeasure process according to the type of the detection target and the distance information; and A vehicle control method for performing the above.

[0150] <Appendix 2> calculating a likelihood of the type of the detection target based on the image information, and causing the work vehicle to execute the countermeasure processing based on the calculated likelihood; 2. A vehicle control method as described in claim 1.

[0151] <Appendix 3> calculating a likelihood of the type of the detection target based on the image information and the distance information, and causing the work vehicle to execute the countermeasure processing based on the calculated likelihood; 2. A vehicle control method as described in claim 1.

[0152] <Appendix 4> The likelihood is calculated so that the shorter the distance, the higher the likelihood. 4. A vehicle control method as described in claim 3.

[0153] <Appendix 5> causing the work vehicle to execute the countermeasure processing when the likelihood exceeds a threshold value; A vehicle control method according to any one of appendixes 2 to 4.

[0154] <Appendix 6> The threshold value is set to a smaller value as the distance is shorter. 6. A vehicle control method as described in appended claim 5.

[0155] <Appendix 7> The threshold value is set to a different value depending on the content of the handling process. 7. A vehicle control method according to claim 5 or 6.

[0156] <Appendix 8> When the countermeasure process is a process of stopping the work vehicle during autonomous driving, the threshold value is set to a first threshold value; When the countermeasure process is a process of decelerating the work vehicle during autonomous driving or a process of issuing warning information from the work vehicle, the threshold value is set to a second threshold value that is greater than the first threshold value. 8. A vehicle control method as described in claim 7.

[0157] <Appendix 9> A sensitivity, which is setting information for making it easier or harder to execute the countermeasure process, can be set; The threshold value is further set according to the sensitivity. A vehicle control method according to any one of appendixes 6 to 8.

[0158] <Appendix 10> The sensitivity is set according to a user's operation. 10. A vehicle control method as described in claim 9.

[0159] <Appendix 11> The threshold value is set based on at least one of a state of a work area in which the work vehicle automatically travels and a work content to be performed by the work vehicle. A vehicle control method according to any one of appendixes 5 to 8.

[0160] <Appendix 12> The threshold value is set according to the type of the detection target. A vehicle control method according to any one of claims 5 to 8.

[0161] <Appendix 13> The work vehicle includes a short-distance mode in which the operation of the work vehicle can be controlled within a predetermined range, and a long-distance mode in which the operation of the work vehicle can be controlled outside the predetermined range, The threshold value is set to one of a threshold value corresponding to the short distance mode and a threshold value corresponding to the long distance mode. A vehicle control method according to any one of claims 5 to 8.

[0162] <Appendix 14> The threshold value is set to a different value depending on whether the detection target is located inside a work area in which the work vehicle automatically travels or whether the detection target is located outside the work area. A vehicle control method according to any one of claims 5 to 8.

[0163] <Appendix 15> The work vehicle includes an independent work mode in which the work vehicle performs work independently and a cooperative work mode in which the work vehicle performs work in cooperation with another work vehicle, The threshold value is set to either a threshold value corresponding to the solo work mode or a threshold value corresponding to the collaborative work mode. A vehicle control method according to any one of claims 5 to 8.

[0164] <Appendix 16> It is possible to set whether or not to cause the work vehicle to execute the countermeasure process for each type of the detection target. A vehicle control method according to any one of appendixes 1 to 15. [Explanation of symbols]

[0165] 1:Autonomous driving system 10: Work vehicle 11: Vehicle control device 14: Work equipment 15: Obstacle detection device 20: Operation terminal 51: Detection control unit 52: Storage section 53: Camera (imaging unit) 54: Obstacle sensor (detection section) 111: Driving processing unit 112: Handling processing unit 211: Setting processing section 212: Output processing section 511: Acquisition processing unit 512: Judgment processing unit 513: Output processing section F: Field R: Target route Ra: Stop control range Rb: Deceleration control range Rc: Notification control range Rd1: First detection range Rd2: Second detection range Rm1: First measurement range Rm2: Second measurement range Th1: Likelihood threshold (threshold) Th2: Likelihood threshold (threshold) Th3: Likelihood threshold (threshold)

Claims

1. When the work vehicle is automatically traveling along the target route, distance information of a distance to a detection target detected by a detection unit is acquired; acquiring image information of an image of the detection target captured by an imaging unit; determining a type of the detection target based on the image information; and causing the work vehicle to execute a countermeasure process according to the type of the detection target and the distance information; and A vehicle control method for performing the above.

2. calculating a likelihood of the type of the detection target based on the image information, and causing the work vehicle to execute the countermeasure processing based on the calculated likelihood; The vehicle control method according to claim 1 .

3. calculating a likelihood of the type of the detection target based on the image information and the distance information, and causing the work vehicle to execute the countermeasure processing based on the calculated likelihood; The vehicle control method according to claim 1 .

4. The likelihood is calculated so that the shorter the distance, the higher the likelihood. The vehicle control method according to claim 3.

5. causing the work vehicle to execute the countermeasure processing when the likelihood exceeds a threshold value; The vehicle control method according to claim 2 .

6. The threshold value is set to a smaller value as the distance is shorter. The vehicle control method according to claim 5.

7. The threshold value is set to a different value depending on the content of the handling process. The vehicle control method according to claim 6.

8. When the countermeasure process is a process of stopping the work vehicle during autonomous driving, the threshold value is set to a first threshold value; When the countermeasure process is a process of decelerating the work vehicle during autonomous driving or a process of issuing warning information from the work vehicle, the threshold value is set to a second threshold value that is greater than the first threshold value. The vehicle control method according to claim 7.

9. A sensitivity, which is setting information for making it easier or harder to execute the countermeasure process, can be set; The threshold value is further set according to the sensitivity. The vehicle control method according to any one of claims 6 to 8.

10. The sensitivity is set according to a user's operation. The vehicle control method according to claim 9.

11. The threshold value is set based on at least one of a state of a work area in which the work vehicle automatically travels and a work content to be performed by the work vehicle. The vehicle control method according to any one of claims 6 to 8.

12. The threshold value is set according to the type of the detection target. The vehicle control method according to claim 5.

13. The work vehicle includes a short-distance mode in which the operation of the work vehicle can be controlled within a predetermined range, and a long-distance mode in which the operation of the work vehicle can be controlled outside the predetermined range, The threshold value is set to one of a threshold value corresponding to the short distance mode and a threshold value corresponding to the long distance mode. The vehicle control method according to claim 5.

14. The threshold value is set to a different value depending on whether the detection target is located inside a work area in which the work vehicle automatically travels or whether the detection target is located outside the work area. The vehicle control method according to claim 5.

15. The work vehicle includes an independent work mode in which the work vehicle performs work independently and a cooperative work mode in which the work vehicle performs work in cooperation with another work vehicle, The threshold value is set to either a threshold value corresponding to the solo work mode or a threshold value corresponding to the collaborative work mode. The vehicle control method according to claim 5.

16. It is possible to set whether or not to cause the work vehicle to execute the countermeasure process for each type of the detection target. The vehicle control method according to any one of claims 1 to 8.

17. When the work vehicle is automatically traveling along the target route, distance information of a distance to a detection target detected by a detection unit is acquired; acquiring image information of an image of the detection target captured by an imaging unit; determining a type of the detection target based on the image information; and causing the work vehicle to execute a countermeasure process according to the type of the detection target and the distance information; and A vehicle control program for causing one or more processors to execute the above.

18. an acquisition processing unit that acquires, when the work vehicle is automatically traveling along a target route, distance information of a distance to a detection target detected by the detection unit and image information of an image of the detection target captured by the imaging unit; a determination processing unit that determines a type of the detection object based on the image information; a countermeasure processing unit that causes the work vehicle to execute a countermeasure process according to the type of the detection target and the distance information; A vehicle control system comprising:

Images