Automated driving system, automated driving method, and automated driving program

Abstract

To provide an automatic driving system, an automatic driving method, and an automatic driving program that can prevent a decrease in work efficiency while ensuring the safety of a work vehicle.SOLUTION: An acquisition processing unit 111 acquires a captured image P1 from a camera 15 installed on a work vehicle 10. A detection processing unit 112 detects an obstacle on the basis of the captured image P1 acquired by the acquisition processing unit 111. When an obstacle is detected by the detection processing unit 112, a reception processing unit 217 receives a travel stop instruction to stop the automatic travel of the work vehicle 10 or a travel continuation instruction to continue the automatic travel. A travel processing unit 113 stops the automatic travel of the work vehicle 10 when the reception processing unit 217 receives the travel stop instruction, and continues the automatic travel of the work vehicle 10 when the reception processing unit 217 receives the travel continuation instruction.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to an automatic driving system, an automatic driving method, and an automatic driving program for automatically driving a work vehicle.

Background Art

[0002] When a work vehicle detects an obstacle while automatically driving along a preset driving route in a field, it has a function to stop the automatic driving. Conventionally, an operator of a work vehicle can switch between a normal mode in which the automatic driving of the work vehicle is stopped when the work vehicle detects an obstacle and a release mode in which the work vehicle continues to drive even when the work vehicle detects an obstacle (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the above conventional technology, when the release mode is set, the automatic driving continues even if the work vehicle detects an obstacle, so there is a risk that the work vehicle will come into contact with the obstacle. Thus, in a system that uniformly stops the automatic driving when the work vehicle detects an obstacle, the work efficiency decreases, while in a system that allows the automatic driving to continue even when the work vehicle detects an obstacle, the safety of the work vehicle decreases.

[0005] An object of the present invention is to provide an automatic driving system, an automatic driving method, and an automatic driving program capable of preventing a decrease in work efficiency while ensuring the safety of a work vehicle.

Means for Solving the Problems

[0006] The automated driving system according to the present invention comprises an acquisition processing unit, a detection processing unit, a reception processing unit, and a driving processing unit. The acquisition processing unit acquires captured images from an imaging unit installed on a work vehicle. The detection processing unit detects obstacles based on the captured images acquired by the acquisition processing unit. When the detection processing unit detects an obstacle, the reception processing unit receives a driving stop instruction to stop the automated driving of the work vehicle or a driving continue instruction to continue the automated driving of the work vehicle. The driving processing unit stops the automated driving of the work vehicle when the reception processing unit receives a driving stop instruction, and continues the automated driving of the work vehicle when the reception processing unit receives a driving continue instruction.

[0007] The automatic driving method according to the present invention is a method in which one or more processors acquire captured images from an imaging unit installed on a work vehicle, detect obstacles based on the acquired captured images, receive a stop-driving instruction to stop the automatic driving of the work vehicle or a continue-driving instruction to continue the automatic driving when an obstacle is detected, and stop the automatic driving of the work vehicle when a stop-driving instruction is received, and continue the automatic driving of the work vehicle when a continue-driving instruction is received.

[0008] The automatic driving program according to the present invention is a program that causes one or more processors to execute the following: acquiring captured images from an imaging unit installed on a work vehicle; detecting obstacles based on the acquired captured images; receiving a stop instruction to stop the automatic driving of the work vehicle or a continue instruction to continue the automatic driving when an obstacle is detected; and stopping the automatic driving of the work vehicle when a stop instruction is received, and continuing the automatic driving of the work vehicle when a continue instruction is received. [Effects of the Invention]

[0009] According to the present invention, it is possible to provide an automated driving system, an automated driving method, and an automated driving program that can prevent a decrease in work efficiency while ensuring the safety of the work vehicle. [Brief explanation of the drawing]

[0010] [Figure 1] Figure 1 is a block diagram showing the configuration of an automated driving system according to an embodiment of the present invention. [Figure 2] Figure 2 is an external view showing an example of a work vehicle according to an embodiment of the present invention. [Figure 3] Figure 3 shows an example of a travel path for a work vehicle according to an embodiment of the present invention. [Figure 4] Figure 4 shows the position of a camera installed on a work vehicle according to an embodiment of the present invention. [Figure 5A] Figure 5A shows an example of a camera layout setting screen displayed on an operating terminal according to an embodiment of the present invention. [Figure 5B] Figure 5B shows an example of a camera layout setting screen displayed on an operating terminal according to an embodiment of the present invention. [Figure 6A] Figure 6A shows an example of a camera image displayed on an operating terminal according to an embodiment of the present invention. [Figure 6B] Figure 6B shows an example of a camera image displayed on an operating terminal according to an embodiment of the present invention. [Figure 7A] Figure 7A shows an example of a camera image displayed on an operating terminal according to an embodiment of the present invention. [Figure 7B] Figure 7B shows an example of a camera image displayed on an operating terminal according to an embodiment of the present invention. [Figure 8A] Figure 8A shows an example of a camera image displayed on an operating terminal according to an embodiment of the present invention. [Figure 8B] Figure 8B shows an example of a camera image displayed on an operating terminal according to an embodiment of the present invention. [Figure 9]FIG. 9 is a diagram showing an example of a travel selection screen displayed on an operation terminal according to an embodiment of the present invention. [Figure 10] FIG. 10 is a flowchart showing an example of the procedure of an automatic driving process executed by an automatic driving system according to an embodiment of the present invention. [Figure 11] FIG. 11 is a diagram showing an example of a travel selection screen displayed on an operation terminal according to an embodiment of the present invention.

Mode for Carrying Out the Invention

[0011] The following embodiments are an example of embodying the present invention and do not limit the technical scope of the present invention.

[0012] As shown in FIG. 1, an automatic driving system 1 according to an embodiment of the present invention includes a work vehicle 10 and an operation terminal 20. The work vehicle 10 and the operation terminal 20 can communicate via a communication network N1. For example, the work vehicle 10 and the operation terminal 20 can communicate via a mobile phone line network, a packet line network, or a wireless LAN.

[0013] In the present embodiment, a case where the work vehicle 10 is a tractor will be described as an example. As another embodiment, the work vehicle 10 may be a rice transplanter, a combine, a construction machine, a snowplow, or the like. The work vehicle 10 is a so-called robot tractor having a configuration capable of automatically traveling (autonomously traveling) along a preset travel route R within a field F (see FIG. 3). For example, the work vehicle 10 can automatically travel along a travel route R generated in advance for the field F based on the position information of the current position of the work vehicle 10 calculated by the positioning device 17.

[0014] For example, the work vehicle 10 reciprocates parallel from the work start position S to the work end position G in the work area of the field F shown in FIG. 3. The outer peripheral side of the field F is, for example, a headland area, and the work vehicle 10 makes a turning travel. The travel route R is not limited to the route shown in FIG. 3 and is appropriately set according to the work content.

[0015] [Work vehicle 10] As shown in FIGS. 1 and 2, the work vehicle 10 includes a vehicle control device 11, a storage unit 12, a traveling device 13, a working machine 14, a camera 15, a communication unit 16, a positioning device 17, and a detection processing device 19. The vehicle control device 11 is electrically connected to the storage unit 12, the traveling device 13, the working machine 14, the positioning device 17, the detection processing device 19, and the like. Note that the vehicle control device 11 and the positioning device 17 may be capable of wireless communication. Also, the camera 15 is electrically connected to the detection processing device 19.

[0016] The storage unit 12 is a non-volatile storage unit such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive) that stores various types of information. The storage unit 12 stores control programs such as an automatic driving program for causing the vehicle control device 11 to execute the automatic driving process (see FIG. 10) described later. For example, the automatic driving program is non-temporarily recorded on a computer-readable recording medium such as a CD or a DVD, and is read by a predetermined reading device (not shown) and stored in the storage unit 12. Note that the automatic driving program may be downloaded from a server (not shown) to the work vehicle 10 via the communication network N1 and stored in the storage unit 12. Also, the storage unit 12 stores data of the travel route R generated in the operation terminal 20, determination data for determining obstacles, and the like. The determination data is data for determining a human, which is an example of an obstacle, and is, for example, data indicating human characteristics, data of an image (sample image) indicating a human part or the whole, and the like. The determination data may be stored in the detection processing device 19. Also, the storage unit 12 may store work information (turning mode, work order, etc.). Note that the obstacle of the present invention is an object that obstructs the travel of the work vehicle 10, and is, for example, a human, an animal, a device (such as another work vehicle), and the like.

[0017] The running gear 13 is the drive unit that moves the work vehicle 10. As shown in Figure 2, the running gear 13 includes an engine 131, front wheels 132, rear wheels 133, transmission 134, front axle 135, rear axle 136, steering wheel 137, etc. The front wheels 132 and rear wheels 133 are provided on the left and right sides of the work vehicle 10, respectively. Furthermore, the running gear 13 is not limited to a wheel type with front wheels 132 and rear wheels 133, but may also be a crawler type with crawlers provided on the left and right sides of the work vehicle 10.

[0018] The engine 131 is a power source such as a diesel engine or gasoline engine that is driven using fuel supplied to a fuel tank (not shown). The running gear 13 may be equipped with an electric motor as a power source together with the engine 131, or in place of the engine 131. A generator (not shown) is connected to the engine 131, and power is supplied from the generator to electrical components such as the vehicle control device 11 and the battery installed on the work vehicle 10. The battery is charged by the power supplied from the generator. The vehicle control device 11 and electrical components such as the positioning device 17 installed on the work vehicle 10 can be driven by the power supplied from the battery even after the engine 131 is stopped.

[0019] The driving force of the engine 131 is transmitted to the front wheels 132 via the transmission 134 and front axle 135, and to the rear wheels 133 via the transmission 134 and rear axle 136. The driving force of the engine 131 is also transmitted to the work equipment 14 via the PTO shaft (not shown). When the work vehicle 10 is driving automatically, the travel device 13 performs driving operations according to the commands of the vehicle control device 11.

[0020] The implement 14 is, for example, a lawnmower, tiller, plow, fertilizer spreader, or seed planter, and is detachable from the work vehicle 10. This allows the work vehicle 10 to perform various tasks using each of the implements 14. In this embodiment, the case where the implement 14 is a lawnmower will be used as an example for explanation.

[0021] For example, the implement 14 is mounted offset to one side (left or right) relative to the work vehicle 10. For example, the work vehicle 10 can perform tasks such as mowing by driving through a field with a directly mounted implement 14 mounted offset to one side (left or right). Note that the implement 14 is not limited to a directly mounted implement fixed to the work vehicle 10 (see Figure 2), but may also be a towed implement pulled by the work vehicle 10.

[0022] Camera 15 is a digital camera that captures images of a subject and outputs them as digital image data. Camera 15 continuously captures images of the subject at a predetermined frame rate, generates frame images of a predetermined resolution, and transmits them sequentially to the detection processing device 19. Camera 15 is an example of the imaging unit of the present invention.

[0023] In this embodiment, five cameras 15 are installed in different locations on the work vehicle 10. Specifically, as shown in Figure 4, camera 15F (hereinafter also referred to as "camera 1") is installed on the front of the work vehicle 10, camera 15B (hereinafter also referred to as "camera 2") is installed on the rear of the work vehicle 10, camera 15R (hereinafter also referred to as "camera 3") is installed on the right side of the work vehicle 10, camera 15L (hereinafter also referred to as "camera 4") is installed on the left side of the work vehicle 10, and camera 15C (hereinafter also referred to as "camera 5") is installed on the front right side of the work machine 14. The cameras 15 may be attached to the work vehicle 10 by means of double-sided tape, for example.

[0024] Furthermore, each camera 15 is configured with a predetermined imaging range (detection area) that it can capture. For example, camera 15F captures the detection area K1 in front of the work vehicle 10, camera 15B captures the detection area K2 behind the work vehicle 10, camera 15R captures the detection area K3 to the right of the work vehicle 10, camera 15L captures the detection area K4 to the left of the work vehicle 10, and camera 15C captures the detection area K5 in front of the right side of the work machine 14. Each camera 15 captures each detection area at a predetermined frame rate and transmits the captured images sequentially to the detection processing device 19. The detection processing device 19 transmits the captured images and the detection results (judgment results) described later to the vehicle control device 11 and the operation terminal 20.

[0025] The detection processing unit 19 becomes able to communicate with the camera 15 when the camera 15 is electrically connected to the work vehicle 10. Once the detection processing unit 19 becomes able to communicate with the camera 15, it acquires the number of cameras 15 and the identification information (device information) of each camera 15. The detection processing unit 19 also outputs the acquired number information and identification information of the cameras 15 to the vehicle control device 11, and the vehicle control device 11 outputs the number information and identification information to the operation terminal 20. The operator can add cameras 15 or change their installation location.

[0026] The steering wheel 137 is an operating unit operated by an operator or a vehicle control device 11. For example, in the travel device 13, the angle of the front wheels 132 is changed by a hydraulic power steering mechanism (not shown) or the like in response to the operation of the steering wheel 137 by the vehicle control device 11, thereby changing the direction of travel of the work vehicle 10.

[0027] In addition to the steering wheel 137, the running gear 13 is equipped with a shift lever (not shown), accelerator, brakes, etc., which are operated by the vehicle control device 11. In the running gear 13, the gears of the transmission 134 are switched to forward gear or reverse gear, etc., in response to the operation of the shift lever by the vehicle control device 11, and the driving mode of the work vehicle 10 is switched to forward or reverse, etc. The vehicle control device 11 also controls the rotational speed of the engine 131 by operating the accelerator. The vehicle control device 11 also controls the rotation of the front wheels 132 and rear wheels 133 using electromagnetic brakes by operating the brakes.

[0028] The positioning device 17 is a communication device comprising a positioning control unit 171, a memory unit 172, a communication unit 173, and a positioning antenna 174. For example, as shown in Figure 2, the positioning device 17 is installed on top of the cabin 18 where the operator sits. However, the installation location of the positioning device 17 is not limited to the cabin 18. Furthermore, the positioning control unit 171, memory unit 172, communication unit 173, and positioning antenna 174 of the positioning device 17 may be distributed and arranged at different locations on the work vehicle 10. As mentioned above, the positioning device 17 is connected to the battery, and the positioning device 17 can operate even when the engine 131 is stopped. In addition, the positioning device 17 may be replaced with, for example, a mobile phone terminal, a smartphone, or a tablet terminal.

[0029] The positioning control unit 171 is a computer system comprising one or more processors and storage memory such as non-volatile memory and RAM. The storage unit 172 is a non-volatile memory that stores a program for causing the positioning control unit 171 to perform positioning processing, and data such as positioning information and movement information. For example, the program is non-temporarily recorded on a computer-readable recording medium such as a CD or DVD, read by a predetermined reading device (not shown), and stored in the storage unit 172. Alternatively, the program may be downloaded from a server (not shown) to the positioning device 17 via a communication network N1 and stored in the storage unit 172.

[0030] The communication unit 173 is a communication interface for connecting the positioning device 17 to the communication network N1 by wire or wireless connection and for performing data communication with external devices such as base station servers via the communication network N1 in accordance with a predetermined communication protocol.

[0031] The positioning antenna 174 is an antenna that receives radio waves (GNSS signals) transmitted from satellites.

[0032] The positioning control unit 171 calculates the current position of the work vehicle 10 based on the GNSS signals received by the positioning antenna 174 from satellites. For example, when the work vehicle 10 is automatically driving within field F, the positioning antenna 174 receives radio waves (transmission time, orbital information, etc.) transmitted from multiple satellites. The positioning control unit 171 then calculates the distance between the positioning antenna 174 and each satellite, and calculates the current position (latitude and longitude) of the work vehicle 10 based on the calculated distance. Alternatively, the positioning control unit 171 may perform positioning using a real-time kinematic method (RTK-GPS positioning method (RTK method)) which calculates the current position of the work vehicle 10 using correction information corresponding to a base station (reference station) close to the work vehicle 10. In this way, the work vehicle 10 automatically drives using positioning information obtained using the RTK method.

[0033] The vehicle control device 11 and the detection processing device 19 have control devices such as a CPU, ROM, and RAM. The CPU is a processor that performs various arithmetic operations. The ROM is a non-volatile storage unit in which control programs such as a BIOS and OS for causing the CPU to perform various arithmetic operations are pre-stored. The RAM is a volatile or non-volatile storage unit that stores various information and is used as a temporary storage memory (work area) for various processes performed by the CPU. The vehicle control device 11 and the detection processing device 19 control the work vehicle 10 by executing various control programs pre-stored in the ROM or storage unit 12 using the CPU.

[0034] The detection processing unit 19 acquires an image from the camera 15 and determines whether or not an obstacle (e.g., a person) is included in the detection area based on the image. Specifically, as shown in Figure 1, the detection processing unit 19 includes various processing units such as an acquisition processing unit 111 and a detection processing unit 112. The detection processing unit 19 functions as these various processing units by executing various processes according to the automatic driving program using a CPU. Furthermore, some or all of the processing units may be composed of electronic circuits. Note that the automatic driving program may be a program that causes multiple processors to function as processing units.

[0035] The acquisition processing unit 111 acquires captured images from one or more cameras 15. For example, the acquisition processing unit 111 sequentially acquires captured images of detection areas K1 to K5 frame by frame from each of the five cameras 15F, 15B, 15R, 15L, and 15C installed on the work vehicle 10 and work machine 14. The acquisition processing unit 111 stores the acquired captured images in the storage unit 12 along with the time of acquisition.

[0036] Furthermore, the acquisition processing unit 111 outputs the image data of the acquired image to the operation terminal 20. The acquisition processing unit 111 is an example of the acquisition processing unit of the present invention.

[0037] The detection processing unit 112 detects obstacles (e.g., people) in the captured image acquired by the acquisition processing unit 111. Specifically, the detection processing unit 112 determines whether or not a person is included in the detection area based on the captured image. For example, the detection processing unit 112 performs image analysis on the captured image and compares it with the determination data stored in the storage unit 12 or the detection processing device 19 to determine the presence or absence of a person in the detection area. The detection processing unit 112 also performs the determination process for each of the captured images acquired sequentially by the acquisition processing unit 111. For example, the detection processing unit 112 performs the determination process for the captured image from camera 15F, then for the captured image from camera 15B, then for the captured image from camera 15R, then for the captured image from camera 15L, and then for the captured image from camera 15C. The detection processing unit 112 transmits the detection result (determination result) to the operation terminal 20. The detection processing unit 112 is an example of the detection processing unit of the present invention.

[0038] As shown in Figure 1, the vehicle control device 11 includes various processing units such as a driving processing unit 113. The vehicle control device 11 functions as these various processing units by executing various processes according to the automatic driving program using the CPU. Some or all of these processing units may be composed of electronic circuits. The automatic driving program may be a program that causes multiple processors to function as processing units.

[0039] The driving processing unit 113 controls the movement of the work vehicle 10. Specifically, when the driving processing unit 113 receives a work start instruction from the operation terminal 20, it starts the automatic movement of the work vehicle 10. For example, when an operator presses the work start button on the operation screen of the operation terminal 20, the operation terminal 20 outputs a work start instruction to the work vehicle 10. When the driving processing unit 113 receives the work start instruction from the operation terminal 20, it starts the automatic movement of the work vehicle 10. As a result, the work vehicle 10 starts moving automatically according to the travel path R and starts working with the implement 14. The travel path R on which the work vehicle 10 travels is generated, for example, by the operation terminal 20. The work vehicle 10 receives the travel path R from the operation terminal 20 and moves automatically within the field F according to the travel path R.

[0040] Furthermore, when the driving processing unit 113 receives a driving stop instruction from the operation terminal 20, it stops the automatic driving of the work vehicle 10. For example, if an operator presses the driving stop button on the operation screen of the operation terminal 20, the operation terminal 20 outputs a driving stop instruction to the work vehicle 10. Also, for example, if the detection processing unit 112 detects a person, and the operator does not give either a driving stop instruction or a driving continue instruction on the operation screen of the operation terminal 20 for a predetermined time, the operation terminal 20 outputs a driving stop instruction to the work vehicle 10. When the driving processing unit 113 receives a driving stop instruction from the operation terminal 20, it stops the automatic driving of the work vehicle 10. As a result, the work vehicle 10 stops automatic driving and stops the work performed by the work machine 14. Specific examples of driving stop instructions and driving continue instructions will be described later.

[0041] Furthermore, when the driving processing unit 113 receives a driving restart instruction from the operation terminal 20, it restarts the automatic driving of the work vehicle 10. For example, if the detection processing unit 112 detects a person and the work vehicle 10 stops driving, and then the operator presses the driving restart button on the operation screen of the operation terminal 20, the operation terminal 20 outputs a driving restart instruction to the work vehicle 10. When the driving processing unit 113 receives the driving restart instruction from the operation terminal 20, it restarts the automatic driving of the work vehicle 10. As a result, the work vehicle 10 resumes automatic driving again according to the driving path R. The driving processing unit 113 is an example of the driving processing unit of the present invention.

[0042] [Operating terminal 20] As shown in Figure 1, the operating terminal 20 is an information processing device comprising an operation control unit 21, a storage unit 22, an operation display unit 23, and a communication unit 24, etc. The operating terminal 20 may be composed of a mobile device such as a tablet or a smartphone.

[0043] The communication unit 24 is a communication interface for connecting the operating terminal 20 to the communication network N1 by wire or wireless connection and for performing data communication with one or more external devices such as work vehicles 10 via the communication network N1 in accordance with a predetermined communication protocol.

[0044] The operation display unit 23 is a user interface comprising a display unit such as a liquid crystal display or an organic EL display that displays various information, and an operation unit such as a touch panel, mouse, or keyboard that accepts operations. The operator can register various information (such as work vehicle information, field information, and work information described later) by operating the operation unit on the operation screen displayed on the display unit. The operator can also issue work start instructions, travel stop instructions, and travel restart instructions to the work vehicle 10 by operating the operation unit. Furthermore, the operator can understand the driving status of the work vehicle 10, which is automatically traveling within the field F according to the travel route R, by observing the travel trajectory displayed on the operation terminal 20 from a location away from the work vehicle 10.

[0045] The storage unit 22 is a non-volatile storage unit such as an HDD or SSD that stores various types of information. The storage unit 22 stores control programs, such as an automatic driving program, which causes the operation control unit 21 to execute the automatic driving process (see Figure 10) described later. For example, the automatic driving program is non-temporarily recorded on a computer-readable recording medium such as a CD or DVD, read by a predetermined reading device (not shown), and stored in the storage unit 22. Alternatively, the automatic driving program may be downloaded from a server (not shown) to the operation terminal 20 via a communication network N1 and stored in the storage unit 22.

[0046] The operation control unit 21 includes control devices such as a CPU, ROM, and RAM. The CPU is a processor that performs various arithmetic operations. The ROM is a non-volatile memory unit that stores control programs such as a BIOS and OS in advance to allow the CPU to perform various arithmetic operations. The RAM is a volatile or non-volatile memory unit that stores various information and is used as a temporary memory (work area) for the various processes performed by the CPU. The operation control unit 21 controls the operation terminal 20 by executing various control programs stored in advance in the ROM or memory unit 22 using the CPU.

[0047] As shown in Figure 1, the operation control unit 21 includes various processing units such as a vehicle setting processing unit 211, a field setting processing unit 212, a work setting processing unit 213, a route generation processing unit 214, an output processing unit 215, a display processing unit 216, and a reception processing unit 217. The operation control unit 21 functions as these various processing units by executing various processes according to the control program using the CPU. Some or all of these processing units may be composed of electronic circuits. The control program may be a program that causes multiple processors to function as processing units.

[0048] The vehicle setting processing unit 211 sets information related to the work vehicle 10 (hereinafter referred to as "work vehicle information"). The vehicle setting processing unit 211 sets information such as the model of the work vehicle 10, the location on which the positioning antenna 174 is attached to the work vehicle 10, the type of work equipment 14, the size and shape of the work equipment 14, the position of the work equipment 14 relative to the work vehicle 10, the vehicle speed and engine speed of the work vehicle 10 during work, and the vehicle speed and engine speed of the work vehicle 10 while turning, by having the operator perform an operation to register this information on the operation terminal 20.

[0049] The field setting processing unit 212 sets information related to field F (hereinafter referred to as field information). The field setting processing unit 212 sets information such as the location and shape of field F, the work start position S where work begins and the work end position G where work ends (see Figure 3), and the work direction by performing an operation to register this information on the operation terminal 20.

[0050] The working direction refers to the direction in which the work vehicle 10 is driven while the work machine 14 is working in the work area, which is the area from field F excluding non-working areas such as headlands and uncultivated land.

[0051] Information on the location and shape of field F can be automatically acquired, for example, by having an operator ride in the work vehicle 10 and drive it in a circle around the perimeter of field F, while recording the changes in the position information of the positioning antenna 174 during that time. Alternatively, the location and shape of field F can also be acquired based on a polygon obtained by having an operator operate the operation terminal 20 to specify multiple points on the map displayed on the terminal 20. The area identified by the acquired location and shape of field F is the area in which the work vehicle 10 can travel (driving area).

[0052] The work setting processing unit 213 sets information regarding how the work will be performed (hereinafter referred to as work information). The work setting processing unit 213 is configured to set, as work information, whether or not there will be coordinated work between the work vehicle 10 (unmanned tractor) and the manned work vehicle 10, the number of skips which is the number of work paths to be skipped when the work vehicle 10 turns in the headland, the width of the headland, and the width of the non-cultivated land.

[0053] The route generation processing unit 214 generates a travel route R, which is a route for the work vehicle 10 to travel automatically, based on the setting information. The travel route R is, for example, a work route from a work start position S to a work end position G (see Figure 3). The travel route R shown in Figure 3 is a route for the work vehicle 10 to travel back and forth in parallel within the work area of ​​the field F. The route generation processing unit 214 can generate and store the travel route R of the work vehicle 10 based on the setting information set by the vehicle setting processing unit 211, the field setting processing unit 212, and the work setting processing unit 213.

[0054] Specifically, the route generation processing unit 214 generates a travel route R (see Figure 3) based on the work start position S and work end position G registered in the field setting. The travel route R is not limited to the route shown in Figure 3.

[0055] The work vehicle 10 is configured to autonomously travel along the travel route R while detecting its current position using the positioning antenna 174. The data of the travel route R generated in the operation terminal 20 is transferred to the work vehicle 10 and stored in the storage unit 12. The current position of the work vehicle 10 usually coincides with the position of the positioning antenna 174.

[0056] The work vehicle 10 according to this embodiment travels in a roughly rectangular field F as shown in Figure 3. The work vehicle 10 is configured to automatically travel when its current location is within field F, and is configured not to automatically travel when its current location is outside field F (e.g., on a public road). Furthermore, the work vehicle 10 is configured to automatically travel, for example, when its current location coincides with the work start position S.

[0057] When the work vehicle 10's current position coincides with the work start position S, and the operator presses the work start button on the operation screen to give a work start instruction, the travel processing unit 113 automatically starts driving and begins work with the work machine 14 (see Figure 2). In other words, the operation control unit 21 permits the automatic driving of the work vehicle 10 on the condition that its current position coincides with the work start position S. However, the conditions for permitting the automatic driving of the work vehicle 10 are not limited to the above conditions.

[0058] The output processing unit 215 outputs information about the travel route R generated by the route generation processing unit 214 to the work vehicle 10. Furthermore, the output processing unit 215 can instruct the work vehicle 10 to start and stop automatic driving by transmitting control signals to the work vehicle 10 via the communication unit 24. This enables the work vehicle 10 to operate automatically.

[0059] For example, the driving processing unit 113 automatically drives the work vehicle 10 from the work start position S to the work end position G based on the driving path R acquired from the operation terminal 20. Furthermore, once the work vehicle 10 has finished its work, the driving processing unit 113 may automatically drive it from the work end position G to the entrance of the field F. While the work vehicle 10 is moving automatically, the operation control unit 21 can receive the status of the work vehicle 10 (position, speed, etc.) from the work vehicle 10 and display it on the operation display unit 23.

[0060] The display processing unit 216 displays various information on the operation display unit 23. For example, the display processing unit 216 displays on the operation display unit 23 an operation screen for registering work vehicle information, field information, work information, etc., a setting screen D1 (see Figure 5) for setting the layout of the image display area L1 that displays each of the images captured by the multiple cameras 15 installed on the work vehicle 10, a driving status screen D2 (see Figures 6 to 8) that displays the captured images, and a driving selection screen D3 (see Figure 9) for selecting a driving stop instruction to stop the work vehicle 10 from driving or a driving continue instruction to continue driving.

[0061] The reception processing unit 217 receives various operations from the operator. For example, the reception processing unit 217 receives an operation from the operator to set the layout of the image display field L1 (an example of a setting operation in the present invention). The reception processing unit 217 also receives a work start instruction from the operator to start work on the work vehicle 10. The reception processing unit 217 also receives a drive stop instruction from the operator to stop the work vehicle 10 while it is automatically driving. The reception processing unit 217 also receives a drive stop instruction or a drive continue instruction from the operator when the work vehicle 10 detects a person. The reception processing unit 217 also receives a drive restart instruction from the operator to resume the movement of the work vehicle 10 that has stopped. When the reception processing unit 217 receives any of the above instructions, the output processing unit 215 outputs the above instructions to the work vehicle 10. The reception processing unit 217 is an example of a reception processing unit in the present invention.

[0062] When the driving processing unit 113 of the work vehicle 10 receives a work start instruction from the operation terminal 20, it starts the work vehicle 10 moving and performing work. When the driving processing unit 113 receives a driving stop instruction from the operation terminal 20, it stops the work vehicle 10 moving and performing work. When the driving processing unit 113 receives a driving continue instruction from the operation terminal 20, it continues the work vehicle 10 moving and performing work. When the driving processing unit 113 receives a driving restart instruction from the operation terminal 20, it restarts the work vehicle 10 moving and performing work.

[0063] Here, we will explain a specific example. For example, as shown in Figure 4, if five cameras 15F, 15B, 15R, 15L, and 15C (cameras 1 to 5) are installed on the work vehicle 10, the display processing unit 216 obtains the number of cameras 15 to acquire from the work vehicle 10 and their identification information, and then displays a setting screen D1 for setting the layout of the image display area L1 on the operation display unit 23. For example, as shown in Figure 5A, the display processing unit 216 displays five image display areas L1, each displaying the captured images of camera 15F (camera 1), camera 15B (camera 2), camera 15R (camera 3), camera 15L (camera 4), and camera 15C (camera 5), ​​in any order. Alternatively, as shown in Figure 5B, the display processing unit 216 may arrange the five image display areas L1 in positions corresponding to the installation layout of the cameras 15 on the work vehicle 10. Furthermore, the operator can change the layout of the image display area L1 on the setting screen D1. For example, the operator can place each image display area L1 in the settings screen D1 in the desired position by dragging and dropping.

[0064] Furthermore, it is desirable that the display processing unit 216 displays an image (camera placement image Pc) that identifies the placement position of each camera 15 relative to the work vehicle 10, as shown on the left side of the setting screen D1. This allows the operator to easily grasp the correspondence between the placement position of each camera 15 and the layout of each image display field L1. Note that the display processing unit 216 may omit the camera placement image Pc on the setting screen D1.

[0065] The display processing unit 216 arranges and displays multiple captured images P1 according to the setting operation on the setting screen D1. For example, when the layout of the image display area L1 shown in Figure 5A is set, the display processing unit 216 displays the camera image display area A1, which displays the captured images from each camera 15, on the driving status screen D2, which displays the status of the automatic driving of the work vehicle 10, as shown in Figure 6A. The driving status screen D2 displays the current status of the work vehicle 10 during automatic driving in real time. The operator can grasp the current driving status and work status of the work vehicle 10 on the driving status screen D2. The display processing unit 216 displays each image display area L1 in the camera image display area A1 according to the layout set on the setting screen D1. Figure 6B shows an example of the driving status screen D2 with captured images P1 displayed in each image display area L1. The captured images P1 displayed in each image display area L1 are updated in real time.

[0066] When the image display area L1 is set to the layout shown in Figure 5B, the display processing unit 216 displays each image display area L1 as shown in Figure 7A. Figure 7B shows an example of a driving status screen D2 in which the captured image P1 is displayed in each image display area L1.

[0067] If an obstacle (in this case, a person) is detected in at least one of the images captured by the five cameras 15, the operation terminal 20 performs the following processing. For example, if the detection processing unit 112 of the work vehicle 10 detects a person in an image acquired from camera 15L (camera 4), the detection processing unit 112 outputs a determination result indicating that a person has been detected and identification information of camera 15L (camera 4) corresponding to that image to the operation terminal 20. In addition, the acquisition processing unit 111 outputs the image data of the images acquired from each camera 15 to the operation terminal 20.

[0068] When the display processing unit 216 of the operating terminal 20 acquires the image data from the work vehicle 10, it displays the captured image P1 in each image display field L1. Furthermore, when the display processing unit 216 acquires the judgment result and the identification information from the work vehicle 10, it enlarges and displays the image display field L1 and the captured image P1 corresponding to camera 15L (camera 4), as shown in Figure 8A. As a result, the image of the person detected as an obstacle is displayed in an enlarged view.

[0069] In another embodiment, the display processing unit 216 may highlight and display the image display field L1 corresponding to camera 15L (camera 4) among the multiple image display fields L1. For example, as shown in Figure 8B, the display processing unit 216 displays the frame of the image display field L1 corresponding to camera 15L (camera 4) with a thick line. The display processing unit 216 may also display the image display field L1 corresponding to camera 15L (camera 4) in color, illuminate, or blink. Alternatively, the display processing unit 216 may display only the image display field L1 corresponding to camera 15L (camera 4) in camera image display field A1, and hide the image display fields L1 corresponding to the other cameras 15.

[0070] Furthermore, when the display processing unit 216 obtains a determination result indicating that a person has been detected from the work vehicle 10, it displays a driving selection screen D3 on the operation display unit 23, which allows the user to select either a driving stop instruction to stop the work vehicle 10 from moving or a driving continue instruction to continue moving. Figure 9 shows an example of the driving selection screen D3. The display processing unit 216 displays on the driving selection screen D3 a message indicating that an obstacle has been detected, a message allowing the user to select whether to stop or continue moving, a driving stop button B1 to stop moving, and a driving continue button B2 to continue moving. When the operator presses the driving stop button B1 on the driving selection screen D3, the reception processing unit 217 receives the driving stop instruction. When the reception processing unit 217 receives the driving stop instruction, the output processing unit 215 outputs the driving stop instruction to the work vehicle 10. When the driving processing unit 113 of the work vehicle 10 obtains the driving stop instruction from the operation terminal 20, it stops the movement and work of the work vehicle 10.

[0071] In response, when the operator presses the "Continue Driving" button B2 on the driving selection screen D3, the reception processing unit 217 receives the instruction to continue driving. Upon receiving the instruction to continue driving, the output processing unit 215 outputs the instruction to the work vehicle 10. When the driving processing unit 113 of the work vehicle 10 receives the instruction to continue driving from the operation terminal 20, it does not stop the driving and work of the work vehicle 10, but continues driving and working. Alternatively, the output processing unit 215 may not output the instruction to continue driving to the work vehicle 10 when the reception processing unit 217 receives the instruction to continue driving. In this case, the driving processing unit 113 does not perform the process of stopping the driving of the work vehicle 10, and as a result, driving and work continue.

[0072] Here, it is conceivable that the operator does not press either the stop-driving button B1 or the continue-driving button B2 on the driving selection screen D3. For example, the operator may not notice that an obstacle has been detected or that the driving selection screen D3 has been displayed. In this case, in order to avoid the risk of the work vehicle 10 coming into contact with the obstacle, the output processing unit 215 outputs the stop-driving instruction to the work vehicle 10. Specifically, if the reception processing unit 217 does not receive either the stop-driving instruction or the continue-driving instruction, the driving processing unit 113 stops the automatic driving of the work vehicle 10. For example, if the operator does not press either the stop-driving button B1 or the continue-driving button B2 within a predetermined time after the display processing unit 216 displays the driving selection screen D3, or after the detection processing unit 112 detects a person, the output processing unit 215 outputs the stop-driving instruction to the work vehicle 10.

[0073] Thus, when the detection processing unit 112 detects an obstacle (a person), and the reception processing unit 217 does not receive either the instruction to stop driving or the instruction to continue driving within a predetermined time, the driving processing unit 113 stops the automatic driving of the work vehicle 10. In other words, if the work vehicle 10 detects an obstacle and does not receive instructions from the operator, it stops driving and performing its work.

[0074] Here, the predetermined time may be a fixed time set in advance, or it may be a time corresponding to the size of the obstacle when the obstacle is detected. For example, if the size of the obstacle is large in the captured image, the distance between the work vehicle 10 and the obstacle will be closer compared to when the size of the obstacle in the captured image is small. Therefore, the operation control unit 21 sets the predetermined time to be shorter the larger the size of the obstacle in the captured image. Also, the operation control unit 21 sets the predetermined time to be longer the smaller the size of the obstacle in the captured image. In this way, when the distance between the work vehicle 10 and the obstacle is close, the reception time for the driving stop instruction and the driving continue instruction can be shortened, and when the distance between the work vehicle 10 and the obstacle is long, the reception time for the driving stop instruction and the driving continue instruction can be lengthened. The driving processing unit 113 may make the work vehicle 10 drive at a reduced speed during the predetermined time.

[0075] The operation control unit 21 may also notify the operator of information indicating that the work vehicle 10 has detected an obstacle. For example, the operation control unit 21 may output an audio (warning sound) when the work vehicle 10 detects an obstacle.

[0076] After the work vehicle 10 has stopped due to the aforementioned stop command, when the operator issues the restart command on the operation terminal 20, the reception processing unit 217 receives the restart command, and the output processing unit 215 outputs the restart command to the work vehicle 10. When the work vehicle 10's driving processing unit 113 receives the restart command from the operation terminal 20, it restarts the work vehicle 10's driving and work.

[0077] The operating terminal 20 may also be able to access a website (agricultural support site) for agricultural support services provided by a server (not shown) via the communication network N1. In this case, the operating terminal 20 can function as an operating terminal for the server by having a browser program executed by the operation control unit 21. The server then comprises the processing units described above and executes each of the processes.

[0078] In another embodiment, the functions of the vehicle control device 11 and the detection processing device 19 described above may be included in the operation control unit 21 of the operation terminal 20. That is, for example, the operation control unit 21 may acquire image data of captured images from each camera 15 of the work vehicle 10 and perform the process of detecting obstacles. Also, the functions of the detection processing device 19 (acquisition processing unit 111 and detection processing unit 112) may be included in the vehicle control device 11.

[0079] [Automatic driving process] Hereinafter, with reference to Figure 10, an example of the automatic driving process performed by the vehicle control device 11, the detection processing device 19, and the operation control unit 21 will be described. For example, the automatic driving process is started by the vehicle control device 11, the detection processing device 19, and the operation control unit 21 when the work vehicle 10 starts automatic driving.

[0080] Furthermore, the present invention may be considered as an invention of an automatic driving method in which a vehicle control device 11, a detection processing device 19, and an operation control unit 21 execute part or all of the automatic driving process, or as an invention of an automatic driving program that causes the vehicle control device 11, the detection processing device 19, and the operation control unit 21 to execute part or all of the automatic driving method. In addition, the automatic driving process may be executed by one or more processors.

[0081] In step S1, the vehicle control device 11 instructs the work vehicle 10 to start work. For example, when an operator presses the work start button on the operation screen of the operation terminal 20, the operation control unit 21 outputs a work start instruction to the work vehicle 10. When the vehicle control device 11 receives the work start instruction from the operation terminal 20, it starts the work vehicle 10 to travel automatically. As a result, the work vehicle 10 starts traveling automatically according to the travel path R and begins work with the work machine 14.

[0082] Next, in step S2, the detection processing device 19 acquires captured images from each camera 15 installed on the work vehicle 10. For example, the detection processing device 19 acquires captured images (frame images) continuously from each of the five cameras 15F, 15B, 15R, 15L, and 15C at a predetermined frame rate.

[0083] Next, in step S3, the detection processing device 19 detects an obstacle (e.g., a person) in the acquired image. Specifically, the detection processing device 19 determines whether or not a person is included in the detection area based on the image. If the detection processing device 19 detects a person (S3: Yes), the process proceeds to step S4. On the other hand, if the detection processing device 19 does not detect a person (S3: No), the process proceeds to step S31.

[0084] In step S31, the vehicle control device 11 determines whether the work vehicle 10 has completed its work. If the work vehicle 10 has completed its work (S31: Yes), the process ends. On the other hand, if the work vehicle 10 has not completed its work (S31: No), the process returns to step S2. In this way, if the detection processing device 19 does not detect a person, it continues to drive and perform the work while acquiring captured images until the predetermined work is completed.

[0085] Furthermore, the detection processing device 19 outputs the image data of the acquired image, the identification information of the camera 15 corresponding to the image, and a determination result indicating whether or not a human is included in the image to the operation terminal 20. When the operation control unit 21 acquires the image data from the detection processing device 19 of the work vehicle 10, it displays the image P1 in each image display field L1 (see Figure 6B).

[0086] In step S4, the operation control unit 21 enlarges and displays the captured image including the person. For example, if a person is detected in the captured image corresponding to camera 15L (camera 4), the operation control unit 21 enlarges the image display area L1 corresponding to camera 15L (camera 4) and displays the captured image P1 enlarged, as shown in Figure 8A. As a result, the image of the person detected as an obstacle is displayed enlarged.

[0087] Next, in step S5, the operation control unit 21 displays a driving selection screen D3 on the operation display unit 23, which allows the user to select either a driving stop instruction to stop the work vehicle 10 from moving or a driving continue instruction to continue moving (see Figure 9).

[0088] Next, in step S6, the operation control unit 21 determines whether or not it has received the command to stop driving from the operator on the driving selection screen D3. For example, if the operator presses the stop driving button B1 on the driving selection screen D3 shown in Figure 9, the operation control unit 21 receives the command to stop driving (S6: Yes), and the process proceeds to step S9. On the other hand, if the operator does not press the stop driving button B1 on the driving selection screen D3 shown in Figure 9 (S6: No), the process proceeds to step S7.

[0089] Next, in step S7, the operation control unit 21 determines whether or not it has received the instruction to continue driving from the operator on the driving selection screen D3. For example, if the operator presses the continue driving button B2 on the driving selection screen D3 shown in Figure 9, the operation control unit 21 receives the instruction to continue driving (S7: Yes), and the process proceeds to step S71. On the other hand, if the operator does not press the continue driving button B2 on the driving selection screen D3 shown in Figure 9 (S7: No), the process proceeds to step S8.

[0090] In step S71, the operation control unit 21 does not stop the movement and operation of the work vehicle 10, but allows it to continue moving and working. After that, the process returns to step S2.

[0091] In step S8, the operation control unit 21 determines whether a predetermined time has elapsed. Specifically, the operation control unit 21 determines whether a predetermined time has elapsed since the operation control unit 21 displayed the driving selection screen D3 or since the detection processing device 19 detected a person, without receiving either the driving stop instruction or the driving continue instruction. If the predetermined time has elapsed without receiving either the driving stop instruction or the driving continue instruction (S8: Yes), the process proceeds to step S9. If the predetermined time has not elapsed (S8: No), the process returns to step S6.

[0092] In step S9, the operation control unit 21 outputs the stop-travel instruction to the work vehicle 10, and the vehicle control device 11 stops the movement and operation of the work vehicle 10.

[0093] In step S10, the operation control unit 21 determines whether or not it has received a command from the operator to resume driving. If the operation control unit 21 receives the command to resume driving from the operator (S10: Yes), the process returns to step S2. On the other hand, if the operation control unit 21 does not receive the command to resume driving from the operator (S10: No), the process ends. The vehicle control device 11, the detection processing device 19, and the operation control unit 21 repeatedly execute the processes from steps S1 to S10 until the work vehicle 10 completes its work.

[0094] As described above, the automated driving system 1 according to this embodiment acquires captured images P1 from a camera 15 installed on the work vehicle 10 and detects obstacles based on the captured images P1. When an obstacle is detected, the automated driving system 1 receives either a stop instruction to stop the automated driving of the work vehicle 10 or a continue instruction to continue the automated driving. The automated driving system 1 stops the automated driving of the work vehicle 10 when it receives a stop instruction, and continues the automated driving of the work vehicle 10 when it receives a continue instruction. This allows the operator to choose whether to stop or continue the automated driving when the work vehicle 10 detects an obstacle while it is automated. For example, if the obstacle does not obstruct the automated driving, the operator can give an instruction to continue the automated driving, and the work vehicle 10 will continue to drive without stopping, thus preventing a decrease in work efficiency. Also, if the obstacle does obstruct the automated driving, the operator can stop the automated driving and remove the obstacle to ensure safety. The operator can then resume automated driving after ensuring the safety of the work vehicle 10. Therefore, it is possible to ensure the safety of the work vehicle 10 while preventing a decrease in work efficiency.

[0095] Furthermore, if the work vehicle 10 detects an obstacle while automatically driving, and the operator does not perform either an operation to stop or continue the automatic driving within a predetermined time, the automatic driving system 1 will stop the automatic driving. This prevents a situation where the work vehicle 10 continues to drive automatically and comes into contact with an obstacle without the operator noticing.

[0096] The present invention is not limited to the embodiments described above, and may also be subject to the following embodiments.

[0097] In the above-described embodiment, when an obstacle (person) is detected by the detection processing unit 112, if the reception processing unit 217 does not receive either the instruction to stop driving or the instruction to continue driving within a predetermined time, the driving processing unit 113 stops the automatic driving of the work vehicle 10. In another embodiment, when an obstacle is detected by the detection processing unit 112, if the reception processing unit 217 does not receive either the instruction to stop driving or the instruction to continue driving within a predetermined distance between the work vehicle 10 and the obstacle, the driving processing unit 113 may stop the automatic driving of the work vehicle 10.

[0098] The operation control unit 21 may calculate the change in distance between the work vehicle 10 and the obstacle based on a series of frame images acquired from the work vehicle 10. For example, as shown in Figure 11, the operation control unit 21 can calculate the distance based on the size of the frame W1 surrounding the person in the captured image P1.

[0099] Here, the predetermined distance may be a predetermined fixed distance, or it may be a distance corresponding to the size of the obstacle when the obstacle is detected. For example, if the size of the obstacle is large in the captured image, the distance between the work vehicle 10 and the obstacle will be closer compared to when the size of the obstacle in the captured image is small. Therefore, the operation control unit 21 sets the predetermined distance to be shorter the larger the size of the obstacle in the captured image. Conversely, the operation control unit 21 sets the predetermined distance to be longer the smaller the size of the obstacle in the captured image. This allows the reception time for the stop-driving instruction and the continue-driving instruction to be shortened when the distance between the work vehicle 10 and the obstacle is short, and the reception time for the stop-driving instruction and the continue-driving instruction to be lengthened when the distance between the work vehicle 10 and the obstacle is long. The driving processing unit 113 may also make the work vehicle 10 slow down during the predetermined distance.

[0100] In another embodiment, for example, if the detection processing unit 112 detects a specific person and the operator gives an instruction to continue driving, and the detection processing unit 112 subsequently detects the same person again, the work vehicle 10 may continue driving automatically without obtaining an instruction from the operator to continue driving. This improves convenience because, for example, each time the same obstacle that does not hinder automatic driving is detected, it is no longer necessary to ask the operator to select an instruction to stop driving or to continue driving.

[0101] Furthermore, in this embodiment, obstacle detection and determination were performed based on images captured by a camera, but in other embodiments, a camera and a LiDAR (obstacle detection sensor) may be used in combination. Specifically, obstacle detection may be performed by LiDAR and obstacle determination by the camera. The obstacle detection sensor may be a sensor that utilizes ultrasound or the like.

[0102] [Notes on the invention] <Note 1> An acquisition processing unit that acquires captured images from an imaging unit installed on a work vehicle, A detection processing unit that detects obstacles based on the captured image acquired by the acquisition processing unit, When the detection processing unit detects the obstacle, a receiving processing unit receives a stop instruction to stop the automatic driving of the work vehicle or a continue driving instruction to continue the automatic driving; A driving processing unit which stops the automatic driving of the work vehicle when the reception processing unit receives the instruction to stop driving, and continues the automatic driving of the work vehicle when the reception processing unit receives the instruction to continue driving, An automated driving system equipped with [the following features].

[0103] <Note 2> If the reception processing unit does not receive either the instruction to stop driving or the instruction to continue driving, the driving processing unit will stop the automatic driving of the work vehicle. The automated driving system described in Appendix 1.

[0104] <Note 3> If the detection processing unit detects an obstacle, and the reception processing unit does not receive either the instruction to stop driving or the instruction to continue driving within a predetermined time, the driving processing unit will stop the automatic driving of the work vehicle. The automated driving system described in Appendix 2.

[0105] <Note 4> The predetermined time is set to be shorter the larger the size of the obstacle included in the captured image, and longer the smaller the size of the obstacle included in the captured image. The automated driving system described in Appendix 3.

[0106] <Note 5> When the detection processing unit detects the obstacle, and the reception processing unit does not receive either the instruction to stop driving or the instruction to continue driving before the distance between the work vehicle and the obstacle reaches a predetermined distance, the driving processing unit stops the automatic driving of the work vehicle. The automated driving system described in Appendix 2.

[0107] <Note 6> The predetermined distance is set to be shorter the larger the size of the obstacle included in the captured image, and longer the smaller the size of the obstacle included in the captured image. The automated driving system described in Appendix 5.

[0108] <Note 7> Multiple imaging units are installed at different locations on the work vehicle. The system further includes a display processing unit that displays a plurality of captured images corresponding to a plurality of imaging units acquired by the acquisition processing unit on an operating terminal, in an orderly fashion. The display processing unit highlights and displays the image in which the obstacle is detected among a plurality of captured images. An automated driving system as described in any of the appendices 1 to 6.

[0109] <Note 8> The reception processing unit further receives a setting operation to set the layout of the multiple captured images displayed on the operation terminal. The display processing unit arranges and displays a plurality of the captured images in the operating terminal according to the setting operation. The automated driving system described in Appendix 7.

[0110] <Note 9> One or more processors Acquiring images from an imaging unit installed on a work vehicle, The acquisition of the captured image is used to detect obstacles, When the aforementioned obstacle is detected, the system receives a stop command to stop the automatic driving of the work vehicle or a continue command to continue the automatic driving, When the aforementioned instruction to stop driving is received, the automatic driving of the work vehicle is stopped, and when the aforementioned instruction to continue driving is received, the automatic driving of the work vehicle is continued. An automated driving system that performs this task.

[0111] <Note 10> Acquiring images from an imaging unit installed on a work vehicle, The acquisition of the captured image is used to detect obstacles, When the aforementioned obstacle is detected, the system receives a stop command to stop the automatic driving of the work vehicle or a continue command to continue the automatic driving, When the aforementioned instruction to stop driving is received, the automatic driving of the work vehicle is stopped, and when the aforementioned instruction to continue driving is received, the automatic driving of the work vehicle is continued. An automated driving program that causes one or more processors to execute. [Explanation of Symbols]

[0112] 1: Automated driving system 10: Work vehicles 11: Vehicle control system 13: Running gear 14: Work Machines 15: Camera (imaging unit) 17: Positioning device 19: Detection and processing device 20: Operating terminal 21: Operation Control Unit 23: Operation display section 111: Acquisition Processing Unit 112: Detection Processing Unit 113: Driving section 211: Vehicle setting processing unit 212: Field setting processing unit 213: Work Setting Processing Unit 214: Route generation processing unit 215: Output Processing Unit 216: Display Processing Unit 217: Reception Processing Section

Claims

1. A detection processing unit that detects obstacles using an obstacle detection device installed on a work vehicle, When the detection processing unit detects the obstacle, a display processing unit displays a selection screen on the operating terminal that accepts either a stop command to stop the automatic driving of the work vehicle or a continue command to continue the automatic driving; A reception processing unit that receives an operation from the operator to select either the instruction to stop driving or the instruction to continue driving on the selection screen, A driving processing unit which stops the automatic driving of the work vehicle when the reception processing unit receives the instruction to stop driving, and continues the automatic driving of the work vehicle when the reception processing unit receives the instruction to continue driving, An automated driving system equipped with [the following features].

2. If the reception processing unit does not receive either the instruction to stop driving or the instruction to continue driving, the driving processing unit will stop the automatic driving of the work vehicle. The automated driving system according to claim 1.

3. When the detection processing unit detects an obstacle, and the reception processing unit does not receive either the instruction to stop driving or the instruction to continue driving from the operator on the selection screen within a predetermined time, the driving processing unit stops the automatic driving of the work vehicle. The automatic driving system according to claim 2.

4. When the detection processing unit detects the obstacle, and the receiving processing unit does not receive either the instruction to stop driving or the instruction to continue driving from the operator on the selection screen until the distance between the work vehicle and the obstacle reaches a predetermined distance, the driving processing unit stops the automatic driving of the work vehicle. The automatic driving system according to claim 2.

5. One or more processors, Obstacles are detected by an obstacle detection device installed on the work vehicle, When the aforementioned obstacle is detected, a selection screen is displayed on the operating terminal that accepts either a stop command to stop the automatic movement of the work vehicle or a continue command to continue the automatic movement. The selection screen accepts an operation from the operator to select either the instruction to stop driving or the instruction to continue driving, When the aforementioned instruction to stop driving is received, the automatic driving of the work vehicle is stopped, and when the aforementioned instruction to continue driving is received, the automatic driving of the work vehicle is continued. An automated driving method that performs this task.

6. Obstacles are detected by an obstacle detection device installed on the work vehicle, When the aforementioned obstacle is detected, a selection screen is displayed on the operating terminal that accepts either a stop command to stop the automatic movement of the work vehicle or a continue command to continue the automatic movement. The selection screen accepts an operation from the operator to select either the instruction to stop driving or the instruction to continue driving, When the aforementioned instruction to stop driving is received, the automatic driving of the work vehicle is stopped, and when the aforementioned instruction to continue driving is received, the automatic driving of the work vehicle is continued. An automated driving program that causes one or more processors to execute.

Images