Autonomous driving system, autonomous driving method, and autonomous driving program

Abstract

To provide a self-driving system, a self-driving method, and a self-driving program capable of preventing a decrease in work efficiency while securing safety of a work vehicle.SOLUTION: An acquisition processing section 111 acquires an image P1 from a camera 15 installed on a work vehicle 10. A detection processing section 112 detects an obstacle on the basis of the image P1 acquired by the acquisition processing section 111. A reception processing section 217 receives a driving stop instruction to stop self-driving of the work vehicle 10 or a driving continuity instruction to allow the self-driving to continuously perform self-driving when the detection processing section 112 detects the obstacle. A driving processing section 113 stops the self-driving of the work vehicle 10 when the reception processing section 217 receives the driving stop instruction and allows the work vehicle 10 to continue the self-driving when the reception processing section 217 receives the driving continuity instruction.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an automatic driving system, an automatic driving method, and an automatic driving program for automatically driving a work vehicle. [Background technology]

[0002] A work vehicle has a function to stop automatic travel if an obstacle is detected while the work vehicle is automatically traveling along a predetermined travel route in a field. Conventionally, there is known technology that allows the operator of the work vehicle to switch between a normal mode, in which the automatic travel of the work vehicle is stopped when the work vehicle detects an obstacle, and a deactivation mode, in which the work vehicle continues traveling even if the work vehicle detects an obstacle (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-113938 Summary of the Invention [Problem to be solved by the invention]

[0004] However, with the conventional technology, when the deactivation mode is set, the work vehicle continues to travel automatically even if it detects an obstacle, which poses a risk of the work vehicle coming into contact with the obstacle. In this way, a system that uniformly stops the work vehicle's automatic travel when the work vehicle detects an obstacle reduces work efficiency, while a system that allows the work vehicle to continue traveling automatically even when it detects an obstacle poses a problem of reducing the safety of the work vehicle.

[0005] An object of the present invention is to provide an automatic driving system, an automatic driving method, and an automatic driving program that can prevent a decrease in work efficiency while ensuring the safety of a work vehicle. [Means for solving the problem]

[0006] The automated driving system according to the present invention comprises an acquisition processing unit, a detection processing unit, a reception processing unit, and a driving processing unit. The acquisition processing unit acquires captured images from an imaging unit installed in a work vehicle. The detection processing unit detects an obstacle based on the captured images acquired by the acquisition processing unit. When the detection processing unit detects the obstacle, the reception processing unit accepts a stop driving instruction to stop the automated driving of the work vehicle or a continue driving instruction to continue the automated driving. When the reception processing unit accepts the stop driving instruction, the driving processing unit stops the automated driving of the work vehicle, and when the reception processing unit accepts the continue driving instruction, the driving processing unit continues the automated driving of the work vehicle.

[0007] The automatic driving method of the present invention is a method in which one or more processors acquire an image from an imaging unit installed in a work vehicle, detect an obstacle based on the acquired image, and when the obstacle is detected, accept a stop driving instruction to stop the automatic driving of the work vehicle or a continue driving instruction to continue the automatic driving, and stop the automatic driving of the work vehicle when the stop driving instruction is accepted, or continue the automatic driving of the work vehicle when the continue driving instruction is accepted.

[0008] The automatic driving program of the present invention is a program that causes one or more processors to execute the following steps: acquire an image from an imaging unit installed in a work vehicle; detect an obstacle based on the acquired image; when the obstacle is detected, accept a stop driving instruction to stop the automatic driving of the work vehicle or a continue driving instruction to continue the automatic driving; and stop the automatic driving of the work vehicle when the stop driving instruction is accepted, and continue the automatic driving of the work vehicle when the continue driving instruction is accepted. [Effects of the Invention]

[0009] According to the present invention, it is possible to provide an automatic driving system, an automatic driving method, and an automatic driving program that can prevent a decrease in work efficiency while ensuring the safety of a work vehicle. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a block diagram showing the configuration of an automatic driving system according to an embodiment of the present invention. [Figure 2] FIG. 2 is an external view showing an example of a work vehicle according to an embodiment of the present invention. [Figure 3] FIG. 3 is a diagram showing an example of a travel route of a work vehicle according to an embodiment of the present invention. [Figure 4] FIG. 4 is a diagram showing the positions of cameras installed in a work vehicle according to an embodiment of the present invention. [Figure 5A] FIG. 5A is a diagram showing an example of a camera layout setting screen displayed on the operation terminal according to the embodiment of the present invention. [Figure 5B] FIG. 5B is a diagram showing an example of a camera layout setting screen displayed on the operation terminal according to the embodiment of the present invention. [Figure 6A] FIG. 6A is a diagram showing an example of a camera image displayed on the operation terminal according to the embodiment of the present invention. [Figure 6B] FIG. 6B is a diagram showing an example of a camera image displayed on the operation terminal according to the embodiment of the present invention. [Figure 7A] FIG. 7A is a diagram showing an example of a camera image displayed on the operation terminal according to the embodiment of the present invention. [Figure 7B] FIG. 7B is a diagram showing an example of a camera image displayed on the operation terminal according to the embodiment of the present invention. [Figure 8A] FIG. 8A is a diagram showing an example of a camera image displayed on the operation terminal according to the embodiment of the present invention. [Figure 8B] FIG. 8B is a diagram showing an example of a camera image displayed on the operation terminal according to the embodiment of the present invention. [Figure 9]FIG. 9 is a diagram showing an example of a travel selection screen displayed on the operation terminal according to the embodiment of the present invention. [Figure 10] FIG. 10 is a flowchart showing an example of the procedure of the automatic driving process executed by the automatic driving system according to the embodiment of the present invention. [Figure 11] FIG. 11 is a diagram showing an example of a travel selection screen displayed on the operation terminal according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0011] The following embodiment is an example of the present invention and does not limit the technical scope of the present invention.

[0012] 1, an automated driving system 1 according to an embodiment of the present invention includes a work vehicle 10 and an operation terminal 20. The work vehicle 10 and the operation terminal 20 can communicate with each other via a communication network N1. For example, the work vehicle 10 and the operation terminal 20 can communicate with each other via a mobile phone network, a packet network, or a wireless LAN.

[0013] In this embodiment, the work vehicle 10 will be described as a tractor. In other embodiments, the work vehicle 10 may be a rice transplanter, a combine harvester, construction machinery, a snowplow, or the like. The work vehicle 10 is a so-called robot tractor that is configured to be able to automatically travel (autonomously travel) within a field F (see FIG. 3 ) along a predetermined travel route R. For example, the work vehicle 10 can automatically travel along a travel route R that has been generated in advance for the field F, based on position information of the current position of the work vehicle 10 calculated by a positioning device 17.

[0014] For example, the work vehicle 10 travels back and forth in parallel from a work start position S to a work end position G in the work area of the field F shown in Fig. 3. The outer periphery of the field F is, for example, a headland area, and the work vehicle 10 travels in a turning motion. The travel route R is not limited to the route shown in Fig. 3, but is set appropriately depending on the work content.

[0015] [Work vehicle 10] 1 and 2, the work vehicle 10 includes a vehicle control device 11, a memory unit 12, a traveling device 13, a work implement 14, a camera 15, a communication unit 16, a positioning device 17, and a detection processing device 19. The vehicle control device 11 is electrically connected to the memory unit 12, the traveling device 13, the work implement 14, the positioning device 17, and the detection processing device 19. The vehicle control device 11 and the positioning device 17 may be capable of wireless communication. The camera 15 is electrically connected to the detection processing device 19.

[0016] The storage unit 12 is a non-volatile storage unit such as an HDD (Hard Disk Drive) or SSD (Solid State Drive) that stores various types of information. The storage unit 12 stores control programs, such as an automatic driving program for causing the vehicle control device 11 to execute the automatic driving process (see FIG. 10 ), which will be described later. For example, the automatic driving program is non-temporarily recorded on a computer-readable recording medium such as a CD or DVD, and is read by a predetermined reading device (not shown) and stored in the storage unit 12. The automatic driving program may be downloaded from a server (not shown) to the work vehicle 10 via a communication network N1 and stored in the storage unit 12. The storage unit 12 also stores data on the travel route R generated by the operation terminal 20, determination data for determining obstacles, and the like. The determination data is data for determining a human, which is an example of an obstacle, and may be, for example, data indicating human characteristics, or data of an image (sample image) showing parts or the entire human body. The determination data may be stored in the detection processing device 19. Additionally, work information (turning patterns, work sequences, etc.) may be stored in memory unit 12. Note that an obstacle in the present invention is an object that impedes the travel of work vehicle 10, such as a human being, an animal, or equipment (another work vehicle, etc.).

[0017] The traveling device 13 is a drive unit that causes the work vehicle 10 to travel. As shown in Fig. 2, the traveling device 13 is equipped with an engine 131, front wheels 132, rear wheels 133, a transmission 134, a front axle 135, a rear axle 136, a handlebar 137, etc. The front wheels 132 and rear wheels 133 are provided on the left and right sides of the work vehicle 10, respectively. The traveling device 13 is not limited to a wheel type equipped with front wheels 132 and rear wheels 133, but may also be a crawler type equipped with crawlers provided on the left and right sides of the work vehicle 10.

[0018] The engine 131 is a drive source such as a diesel engine or a gasoline engine that is driven by fuel supplied to a fuel tank (not shown). The traveling device 13 may be equipped with an electric motor as a drive source in addition to or instead of the engine 131. A generator (not shown) is connected to the engine 131, and power is supplied from the generator to electrical components such as the vehicle control device 11 and the battery provided on the work vehicle 10. The battery is charged with power supplied from the generator. The vehicle control device 11, positioning device 17, and other electrical components provided on the work vehicle 10 can be driven by power supplied from the battery even after the engine 131 is stopped.

[0019] The driving force of the engine 131 is transmitted to the front wheels 132 via the transmission 134 and the front axle 135, and to the rear wheels 133 via the transmission 134 and the rear axle 136. The driving force of the engine 131 is also transmitted to the work implement 14 via a PTO shaft (not shown). When the work vehicle 10 performs autonomous driving, the traveling device 13 performs traveling operations in accordance with commands from the vehicle control device 11.

[0020] The work implement 14 is, for example, a brush cutter, cultivator, plow, fertilizer applicator, or seed sowing machine, and is detachable from the work vehicle 10. This allows the work vehicle 10 to perform various tasks using each of the work implements 14. In this embodiment, the work implement 14 will be described as a brush cutter.

[0021] For example, the work implement 14 is mounted offset to either the left or right side of the work vehicle 10. For example, the work vehicle 10 is mounted with a directly mounted work implement 14 offset to either the left or right side and travels within a field to perform work such as mowing. Note that the work implement 14 is not limited to a directly mounted work implement fixed to the work vehicle 10 (see FIG. 2 ), but may also be a towed work implement that is towed by the work vehicle 10.

[0022] Camera 15 is a digital camera that captures images of a subject and outputs them as digital image data. Camera 15 continuously captures images of the subject at a predetermined frame rate, generates frame images with a predetermined resolution, and sequentially transmits them to detection processing device 19. Camera 15 is an example of the imaging unit of the present invention.

[0023] In this embodiment, five cameras 15 are installed in different locations on the work vehicle 10. Specifically, as shown in Fig. 4, camera 15F (hereinafter also referred to as "camera 1") is installed at the front of the work vehicle 10, camera 15B (hereinafter also referred to as "camera 2") is installed at the rear of the work vehicle 10, camera 15R (hereinafter also referred to as "camera 3") is installed at the right side of the work vehicle 10, camera 15L (hereinafter also referred to as "camera 4") is installed at the left side of the work vehicle 10, and camera 15C (hereinafter also referred to as "camera 5") is installed at the front right of the work implement 14. The cameras 15 may also be installed on the work vehicle 10 using, for example, double-sided tape.

[0024] Each camera 15 is also set with a predetermined imaging range (detection area) that it can capture. For example, camera 15F images detection area K1 in front of the work vehicle 10, camera 15B images detection area K2 behind the work vehicle 10, camera 15R images detection area K3 on the right side of the work vehicle 10, camera 15L images detection area K4 on the left side of the work vehicle 10, and camera 15C images detection area K5 in front of the right side of the work implement 14. Each camera 15 captures images of its respective detection area at a predetermined frame rate and sequentially transmits the captured images to the detection processing device 19. The detection processing device 19 transmits the captured images and detection results (determination results) described below to the vehicle control device 11 and the operation terminal 20.

[0025] The detection processing device 19 is able to communicate with the cameras 15 when the cameras 15 are electrically connected to the work vehicle 10. When the detection processing device 19 is able to communicate with the cameras 15, it acquires the number of cameras 15 and the identification information (device information) of each camera 15. The detection processing device 19 also outputs the acquired number information and identification information of the cameras 15 to the vehicle control device 11, and the vehicle control device 11 outputs the number information and identification information to the operation terminal 20. The operator can add cameras 15 or change their installation positions.

[0026] The handle 137 is an operating unit that is operated by an operator or the vehicle control device 11. For example, in the traveling device 13, the angle of the front wheels 132 is changed by a hydraulic power steering mechanism (not shown) or the like in response to operation of the handle 137 by the vehicle control device 11, and the traveling direction of the work vehicle 10 is changed.

[0027] In addition to the handlebars 137, the traveling device 13 is also equipped with a shift lever, accelerator, brake, etc. (not shown) that are operated by the vehicle control device 11. In the traveling device 13, the gear of the transmission 134 is switched to a forward gear, a reverse gear, etc. in response to operation of the shift lever by the vehicle control device 11, and the traveling mode of the work vehicle 10 is switched to forward, reverse, etc. The vehicle control device 11 also operates the accelerator to control the rotation speed of the engine 131. The vehicle control device 11 also operates the brake to brake the rotation of the front wheels 132 and rear wheels 133 using an electromagnetic brake.

[0028] The positioning device 17 is a communication device including a positioning control unit 171, a memory unit 172, a communication unit 173, a positioning antenna 174, and the like. For example, as shown in FIG. 2 , the positioning device 17 is provided above the cabin 18 in which the operator sits. The installation location of the positioning device 17 is not limited to the cabin 18. The positioning control unit 171, the memory unit 172, the communication unit 173, and the positioning antenna 174 of the positioning device 17 may be disposed in different locations in the work vehicle 10. As described above, the battery is connected to the positioning device 17, and the positioning device 17 can operate even when the engine 131 is stopped. The positioning device 17 may be substituted with, for example, a mobile phone terminal, a smartphone, or a tablet terminal.

[0029] The positioning control unit 171 is a computer system including one or more processors and storage memories such as nonvolatile memory and RAM. The storage unit 172 is a nonvolatile memory that stores a program for causing the positioning control unit 171 to execute the positioning process, and data such as positioning information and movement information. For example, the program is non-temporarily recorded on a computer-readable recording medium such as a CD or DVD, and is read by a predetermined reading device (not shown) and stored in the storage unit 172. Note that the program may be downloaded to the positioning device 17 from a server (not shown) via the communication network N1 and stored in the storage unit 172.

[0030] The communication unit 173 is a communication interface that connects the positioning device 17 to the communication network N1 by wire or wirelessly and performs data communication with external devices such as a base station server via the communication network N1 in accordance with a predetermined communication protocol.

[0031] The positioning antenna 174 is an antenna that receives radio waves (GNSS signals) transmitted from satellites.

[0032] The positioning control unit 171 calculates the current position of the work vehicle 10 based on the GNSS signals received from satellites by the positioning antenna 174. For example, when the work vehicle 10 is autonomously traveling within a field F, the positioning antenna 174 receives radio waves (such as transmission time and orbit information) transmitted from each of a plurality of satellites, and the positioning control unit 171 calculates the distance between the positioning antenna 174 and each satellite, and calculates the current position (latitude and longitude) of the work vehicle 10 based on the calculated distance. The positioning control unit 171 may also perform positioning using a real-time kinematic method (RTK-GPS positioning method (RTK method)), which calculates the current position of the work vehicle 10 using correction information corresponding to a base station (reference station) close to the work vehicle 10. In this way, the work vehicle 10 performs autonomous traveling using positioning information obtained by the RTK method.

[0033] The vehicle control device 11 and the detection processing device 19 have control devices such as a CPU, ROM, and RAM. The CPU is a processor that executes various types of arithmetic processing. The ROM is a non-volatile storage unit in which control programs such as a BIOS and an OS that cause the CPU to execute various types of arithmetic processing are stored in advance. The RAM is a volatile or non-volatile storage unit that stores various types of information and is used as temporary storage memory (work area) for the various types of processing executed by the CPU. The vehicle control device 11 and the detection processing device 19 control the work vehicle 10 by the CPU executing various control programs that are stored in advance in the ROM or the storage unit 12.

[0034] The detection processing device 19 acquires a captured image from the camera 15 and determines whether or not an obstacle (e.g., a human) is included in the detection area based on the captured image. Specifically, as shown in FIG. 1, the detection processing device 19 includes various processing units such as an acquisition processing unit 111 and a detection processing unit 112. The detection processing device 19 functions as the various processing units by executing various processes in accordance with the automatic driving program using a CPU. Furthermore, some or all of the processing units may be configured with electronic circuits. The automatic driving program may be a program for causing multiple processors to function as the processing units.

[0035] The acquisition processing unit 111 acquires captured images from one or more cameras 15. For example, the acquisition processing unit 111 acquires captured images of each of the detection areas K1 to K5 in sequence, frame by frame, from each of the five cameras 15F, 15B, 15R, 15L, and 15C installed on the work vehicle 10 and the work implement 14. The acquisition processing unit 111 stores the acquired captured images in the memory unit 12 together with the time of capture.

[0036] The acquisition processing unit 111 also outputs the acquired image data of the captured image to the operation terminal 20. The acquisition processing unit 111 is an example of the acquisition processing unit of the present invention.

[0037] The detection processing unit 112 detects an obstacle (e.g., a human) in the captured image acquired by the acquisition processing unit 111. Specifically, the detection processing unit 112 determines whether or not a human is included in the detection area based on the captured image. For example, the detection processing unit 112 performs image analysis on the captured image and compares it with the determination data stored in the storage unit 12 or the detection processing device 19 to determine the presence or absence of a human in the detection area. The detection processing unit 112 also performs a determination process for each captured image acquired sequentially by the acquisition processing unit 111. For example, the detection processing unit 112 performs the determination process on the image captured by camera 15F, then performs the determination process on the image captured by camera 15B, then performs the determination process on the image captured by camera 15R, then performs the determination process on the image captured by camera 15L, and then performs the determination process on the image captured by camera 15C. The detection processing unit 112 transmits the detection result (determination result) to the operation terminal 20. The detection processing unit 112 is an example of a detection processing unit of the present invention.

[0038] As shown in Fig. 1, the vehicle control device 11 includes various processing units such as a driving processing unit 113. The vehicle control device 11 functions as the various processing units by executing various processes in accordance with the automatic driving program using the CPU. Some or all of the processing units may be configured with electronic circuits. The automatic driving program may be a program that causes multiple processors to function as the processing units.

[0039] The driving processing unit 113 controls the driving of the work vehicle 10. Specifically, the driving processing unit 113 starts the automatic driving of the work vehicle 10 when it receives a work start instruction from the operation terminal 20. For example, when the operator presses the work start button on the operation screen of the operation terminal 20, the operation terminal 20 outputs a work start instruction to the work vehicle 10. When the driving processing unit 113 receives a work start instruction from the operation terminal 20, it starts the automatic driving of the work vehicle 10. As a result, the work vehicle 10 starts automatic driving according to the driving route R and starts work by the work implement 14. Note that the driving route R along which the work vehicle 10 travels is generated, for example, by the operation terminal 20. The work vehicle 10 receives the driving route R from the operation terminal 20 and automatically drives within the field F according to the driving route R.

[0040] Furthermore, the driving processing unit 113 stops the automatic driving of the work vehicle 10 when it receives a stop driving instruction from the operation terminal 20. For example, when the operator presses the stop driving button on the operation screen of the operation terminal 20, the operation terminal 20 outputs a stop driving instruction to the work vehicle 10. Also, for example, when the detection processing unit 112 detects a human being, if the operator does not issue either a stop driving instruction or a continue driving instruction on the operation screen of the operation terminal 20 for a predetermined time, the operation terminal 20 outputs a stop driving instruction to the work vehicle 10. When the driving processing unit 113 receives a stop driving instruction from the operation terminal 20, it stops the automatic driving of the work vehicle 10. As a result, the work vehicle 10 stops automatic driving and stops work by the work equipment 14. Specific examples of stop driving instructions and continue driving instructions will be described later.

[0041] Furthermore, the driving processing unit 113 resumes the automatic driving of the work vehicle 10 when it receives a driving restart instruction from the operation terminal 20. For example, if the detection processing unit 112 detects a human and the work vehicle 10 stops driving, and then the operator presses the driving restart button on the operation screen of the operation terminal 20, the operation terminal 20 outputs a driving restart instruction to the work vehicle 10. When the driving processing unit 113 receives the driving restart instruction from the operation terminal 20, it resumes the automatic driving of the work vehicle 10. As a result, the work vehicle 10 resumes automatic driving again according to the driving route R. The driving processing unit 113 is an example of a driving processing unit of the present invention.

[0042] [Operation terminal 20] 1, the operation terminal 20 is an information processing device including an operation control unit 21, a storage unit 22, an operation display unit 23, and a communication unit 24. The operation terminal 20 may be configured as a mobile terminal such as a tablet terminal or a smartphone.

[0043] The communication unit 24 is a communication interface that connects the operation terminal 20 to the communication network N1 via a wired or wireless connection and performs data communication in accordance with a predetermined communication protocol with external devices such as one or more work vehicles 10 via the communication network N1.

[0044] The operation display unit 23 is a user interface equipped with a display unit such as a liquid crystal display or organic EL display that displays various information, and an operation unit such as a touch panel, mouse, or keyboard that accepts operations. The operator can operate the operation unit to register various information (such as work vehicle information, field information, and work information, which will be described later) on the operation screen displayed on the display unit. The operator can also operate the operation unit to issue instructions to the work vehicle 10 to start work, stop traveling, resume traveling, and so on. Furthermore, from a location away from the work vehicle 10, the operator can grasp the traveling status of the work vehicle 10, which is automatically traveling along the traveling route R within the field F, by looking at the traveling trajectory displayed on the operation terminal 20.

[0045] The storage unit 22 is a non-volatile storage unit such as an HDD or SSD that stores various types of information. The storage unit 22 stores control programs such as an automatic driving program for causing the operation control unit 21 to execute the automatic driving process (see FIG. 10 ), which will be described later. For example, the automatic driving program is non-temporarily recorded on a computer-readable recording medium such as a CD or DVD, and is read by a predetermined reading device (not shown) and stored in the storage unit 22. The automatic driving program may be downloaded to the operation terminal 20 from a server (not shown) via the communication network N1 and stored in the storage unit 22.

[0046] The operation control unit 21 has control devices such as a CPU, a ROM, and a RAM. The CPU is a processor that executes various types of arithmetic processing. The ROM is a non-volatile storage unit in which control programs such as a BIOS and an OS that cause the CPU to execute various types of arithmetic processing are stored in advance. The RAM is a volatile or non-volatile storage unit that stores various types of information and is used as a temporary storage memory (work area) for the various types of processing executed by the CPU. The operation control unit 21 controls the operation terminal 20 by having the CPU execute various control programs that are stored in advance in the ROM or the storage unit 22.

[0047] As shown in Fig. 1, the operation control unit 21 includes various processing units such as a vehicle setting processing unit 211, a field setting processing unit 212, a work setting processing unit 213, a route generation processing unit 214, an output processing unit 215, a display processing unit 216, and a reception processing unit 217. The operation control unit 21 functions as the various processing units by executing various processes in accordance with the control program using the CPU. Some or all of the processing units may be configured with electronic circuits. The control program may be a program for causing multiple processors to function as the processing units.

[0048] The vehicle setting processing unit 211 sets information relating to the work vehicle 10 (hereinafter referred to as work vehicle information). The vehicle setting processing unit 211 sets information such as the model of the work vehicle 10, the position where the positioning antenna 174 is attached on the work vehicle 10, the type of work implement 14, the size and shape of the work implement 14, the position of the work implement 14 relative to the work vehicle 10, the vehicle speed and engine rotation speed of the work vehicle 10 while working, and the vehicle speed and engine rotation speed of the work vehicle 10 while turning, by having the operator perform an operation to register this information on the operation terminal 20.

[0049] The field setting processing unit 212 sets information (hereinafter referred to as field information) about the field F. The field setting processing unit 212 sets information such as the position and shape of the field F, the work start position S where work begins and the work end position G where work ends (see FIG. 3 ), the work direction, etc., by performing a registration operation on the operation terminal 20.

[0050] The working direction refers to the direction in which the work vehicle 10 travels while working with the work implement 14 in the working area, which is the area of the field F excluding non-working areas such as headland and non-cultivated land.

[0051] Information on the position and shape of the field F can be automatically obtained, for example, by having an operator get into the work vehicle 10 and drive it around the perimeter of the field F, recording the changes in position information of the positioning antenna 174 at that time. The position and shape of the field F can also be obtained based on a polygon obtained by the operator operating the operation terminal 20 to specify multiple points on a map displayed on the operation terminal 20. The area specified by the obtained position and shape of the field F is the area in which the work vehicle 10 can be driven (travel area).

[0052] The work setting processing unit 213 sets information relating to how work will be carried out specifically (hereinafter referred to as work information). The work setting processing unit 213 is configured to be able to set, as work information, whether or not cooperative work will occur between the work vehicle 10 (unmanned tractor) and the manned work vehicle 10, the number of skips which is the number of work routes that the work vehicle 10 will skip when turning on the headland, the width of the headland, the width of the non-cultivated land, etc.

[0053] The route generation processing unit 214 generates a travel route R, which is a route along which the work vehicle 10 will automatically travel, based on the setting information. The travel route R is, for example, a work route from a work start position S to a work end position G (see FIG. 3). The travel route R shown in FIG. 3 is a route along which the work vehicle 10 will travel back and forth in parallel within the work area of the field F. The route generation processing unit 214 can generate and store the travel route R for the work vehicle 10 based on the setting information set by the vehicle setting processing unit 211, the field setting processing unit 212, and the work setting processing unit 213.

[0054] Specifically, the path generation processing unit 214 generates a travel path R (see FIG. 3) based on the work start position S and the work end position G registered in the field setting. The travel path R is not limited to the path shown in FIG.

[0055] The work vehicle 10 is configured so that data of the travel route R generated in the operation terminal 20 is transferred to the work vehicle 10 and stored in the memory unit 12, and the current position of the work vehicle 10 is detected by the positioning antenna 174, allowing the work vehicle 10 to travel autonomously along the travel route R. The current position of the work vehicle 10 normally coincides with the position of the positioning antenna 174.

[0056] The work vehicle 10 according to this embodiment travels in a substantially rectangular field F as shown in Fig. 3. The work vehicle 10 is configured to be able to travel automatically when its current position is located within the field F, and is configured not to be able to travel automatically when its current position is located outside the field F (such as on a public road). Furthermore, the work vehicle 10 is configured to be able to travel automatically when, for example, its current position coincides with a work start position S.

[0057] When the current position of the work vehicle 10 matches the work start position S, and the operator presses the work start button on the operation screen to give a work start instruction, the work vehicle 10 starts automatic driving by the driving processing unit 113, and work is started by the work implement 14 (see FIG. 2). In other words, the operation control unit 21 permits automatic driving of the work vehicle 10 on the condition that the current position matches the work start position S. Note that the conditions for permitting automatic driving of the work vehicle 10 are not limited to the above conditions.

[0058] The output processing unit 215 outputs information about the travel route R generated by the route generation processing unit 214 to the work vehicle 10. The output processing unit 215 can also instruct the work vehicle 10 to start and stop autonomous travel, etc., by sending control signals to the work vehicle 10 via the communication unit 24. This makes it possible for the work vehicle 10 to travel autonomously.

[0059] For example, the driving processing unit 113 automatically drives the work vehicle 10 from the work start position S to the work end position G based on the driving route R acquired from the operation terminal 20. Furthermore, when the work vehicle 10 finishes work, the driving processing unit 113 may automatically drive the work vehicle 10 from the work end position G to the entrance of the field F. When the work vehicle 10 is driving automatically, the operation control unit 21 can receive the status of the work vehicle 10 (position, driving speed, etc.) from the work vehicle 10 and display it on the operation display unit 23.

[0060] The display processing unit 216 displays various types of information on the operation display unit 23. For example, the display processing unit 216 displays on the operation display unit 23 an operation screen for registering work vehicle information, field information, work information, etc., a setting screen D1 (see FIG. 5) for setting the layout of the image display field L1 that displays images captured by each of the multiple cameras 15 installed on the work vehicle 10, a traveling status screen D2 (see FIGS. 6 to 8) that displays the captured images, a traveling selection screen D3 (see FIG. 9) for selecting a traveling stop instruction to stop the traveling of the work vehicle 10 or a traveling continue instruction to continue traveling, etc.

[0061] The reception processing unit 217 accepts various operations from the operator. For example, the reception processing unit 217 accepts an operation from the operator to set the layout of the image display field L1 (an example of a setting operation of the present invention). The reception processing unit 217 also accepts a work start instruction from the operator to cause the work vehicle 10 to start work. The reception processing unit 217 also accepts a travel stop instruction from the operator to stop the travel of the work vehicle 10 that is currently traveling automatically. The reception processing unit 217 also accepts an instruction from the operator to stop the travel of the work vehicle 10 or an instruction to continue traveling of the work vehicle 10 from the operator when the work vehicle 10 detects a human. The reception processing unit 217 also accepts a travel resume instruction from the operator to restart the travel of the work vehicle 10 that has stopped traveling. When the reception processing unit 217 accepts each of the above instructions, the output processing unit 215 outputs each of the above instructions to the work vehicle 10. The reception processing unit 217 is an example of a reception processing unit of the present invention.

[0062] When the driving processing unit 113 of the work vehicle 10 receives a work start instruction from the operation terminal 20, it causes the work vehicle 10 to start driving and working. Furthermore, when the driving processing unit 113 receives a driving stop instruction from the operation terminal 20, it causes the work vehicle 10 to stop driving and working. Furthermore, when the driving processing unit 113 receives a driving continue instruction from the operation terminal 20, it causes the work vehicle 10 to continue driving and working. Furthermore, when the driving processing unit 113 receives a driving restart instruction from the operation terminal 20, it causes the work vehicle 10 to resume driving and working.

[0063] Here, a specific example will be described. For example, in the case where five cameras 15F, 15B, 15R, 15L, and 15C (cameras 1 to 5) are installed on the work vehicle 10 as shown in FIG. 4, the display processing unit 216 acquires the number and identification information of the cameras 15 from the work vehicle 10 and then displays a setting screen D1 on the operation display unit 23 for setting the layout of the image display fields L1. For example, as shown in FIG. 5A, the display processing unit 216 displays five image display fields L1, each displaying images captured by camera 15F (camera 1), camera 15B (camera 2), camera 15R (camera 3), camera 15L (camera 4), and camera 15C (camera 5), in any order. Furthermore, as shown in FIG. 5B, the display processing unit 216 may arrange the five image display fields L1 in positions corresponding to the installation layout of the cameras 15 on the work vehicle 10. Furthermore, the operator can change the layout of the image display fields L1 on the setting screen D1. For example, the operator can place each image display field L1 in a desired position on the setting screen D1 by using a drag-and-drop operation or the like.

[0064] Furthermore, it is desirable that the display processing unit 216 display an image (camera layout image Pc) that makes it possible to identify the layout position of each camera 15 relative to the work vehicle 10, as shown on the left side of the setting screen D1. This allows the operator to easily understand the correspondence between the layout position of each camera 15 and the layout of each image display field L1. Note that the display processing unit 216 may omit the camera layout image Pc from the setting screen D1.

[0065] The display processing unit 216 arranges and displays multiple captured images P1 in accordance with the setting operation on the setting screen D1. For example, when the layout of the image display field L1 shown in FIG. 5A is set, the display processing unit 216 displays a camera image display field A1 displaying images captured by each camera 15 on a driving status screen D2 that displays the status of the automatic driving of the work vehicle 10, as shown in FIG. 6A. The current status of the work vehicle 10 during automatic driving is displayed in real time on the driving status screen D2. The operator can grasp the current driving status and work status of the work vehicle 10 on the driving status screen D2. The display processing unit 216 displays each image display field L1 in the camera image display field A1 in accordance with the layout set on the setting screen D1. FIG. 6B shows an example of the driving status screen D2 in which a captured image P1 is displayed in each image display field L1. The captured image P1 displayed in each image display field L1 is updated in real time.

[0066] When the image display field L1 is set to the layout shown in Fig. 5B, the display processing unit 216 displays each image display field L1 shown in Fig. 7A. Fig. 7B shows an example of the traveling state screen D2 in which the captured image P1 is displayed in each image display field L1.

[0067] Here, when an obstacle (here, a human) is detected in at least one of the images captured by the five cameras 15, the operation terminal 20 executes the following process. For example, when the detection processing unit 112 of the work vehicle 10 detects a human in an image acquired from camera 15L (camera 4), the detection processing unit 112 outputs a determination result indicating that a human has been detected and identification information of camera 15L (camera 4) corresponding to the image to the operation terminal 20. In addition, the acquisition processing unit 111 outputs image data of the image acquired from each camera 15 to the operation terminal 20.

[0068] When the display processing unit 216 of the operation terminal 20 acquires the image data from the work vehicle 10, it displays the captured image P1 in each image display field L1. Furthermore, when the display processing unit 216 acquires the determination result and the identification information from the work vehicle 10, it enlarges and displays the image display field L1 and captured image P1 corresponding to the camera 15L (camera 4), as shown in Fig. 8A. This causes an enlarged image of a person detected as an obstacle to be displayed.

[0069] In another embodiment, the display processing unit 216 may highlight and display the image display field L1 corresponding to camera 15L (camera 4) among the multiple image display fields L1. For example, as shown in FIG. 8B, the display processing unit 216 may display the frame of the image display field L1 corresponding to camera 15L (camera 4) with a thick line. The display processing unit 216 may also display the image display field L1 corresponding to camera 15L (camera 4) in color, light up, or blink. The display processing unit 216 may also display only the image display field L1 corresponding to camera 15L (camera 4) in the camera image display field A1, and hide the image display fields L1 corresponding to the other cameras 15L.

[0070] Furthermore, when the display processing unit 216 acquires a determination result indicating that a human has been detected from the work vehicle 10, it causes the operation display unit 23 to display a travel selection screen D3, which allows the operator to select either a stop travel instruction to stop the travel of the work vehicle 10 or a continue travel instruction to continue travel. FIG. 9 shows an example of the travel selection screen D3. The display processing unit 216 displays, on the travel selection screen D3, a message indicating that an obstacle has been detected, a message prompting the operator to select whether to stop or continue travel, a stop travel button B1 to stop travel, and a continue travel button B2 to continue travel. When the operator presses the stop travel button B1 on the travel selection screen D3, the reception processing unit 217 accepts the stop travel instruction. When the reception processing unit 217 accepts the stop travel instruction, the output processing unit 215 outputs the stop travel instruction to the work vehicle 10. When the travel processing unit 113 of the work vehicle 10 acquires the stop travel instruction from the operation terminal 20, it causes the work vehicle 10 to stop traveling and perform operations.

[0071] In response to this, when the operator presses the continue traveling button B2 on the travel selection screen D3, the reception processing unit 217 receives the continue traveling instruction. When the reception processing unit 217 receives the continue traveling instruction, the output processing unit 215 outputs the continue traveling instruction to the work vehicle 10. When the travel processing unit 113 of the work vehicle 10 acquires the continue traveling instruction from the operation terminal 20, it does not stop the travel and work of the work vehicle 10, but continues travel and work. Note that when the reception processing unit 217 receives the continue traveling instruction, the output processing unit 215 may be configured not to output the continue traveling instruction to the work vehicle 10. As a result, the travel processing unit 113 does not perform processing to stop the travel of the work vehicle 10, and as a result, travel and work continue.

[0072] Here, it is conceivable that the operator will not press either the stop traveling button B1 or the continue traveling button B2 on the traveling selection screen D3. For example, it is conceivable that the operator will not notice that an obstacle has been detected or that the traveling selection screen D3 has been displayed. In this case, to avoid the risk of the work vehicle 10 coming into contact with the obstacle, the output processing unit 215 outputs the stop traveling instruction to the work vehicle 10. Specifically, if the reception processing unit 217 does not accept either the stop traveling instruction or the continue traveling instruction, the traveling processing unit 113 stops the automatic traveling of the work vehicle 10. For example, if the operator does not press either the stop traveling button B1 or the continue traveling button B2 within a predetermined time after the display processing unit 216 displays the traveling selection screen D3 or after the detection processing unit 112 detects a human, the output processing unit 215 outputs the stop traveling instruction to the work vehicle 10.

[0073] In this way, when an obstacle (human) is detected by the detection processing unit 112, if the reception processing unit 217 does not receive either the instruction to stop traveling or the instruction to continue traveling before a predetermined time has elapsed, the traveling processing unit 113 stops the automatic traveling of the work vehicle 10. In other words, when the work vehicle 10 detects an obstacle but does not receive an instruction from the operator, it stops traveling and working.

[0074] Here, the predetermined time may be a fixed time set in advance, or may be a time corresponding to the size of the obstacle when it is detected. For example, when the size of the obstacle in the captured image is large, the distance between the work vehicle 10 and the obstacle is closer than when the size of the obstacle in the captured image is small. Therefore, the operation control unit 21 sets the predetermined time shorter as the size of the obstacle in the captured image increases. Furthermore, the operation control unit 21 sets the predetermined time longer as the size of the obstacle in the captured image decreases. In this way, when the distance between the work vehicle 10 and the obstacle is short, the time period for accepting the stop traveling instruction and the continue traveling instruction can be shortened, and when the distance between the work vehicle 10 and the obstacle is long, the time period for accepting the stop traveling instruction and the continue traveling instruction can be lengthened. Note that the travel processing unit 113 may decelerate the work vehicle 10 during the predetermined time period.

[0075] The operation control unit 21 may notify the operator of information indicating that the work vehicle 10 has detected an obstacle. For example, the operation control unit 21 may output a sound (warning sound) when the work vehicle 10 detects an obstacle.

[0076] After the work vehicle 10 has stopped in response to the travel stop instruction, when the operator issues the travel restart instruction on the operation terminal 20, the reception processing unit 217 receives the travel restart instruction, and the output processing unit 215 outputs the travel restart instruction to the work vehicle 10. When the travel processing unit 113 of the work vehicle 10 receives the travel restart instruction from the operation terminal 20, it causes the work vehicle 10 to resume travel and work.

[0077] The operation terminal 20 may be able to access a website (agricultural support site) for an agricultural support service provided by a server (not shown) via the communication network N1. In this case, the operation terminal 20 can function as an operation terminal for the server by executing a browser program by the operation control unit 21. The server is provided with the above-mentioned processing units and executes each process.

[0078] In another embodiment, the functions of the vehicle control device 11 and the detection processing device 19 described above may be included in the operation control unit 21 of the operation terminal 20. That is, for example, the operation control unit 21 may acquire image data of captured images from each camera 15 of the work vehicle 10 and execute processing to detect obstacles. Furthermore, the functions of the detection processing device 19 (acquisition processing unit 111 and detection processing unit 112) may be included in the vehicle control device 11.

[0079] [Automatic driving processing] 10, an example of the automatic driving process executed by the vehicle control device 11, the detection processing device 19, and the operation control unit 21 will be described below. For example, the automatic driving process is started by the vehicle control device 11, the detection processing device 19, and the operation control unit 21 when the work vehicle 10 starts automatic driving.

[0080] The present invention may be understood as an invention of an automatic driving method in which the vehicle control device 11, the detection processing device 19, and the operation control unit 21 execute part or all of the automatic driving processing, or as an invention of an automatic driving program for causing the vehicle control device 11, the detection processing device 19, and the operation control unit 21 to execute part or all of the automatic driving method. Furthermore, the automatic driving processing may be executed by one or more processors.

[0081] In step S1, the vehicle control device 11 causes the work vehicle 10 to start work. For example, when the operator presses the work start button on the operation screen of the operation terminal 20, the operation control unit 21 outputs a work start instruction to the work vehicle 10. When the vehicle control device 11 acquires the work start instruction from the operation terminal 20, it causes the work vehicle 10 to start automatic traveling. As a result, the work vehicle 10 starts automatic traveling according to the travel route R, and starts work by the work equipment 14.

[0082] Next, in step S2, the detection processing device 19 acquires captured images from each camera 15 installed on the work vehicle 10. For example, the detection processing device 19 acquires captured images (frame images) continuously at a predetermined frame rate from each of the five cameras 15F, 15B, 15R, 15L, and 15C.

[0083] Next, in step S3, the detection processing device 19 detects an obstacle (e.g., a human) in the acquired captured image. Specifically, the detection processing device 19 determines whether or not a human is included in the detection area based on the captured image. If the detection processing device 19 detects a human (S3: Yes), the process proceeds to step S4. On the other hand, if the detection processing device 19 does not detect a human (S3: No), the process proceeds to step S31.

[0084] In step S31, the vehicle control device 11 determines whether the work vehicle 10 has finished work. If the work vehicle 10 has finished work (S31: Yes), the process ends. On the other hand, if the work vehicle 10 has not finished work (S31: No), the process returns to step S2. In this way, if the detection processing device 19 does not detect a person, it continues driving and working while acquiring captured images until the specified work is completed.

[0085] Furthermore, the detection processing device 19 outputs image data of the acquired captured image, identification information of the camera 15 corresponding to the captured image, and a determination result indicating whether or not a human is included in the captured image to the operation terminal 20. When the operation control unit 21 acquires the image data from the detection processing device 19 of the work vehicle 10, it displays the captured image P1 in each image display field L1 (see FIG. 6B).

[0086] In step S4, the operation control unit 21 enlarges and displays the captured image including a person. For example, when a person is detected in the captured image corresponding to the camera 15L (camera 4), the operation control unit 21 enlarges the image display field L1 corresponding to the camera 15L (camera 4) and enlarges and displays the captured image P1, as shown in Fig. 8A. As a result, the image of the person detected as an obstacle is enlarged and displayed.

[0087] Next, in step S5, the operation control unit 21 causes the operation display unit 23 to display a travel selection screen D3 for selecting a travel stop instruction to stop the travel of the work vehicle 10 or a travel continue instruction to continue the travel (see FIG. 9).

[0088] Next, in step S6, the operation control unit 21 determines whether or not the operation stop instruction has been received from the operator on the operation selection screen D3. For example, when the operator presses the operation stop button B1 on the operation selection screen D3 shown in Fig. 9, the operation control unit 21 receives the operation stop instruction (S6: Yes), and the process proceeds to step S9. On the other hand, when the operator does not press the operation stop button B1 on the operation selection screen D3 shown in Fig. 9 (S6: No), the process proceeds to step S7.

[0089] Next, in step S7, the operation control unit 21 determines whether or not the operation control unit 21 has received the instruction to continue traveling from the operator on the traveling selection screen D3. For example, when the operator presses the continue traveling button B2 on the traveling selection screen D3 shown in Fig. 9, the operation control unit 21 receives the instruction to continue traveling (S7: Yes), and the process proceeds to step S71. On the other hand, when the operator does not press the continue traveling button B2 on the traveling selection screen D3 shown in Fig. 9 (S7: No), the process proceeds to step S8.

[0090] In step S71, the operation control unit 21 does not stop the travel and work of the work vehicle 10, but allows the travel and work to continue. Thereafter, the process returns to step S2.

[0091] In step S8, the operation control unit 21 determines whether a predetermined time has elapsed. Specifically, the operation control unit 21 determines whether the predetermined time has elapsed without receiving either the stop traveling instruction or the continue traveling instruction since the operation control unit 21 displayed the traveling selection screen D3 or since the detection processing device 19 detected a human. If the predetermined time has elapsed without receiving either the stop traveling instruction or the continue traveling instruction (S8: Yes), the process proceeds to step S9. If the predetermined time has not elapsed (S8: No), the process returns to step S6.

[0092] In step S9, the operation control unit 21 outputs the travel stop instruction to the work vehicle 10, and the vehicle control device 11 causes the work vehicle 10 to stop traveling and working.

[0093] In step S10, the operation control unit 21 determines whether or not a traveling restart instruction has been received from the operator. If the operation control unit 21 receives the traveling restart instruction from the operator (S10: Yes), the process returns to step S2. On the other hand, if the operation control unit 21 does not receive the traveling restart instruction from the operator (S10: No), the process ends. The vehicle control device 11, detection processing device 19, and operation control unit 21 repeatedly execute the processes of steps S1 to S10 until the work vehicle 10 finishes work.

[0094] As described above, the autonomous driving system 1 according to this embodiment acquires a captured image P1 from the camera 15 installed on the work vehicle 10 and detects an obstacle based on the captured image P1. Furthermore, when the autonomous driving system 1 detects an obstacle, it accepts a stop driving instruction to stop the autonomous driving of the work vehicle 10 or a continue driving instruction to continue the autonomous driving. The autonomous driving system 1 stops the autonomous driving of the work vehicle 10 when it accepts the stop driving instruction, and continues the autonomous driving of the work vehicle 10 when it accepts the continue driving instruction. This allows the operator to choose whether to stop or continue the autonomous driving when the work vehicle 10 detects an obstacle during autonomous driving. For example, if the obstacle does not interfere with the autonomous driving, the operator can issue an instruction to continue the autonomous driving, which causes the work vehicle 10 to continue the autonomous driving without stopping, thereby preventing a decrease in work efficiency. Furthermore, if the obstacle interferes with the autonomous driving, the operator can ensure safety by stopping the autonomous driving and removing the obstacle. The operator can then resume the autonomous driving once the safety of the work vehicle 10 has been ensured. Therefore, it is possible to ensure the safety of the work vehicle 10 while preventing a decrease in work efficiency.

[0095] Furthermore, if the work vehicle 10 detects an obstacle while it is traveling automatically, and the operator does not perform either an operation to stop the automatic traveling or an operation to continue the automatic traveling within a predetermined time, the automatic traveling system 1 stops the automatic traveling. This makes it possible to avoid a situation in which the work vehicle 10 continues traveling automatically and hits an obstacle without the operator noticing.

[0096] The present invention is not limited to the above-described embodiment, and may be embodied in the following manner.

[0097] In the above-described embodiment, when an obstacle (human) is detected by the detection processing unit 112, if the reception processing unit 217 does not accept either the stop traveling instruction or the continue traveling instruction before a predetermined time has elapsed, the traveling processing unit 113 stops the automatic traveling of the work vehicle 10. In another embodiment, when an obstacle is detected by the detection processing unit 112, if the reception processing unit 217 does not accept either the stop traveling instruction or the continue traveling instruction before the distance between the work vehicle 10 and the obstacle becomes a predetermined distance, the traveling processing unit 113 may stop the automatic traveling of the work vehicle 10.

[0098] The operation control unit 21 may calculate the change in the distance between the work vehicle 10 and an obstacle based on successive frame images acquired from the work vehicle 10. For example, as shown in Fig. 11, the operation control unit 21 can calculate the distance based on the size of a frame W1 that surrounds a person in a captured image P1.

[0099] Here, the predetermined distance may be a fixed distance set in advance, or a distance according to the size of the obstacle when it is detected. For example, when the size of the obstacle in the captured image is large, the distance between the work vehicle 10 and the obstacle is closer than when the size of the obstacle in the captured image is small. Therefore, the operation control unit 21 sets the predetermined distance shorter as the size of the obstacle in the captured image increases. Furthermore, the operation control unit 21 sets the predetermined distance longer as the size of the obstacle in the captured image decreases. As a result, when the distance between the work vehicle 10 and the obstacle is short, the time period for accepting the stop traveling instruction and the continue traveling instruction can be shortened, and when the distance between the work vehicle 10 and the obstacle is long, the time period for accepting the stop traveling instruction and the continue traveling instruction can be lengthened. Note that the travel processing unit 113 may decelerate the work vehicle 10 for the predetermined distance.

[0100] In another embodiment, for example, if the detection processing unit 112 detects a specific person and the operator issues an instruction to continue driving, and if the detection processing unit 112 subsequently detects the same person again, the work vehicle 10 may continue autonomous driving without obtaining an instruction to continue driving from the operator. This eliminates the need to request the operator to select between an instruction to stop driving or an instruction to continue driving every time the same obstacle that does not interfere with autonomous driving is detected, for example, thereby improving convenience.

[0101] In addition, in this embodiment, obstacle detection and determination are performed based on images captured by a camera, but in other embodiments, a camera and a Lidar (obstacle detection sensor) may be used together. Specifically, obstacle detection may be performed by Lidar, and obstacle determination may be performed by a camera. The obstacle detection sensor may be a sensor that uses ultrasonic waves or the like. [Explanation of symbols]

[0102] 1:Automated driving system 10: Work vehicle 11: Vehicle control device 13: Running gear 14: Work equipment 15: Camera (imaging unit) 17: Positioning device 19: Detection processing device 20: Operation terminal 21: Operation control section 23: Operation display section 111: Acquisition processing unit 112: Detection processing unit 113: Driving processing unit 211: Vehicle setting processing unit 212: Field setting processing unit 213: Work setting processing section 214: Route generation processing unit 215: Output processing section 216: Display processing unit 217: Reception processing unit

Claims

1. an acquisition processing unit that acquires captured images from an imaging unit installed in the work vehicle; a detection processing unit that detects an obstacle based on the captured image acquired by the acquisition processing unit; a display processing unit that, when the obstacle is detected by the detection processing unit, displays on an operation terminal a selection screen that accepts a stop driving instruction to stop the automatic driving of the work vehicle or a continue driving instruction to continue the automatic driving; a reception processing unit that receives an operation from an operator on the selection screen to select the travel stop instruction or the travel continue instruction; a driving processing unit that stops the automatic driving of the work vehicle when the reception processing unit receives the driving stop instruction, and continues the automatic driving of the work vehicle when the reception processing unit receives the driving continue instruction; An autonomous driving system equipped with

2. The display processing unit: When the obstacle is not detected by the detection processing unit, a driving status screen that displays the automatic driving status of the work vehicle and the captured image are displayed side by side on the operation terminal, When the obstacle is detected by the detection processing unit, the selection screen and the captured image including the obstacle are displayed side by side on the operation terminal. The automated driving system according to claim 1 .

3. The display processing unit displays the selection screen when a person is detected by the detection processing unit, and does not display the selection screen when a person is not detected by the detection processing unit. The automated driving system according to claim 1 .

4. When the reception processing unit does not receive either the travel stop instruction or the travel continue instruction, the travel processing unit stops the automatic travel of the work vehicle. The automated driving system according to claim 1 .

5. When the obstacle is detected by the detection processing unit, if the reception processing unit does not receive either the travel stop instruction or the travel continue instruction from the operator on the selection screen until a predetermined time has elapsed, the travel processing unit stops the automatic travel of the work vehicle. The automated driving system according to claim 4.

6. The predetermined time is set to a shorter time as the size of the obstacle included in the captured image increases, and the predetermined time is set to a longer time as the size of the obstacle included in the captured image decreases. The automated driving system according to claim 5 .

7. When the obstacle is detected by the detection processing unit, if the reception processing unit does not receive either the travel stop instruction or the travel continue instruction from the operator on the selection screen until the distance between the work vehicle and the obstacle reaches a predetermined distance, the travel processing unit stops the automatic travel of the work vehicle. The automated driving system according to claim 4.

8. The predetermined distance is set to a shorter distance as the size of the obstacle included in the captured image increases, and the predetermined distance is set to a longer distance as the size of the obstacle included in the captured image decreases. The automated driving system according to claim 7.

9. A plurality of the imaging units are installed at different locations on the work vehicle, the display processing unit causes the operation terminal to display, side by side, the plurality of captured images corresponding to the plurality of imaging units acquired by the acquisition processing unit; the display processing unit highlights and displays the captured image in which the obstacle is detected among the plurality of captured images. The automatic driving system according to any one of claims 1 to 8.

10. the reception processing unit further receives a setting operation for setting a layout of the plurality of captured images to be displayed on the operation terminal; the display processing unit arranges and displays the plurality of captured images in accordance with the setting operation on the operation terminal. The automated driving system according to claim 9.

11. one or more processors, Acquiring a captured image from an imaging unit installed in a work vehicle; Detecting an obstacle based on the acquired captured image; When the obstacle is detected, displaying on the operation terminal a selection screen for receiving a stop instruction to stop the automatic traveling of the work vehicle or a continue instruction to continue the automatic traveling; receiving an operation from an operator on the selection screen to select the travel stop instruction or the travel continue instruction; Stopping the automatic traveling of the work vehicle when the traveling stop instruction is received, and continuing the automatic traveling of the work vehicle when the traveling continue instruction is received. An autonomous driving system that performs the following:

12. Acquiring a captured image from an imaging unit installed in a work vehicle; Detecting an obstacle based on the acquired captured image; When the obstacle is detected, displaying on the operation terminal a selection screen for receiving a stop instruction to stop the automatic traveling of the work vehicle or a continue instruction to continue the automatic traveling; receiving an operation from an operator on the selection screen to select the travel stop instruction or the travel continue instruction; Stopping the automatic traveling of the work vehicle when the traveling stop instruction is received, and continuing the automatic traveling of the work vehicle when the traveling continue instruction is received. An automated driving program for executing the above on one or more processors.

Images