Work vehicles

Abstract

Compared to detecting obstacles using only devices, this method reduces the number of devices needed to detect obstacles while also addressing blind spots. [Solution] A terminal (46) that can communicate with the vehicle body (2) and transmit control signals to the vehicle body (2) to control the vehicle body, and which can be carried by an operator outside the vehicle body (2), and a control unit (100) that identifies an area (A1) where the operator can visually monitor the vehicle body (2) and an area (A2) where the operator cannot monitor, based on the operator's position and the position of the vehicle body (2). As a result, the area that can be monitored (A1) can be monitored visually, and compared to the case where obstacles are detected only by devices, it is possible to reduce the number of devices that detect obstacles while dealing with blind spots.

Description

Technical Field

[0001] The present invention relates to a work vehicle that performs work in a field.

Background Art

[0002] In work vehicles such as tractors and rice transplanters, so-called robotic agricultural machines that automatically perform work while performing automatic driving and autonomous driving are known. When performing automatic driving, a technique for remotely operating and controlling a lighting device using an image projected by an imaging device and an obstacle sensor is known (Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] (Problems of the Prior Art) When detecting obstacles by remote operation as in the prior art, the capabilities of imaging devices and sensors are constant. Therefore, in order to eliminate a so-called blind spot, which is a range where obstacles cannot be detected, it is necessary to install many imaging devices and sensors, and it is necessary to transmit the captured images of the installed imaging devices and the like to a remote operation terminal at any time. Therefore, the capacity of communication data also increases, the communication load also increases, and it can also cause communication failures.

[0005] The technical problem of the present invention is to suppress devices for detecting obstacles while dealing with blind spots as compared with the case of detecting obstacles only by devices.

Means for Solving the Problems

[0006] To solve the aforementioned technical problems, the invention described in claim 1 is a work vehicle comprising: a vehicle body (2); work implements (13, 16) supported by the vehicle body (2) for performing work in a field; a terminal (46) capable of communicating with the vehicle body (2) and transmitting control signals to the vehicle body (2) to control the vehicle body, wherein the terminal (46) is portable by the worker outside the vehicle body (2); and a control unit (100) that identifies an area (A1) in which the worker can visually monitor the vehicle body (2) and an area (A2) in which the worker cannot monitor the vehicle body (2) based on the worker's position and the position of the vehicle body (2).

[0007] The invention described in claim 2 is a work vehicle according to claim 1, comprising: a first positioning device (41) for measuring the position of the vehicle body (2); a second positioning device (SN1) for measuring the position of the terminal (46); and a control unit (100) that allows an operator carrying the terminal (46) to identify the unmonitored area (A2) based on the positioning result of the first positioning device (41) and the positioning result of the second positioning device (SN1).

[0008] The invention described in claim 3 is a work vehicle according to claim 1, further comprising a control unit (100) that changes the obstacle detection area (A5) for detecting obstacles in the movement of the vehicle body (2) to cover the unmonitored area (A2).

[0009] The invention described in claim 4 is a work vehicle according to claim 3, comprising a control unit (100) that identifies the worker's field of view (A3) based on the direction of the worker's face and changes the obstacle detection area (A5) to cover non-visible areas not included in the field of view (A3).

[0010] The invention described in claim 5 is a work vehicle according to claim 4, comprising a control unit (100) that detects the gaze of the worker, determines based on the gaze detection result whether the worker is monitoring the vehicle body (2), looking at the terminal (46), or looking at neither the vehicle body (2) nor the terminal (46), identifies the visible area (A3) within the monitorable area (A1) based on the direction of the worker's face when the worker is monitoring the vehicle body (2), and considers the monitorable area (A1) as the non-visible area when the worker is looking at the terminal (46) or when neither the vehicle body (2) nor the terminal (46) is being looked at.

[0011] The invention described in claim 6 is a work vehicle according to claim 1, comprising a control unit (100) that performs automatic driving control to drive the vehicle body (2) along a predetermined driving path, wherein the control unit (100) performs the automatic driving control when the distance (La) between the vehicle body (2) and the terminal (46) is less than or equal to a predetermined distance (Lb).

[0012] The invention described in claim 7 is a work vehicle according to claim 6, comprising a control unit (100) that detects the gaze of the worker and, based on the gaze detection result, executes the automatic driving control when the worker is monitoring the vehicle body (2).

[0013] The invention described in claim 8 is a work vehicle according to claim 6 or 7, comprising a control unit (100) provided at the terminal (46) that performs the automatic driving control when input is continuously received from an input unit that instructs the execution of the automatic driving control. [Effects of the Invention]

[0014] According to the invention described in claim 1, by identifying areas (A1) where the worker can visually monitor the vehicle body (2) and areas (A2) where the worker cannot monitor the vehicle body (2) based on the worker's position and the vehicle body (2), the monitorable area (A1) can be visually monitored, and compared to the case where obstacles are detected only by devices, it is possible to reduce the number of devices used to detect obstacles while addressing blind spots.

[0015] According to the invention described in claim 2, in addition to the effects described in claim 1, the first positioning device (41) and the second positioning device (SN1) can automatically identify areas (A2) that are not monitored by the worker carrying the terminal (46).

[0016] According to the invention described in claim 3, in addition to the effects described in claim 1, by changing the obstacle detection area (A5) to cover the unmonitored area (A2), the ability to detect obstacles in the unmonitored area (A2) is improved, and safety is enhanced.

[0017] According to the invention described in claim 4, in addition to the effects described in claim 3, by changing the obstacle detection area (A5) to cover the non-visible area, the ability to detect obstacles in the non-visible area is improved, and safety is enhanced.

[0018] According to the invention described in claim 5, in addition to the effects described in claim 4, the monitoring state can be detected visually from the worker's line of sight, and the monitoring state and non-monitoring state can be detected at the terminal (46).

[0019] According to the invention described in claim 6, in addition to the effects described in claim 1, by performing automatic driving control when the distance (La) between the vehicle body (2) and the terminal (46) is less than or equal to the distance (Lb), automatic driving can be performed within a range that is easily visible to a human, thereby improving safety.

[0020] According to the invention described in claim 7, in addition to the effects described in claim 6, by executing automatic driving control when an operator is monitoring the vehicle body (2), automatic driving can be reliably performed under human supervision, thereby improving safety.

[0021] According to the invention described in claim 8, in addition to the effects described in claim 6 or 7, by executing the automatic driving control when the input of the automatic driving control continues to be received, the automatic driving is performed in a situation where the operator is paying attention to the input, and the safety is improved.

Brief Description of Drawings

[0022] [Figure 1] FIG. 1 is a side view of a combine as an example of a work vehicle according to an embodiment of the present invention. [Figure 2] FIG. 2 is a front view of the work vehicle of FIG. 1. [Figure 3] FIG. 3 is a plan view of the work vehicle of FIG. 1. [Figure 4] FIG. 4 is a rear view of the work vehicle of FIG. 1. [Figure 5] FIG. 5 is a functional block diagram of a control means according to an embodiment. [Figure 6] FIG. 6 is an explanatory diagram of an example of a monitorable area and a non-monitorable area according to an embodiment. [Figure 7] FIG. 7 is an explanatory diagram of an example of the relationship between a blind spot and an obstacle detection range according to an embodiment. FIG. 7(A) is an explanatory diagram when the detection range of an obstacle is expanded to the blind spot side, and FIG. 7(B) is an explanatory diagram when the detection range of an obstacle is expanded in the traveling direction. [Figure 8] FIG. 8 is an explanatory diagram of another example of a work vehicle according to an embodiment. [Figure 9] FIG. 9 is an explanatory diagram of a main part of a raking unit and a collection box.

Embodiments for Carrying Out the Invention

[0023] Next, an example of an embodiment, which is a specific example of an embodiment of the present invention, will be described with reference to the drawings. However, the present invention is not limited to the following examples. In the description of the embodiment, the left and right directions are referred to as left and right, respectively, toward the forward direction of the machine body, and the forward direction is referred to as front and the backward direction is referred to as rear for explanation. In the following description using the drawings, illustrations other than the members necessary for the explanation are appropriately omitted for ease of understanding.

[0024] Figure 1 is a side view of a combine harvester, which is an example of a work vehicle according to an embodiment of the present invention. Figure 2 is a front view of the work vehicle shown in Figure 1. Figure 3 is a plan view of the work vehicle shown in Figure 1. Figure 4 is a rear view of the work vehicle shown in Figure 1.

[0025] In Figures 1 to 4, a combine harvester 1, as an example of a work vehicle according to an embodiment of the present invention, has a body 2. A pair of left and right running gears 11 are arranged at the bottom of the body 2. The running gears 11 in this embodiment are, as an example, composed of so-called crawler tracks. A cabin 12, as an example of a passenger compartment, is installed on the right front of the body 2. A harvesting device 13 (an example of a work machine, an example of a cutting device) for harvesting crops in the field is arranged at the front of the body 2. Behind the harvesting device 13 is a conveying device 14 for transporting the harvested grain. Behind the conveying device 14 is a threshing device 16 for threshing the grain transported by the conveying device 14. To the right of the threshing device 16 is a grain tank 17 (an example of a container) for storing the grain processed by the threshing device 16. A discharge device 18 is connected to the rear of the grain tank 17 to discharge the grain from the grain tank 17 to a container (not shown) on a truck outside the field. A straw discharge device 19 for discharging straw is located at the rear of the vehicle body 2. The straw discharge device 19 in this embodiment is a device that temporarily stores the straw, allowing it to be discharged in bundles, a so-called dropper. That is, when the dropper function is operating, the straw can be discharged in bundles, and when the dropper function is stopped, the harvested straw is discharged as is, without being bundled.

[0026] In the combine harvester 1 of this embodiment, cameras 31, 32, and 33, which are examples of detection devices and examples of imaging devices, are installed on the front and both sides of the vehicle body 2. Cameras 31 to 33 can capture images of the front and both sides of the vehicle body 2. Although cameras have been given as an example of a detection device, the system is not limited to cameras, and obstacle sensors or distance meters that detect crops or obstacles using reflected electromagnetic waves (light, infrared rays, etc.) can also be used. In this embodiment, each camera 31 to 33 is provided with an actuator, or so-called pan / tilt function, for changing and adjusting its orientation (shooting direction). Therefore, the imaging range of cameras 31 to 33 is configured to be changeable.

[0027] Furthermore, the combine harvester 1 of this embodiment has a receiver 41 installed on the top surface of the cabin 12, which is an example of a current position measurement device and an example of a first positioning device. The receiver 41 can receive signals from a satellite 42 for GNSS (Global Navigation Satellite System) and measure the current position of the combine harvester 1. In this embodiment, in addition to the receiver 41, an IMU (Inertial Measurement Unit) (not shown) is provided. The IMU can measure acceleration and angular velocity to detect the attitude of the combine harvester 1 (left-right tilt and front-back tilt). Therefore, by correcting the measurement results of the GNSS receiver with the IMU, the current position can be measured with greater accuracy compared to when the current position is measured using only the GNSS method.

[0028] Therefore, the combine harvester 1 of this embodiment can operate autonomously (automatic driving, unmanned driving) using GNSS, or it can be driven by an operator riding in the cabin 12 according to their commands (manual driving, manned driving). In addition, during autonomous driving, the operator who operates the tablet terminal (an example of a terminal) 46 capable of wireless communication with the combine harvester 1 can be outside the combine harvester 1 (outside or inside the field), or can be riding in the cabin 12 while carrying the tablet terminal 46.

[0029] The combine harvester 1 and the tablet terminal 46 are configured to communicate via a communication line 47. The communication line 47 can be any wireless or wired line, such as a telephone line, internet line, LAN line, or Bluetooth®. A server 48, which is an example of an information processing device, is connected to the communication line 47. The server 48 can send and receive information (communicate) with the combine harvester 1 and the tablet terminal 46. It is also possible to configure the combine harvester 1 and the tablet terminal 46 to communicate directly without using the communication line 47 (ad hoc network). The tablet terminal 46 of this embodiment incorporates a GNSS receiver SN1 (see Figure 5) and a three-dimensional acceleration sensor SN2 (see Figure 5) as examples of second positioning devices. Therefore, the current location of the tablet terminal 46, that is, the current location of the worker carrying the tablet terminal 46, can be determined.

[0030] (Description of the control unit) Figure 5 is a functional block diagram of the control means in the embodiment. In the block diagram of Figure 5, elements unrelated to the description of the embodiments of the present invention are omitted from the illustration and description. (Description of the control unit of tablet device 46) In Figure 5, the control unit (an example of a control means) 200 of the tablet terminal 46 in this embodiment is composed of a small information processing device, a so-called microcomputer. Therefore, the control unit 200 can realize various functions by executing an operating system stored in ROM, RAM, etc., or a work instruction program AP1, etc.

[0031] The control unit 200 of the tablet terminal 46 in this embodiment receives signals from signal output elements such as the touch panel 46b, GNSS receiver SN1, 3D acceleration sensor SN2, camera 46c, other sensors (not shown), various switches, and buttons. Signals transmitted from the control unit 100 of the combine harvester 1 are also input. The touch panel 46b, which is an example of a display unit and an example of an input unit, detects input according to the position where the operator touches it with their finger.

[0032] The GNSS receiver SN1 receives signals from the GNSS satellite 42 and determines the current position of the tablet terminal 46 by correcting it with the attitude of the tablet terminal 46 (direction of gravity and horizontal orientation) measured by the 3D acceleration sensor SN2. The 3D acceleration sensor SN2 detects acceleration acting on the tablet device 46, including acceleration due to gravity. Therefore, the orientation of the tablet device 46 (direction of gravity and horizontal orientation) can be determined from the detected 3D acceleration. Camera 46c captures images from the touch panel 46b of the tablet terminal 46. Therefore, while the worker is operating the tablet terminal 46, the worker's face can be captured.

[0033] The control unit 200 of the tablet terminal 46 in this embodiment outputs control signals to controlled elements such as the touch panel 46b. The control unit 200 of the tablet terminal 46 in this embodiment can also output control signals, image information, etc. to the combine harvester 1. The control unit 200 of the tablet terminal 46 in this embodiment has the following functional modules (program modules). The work instruction program AP1 is a program for issuing work instructions to the combine harvester 1 from a tablet terminal 46. In this embodiment, the work instruction program AP1 displays images of the combine harvester 1's work status and blind spots (unmonitored area A2) on the touch panel 46b, as well as input buttons such as a work execution button and an emergency stop button. Therefore, the work status displayed includes whether the combine harvester 1 is stopped, working, driving on the road, detecting or avoiding an obstacle, etc. In addition, the operator can issue work execution instructions by touching the work execution button displayed on the touch panel 46b with their finger.

[0034] (Description of the control unit of Combine 1) In Figure 5, the control unit (an example of a control means) 100 of the combine harvester 1 in this embodiment is composed of a small information processing device, a so-called microcomputer. Therefore, the control unit 100 can realize various functions by executing programs stored in ROM, RAM, etc.

[0035] The control unit 100 of the combine harvester 1 in this embodiment receives signals from the receiver 41, cameras 31-33, distance measuring sensor (rangefinder) SN0, other sensors (not shown), various switches, buttons, and other signal output elements. It also receives control signals transmitted from the tablet terminal 46. The receiver 41 receives signals from a GNSS satellite 42 to measure the vehicle's current position, corrects this position using the vehicle's attitude measured by an inertial measurement unit (IMU), and then measures the vehicle's current position. Cameras 31-33 capture images of the area outside the vehicle body 2. Therefore, images including fields, crops, obstacles, etc., are captured. The distance measuring sensor SN0 measures the distance to obstacles, dikes, and other objects. While the distance measuring sensors SN0 are installed near cameras 31-33, they can also be installed in locations separate from the cameras. Furthermore, depending on the performance of the distance measuring sensors SN0, the number of sensors can be increased or decreased relative to the number of cameras 31-33.

[0036] The control unit 100 of the combine harvester 1 in this embodiment outputs control signals to controlled elements such as the engine (not shown), the traveling device 11, the harvesting device 13, the conveying device 14, the threshing device 16, the discharge device 18, the straw discharge device 19, and the actuators 31a to 33a of the cameras 31 to 33. The control unit 100 in this embodiment can also transmit information such as image information and signals to the tablet terminal 46.

[0037] The control unit 100 of this embodiment has the following functional modules (program modules). The first positioning means 101 measures the current position of the combine harvester 1 based on the positioning result of the receiver 41. The second positioning means 102 obtains the current location of the tablet terminal 46, which has been determined by the tablet terminal 46 based on the positioning result of the GNSS receiver SN1 of the tablet terminal 46, from the tablet terminal 46.

[0038] Figure 6 is an explanatory diagram illustrating an example of a monitorable and non-monitoring area in the embodiment. The monitorable area identification means 103 identifies areas where the worker can visually monitor the vehicle body 2 (monitored area A1) and areas where the worker cannot monitor it (unmonitored area A2), based on the worker's position and the vehicle body 2's position. In this embodiment, the monitorable area identification means 103 identifies the unmonitored area A2, or so-called blind spot, for the worker carrying the tablet terminal 46, based on the current position of the combine harvester 1 measured by the first positioning means 101 and the current position of the tablet terminal 46 measured by the second positioning means 102. In this embodiment, virtual lines L1 to L4 are generated connecting the worker's position P2 to the four corners of the combine harvester 1, with respect to the combine harvester 1's position P1 and the worker's position P2. The widest area enclosed by virtual lines L1 to L4 (in Figure 6, the area enclosed by virtual lines L2 and L4), which lies on the opposite side of the combine harvester 1 from the worker's position P2 (i.e., the area beyond the combine harvester 1 from the worker's perspective), is identified as the unmonitored area A2. The area other than the unmonitored area A2 is identified as the monitorable area A1. Therefore, the unmonitored area A2 in this embodiment corresponds to the shadow area (projection area) of the combine harvester 1, assuming that the light source is at the worker's position. Consequently, objects (crops or obstacles) in the unmonitored area A2 cannot be seen by the worker because the combine harvester 1 in front of them obstructs their view.

[0039] The vehicle distance calculation means 104 calculates the distance La between the combine harvester 1 and the tablet terminal 46 based on the current position of the combine harvester 1 determined by the first positioning means 101 and the current position of the tablet terminal 46 determined by the second positioning means 102. The worker orientation detection means 105 detects the orientation (direction) of the tablet terminal 46 based on the detection result of the 3D acceleration sensor SN2 built into the tablet terminal 46. In this embodiment, the worker orientation detection means 105 detects (estimates, assumes) the detected orientation of the tablet terminal 46 as the orientation of the worker carrying the tablet terminal 46.

[0040] The gaze detection means 106 detects the worker's gaze based on the image from the camera 46c built into the tablet terminal 46. That is, when a worker is remotely operating the combine harvester 1, they will be holding the tablet terminal 46 in their hand, and their face will almost always be visible in the camera 46c. Therefore, it is possible to detect the worker's gaze from the image from the camera 46c. The gaze detection means 106 in this embodiment detects whether the worker's gaze is directed towards the combine harvester 1's body 2 (monitoring the combine harvester 1 = manned monitoring state), towards the tablet terminal 46 (looking at the tablet terminal 46 = tablet monitoring state), or not looking at either the body 2 or the tablet terminal 46 (unmonitored state). If the gaze cannot be detected, for example, because the face is not visible in the camera 46c, it is presumed that the worker is not looking at either the body 2 or the tablet terminal 46 (unmonitored).

[0041] The viewing area identification means 107 identifies the worker's viewing area A3 based on the detection result of the worker orientation detection means 105 and the detection result of the gaze detection means 106. Therefore, the viewing area A3 is identified based on the worker's orientation, gaze, and a predetermined field of view of the worker (for example, 150 degrees to the left and right). Therefore, the area within the monitorable area A1 that overlaps with the worker's visible area A3 is the area that the worker actually sees, and the behavior of combine harvester 1 and obstacles can be visually confirmed. The area within the monitorable area A1 that does not overlap with the visible area A3 is an invisible area that is not actually seen by the worker. In addition, even if the unmonitored area A2 overlaps with the visible area A3, it is actually not visible or can not be seen because combine harvester 1 obstructs the view. Furthermore, the visible area A3 changes when the direction the worker is facing or their line of sight changes, and the unmonitored area A2 (and the monitorable area A1) also changes when combine harvester 1 moves (travels) or the worker moves.

[0042] The image information acquisition means 108 acquires images taken by cameras 31-33. Figure 7 is an explanatory diagram illustrating an example of the relationship between the blind spot and the obstacle detection range in the embodiment. Figure 7(A) is an explanatory diagram when the obstacle detection range is extended towards the blind spot, and Figure 7(B) is an explanatory diagram when the obstacle detection range is extended in the direction of travel. The obstacle detection means 109 detects obstacles based on images from cameras 31-33. Therefore, it detects obstacles that may hinder movement, such as objects left in the field (farm tools, containers, bags, etc.), people, utility poles, rocks, and depressions. In this embodiment, the obstacle detection means 109 performs standard obstacle detection in the monitorable area A1, and detects obstacles in the non-monitoring area A2 over a wider area A5a (longer distance) than the standard obstacle detection area A5 (see Figure 7(A)). Therefore, in blind spots that are out of the worker's line of sight, the obstacle detection area A5 is expanded to ensure safety. Furthermore, by expanding the detection area only in the blind spots compared to expanding it across the entire area, the processing load for obstacle detection is reduced.

[0043] While the example given illustrates expanding the obstacle detection area A5 to the extended area A5a in the blind spot, the system is not limited to this. Since obstacles in front of the combine harvester 1 in its direction of travel tend to be particularly problematic, it is also possible to expand the obstacle detection area A5 to the area A5a' corresponding to the blind spot and in front of the combine harvester 1's direction of travel, as shown in Figure 7(B). In this case, the expansion of the obstacle detection area A5a' forward in the blind spot and in front of the direction of travel may result in it overlapping with the monitorable area A1. Furthermore, in both the tablet monitoring state and the non-monitoring state, neither the monitorable area A1 nor the non-monitoring area A2 is visually inspected, so the obstacle detection area is operated entirely within the standard range. In other words, the obstacle detection area in the non-monitoring area A2 becomes narrower compared to the manned monitoring (direct monitoring) state. To put it another way, in this embodiment, it is also possible to configure the system so that the obstacle detection area in the non-monitoring area A2 is expanded only in the manned monitoring state. Alternatively, in the manned monitoring state, the obstacle detection area can be narrowed in the monitorable area A1 and the visible area A3, meaning that the obstacle detection area is wider in the tablet monitoring state and the non-monitoring state.

[0044] The detection area adjustment means 110 adjusts the area in which obstacles are detected. In this embodiment, among the cameras 31 to 33, for cameras whose imaging area A4 partially overlaps with the unmonitored area A2 in the reference orientation, the actuator of that camera is controlled so that the overlapping range between the imaging area A4 and the unmonitored area A2 is maximized (covered). When the unmonitored area A2 changes as the combine harvester 1 moves, the detection area adjustment means 110 activates the actuator to change the imaging area A4 of the camera as well (to follow the unmonitored area A2).

[0045] Furthermore, for cameras that do not overlap with the unmonitored area A2, i.e., cameras that capture images of the monitorable area A1, it is preferable that cameras capturing images of areas that do not overlap with the worker's visible area A3 (non-visible areas) adjust the imaging area A4 to cover the non-visible areas. Therefore, it is preferable that cameras capturing images of non-visible areas pan and tilt using an actuator while taking pictures to capture images of a wide area, so that the obstacle detection means 109 can detect obstacles over a wide area. In this embodiment, an example was given in which the imaging area A4 of cameras 31-33 is changed by an actuator, but the system is not limited to this. For example, it is also possible to use a wide-angle lens or a fisheye lens to capture an image with a wide field of view, and to eliminate the pan / tilt function. Furthermore, the obstacle detection means 109 can extract a portion corresponding to the unmonitored area A2 from the wide field of view image and provide it to the operator, or it is possible to perform obstacle detection with high accuracy on the extracted portion and with normal accuracy on the remaining portion.

[0046] The image information transmission means 111 transmits images of the unmonitored area A2 to the tablet terminal 46. In this embodiment, in order to reduce the communication load, images from the camera corresponding to the unmonitored area A2 are transmitted to the tablet terminal 46, rather than images from all cameras 31 to 33. Therefore, the worker can check for obstacles, crops, etc. in the unmonitored area A2 using the image displayed on the touch panel 46b, and can directly check the monitorable area A1 with their eyes.

[0047] The automatic work control means 112 includes a travel control means 112a and a work machine control means 112b, and controls the automatic travel of the combine harvester 1. In this embodiment, when the work execution button is pressed on the tablet terminal 46, an automatic work operation is performed while the combine harvester travels automatically. The travel control means 112a controls the travel device 11 based on the current position of the combine harvester 1 and predetermined work path data, causing the combine harvester 1 to travel automatically (autonomously) along the work path. During operation, the work machine control means 112b controls the work machines, such as the harvesting device 13, threshing device 16, conveying device 14, discharge device 18, and straw discharge device 19, to perform the work.

[0048] In this embodiment, the automatic work control means 112 performs automatic operation when the distance La between the combine harvester 1 and the tablet terminal 46, calculated by the vehicle distance calculation means 104, is less than or equal to a predetermined distance Lb. Therefore, if the distance La is greater than the distance Lb, automatic operation and automatic operation will not be performed even if an instruction to perform work is given on the tablet terminal 46. Also, if the distance La exceeds the distance Lb during automatic operation, the combine harvester 1 will stop automatic operation and automatic operation. If the combine harvester 1 is operated automatically when the operator is too far away from it to monitor it visually, the safety of the combine harvester 1 cannot be ensured. In this embodiment, the combine harvester 1 is capable of automatic operation within a distance Lb that can be visually confirmed by the operator, thus ensuring safety.

[0049] Furthermore, based on the detection results of the gaze detection means 106, it is also possible to control the combine harvester 1 so that it can operate automatically when an operator is monitoring it. In this way, the combine harvester 1 can operate automatically while an operator is reliably monitoring it, making it easier to ensure safety. In reality, workers may also check the tablet terminal 46, making it difficult for them to constantly keep their eyes on the combine harvester 1. Therefore, it is preferable that, after detecting that the worker's gaze is directed towards the combine harvester 1, the system continues automatic operation even if the worker's gaze shifts away from the combine harvester 1, as long as the worker's gaze is directed towards the combine harvester 1 at least once within a predetermined period (for example, within 3 minutes). In other words, safety can be ensured by controlling the system to stop automatic operation if the combine harvester 1 is not monitored for a predetermined period of time or longer. Alternatively, the system can be configured to stop automatic operation if a blind spot (unmonitored area A2) exists in front of the combine harvester 1 in its direction of travel, meaning that automatic operation is only possible when an operator is monitoring the combine harvester 1 from the front.

[0050] Furthermore, the automatic work control means 112 starts work when the work execution button is pressed on the tablet terminal 46 and stops work when the input is made again, but is not limited to this. For example, it is also possible to execute automatic work when the work execution button (an example of an input unit that instructs the execution of automatic driving control) is continuously pressed, that is, when the work execution button is kept pressed. Therefore, it is also possible to control the combine 1 so that it stops when the finger is released from the work execution button. In this way, if the worker's attention decreases and the finger is released from the tablet terminal 46, the combine 1 will stop. Thus, the combine 1 will automatically drive while the worker is concentrating on the work, making it easier to ensure safety.

[0051] Furthermore, if the obstacle detection means 109 detects an obstacle, and the obstacle is located in the direction of travel of the combine harvester 1, the automatic work control means 112a will stop the combine harvester 1 at a position where the distance to the obstacle is a predetermined stopping distance, based on the distance to the obstacle detected by the distance measuring sensor SN0. Note that the distance to the obstacle is not limited to being measured by the distance measuring sensor SN0, but can also be measured by analyzing images from cameras 31 to 33.

[0052] The distance measurement fault diagnosis means 113 diagnoses whether or not the distance measurement sensor SN0 is malfunctioning. In this embodiment, the distance measurement fault diagnosis means 113 performs the diagnosis when the vehicle starts automatic driving from a storage location such as a barn or warehouse. The distance measurement fault diagnosis means 113 of this embodiment observes a simulated obstacle for fault diagnosis using the distance measurement sensor SN0 and cameras 31-33, respectively, and can diagnose a fault by comparing the distance measurement result of the distance measurement sensor SN0 with the distance measurement result obtained by image analysis of cameras 31-33. For example, if the two measurement results deviate by a predetermined discrimination value (e.g., 10%) or more, a fault can be diagnosed, and if they fall within the discrimination value, it can be diagnosed as normal (not faulty). If a fault is detected, a control signal is sent to the automatic work control means 112 to prevent the automatic work from starting.

[0053] Furthermore, if the distance measured by the distance measuring sensor SN0 is shorter than the distance measured by cameras 31-33 and deviates by more than the discrimination value, it is possible to not diagnose a malfunction, but rather diagnose that there may be an object other than the simulated obstacle between the distance measuring sensor SN0 and the simulated obstacle, and to notify the operator to check the surroundings and perform a re-diagnosis. If the same result is obtained in the re-diagnosis, it is also possible to diagnose a malfunction of the distance measuring sensor SN0. Furthermore, if cameras 31-33 fail to detect a simulated obstacle, it is possible to not immediately diagnose a malfunction, but instead diagnose that there may be dirt on the camera lens and notify the worker to check cameras 31-33 and prompt them to re-diagnose.

[0054] Furthermore, while it is preferable to perform fault diagnosis with the combine harvester 1 stopped, the procedure is not limited to this. For example, it is possible to perform measurements and diagnoses multiple times while the combine harvester 1 is automatically moving little by little, within the range where a simulated obstacle can be observed by the distance measuring sensor SN0 and cameras 31-33, and then arrive at the final diagnostic result.

[0055] The simulated obstacles can be predetermined and used. For example, the system can read specific images that are pasted or hung on the walls of barns, warehouses, or parking lots, or that are placed or painted on the road. It can also be used to attach specific images to spare work equipment stored in a barn. By embedding (encrypting) information such as location, distance, model, and orientation into specific images, for example, as barcodes, 2D codes, or AR markers, it becomes easier to identify simulated obstacles from images read by cameras 31-33, and it is also possible to obtain additional information from the images. Therefore, it is also possible to use a marker indicating the parking position of a work vehicle as a specific image.

[0056] Furthermore, while it is desirable to install a number of simulated obstacles corresponding to the number of distance measuring sensors SN0 installed, if it is difficult to install simulated obstacles for all distance measuring sensors SN0, it is also possible to install simulated obstacles for high-priority distance measuring sensors SN0 (for example, those that detect the front and rear of the vehicle 2) and not install them for low-priority distance measuring sensors SN0 (for example, those that detect the sides). In addition, it is also possible to set up a system where a single simulated obstacle is used to diagnose the front distance measuring sensor SN0 when entering the barn and the rear distance measuring sensor SN0 when leaving the barn.

[0057] Therefore, in the distance measurement fault diagnosis means 113 of this embodiment, in addition to fault diagnosis of the distance measurement sensor SN0 itself, diagnosis of mud adhesion to cameras 31-33 is also performed. In conventional configurations, sensor inspection is required before starting autonomous driving, and autonomous driving cannot begin unless the normal operation of the sensors is confirmed through inspection. Therefore, in conventional technology, an operator had to go to the storage location to perform the inspection work, which resulted in poor work efficiency. In contrast, in this embodiment, the diagnosis of faults in the distance measuring sensor SN0 is automated, improving work efficiency compared to conventional methods.

[0058] While the example illustrates a method of diagnosing a malfunction of the distance measuring sensor SN0 using cameras 31-33 and simulated obstacles, the system is not limited to this. Instead of detecting simulated obstacles with cameras 31-33, it is also possible to detect a worker, read the worker's movements (motions), and perform driving control (starting and stopping the vehicle 2, etc.). When driving control is performed using the worker's movements, lever operation of the vehicle 2 becomes ineffective. Furthermore, it is desirable to notify the surroundings that driving control is in operation using voice guidance, buzzers, lights, etc., when driving control is performed using the worker's movements.

[0059] (Operation of the embodiment) In the combine harvester 1 of the embodiment with the above configuration, obstacles are detected based on images from cameras 31 to 33. At this time, the operator also monitors the visible area A3, and if the operator sees an obstacle, they can operate the tablet terminal 46 to stop the combine harvester 1 and remove the obstacle, or change the travel path of the combine harvester 1 to avoid the obstacle. Furthermore, for the unmonitored area A2, images from cameras 31 to 33 can be transmitted to the tablet terminal 46 for the operator to check on the tablet terminal 46. Therefore, obstacles can be identified even in the unmonitored area A2, which is a blind spot for the operator. Thus, compared to detecting obstacles using only sensors and cameras, it is possible to reduce the number of devices required to detect obstacles while addressing the operator's blind spots.

[0060] Furthermore, in this embodiment, not all images from cameras 31-33 are transmitted to the tablet terminal 46; only camera images of the blind spot, the unmonitored area A2, are transmitted. Therefore, the communication load is reduced compared to when all camera images are transmitted. Furthermore, in this embodiment, the unmonitored area A2 is automatically identified from the current position of the combine harvester 1 and the current position of the worker. Therefore, the burden on the worker is reduced compared to when the worker specifies the unmonitored area A2.

[0061] Furthermore, in the combine harvester 1 of this embodiment, the imaging area A4 of cameras 31-33 moves to cover the unmonitored area A2. Therefore, compared to a case where the imaging area A4 does not change, the obstacle detection area (imaging area A4) in which obstacles can be detected changes according to the operator's blind spot, making it easier to identify obstacles. Consequently, safety is more easily ensured. Furthermore, in the combine harvester 1 of this embodiment, the visible area A3 and the non-visible area are identified according to the worker's line of sight, and the imaging area A4 changes according to the visible area A3. Therefore, cameras 31-33 can automatically cover areas that the worker is not currently seeing, making it easier to ensure safety.

[0062] In this embodiment, if a manned monitoring state or a tablet monitoring state is detected, it is possible to determine and detect that the worker is in a monitoring state. When the worker is in a monitoring state, it is unlikely that they are performing other tasks, and it is highly likely that the load other than communication between the combine 1 and the tablet terminal 46 has decreased. Therefore, it is possible to control the system to increase the communication load (communication volume and frequency) between the combine 1 and the tablet terminal 46. In addition, when the worker is intently watching the behavior of the combine 1 in a monitoring state, it is possible to make the distance Lb used to determine when to stop automatic work longer compared to the non-monitoring state. In response to the increased distance Lb, the tablet terminal 46 is also equipped with both a radio wave method capable of short-range communication (e.g., wireless LAN) and a radio wave method capable of long-range communication (e.g., mobile phone network). When the distance Lb is long, it is possible to transmit and receive using the radio wave capable of long-range communication, and when the distance Lb is short, it is possible to transmit and receive using the radio wave capable of short-range communication. In situations where increasing the communication load is acceptable, communication should be conducted over long distances, or both long and short distances. In situations where increasing the communication load is not feasible, it is also possible to use radio waves for short-range communication.

[0063] Furthermore, the determination of the monitoring state is not limited to line of sight; it is also possible to determine the monitoring state from the detection results of the 3D acceleration sensor SN2, which indicates that the tablet terminal 46 is in a specific posture (the range of angles in which a person holds the tablet terminal when using it).

[0064] In the embodiment of the combine harvester 1, the obstacle detection process is shown as being centrally processed by the control unit 100, but the invention is not limited to this. It is also possible to use a distributed processing method that utilizes multiple information processing devices connected to a network.

[0065] Figure 8 is an explanatory diagram of another example of the work vehicle according to the embodiment. Furthermore, although the embodiment uses a combine harvester 1 as an example of a work vehicle, it is not limited to this. It can be applied to work vehicles such as tractors, rice transplanters, cultivators (pest control machines), and riding lawnmowers. In Figure 8, it can also be applied to cultivator 300, which is an example of a cultivator used to create furrows in fields and cut grass, as another example of a work vehicle. In Figure 8, the cultivator 300 comprises a vehicle body 301, a sensor unit 302 housing a camera, a distance measuring sensor, a GNSS receiver, etc., a power supply unit 303 containing a power supply and control board, etc., a tank 304 containing chemicals, a spray nozzle 306 for spraying chemicals, a raking unit 307 for raking up soil in the field, and a collection box 308 for collecting attached materials (grass, snails, etc.) attached to the raking unit 307. The spray nozzle 306 sprays chemicals when pests (clusters of planthoppers, stink bugs, etc.) are detected by the camera in the sensor unit 302.

[0066] Figure 9 is an explanatory diagram of the main parts of the scraping unit and the collection box. In Figure 9, the raking unit 307 has a rotating shaft 307a and a claw portion 307b that extends spirally radially outward from the rotating shaft 307a. The claw portion 307b extends in a direction toward the downstream side in the direction of rotation as it moves radially outward. Therefore, when the rotating shaft 307a rotates, the outer end of the claw portion 307b extends upward on the side that rotates from bottom to top (front side). Consequently, grass, snails, etc. are caught and raked up from the field when it rotates. Also, when the rotating shaft 307a rotates, the claw portion 307b passes the position of the entrance 308a of the rear collection box 308 on the side that rotates from top to bottom (rear side). Therefore, the grass, etc. raked up by the claw portion 307b is collected in a manner that it is thrown into the inside of the collection box 308. [Explanation of symbols]

[0067] 1. Work vehicles, 2 car bodies, 13,16 Work equipment, 41 First positioning device, 46 devices, 100 Control unit, A1 Area that can be monitored visually, A2 Unmonitorable area, A3 viewing area, A5 Obstacle detection area, La The distance between the vehicle body and the terminal, Lb is a predetermined distance. SN1: Second positioning device.

Claims

1. Body (2) and Supported by the vehicle body (2), the implements (13, 16) perform work in the field. A terminal (46) capable of communicating with the vehicle body (2) and transmitting control signals to the vehicle body (2) to control the vehicle body, wherein the terminal (46) is portable by the operator outside the vehicle body (2), A control unit (100) that identifies an area (A1) in which the worker can visually monitor the vehicle body (2) and an area (A2) in which the worker cannot monitor the vehicle body (2), based on the position of the worker and the position of the vehicle body (2), A work vehicle equipped with [a specific feature / equipment].

2. A first positioning device (41) for measuring the position of the vehicle body (2), A second positioning device (SN1) for measuring the position of the terminal (46), Based on the positioning result of the first positioning device (41) and the positioning result of the second positioning device (SN1), the control unit (100) allows the worker carrying the terminal (46) to identify the area that cannot be monitored (A2), A work vehicle according to claim 1, comprising:

3. The control unit (100) changes the obstacle detection area (A5) for detecting obstacles in the movement of the vehicle body (2) to cover the unmonitored area (A2), A work vehicle according to claim 1, comprising:

4. The control unit (100) identifies the worker's field of view (A3) based on the direction of the worker's face, and changes the obstacle detection area (A5) to cover the non-visible area not included in the field of view (A3). A work vehicle according to claim 3, comprising:

5. The control unit (100) detects the gaze of the worker and, based on the gaze detection result, determines whether the worker is monitoring the vehicle body (2), looking at the terminal (46), or looking at neither the vehicle body (2) nor the terminal (46). If the worker is monitoring the vehicle body (2), it identifies the visible area (A3) within the monitorable area (A1) based on the direction of the worker's face, and if the worker is looking at the terminal (46) or neither the vehicle body (2) nor the terminal (46), it considers the monitorable area (A1) to be the non-visible area. A work vehicle according to claim 4, comprising:

6. The control unit (100) executes automatic driving control to drive the vehicle body (2) along a predetermined driving path, wherein the control unit (100) executes the automatic driving control when the distance (La) between the vehicle body (2) and the terminal (46) is less than or equal to a predetermined distance (Lb), A work vehicle according to claim 1, comprising:

7. The control unit (100) detects the gaze of the worker and, based on the gaze detection result, executes the automatic driving control when the worker is monitoring the vehicle body (2). A work vehicle according to claim 6, comprising:

8. The control unit (100) that executes the automatic driving control when input is continuously received from the input unit provided in the terminal (46) that instructs the execution of the automatic driving control, A work vehicle according to claim 6 or 7, comprising:

Images