Work vehicles and systems for controlling them

Abstract

To increase an efficiency of a harvesting work by a work vehicle.SOLUTION: A system for controlling a work vehicle having a chassis comprises: a plurality of sensors 13 for detecting surroundings of the chassis; a peripheral information generation unit 14a for generating peripheral information 15 indicating a peripheral condition of the chassis based on an output signal of one sensor 13; a specifying unit 14b for specifying an insensitive region in the peripheral information; a determination unit 14c for determining whether the insensitive region is entirely located within a complementary region that can be complemented by another sensor; and a controlling unit 12 that controls an automatic travel of the chassis. When the determination unit 14c determines that the insensitive region is located within the complementary region, the controlling unit 12 continues the automatic travel of the chassis; and when the determination unit 14c determines that the insensitive region is not located within the complementary region, the controlling unit 12 stops the automatic travel of the chassis.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to a work vehicle and a system for controlling the work vehicle.

Background Art

[0002] Patent Document 1 discloses a work vehicle including an active sensor equipped with a LiDAR sensor and an imaging device. When an abnormality such as dirt occurs on the sensor surface of the active sensor, information around the work vehicle can be detected by an image processing device connected to the imaging device.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When an abnormality such as dirt occurs on the surface of the sensor, a dead zone appears in the surrounding information generated based on the output of the sensor. When a dead zone appears, it may hinder the automatic driving of the vehicle body, so it is preferable to stop the automatic driving. However, depending on the position and size of the dead zone, even when a dead zone appears, it may not hinder the automatic driving of the vehicle body. Even in this case, stopping the automatic driving may deteriorate the efficiency of the harvesting operation.

[0005] An object of the present invention is to improve the efficiency of the harvesting operation of the work vehicle.

Means for Solving the Problems

[0006] The system according to the present invention is a system for controlling a work vehicle equipped with a machine, comprising: a plurality of sensors for detecting the surroundings of the machine; a surrounding information generation unit that generates surrounding information indicating the conditions around the machine based on the output signal of one of the sensors; a specification unit that identifies a dead zone in the surrounding information; a determination unit that determines whether the entire dead zone is located within a complementary zone that can be complemented by other sensors; and a control unit that controls the automatic driving of the machine. The sensor is a camera, and the surrounding information generation unit generates the surrounding information, which is a captured image showing the conditions around the aircraft, based on the output signal of one of the cameras, and a part of the captured image is the complementary region. The control unit continues the automatic driving of the aircraft when the determination unit determines that the aircraft is located inside the complementary region, and stops the automatic driving of the aircraft when the determination unit determines that the aircraft is not located inside the complementary region.

[0007] According to the above feature configuration, it is possible to determine whether the location of a dead zone appearing in the surrounding information interferes with the machine's automatic driving. By controlling the machine's automatic driving according to this determination result, excessive stopping of the automatic driving can be suppressed, thereby improving the efficiency of harvesting work. Furthermore, this configuration allows for the determination of whether any dead zones in the surrounding information captured by the camera interfere with the machine's automatic movement. By controlling the machine's automatic movement according to this determination, the efficiency of harvesting work can be improved. Furthermore, in the present invention, the camera comprises a first camera and a second camera adjacent to the first camera, wherein the shooting range of the first camera and the shooting range of the second camera are different, and the interpolation region in the image captured by the first camera is preferably a region where the shooting range of the first camera and the second camera overlap.

[0008] In the present invention, when the determination unit determines that the machine is not located within the complementary region, the control unit preferably stops the automatic movement of the machine at the edge of the field.

[0009] With this configuration, the machine does not stop in the center of the field, allowing it to be stopped in a position that makes it easy for the operator to perform maintenance tasks such as checking the sensor status.

[0010]

[0011]

[0012] In the present invention, the camera comprises a first camera and a second camera adjacent to the first camera, and the surrounding information includes first surrounding information generated based on the output signal of the first camera and second surrounding information generated based on the output signal of the second camera, and it is preferable that the control unit controls the automatic driving of the machine to stop when the identification unit identifies the dead zone in the first surrounding information and the second surrounding information.

[0013] According to this configuration, if a dead zone occurs in the first surrounding image captured by the first camera and the second surrounding information captured by the second camera adjacent to the first camera, it can be determined that there is an impediment to the aircraft's automatic navigation. By controlling the aircraft's automatic navigation according to this determination, automatic navigation in a state where automatic navigation is impediment can be avoided.

[0014] In the present invention, the control unit further comprises a size determination unit that determines whether the size of the insensitive area is less than or equal to a predetermined threshold, and when the determination unit determines that the insensitive area is not located inside the complementary area and that the size determination unit determines that it is less than or equal to the threshold, the control unit preferably continues the automatic driving of the machine.

[0015] This configuration allows for the determination of whether the location and size of a dead zone in the surrounding information interfere with the machine's automatic movement. By controlling the machine's automatic movement according to this determination, the efficiency of harvesting work can be improved.

[0016] The features of the work vehicle according to the present invention include: a machine body; a plurality of sensors for detecting the surroundings of the machine body; a surrounding information generation unit that generates surrounding information indicating the conditions around the machine body based on the output signal of one of the sensors; a specification unit that identifies a dead zone in the surrounding information; a determination unit that determines whether the entire dead zone is located within a complementary zone that can be complemented by other sensors; and a control unit that controls the automatic driving of the machine body. The sensor is a camera, and the surrounding information generation unit generates the surrounding information, which is a captured image showing the conditions around the aircraft, based on the output signal of one of the cameras, and a part of the captured image is the complementary region.When the determination unit determines that the position is inside the completion area, the control unit continues the automatic flight of the aircraft, and when the determination unit determines that the position is not inside the completion area, the control unit stops the automatic flight of the aircraft.

[0017] According to the above characteristic configuration, it is possible to determine whether the position of the insensitive area appearing in the surrounding information hinders the automatic flight of the aircraft, and by controlling the automatic flight of the aircraft according to this determination result, excessive stopping of the automatic flight can be suppressed and the efficiency of the harvesting operation can be increased.

Brief Description of the Drawings

[0018] [Figure 1] It is a right side view of the body of a conventional combine. [Figure 2] It is a plan view of the body of a conventional combine. [Figure 3] It is a block diagram showing a configuration related to the control of the aircraft. [Figure 4] It is a flowchart showing a processing procedure related to the control of the aircraft. [Figure 5] It is a diagram showing an example of the specific processing of the insensitive area. [Figure 6] It is a diagram showing another example of the specific processing of the insensitive area. [Figure 7] It is a diagram showing an example of the automatic flight of the aircraft.

Modes for Carrying Out the Invention

[0019] Embodiments of the present invention will be described based on the drawings. In the following description, the direction of arrow F is the "front side of the aircraft", the direction of arrow B is the "rear side of the aircraft", the direction of arrow L is the "left side of the aircraft", and the direction of arrow R is the "right side of the aircraft". The direction of arrow U is "up" and the direction of arrow D is "down".

[0020] 〔Overall Configuration of the Combine〕 Figures 1 and 2 show a standard combine harvester, which is an example of a work vehicle. The body V of this combine harvester 1 is equipped with a crawler-type running gear 2. The front of the body V is equipped with a cutting unit 3 for cutting planted grain stalks in the field. The cutting unit 3 has a cutting width that is wider than the width of the machine body.

[0021] In the machine body V, the driver unit 4 is located behind the harvesting unit 3. The driver unit 4 is located on the right side of the front of the machine body V. To the left of the driver unit 4, a conveying unit 5 is provided for transporting the harvested crops harvested by the harvesting unit 3.

[0022] A threshing device 6 is provided behind the conveying unit 5 for threshing the harvested crops conveyed by the conveying unit 5. A straw disposal device 7 is provided behind the threshing device 6 for cutting the discarded straw.

[0023] A grain tank 8 for storing grain obtained by the threshing device 6 is provided behind the driver's unit 4 and to the right of the threshing device 6. Behind the grain tank 8, a discharge device 9 for discharging the grain stored in the grain tank 8 to the outside is provided. The discharge device 9 is rotatable around a pivot axis that extends in the vertical direction.

[0024] A satellite positioning module 10 is provided at the front of the driver's unit 4. The satellite positioning module 10 receives positioning signals from artificial satellites used in GNSS (Global Navigation Satellite System, such as GPS, QZSS, Galileo, GLONASS, BeiDou, etc.). Based on the received positioning signals, the satellite positioning module 10 generates positioning data indicating the vehicle's position.

[0025] The aircraft V is equipped with a communication unit 11 (see Figure 3) that can connect to an external communication network. The communication unit 11 is configured to communicate with an external server or the like through the communication network.

[0026] The machine V is configured to be able to travel automatically by the control unit 12 (see Figure 3), and is configured to be able to travel by the travel device 2 while the harvesting unit 3 harvests the planted grain stalks in the field.

[0027] 〔camera〕 As shown in Figures 1, 2, and 3, the aircraft V is equipped with multiple cameras 13 as sensors. Specifically, the aircraft V is equipped with a front camera 13f, a right camera 13r, a left camera 13l, and a rear camera 13b as multiple cameras 13. The output signals from these four cameras 13 are output to the surrounding information processing device 14 (surrounding information generation unit 14a). The surrounding information processing device 14 (surrounding information generation unit 14a) generates a captured image 15 (an example of surrounding information) based on the output signals from the cameras 13.

[0028] The four cameras 13 may primarily capture fields, ridges, roads, parts of the aircraft V, and the area surrounding the aircraft V. In this embodiment, the captured images 15 are still images. However, the captured images 15 may also be videos and images that display still images in sequence. The shooting range 13a (angle of view) of each camera 13 is 180° or more horizontally (e.g., 195°) and 100° or more vertically (e.g., 120°) with respect to the optical axis (shooting direction) when the optical axis is pointed horizontally.

[0029] [Front camera] As shown in Figures 1 and 2, the front camera 13f is located in the center of the aircraft body V in the left-right direction. The front camera 13f is installed at the front of the driver's unit 4.

[0030] The front camera 13f is pointed diagonally downwards and forwards, in other words, it is pointed towards the harvesting unit 3. The front camera 13f is positioned so that its shooting range 13a includes the entire harvesting unit 3 and the area surrounding the harvesting unit 3 (part of the field). In other words, the shooting range 13a of the front camera 13f includes the area surrounding the harvesting unit 3, which includes part of the conveying unit 5, the area in front of the harvesting unit 3, the area to the right of the harvesting unit 3, the area to the left of the harvesting unit 3, and the area to the left of the conveying unit 5.

[0031] [Right camera] As shown in Figures 1 and 2, the right camera 13r is located in the center of the aircraft body V in the front-rear direction. The right camera 13r is located on the right side of the control unit 4. More specifically, the right camera 13r is located on the right side of the rear of the control unit 4.

[0032] The right camera 13r is pointed diagonally downward to the right, or in other words, to the right of the driver's unit 4. The right camera 13r is positioned so that its shooting range 13a includes the area to the right of the driver's unit 4, the area to the right of the grain tank 8, and the area to the right of the harvesting unit 3.

[0033] [Left camera] As shown in Figures 1 and 2, the left camera 13l is located in the center of the machine body V in the front-to-back direction. The left camera 13l is located on the left side of the threshing machine 6. More specifically, the left camera 13l is located on the left side of the front of the threshing machine 6.

[0034] The left camera 13l is pointed diagonally downward to the left, or in other words, to the left of the threshing device 6. The left camera 13l is positioned so that its shooting range 13a includes the area to the left of the threshing device 6, the area to the left of the conveying unit 5, the area in front of the cutting unit 3, and the area to the left of the cutting unit 3.

[0035] [Rear camera] As shown in Figures 1 and 2, the rear camera 13b is located in the center of the aircraft body V in the left-right direction. The rear camera 13b is installed at the rear of the grain tank 8.

[0036] The rear camera 13b is pointed diagonally downward and rearward, in other words, it is pointed towards the rear of the straw removal device 7. The rear camera 13b is positioned so that its shooting range 13a includes the entire straw removal device 7 and the area surrounding the straw removal device 7 (part of the field). In other words, the shooting range 13a of the rear camera 13b includes the area behind the straw removal device 7, the area to the right of the straw removal device 7, the area to the left of the straw removal device 7, the area to the right of the grain tank 8, and the area to the left of the threshing device 6, as the area surrounding the straw removal device 7.

[0037] As described above, the aircraft V is equipped with multiple cameras 13 in a clockwise direction in a plan view, in the order of front camera 13f, right camera 13r, rear camera 13b, and left camera 13l. The front camera 13f is adjacent to the right camera 13r in a clockwise direction. The front camera 13f is adjacent to the left camera 13l in a counterclockwise direction.

[0038] The right camera 13r is adjacent to the rear camera 13b in a clockwise direction. The right camera 13r is adjacent to the front camera 13f in a counterclockwise direction. The rear camera 13b is adjacent to the left camera 13l in a clockwise direction. The rear camera 13b is adjacent to the right camera 13r in a counterclockwise direction. The left camera 13l is adjacent to the front camera 13f in a clockwise direction. The left camera 13l is adjacent to the rear camera 13b in a counterclockwise direction.

[0039] The front camera 13f and the right camera 13r have shooting ranges 13a that include the area in front of the harvesting unit 3 and the area to the right of the harvesting unit 3. In other words, the shooting ranges 13a of the front camera 13f and the right camera 13r overlap. Therefore, the front camera 13f is adjacent to the right camera 13r in the shooting range 13a.

[0040] The front camera 13f and the left camera 13l have shooting ranges 13a that include the area in front of the harvesting unit 3, the area to the left of the harvesting unit 3, and the area to the left of the conveying unit 5. In other words, the shooting ranges 13a of the front camera 13f and the left camera 13l overlap. Therefore, the front camera 13f is adjacent to the left camera 13l in the shooting range 13a.

[0041] The rear camera 13b and the left camera 13l include the area to the left of the threshing machine 6 in their shooting range 13a. In other words, the shooting ranges 13a of the rear camera 13b and the left camera 13l overlap. Therefore, the rear camera 13b is adjacent to the left camera 13l in the shooting range 13a.

[0042] The rear camera 13b and the right camera 13r have a shooting range 13a that includes the area to the right of the grain tank 8. In other words, the shooting ranges 13a of the rear camera 13b and the right camera 13r overlap. Therefore, the rear camera 13b is adjacent to the right camera 13r in the shooting range 13a.

[0043] [Control Unit] As shown in Figure 3, the control unit 12 is located on the vehicle V. The control unit 12 includes a vehicle position calculation unit 12a, a storage unit 12b, a route calculation unit 12c, a driving control unit 12d, and an obstacle processing unit 12e. The control unit 12 is equipped with a microcomputer and performs control of the vehicle V according to a pre-set program.

[0044] The vehicle position calculation unit 12a calculates the position coordinates of the aircraft V over time based on positioning data generated by the satellite positioning module 10. The storage unit 12b has pre-stored date, time, field information, and field map acquired from the communication unit 11. The storage unit 12b is composed of, for example, an HDD, ROM, or non-volatile memory.

[0045] The route calculation unit 12c calculates a travel route L for the automatic driving of the machine V based on the time-series position coordinates of the machine V calculated by the self-position calculation unit 12a and the field map stored in the storage unit 12b.

[0046] The driving control unit 12d sets the next driving path L from among the driving paths L calculated by the path calculation unit 12c. Then, the driving control unit 12d controls the automatic driving of the machine V based on the position coordinates of the machine V calculated by the self-position calculation unit 12a and the driving path L set by the path calculation unit 12c. Specifically, the driving control unit 12d controls the driving device 2 and the harvesting unit 3 so that the machine V drives along the set driving path L. Then, the driving control unit 12d operates the harvesting unit 3 when the machine V is driving along the driving path L. In this way, the control unit 12 controls the automatic driving of the machine V.

[0047] The obstacle processing unit 12e detects the location and type of obstacle. The obstacle processing unit 12e is a learning-type unit that has been trained to output obstacle information from the captured image 15 input from the surrounding information processing unit 14. In this embodiment, the obstacle processing unit 12e is a trained unit using deep learning, and the obstacle information output based on the captured image 15 includes the location and type of the obstacle.

[0048] The driving control unit 12d controls the driving device 2 based on obstacle information from the obstacle processing unit 12e when the machine V is moving.

[0049] The control unit 12 outputs the machine information of the combine harvester 1 stored in the memory unit 12b to the surrounding information processing device 14. The machine information includes driving data such as the vehicle speed of the machine V, and data indicating the operating status such as forward, reverse, turning, and stopping.

[0050] Control devices such as a steering control device (not shown) and a main gear shift lever (not shown) are connected to the control unit 12. The control unit 12 generates machine information based on the operations input to these control devices. Alternatively, the control unit 12 may generate machine information based on the outputs of sensors (not shown) provided on these control devices, the harvesting unit 3, the discharge device 9, etc.

[0051] [Surrounding Information Processing Device] The configuration of the surrounding information processing device 14 will be described below with reference to the block diagram in Figure 3.

[0052] The aircraft V is equipped with an ambient information processing device 14. The ambient information processing device 14 comprises an ambient information generation unit 14a, a specification unit 14b, a determination unit 14c, and a size determination unit 14d. The ambient information processing device 14 is equipped with a microcomputer and processes ambient information according to a pre-set program. For example, the ambient information processing device 14 is located in the operation unit 4.

[0053] The surrounding information generation unit 14a generates an image 15 (an example of surrounding information) that shows the situation around the aircraft V based on the output signals from the front camera 13f, right camera 13r, left camera 13l, and rear camera 13b. Specifically, the surrounding information generation unit 14a generates one image 15 based on the output signal of one camera 13.

[0054] The surrounding information generation unit 14a is connected to the front camera 13f, the right camera 13r, the left camera 13l, and the rear camera 13b. The surrounding information generation unit 14a generates a front image 15f based on the output signal output from the front camera 13f. The surrounding information generation unit 14a generates a right image 15r based on the output signal output from the right camera 13r. The surrounding information generation unit 14a generates a left image 15l based on the output signal output from the left camera 13l. The surrounding information generation unit 14a generates a rear image 15b based on the output signal output from the rear camera 13b.

[0055] The identification unit 14b identifies a dead region 16 in the captured image 15 input from the surrounding information generation unit 14a. The dead region 16 is an area in the captured image 15 where information about the surroundings of the aircraft V is not shown. In this embodiment, the identification unit 14b can identify the dead region 16 as an area in the captured image 15 where the camera 13 is shown to be dirty due to the attachment of mud, straw, etc.

[0056] The determination unit 14c determines whether the entire dead zone 16 is located inside the interpolation zone 17. The interpolation zone 17 is the area in the captured image 15 that can be interpolated by another camera 13. Details of the determination by the determination unit 14c will be described later.

[0057] [Processing related to aircraft control] Figure 4 is a flowchart showing the processing procedure for controlling the aircraft V. When the control unit 12 starts the automatic movement of the aircraft V, the output signal from the camera 13 is input to the surrounding information generation unit 14a (step S01). When the surrounding information generation unit 14a generates the captured image 15 (step S02), the captured image 15 is output to the identification unit 14b. The output signals from the cameras 13 are output to the surrounding information generation unit 14a at predetermined time intervals. The output signals from the four cameras 13 may be input to the surrounding information generation unit 14a simultaneously, or they may be input at different timings.

[0058] If the identification unit 14b does not identify a dead zone 16 in the captured image 15 (Step S03; No), the control unit 12 continues the automatic movement of the machine V (Step S07). If the identification unit 14b identifies a dead zone 16 in the captured image 15 (Step S03; Yes), the captured image 15 is output to the determination unit 14c.

[0059] If the determination unit 14c determines that the entire dead zone 16 is located inside the interpolation zone 17 (Step S04; Yes), and the identification unit 14b identifies the dead zone 16 in the image 15 captured by the adjacent camera 13 (Step S05; Yes), the control unit 12 stops the automatic movement of the aircraft V (Step S08). This makes it possible to determine that there is an impediment to the automatic movement of the aircraft V.

[0060] If the determination unit 14c determines that the entire dead zone 16 is located inside the interpolation zone 17 (Step S04; Yes), and the identification unit 14b does not identify the dead zone 16 in the image 15 captured by the adjacent camera 13 (Step S05; No), the control unit 12 continues the automatic driving of the aircraft V (Step S07).

[0061] If the determination unit 14c determines that the entire dead zone 16 is not located inside the interpolation zone 17 (step S04; No), the captured image 15 is output to the size determination unit 14d. If the size determination unit 14d determines that the size of the dead zone 16 is below a predetermined threshold (step S06; Yes), the control unit 12 continues the automatic movement of the machine V (step S07). This makes it possible to determine that the size of the dead zone 16 is not large enough to interfere with the automatic movement of the machine V, and harvesting work can be performed without stopping the automatic movement of the machine V, thereby increasing the efficiency of the harvesting work.

[0062] If the size determination unit 14d determines that the size of the dead zone 16 is not below a predetermined threshold (step S06; No), the control unit 12 stops the automatic movement of the machine V (step S08).

[0063] [Regarding the process for identifying the dead zone] Next, an example of the process for identifying the dead zone 16 will be described. Figures 5 and 6 show examples of identifying the dead zone 16 in the previously captured image 15f taken by the front camera 13f. The previously captured image 15f shows an obstacle 18, field mud 19 adhering to the lens of the front camera 13f, the harvesting unit 3, and the area surrounding the harvesting unit 3 (part of the field). As part of the field, the previously captured image 15f shows the area in front of the harvesting unit 3, the area to the right of the harvesting unit 3, the area to the left of the harvesting unit 3, and the area to the left of the conveying unit 5.

[0064] The location and type of the obstacle 18 are detected by the obstacle processing unit 12e. In this embodiment, the obstacle processing unit 12e outputs the location of the obstacle 18 and the fact that the obstacle 18 is a person as obstacle information. Based on this obstacle information, the driving control unit 12d decelerates or stops the driving device 2 as necessary.

[0065] The identification unit 14b identifies regions in the previously captured image 15f, generated at predetermined time intervals, where the adhesion of mud 19 is shown without any change in brightness over time, as a first dead region 16a (an example of a dead region 16) and a second dead region 16b (an example of a dead region 16). However, it is not limited to this; for example, the identification unit 14b may identify regions in the previously captured image 15f, generated at predetermined time intervals, where the adhesion of mud 19 is shown without any change in color over time, as the first dead region 16a and the second dead region 16b.

[0066] The identification unit 14b receives input from the control unit 12, including the date, time, field information, vehicle speed, and vehicle information such as the position coordinates of the vehicle V. Based on this information and the temporal change in brightness in the previously captured image 15f, the identification unit 14b identifies a region showing a part of the vehicle V in the previously captured image 15f, a region showing the scenery such as a field, and a dead zone 16. However, the identification unit 14b may have a region showing a part of the vehicle V pre-set and stored in its memory.

[0067] The determination unit 14c determines whether the entirety of the first insensitive region 16a and the second insensitive region 16b are located inside the interpolation region 17.

[0068] The interpolation region 17 represents the region in the previously captured image 15f that can be interpolated by the front camera 13f and the adjacent right camera 13r and left camera 13l. The interpolation region 17 is pre-set and stored in the determination unit 14c. In this embodiment, the interpolation region 17 is located on both the left and right sides of the previously captured image 15f and is an area that is approximately 20% of the total width in the vertical direction and the width in the horizontal direction of the previously captured image 15f. However, it is not limited to this, and for example, the interpolation region 17 may be the region in the previously captured image 15f where the left camera 13l and right camera 13r and the shooting range 13a overlap. In other words, the interpolation region 17 may be the region in front of the harvesting unit 3 and the region to the right of the harvesting unit 3. Alternatively, the interpolation region 17 may be the region in front of the harvesting unit 3 and the region to the left of both the harvesting unit 3 and the conveying unit 5.

[0069] As shown in Figure 5, if the determination unit 14c determines that the entire first dead zone 16a is located inside a preset interpolation zone 17, the determination unit 14c outputs a signal to the control unit 12. Also, if the determination unit 14c determines that the entire second dead zone 16b is not located inside a preset interpolation zone 17, the determination unit 14c outputs a signal to the size determination unit 14d. If the size determination unit 14d determines that the size of the second dead zone 16b is not below a threshold, it outputs a signal to the control unit 12. The driving control unit 12d decelerates or stops the driving device 2 according to these signals. As a result, the control unit 12 stops the automatic driving of the machine V.

[0070] As shown in Figure 6, if the determination unit 14c determines that the entire first dead zone 16a is located inside the interpolation zone 17, the determination unit 14c outputs a signal to the control unit 12. Also, if the determination unit 14c determines that the entire second dead zone 16b is not located inside the interpolation zone 17, the determination unit 14c outputs a signal to the size determination unit 14d. If the size determination unit 14d determines that the size of the second dead zone 16b is below a threshold, it outputs a signal to the control unit 12. The control unit 12 continues the automatic driving of the machine V according to these signals.

[0071] Figure 7 illustrates an example where the field has a rectangular shape. It shows the outer perimeter area LO, where the machine V manually traveled during the initial circular run around the outer edge of the field, and the harvested area G1 and unharvested area G2, located inside the outer perimeter area LO. In the illustrated example, the machine V completes three circular runs. The machine V then automatically travels within the area inside the outer perimeter area LO. The machine V's travel path L is shown within the area inside the outer perimeter area LO.

[0072] Figure 7 shows an example in which, when the aircraft V is automatically traveling to position P1, the determination unit 14c determines that the entire dead zone 16 is not located inside the interpolation zone 17, and the size determination unit 14d determines that the size of the dead zone 16 is not below a predetermined threshold.

[0073] The control unit 12 continues the automatic movement of the machine V to the edge GE of the field. When the machine V arrives at the edge GE of the field (P2), the control unit 12 stops the automatic movement of the machine V. As a result, the machine V does not stop in the center of the field, so it can be stopped in a position where it is easy for the operator to perform maintenance work such as checking the status of the camera 13. In this embodiment, the edge GE of the field is the outer edge of the field and its surroundings. However, the edge GE of the field may also be the outer region LO.

[0074] The “first camera” described in the claims is any one camera 13 from the front camera 13f, right camera 13r, left camera 13l, and rear camera 13b in the embodiment. The “second camera” is a camera 13 adjacent to the “first camera.” If there is one camera 13 adjacent to the “first camera,” there is one “second camera.” If there are multiple cameras 13 adjacent to the “first camera,” there are multiple “second cameras.” For example, if the front camera 13f corresponds to the “first camera,” then the left camera 13l and right camera 13r adjacent to the front camera 13f correspond to the “second cameras.” In this case, the “first surrounding information” described in the claims corresponds to the front captured image 15f. Also, the “second surrounding information” corresponds to the right captured image 15r and the left captured image 15l.

[0075] When the right camera 13r corresponds to the "first camera," the front camera 13f and rear camera 13b adjacent to the right camera 13r correspond to the "second camera." In this case, the "first surrounding information" corresponds to the right captured image 15r. The "second surrounding information" corresponds to the front captured image 15f and rear captured image 15b.

[0076] When the rear camera 13b corresponds to the "first camera," the right camera 13r and left camera 13l, which are adjacent to the rear camera 13b, correspond to the "second camera." In this case, the "first surrounding information" corresponds to the rear captured image 15b. The "second surrounding information" corresponds to the right captured image 15r and left captured image 15l.

[0077] If the left camera 13l corresponds to the "first camera," then the rear camera 13b and front camera 13f, which are adjacent to the left camera 13l, correspond to the "second camera." In this case, the "first surrounding information" corresponds to the left captured image 15l. Also, the "second surrounding information" corresponds to the rear captured image 15b and front captured image 15f.

[0078] [Another embodiment] The present invention is not limited to the embodiments described above. For example, it may be configured as in the following alternative embodiments. In the alternative embodiments described below, components identical to those in the embodiments are denoted by the same numbers and reference numerals as in the embodiments.

[0079] [1] In the above embodiment, the aircraft V is provided with a camera 13 as a sensor. However, the aircraft V may also be provided with a LiDAR unit (not shown) as a sensor. In this case, the surrounding information generation unit 14a generates output data (an example of surrounding information) based on the output signal of the LiDAR unit (not shown). In this case, the control unit 12 does not need to include an obstacle processing unit 12e.

[0080] Furthermore, the identification unit 14b identifies areas in the output data of the LiDAR unit (not shown) where no surrounding information is shown as dead regions 16. For example, the identification unit 14b may identify areas in the output data generated at predetermined time intervals where the adhesion of mud 19 without change over time is shown as dead regions 16. The identification unit 14b may also identify areas in the output data generated at predetermined time intervals that do not change over time as dead regions 16 due to a failure of the light receiving sensor of the LiDAR unit (not shown).

[0081] [2] In the above embodiment, four cameras 13 are provided on the aircraft V. However, the number of cameras 13 is not limited to this, and can be any number as long as it is multiple. For example, the number of cameras 13 may be two or three, or four or more.

[0082] [3] In the above embodiment, the identification unit 14b identifies areas in the captured images 15 generated at predetermined time intervals where the adhesion of mud 19 does not change over time as a dead region 16. However, it is not limited to this, and for example, the identification unit 14b may identify areas in the captured images 15 generated at predetermined time intervals where there is no change over time due to a malfunction of the light receiving sensor of the camera 13 or a malfunction of the ambient information generation unit 14a as a dead region 16.

[0083] [4] If the determination unit 14c determines that the entire insensitive area 16 is located inside the complementary area 17 (step S04; Yes), the control unit 12 may continue the automatic driving of the machine V without performing the process in step S05 (step S07).

[0084] [5] If the determination unit 14c determines that the entire insensitive area 16 is not located inside the complementary area 17 (step S04; No), the control unit 12 may stop the automatic movement of the machine V without performing the process in step S06 (step S08).

[0085] [6] In the above embodiment, the control unit 12 includes an obstacle processing unit 12e. Alternatively, the surrounding information processing device 14 may include an obstacle processing unit 12e. In this case, the surrounding information processing device 14 does not need to include a specific unit 14b. The obstacle processing unit 12e may identify obstacles 18 and dead zones 16 in the captured image 15 input from the surrounding information generation unit 14a using a trained unit using deep learning.

[0086] [7] The device is equipped with a display unit (not shown), and the output results of the ambient information processing device 14 may be displayed on the display unit (not shown).

[0087] [8] In the above embodiment, a conventional combine harvester is shown as an example of a work vehicle. However, the system is not limited to this, and can be applied to a system for controlling work vehicles such as self-propelled combine harvesters, tractors, rice transplanters, paddy field direct seeders, corn harvesters, potato harvesters, carrot harvesters, and construction work machines.

[0088] [9] The running gear 2 may be wheeled or semi-crawler type.

[0089]

[10] Some or all of the surrounding information generation unit 14a, the identification unit 14b, the determination unit 14c, and the size determination unit 14d may be located outside the combine harvester 1. For example, some or all of the surrounding information generation unit 14a, the identification unit 14b, the determination unit 14c, and the size determination unit 14d may be located outside the combine harvester 1 in a PC, mobile information terminal, or management server.

[0090]

[11] The configurations disclosed in the above embodiments (including other embodiments, the same applies hereinafter) can be applied in combination with configurations disclosed in other embodiments, insofar as they do not cause a contradiction, and the embodiments disclosed herein are illustrative, and the embodiments of the present invention are not limited thereto and can be modified as appropriate without departing from the object of the present invention. [Industrial applicability]

[0091] The present invention is applicable to work vehicles and systems for controlling work vehicles. [Explanation of Symbols]

[0092] 1: Combine harvester (work vehicle) 12: Control Unit 13: Camera (sensor) 13f: Front camera (Camera 1, Camera 2) 13r: Right camera (1st camera, 2nd camera) 13l: Left camera (Camera 1, Camera 2) 13b: Rear camera (1st camera, 2nd camera) 14a: Surrounding Information Generation Unit 14b: Specific part 14c: Judgment section 14d: Size determination unit 15: Captured image (surrounding information) 15f: Previously captured image (1st surrounding information, 2nd surrounding information) 15r: Right-hand image (first surrounding information, second surrounding information) 15l: Left image (first surrounding information, second surrounding information) 15b: Post-capture images (first surrounding information, second surrounding information) 16: Dead area 16a: 1st insensitive area 16b:Second blind area 17: Complement area V: Aircraft

Claims

1. A system for controlling a work vehicle equipped with a machine, Multiple sensors for detecting the surroundings of the aforementioned aircraft, A surrounding information generation unit generates surrounding information indicating the conditions around the aircraft based on the output signal of one of the sensors, A unit for identifying a dead zone in the aforementioned ambient information, A determination unit that determines whether the entire insensitive region is located within a complementary region that can be complemented by other sensors, The aircraft comprises a control unit that controls the automatic movement of the aircraft, The aforementioned sensor is a camera, The surrounding information generation unit generates the surrounding information, which is a captured image showing the conditions around the aircraft, based on the output signal of one of the cameras. A portion of the captured image is the interpolation region, When the determination unit determines that the vehicle is located within the aforementioned complementary region, the control unit continues the automatic movement of the vehicle. A system in which the control unit stops the automatic movement of the machine when the determination unit determines that the machine is not located inside the aforementioned complementary region.

2. The camera comprises a first camera and a second camera adjacent to the first camera, The shooting range of the first camera and the shooting range of the second camera are different. The system according to claim 1, wherein the interpolation region in the image captured by the first camera is a region whose shooting range overlaps with that of the second camera.

3. The system according to claim 1, wherein when the determination unit determines that the machine is not located within the complementary region, the control unit stops the automatic movement of the machine at the edge of the field.

4. The camera comprises a first camera and a second camera adjacent to the first camera. The surrounding information includes first surrounding information generated based on the output signal of the first camera and second surrounding information generated based on the output signal of the second camera. The system according to claim 1, wherein when the identifying unit identifies the dead zone in the first and second surrounding information, the control unit controls the automatic driving of the machine to stop.

5. The system further includes a size determination unit that determines whether the size of the insensitive region is below a predetermined threshold, The system according to claim 1, wherein the control unit continues the automatic driving of the machine when the determination unit determines that the machine is not located inside the complementary region and the size determination unit determines that the size is below the threshold.

6. The aircraft and, Multiple sensors for detecting the surroundings of the aforementioned aircraft, A surrounding information generation unit generates surrounding information indicating the conditions around the aircraft based on the output signal of one of the sensors, A unit for identifying a dead zone in the aforementioned ambient information, A determination unit that determines whether the entire insensitive region is located within a complementary region that can be complemented by other sensors, The aircraft comprises a control unit that controls the automatic movement of the aircraft, The aforementioned sensor is a camera, The surrounding information generation unit generates the surrounding information, which is a captured image showing the conditions around the aircraft, based on the output signal of one of the cameras. A portion of the captured image is the interpolation region, When the determination unit determines that the vehicle is located within the aforementioned complementary region, the control unit continues the automatic movement of the vehicle. The control unit stops the automatic movement of the work vehicle when it determines that the work vehicle is not located inside the aforementioned complementary region.

Images