Control system and work machine

Abstract

The invention provides a control system and a work machine which perform obstacle avoidance control using a plurality of sensors with high reliability. A control system for a work machine is provided with: a plurality of sensors (2) for sensing an obstacle; a travel state detection unit (51) that detects the travel state of the work machine; an avoidance control unit (64) that executes avoidance control, which is control for avoiding interference between the work machine and the obstacle, on the basis of the detection results of the plurality of sensors (2); and a distance changing unit (52) that changes at least one of the detectable distance and the control execution distance of each of the plurality of sensors (2) with respect to the obstacle. The avoidance control unit (64) executes the avoidance control when the distance between the work machine and the obstacle is equal to or less than the control execution distance, and the distance changing unit (52) changes the detectable distance and / or the control execution distance on the basis of the travel state detected by the travel state detection unit (51).

Description

Technical Field

[0001] The present invention relates to a control system for controlling a working machine and the working machine. Background Art

[0002] Work machines in recent years are equipped with an obstacle sensor for detecting an obstacle, and have an avoidance control function for executing avoidance steering or emergency stopping in order to avoid collision with the obstacle when the obstacle sensor detects an obstacle.

[0003] According to Patent Document 1, an automatic driving working vehicle having an avoidance control function is known. In the avoidance control of Patent Document 1, the obstacle detection area of ​​the obstacle sensor can be changed based on the detected working driving state. For example, when the machine body is equipped with a working device with a large lateral width, the lateral width of the obstacle detection area is expanded. In addition, when the machine body turns left (or right), the left side (or right side) of the obstacle detection area is expanded.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2018-113937 Summary of the invention

[0007] Technical problem to be solved by the invention

[0008] Although Patent Document 1 describes the expansion and reduction of the obstacle area of ​​the obstacle sensor, it does not disclose changing the detectable distance of the obstacle sensor relative to the obstacle and the control execution distance (the distance for executing the avoidance control) according to the situation. Changing the detectable distance of the obstacle sensor relative to the obstacle and the control execution distance according to the situation is important for executing the obstacle avoidance control with higher reliability. In the obstacle avoidance control with low reliability, for example, unnecessary parking of the working vehicle due to unnecessary obstacle detection will frequently occur. In addition, although Patent Document 1 describes the change of the obstacle detection area of ​​the sensor that senses the center of the front representing multiple sensors, it does not describe changing the obstacle detection areas of multiple obstacle sensors individually.

[0009] In conventional obstacle avoidance control, there is a problem in which a disadvantage such as the work vehicle being stopped due to the obstacle avoidance control occurs even though there is no interference between the vehicle and the obstacle.

[0010] An object of the present invention is to provide a control system and a working machine that execute obstacle avoidance control using a plurality of sensors with high reliability.

[0011] Technical solutions for solving technical problems

[0012] The control system of the working machine of the present invention is characterized in that it comprises: a plurality of sensors, which respectively sense different directions and detect obstacles relative to the working machine; a driving state detection unit, which detects the driving state of the working machine; an avoidance control unit, which performs avoidance control as a control for avoiding interference between the working machine and the obstacle based on the detection results of the plurality of sensors; a distance change unit, which changes at least one of a detectable distance and a control execution distance of each of the plurality of sensors relative to the obstacle, the avoidance control unit performs the avoidance control when the distance between the working machine and the obstacle is less than the control execution distance, and the distance change unit changes at least one of the detectable distance and the control execution distance based on the driving state detected by the driving state detection unit.

[0013] The working machine of the present invention is characterized in that it is equipped with: multiple sensors, which respectively sense different directions to detect obstacles; a driving state detection unit, which detects the driving state of the machine body; an avoidance control unit, which performs avoidance control as a control for avoiding interference between the machine body and the obstacle based on the detection results of the multiple sensors; a distance change unit, which changes at least one of the detectable distance and the control execution distance of each of the multiple sensors relative to the obstacle, the avoidance control unit performs the avoidance control when the distance between the machine body and the obstacle is less than the control execution distance, and the distance change unit changes at least one of the detectable distance and the control execution distance based on the driving state detected by the driving state detection unit.

[0014] According to this structure, at least one of the detectable distances and control execution distances of each of the plurality of sensors relative to an obstacle is changed by the distance changing unit based on the driving state detected by the driving state detecting unit. Thus, each sensor can detect obstacles with higher reliability, and as a result, it is possible to suppress undesirable situations such as executing avoidance control (stopping the working machine, etc.) even when no interference occurs between the machine body and the obstacle.

[0015] That is, according to the present configuration, it is possible to realize a control system and a working machine that can execute obstacle avoidance control using a plurality of sensors with high reliability.

[0016] Furthermore, in the present invention, it is preferred that the plurality of sensors include a distance measuring sensor and a camera that generates a captured image. According to this structure, more effective obstacle detection can be performed by combining a distance measuring sensor that has the advantages of being able to accurately obtain the distance to an obstacle and having a long obstacle detectable distance and a camera that is less restricted by environmental conditions such as weather and can reliably detect obstacles at a close distance.

[0017] Furthermore, in the present invention, it is preferable that the detectable distance of the distance measuring sensor is set longer than the detectable distance of the camera. According to this configuration, effective obstacle detection can be performed by taking advantage of the obstacle detection of both.

[0018] Furthermore, in the present invention, it is preferred that the plurality of sensors include a front detection sensor that uses the front of the working machine as a detection range and a rear detection sensor that uses the rear of the working machine as a detection range, and when the working machine is moving forward, at least one of the detectable distance and the control execution distance of the rear detection sensor is made shorter than when the working machine is moving backward by the distance changing unit, and when the working machine is moving backward, at least one of the detectable distance and the control execution distance of the front detection sensor is made shorter than when the working machine is moving forward by the distance changing unit. When the machine body is moving forward, the importance of obstacle detection in front of the machine body is higher than that of obstacle detection in the rear of the machine body, so the detection capability of the front detection sensor is increased, and in order to reduce the possibility of detecting obstacles that do not need to be detected, it is preferred to reduce the detection capability of the rear detection sensor. In this structure, considering that the importance of obstacle detection in front of the machine body is higher than that of obstacle detection in the rear of the machine body when the machine body is moving forward, unnecessary stopping, deceleration, etc. of the working machine caused by unnecessary obstacle detection is suppressed.

[0019] Furthermore, in the present invention, it is preferred that the plurality of sensors include a left detection sensor that uses the left side of the working machine as a detection range and a right detection sensor that uses the right side of the working machine as a detection range, and when the working machine turns left, the distance changing unit makes at least one of the detectable distance and the control execution distance of the right detection sensor shorter than when the working machine turns right, and when the working machine turns right, the distance changing unit makes at least one of the detectable distance and the control execution distance of the left detection sensor shorter than when the working machine turns left. When the machine body turns left, the importance of obstacle detection on the left side of the machine body is higher than that of obstacle detection on the right side of the machine body, so the detection capability of the left detection sensor is increased, and in order to reduce the possibility of detecting obstacles that do not need to be detected, it is preferred to reduce the detection capability of the right detection sensor. Similarly, when the machine body turns right, the importance of obstacle detection on the right side of the machine body is higher than that of obstacle detection on the left side of the machine body, so the detection capability of the right detection sensor is increased, and in order to reduce the possibility of detecting obstacles that do not need to be detected, the detection capability of the left detection sensor is reduced. In the present structure, considering that when the machine turns left, obstacle detection on the left side of the machine is more important than obstacle detection on the right side of the machine, and considering that when the machine turns right, obstacle detection on the right side of the machine is more important than obstacle detection on the left side of the machine, unnecessary stopping, deceleration, etc. of the working machine caused by unnecessary obstacle detection is suppressed.

[0020] Furthermore, in the present invention, it is preferred that a working device is provided in a liftable manner at the front of the working machine, and the plurality of sensors include a front detection sensor for detecting the obstacle in front of the working machine, and when the driving state detection unit detects the rising state of the working device, the obstacle detection by the front detection sensor is terminated. When the front of the machine body is partially blocked by the working device in the rising state, it is assumed that the front detection sensor mistakenly detects the working device as an obstacle. According to this structure, such a problem is avoided, and unnecessary stopping and deceleration of the working machine caused by unnecessary obstacle detection is suppressed.

[0021] Furthermore, in the present invention, it is preferred that an obstacle position calculation unit is provided, which calculates the position of the detected obstacle based on the vehicle position of the working machine calculated by the vehicle position calculation unit and the obstacle detection result of the sensor, and ignores the obstacle when the obstacle is located outside the working area (if the working machine is an agricultural working machine, it is a field) that is the driving object of the working machine. The working machine travels in the working area, so even if the detected obstacle is located near the machine body, as long as the position of the obstacle is outside the working area, it is considered that there is substantially no danger of collision. In the present structure taking this situation into consideration, when the obstacle is located outside the working area, unnecessary stopping, deceleration, etc. of the working machine caused by unnecessary obstacle detection is suppressed. Of course, the following structure can also be adopted: when the obstacle is very close to the machine body or when the obstacle is considered to be close to the machine body through obstacle detection over time, as long as there is no strong safety fence separating the inside and outside of the working area, the obstacle is not ignored and avoidance control is performed. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is a left side view of a combine harvester.

[0023] Figure 2 It is a diagram showing peripheral travel.

[0024] Figure 3 This is a schematic diagram showing the detectable distance and control execution distance of each distance measuring sensor as an obstacle sensor.

[0025] Figure 4 This is a schematic diagram showing the detectable distance and control execution distance of each camera as an obstacle sensor.

[0026] Figure 5 It is a block diagram showing the structure of a control system.

[0027] Figure 6 This is a control flow chart showing the flow of obstacle avoidance control.

[0028] Description of Reference Numerals

[0029] 2 Surrounding monitoring sensor group (sensor)

[0030] 19 Machine

[0031] 21mm wave radar

[0032] 21F front millimeter wave radar

[0033] 21B post millimeter wave radar

[0034] 22 cameras

[0035] 22B rear camera

[0036] 22F front camera

[0037] 22L Left Camera

[0038] 22R right camera

[0039] 30 status detection sensor group

[0040] 40 sensor signal processing unit

[0041] 41 first signal processing unit

[0042] 42 second signal processing unit

[0043] 43 Obstacle position calculation unit

[0044] 51 Driving status detection unit

[0045] 52 Distance Change Department

[0046] 53 Vehicle position calculation unit

[0047] 60 Driving control unit

[0048] 64 Avoidance Control Department

[0049] 80 satellite positioning module

[0050] 81 Inertial Measurement Unit

[0051] 100 control units

[0052] H Harvesting unit (operating device)

[0053] LC control execution distance

[0054] LD detectable distance DETAILED DESCRIPTION

[0055] A method for implementing the present invention will be described based on the accompanying drawings. In addition, in the following description, unless otherwise specified, the direction of the arrow F in the figure is set to "front", and the direction of the arrow B is set to "rear". In addition, the direction of the arrow U in the figure is set to "up", and the direction of the arrow D is set to "down". In addition, the direction of the arrow N in the figure is set to "north", the direction of the arrow S is set to "south", the direction of the arrow E is set to "east", and the direction of the arrow W is set to "west". In addition, "left" and "right" are defined based on the front and rear direction line of the working machine moving forward.

[0056] [Overall structure of a combine harvester]

[0057] like Figure 1 As shown, a conventional combine harvester (equivalent to the "working machine" of the present invention) comprises a harvesting section H (equivalent to the "working device" of the present invention), a crawler-type traveling device 11, a driving section 12, a threshing device 13, a grain box 14, a transport section 16, a grain discharge device 18, and a satellite positioning module 80.

[0058] The travel device 11 is attached to the lower part of the machine body 19 of the combine harvester. The travel device 11 is driven by the power from the engine (not shown) mounted on the combine harvester. And the combine harvester can travel by the travel device 11.

[0059] Moreover, the driving|operation part 12, the threshing apparatus 13, and the grain tank 14 are attached to the upper side rather than the traveling device 11. The driving|operation part 12 has the driver's seat 12a. The user (operator) can get on the driving|operation part 12.

[0060] The grain discharge device 18 is provided on the upper side of the grain tank 14. Moreover, the satellite positioning module 80 is attached to the upper surface of the driving part 12.

[0061] The harvesting unit H is mounted on the front of the combine harvester. The transport unit 16 is provided at the rear of the harvesting unit H. The harvesting unit H includes left and right crop dividers 10, a cutter 15, and a reel 17. The harvesting unit H can be raised and lowered together with the transport unit 16 by a lifting mechanism (not shown).

[0062] The left and right crop dividers 10 are provided at the left and right ends of the front end of the harvesting section H. The left and right crop dividers 10 divide the field 5 (see Figure 2 ) are divided into harvesting objects and non-objects. The standing grain stalks on the right side of the grain divider 10 on the left and on the left side of the grain divider 10 on the right are divided as harvesting objects. The standing grain stalks on the left side of the grain divider 10 on the left and on the right side of the grain divider 10 on the right are divided as non-objects.

[0063] The cutter 15 cuts the standing grain stalks for harvesting divided by the left and right grain dividers 10. The reel 17 is driven to rotate around a reel axis 17b along the left and right directions of the machine body, while cutting the standing grain stalks for harvesting. The harvested grain stalks cut by the cutter 15 are transported to the transport unit 16.

[0064] According to this configuration, the harvesting unit H harvests the grains in the field 5 that is the travel target of the combine harvester. The combine harvester can perform harvesting travel in which the travel device 11 travels while the harvesting unit H harvests the grains in the field 5.

[0065] The harvested grain stems harvested by the harvesting section H are transported to the rear of the machine body by the transport section 16. Thus, the harvested grain stems are transported to the threshing device 13.

[0066] The harvested grain stems are threshed in the threshing device 13. The grains obtained by the threshing are stored in the grain tank 14. The grains stored in the grain tank 14 are discharged to the outside of the machine by the grain discharge device 18 as needed.

[0067] In addition, if Figure 1 As shown, a display terminal 4 is arranged on the driving part 12. The display terminal 4 is configured to display various information. In the present embodiment, the display terminal 4 is fixed to the driving part 12. However, the present invention is not limited thereto, and the display terminal 4 may also be configured to be detachable relative to the driving part 12, and the display terminal 4 may also be located outside the combine harvester.

[0068] The combine harvester is configured to perform harvesting work in the field 5 as a work site. Figure 2 As shown, after the peripheral driving is performed, the harvesting driving is performed by automatic driving or manual driving. In addition, the peripheral driving refers to the harvesting driving performed in the peripheral area of ​​the field 5 by manual operation. However, the present invention is not limited to this, and the harvesting driving performed in the peripheral area can also be performed by automatic driving. In addition, all harvesting driving in the field can also be performed by manual driving.

[0069] exist Figure 2 In FIG. 5 , arrows are used to indicate the path along which the combine harvester travels in the peripheral travel. When the harvesting travel along this path is completed, the harvesting travel inside the field 5 is performed. Figure 2 As shown, the outer peripheral driving in the present embodiment is a harvesting driving that goes around the outermost periphery of the field 5. However, the present invention is not limited thereto, and the driving may be carried out around the outermost periphery of the field 5 twice or more.

[0070] [Obstacle detection sensor configuration]

[0071] like Figure 1 , Figure 3 as well as Figure 4As shown, the plurality of sensors 2 for obstacle detection sense different directions respectively to detect obstacles for the combine harvester. In this embodiment, the plurality of sensors 2 include a plurality of millimeter wave radars 21 as distance measuring sensors and a plurality of cameras 22 for generating captured images.

[0072] like Figure 3 and Figure 4 As shown, the millimeter wave radar 21 is composed of a front millimeter wave radar 21F installed at the front end of the cab constituting the driving unit 12 as a front ranging sensor (a type of front detection sensor that uses the front of the machine body as a detection range) and a rear millimeter wave radar 21B installed at the rear end of the threshing device 13 as a rear ranging sensor (a type of rear detection sensor that uses the rear of the machine body as a detection range). The millimeter wave radar 21 is a sensor that irradiates millimeter waves to the detection object area for sensing. As a ranging sensor, in addition to the millimeter wave radar 21, LiDAR (laser radar), sonar, etc. can also be used.

[0073] like Figure 3 and Figure 4 As shown, the camera 22 is composed of a front camera 22F installed at the front end of the cab constituting the driving part 12 (a kind of front detection sensor that uses the front of the machine body as a detection range), a right camera 22R installed at the right side of the cab constituting the driving part 12 (a kind of right detection sensor that uses the right side of the machine body as a detection range), a left camera 22L installed at the left side of the threshing device 13 (a kind of left detection sensor that uses the left side of the machine body as a detection range), and a rear camera 22B installed at the rear end of the threshing device 13 (a kind of rear detection sensor that uses the rear of the machine body as a detection range). A wide-angle lens is installed on the camera 22, and its shooting angle of view is approximately 180 degrees, so the full range of the machine body 19 is substantially covered by these cameras 22. The captured image of the camera 22 is used as an input image of a learning obstacle detection algorithm or other image recognition algorithm in a deep learning method. That is, although not particularly limited, the camera 22 in this embodiment is an AI obstacle recognition sensor.

[0074] exist Figure 3 The detectable distance LD and the control execution distance LC of the front millimeter wave radar 21F and the rear millimeter wave radar 21B are shown in FIG. Figure 4Detectable distance LD and control execution distance LC of the front camera 22F, rear camera 22B, left camera 22L and right camera 22R are shown in FIG. 6. The detectable distance LD and control execution distance LC of each can be set individually. The detectable distance LD is the distance from the machine body 19 that can detect obstacles. The control execution distance LC is the distance from the machine body 19 at which the detected obstacle becomes the object of obstacle avoidance control. The set detectable distance LD and control execution distance LC are changed based on the driving state of the combine harvester (including the morphological state and action state of the working device) as described below. The detectable distance LD and control execution distance LC of the millimeter wave radar 21 and the camera 22 are different based on the detection characteristics, but can also be substantially the same. In addition, the detectable distance LD and control execution distance LC of the front millimeter wave radar 21F and the rear millimeter wave radar 21B can also be different or the same. Similarly, the detectable distance LD and control execution distance LC of the front camera 22F, the rear camera 22B, the left camera 22L, and the right camera 22R can also be different or the same. In addition, when the millimeter-wave radar 21 (or the camera 22) is turned off, the detectable distance LD of the millimeter-wave radar 21 (or the camera 22) is zero, and obstacle detection cannot be performed regardless of the value of the control execution distance LC. Turning off the millimeter-wave radar 21 (or the camera 22) is a specific example of changing the detectable distance LD of the millimeter-wave radar 21 (or the camera 22). In addition, the control execution distance LC of the millimeter-wave radar 21 (or the camera 22) may always be less than the detectable distance LD. In this case, when the detectable distance LD is zero, the control execution distance LC is also zero.

[0075] In this embodiment, for example, when the combine harvester is moving forward, the control execution distance LC of the rear millimeter wave radar 21B and the rear camera 22B is shorter than when the combine harvester is moving backward. In addition, when moving backward, the control execution distance LC of the front millimeter wave radar 21F and the front camera 22F is shorter than when moving forward. Of course, the detectable distance LD can also be changed, or only the detectable distance LD can be changed.

[0076] When the combine harvester turns left, the control execution distance LC of the right camera 22R is shorter than when the combine harvester turns right. In addition, when turning right, the control execution distance LC of the left camera 22L is shorter than when turning left. Of course, the detectable distance LD can also be changed in the same way, or only the detectable distance LD can be changed.

[0077] Furthermore, when the harvesting unit H as the working device rises (the rising state of the working device), the harvesting unit H sometimes enters the sensing range of the front millimeter-wave radar 21F and the front camera 22F, which hinders obstacle detection, so the obstacle detection of the front millimeter-wave radar 21F and the front camera 22F is stopped, that is, the front millimeter-wave radar 21F and the front camera 22F are turned off. In addition, when the front camera 22F is the above-mentioned AI obstacle recognition sensor, it is also possible to have a structure in which the front camera 22F is not turned off even when the harvesting unit H is in the rising state.

[0078]

Description of electronic control system

[0079] Figure 5 The functional block diagram of the electronic control system related to obstacle avoidance of the combine harvester is shown. The electronic control system shown here includes a control unit 100 composed of an ECU, a surrounding monitoring sensor group 2, a state detection sensor group 30 for detecting the states of various control devices, a satellite positioning module 80, and an inertial measurement device 81. Data transmission between them is performed using an in-vehicle LAN (Local Area Network) and a normal wiring harness.

[0080] In this embodiment, the surrounding monitoring sensor group 2, which is simply referred to as the sensor 2, includes the millimeter wave radar 21 and the camera 22 as described above. The millimeter wave radar 21 includes the front millimeter wave radar 21F and the rear millimeter wave radar 21B, and the camera 22 includes the front camera 22F, the rear camera 22B, the left camera 22L, and the right camera 22R. The ranging signal from the millimeter wave radar 21 and the shooting signal (shot image) from the camera 22 are sent to the control unit 100.

[0081] The state detection sensor group 30 includes sensors for detecting the actions of various action devices provided in the combine harvester, such as the vehicle speed, forward and backward operations (forward operation, backward operation, neutral operation), forward and backward actions (forward driving, backward driving, parking), steering operations (left turn operation, right turn operation, neutral operation), steering actions (left turn driving, right turn driving, straight driving), and the lifting position (upward position, downward position) of the harvesting section H. The state detection signal from the state detection sensor group 30 is sent to the control unit 100.

[0082] The satellite positioning module 80 receives signals from artificial satellites GS (see Figure 1) GPS signal. Then, the satellite positioning module 80 generates a positioning signal for calculating the vehicle position of the combine harvester based on the received GPS signal. The generated positioning signal is sent to the control unit 100. It should be noted that the satellite positioning module 80 may not use GPS. For example, the satellite positioning module 80 may also use GNSS other than GPS (GLONASS, Galileo, Michibiki (Japanese Satellite Positioning System), BeiDou, etc.).

[0083] The inertial measurement device 81 detects the angular velocity of the yaw angle of the combine harvester body 19 and the accelerations in three mutually orthogonal axis directions over time. The orientation signal as the detection result of the inertial measurement device 81 is sent to the control unit 100 .

[0084] The control unit 100 includes a sensor signal processing unit 40 , a traveling state detection unit 51 , a distance changing unit 52 , a vehicle position calculation unit 53 , a direction calculation unit 54 , and a traveling control unit 60 .

[0085] The driving state detection unit 51 detects driving state information (including parking state information and working device state information) as the operating state of various operating devices provided in the combine harvester based on the state detection signal from the state detection sensor group 30, and generates the driving state information.

[0086] The distance change unit 52 changes the detectable distance LD and the control execution distance LC of the millimeter-wave radar 21 and the camera 22, respectively, based on the driving state information from the driving state detection unit 51. The detectable distance LD is set to a predetermined distance in each initial setting. The distance change unit 52 can change the detectable distance LD (or the detectable distance LD and the control execution distance LC) to zero by turning off the millimeter-wave radar 21 and the camera 22. For example, when the millimeter-wave radar 21 is turned off, the detectable distance LD (or the detectable distance LD and the control execution distance LC) of the millimeter-wave radar 21 becomes zero. In addition, when the camera 22 is turned off, the detectable distance LD (or the detectable distance LD and the control execution distance LC) of the camera 22 is zero. If the detectable distance LD becomes zero, obstacle detection is not performed.

[0087] The detectable distance LD and control execution distance LC of the millimeter wave radar 21 and the camera 22 are changed according to the content of the received driving state information. An example of the change relationship between the detectable distance LD and the control execution distance LC for the driving state is shown below separately for the camera 22 and the millimeter wave radar 21.

[0088] In the camera 22,

[0089] (1) When the combine harvester is moving forward,

[0090] The control execution distance LC of the front camera 22F is 3 m. The rear camera 22B is turned off, and the detectable distance LD of the rear camera 22B becomes zero. The control execution distance LC of the left camera 22L is 1 m, and the control execution distance LC of the right camera 22R is 1 m.

[0091] (2) When the combine harvester is moving backwards,

[0092] The front camera 22F is turned off, and the detectable distance LD of the front camera 22F is zero. The control execution distance LC of the rear camera 22B is 3 m. The control execution distance LC of the left camera 22L is 1 m, and the control execution distance LC of the right camera 22R is 1 m.

[0093] (3) When the combine harvester turns left,

[0094] The control execution distance LC of the front camera 22F is 3 m. The rear camera 22B is turned off, and the detectable distance LD of the rear camera 22B becomes zero. The control execution distance LC of the left camera 22L is 3 m, and the control execution distance LC of the right camera 22R is 1 m.

[0095] (4) When the combine harvester turns right,

[0096] The control execution distance LC of the front camera 22F is 3m. The rear camera 22B is turned off, and the detectable distance LD of the rear camera 22B becomes zero. The control execution distance LC of the left camera 22L is 1m, and the control execution distance LC of the right camera 22R is 3m.

[0097] In the millimeter wave radar 21,

[0098] (1) When the combine harvester moves forward with the harvesting section H lowered (when harvesting),

[0099] The control execution distance LC of the front millimeter-wave radar 21F is 12 m at maximum. The rear millimeter-wave radar 21B is turned off, and the detectable distance LD of the rear millimeter-wave radar 21B becomes zero.

[0100] (2) When the combine harvester moves forward with the harvesting section H raised (when not traveling for harvesting),

[0101] The front millimeter-wave radar 21F is turned off, and the detectable distance LD of the front millimeter-wave radar 21F is zero. The rear millimeter-wave radar 21B is turned off, and the detectable distance LD of the rear millimeter-wave radar 21B is zero. The front millimeter-wave radar 21F is turned off despite the advance because the rising harvesting section H deteriorates the detection function of the front millimeter-wave radar 21F.

[0102] (3) When the combine harvester with the harvesting section H raised is moving backward (when not traveling for harvesting),

[0103] The front millimeter wave radar 21F is turned off, and the detectable distance LD of the front millimeter wave radar 21F is zero. The detectable distance LD of the rear millimeter wave radar 21B is a maximum of 12 m.

[0104] The above-mentioned values ​​of the control execution distance LC are reference examples and are actually determined based on the detection capability of each sensor, the braking distance at which the machine body 19 can be safely stopped, etc., and can suppress the running stop caused by unnecessary detection.

[0105] The vehicle position calculation unit 53 calculates the position coordinates of the body 19 based on the positioning signal output by the satellite positioning module 80. The calculated position coordinates of the body 19 are used as the vehicle position information by each functional unit of the control unit 100. The orientation calculation unit 54 receives the position coordinates of the body 19 from the vehicle position calculation unit 53. The orientation calculation unit 54 calculates the body orientation of the body 19 based on the detection result of the inertial measurement device 81 and the position coordinates of the body 19 over time. The calculated orientation of the body 19 is used as orientation information by each functional unit of the control unit 100. The position coordinates and orientation of the body 19 are used for setting a target driving path and calculating a position deviation from the set target driving path during automatic driving, and for notification of driving guidance during manual driving.

[0106] The sensor signal processing unit 40 functions as an obstacle detection unit. The sensor signal processing unit 40 receives the ranging signal from the millimeter wave radar 21 and the shooting signal (shot image) from the camera 22 as input signals, and uses the detectable distance LD and the control execution distance LC set (changed or maintained) according to the distance change instruction from the distance change unit 52 as detection conditions to detect obstacles.

[0107] The sensor signal processing unit 40 includes a first signal processing unit 41, a second signal processing unit 42, and an obstacle position calculation unit 43. If the first signal processing unit 41 detects an obstacle within the control execution distance LC based on the ranging signal from each millimeter wave radar 21, obstacle information including the position and size of the detected obstacle is generated as an obstacle detection result. The second signal processing unit 42 uses the shooting signal (shot image) from the camera 22 as an input signal to determine whether an obstacle exists in the shot image. If the obstacle deemed to exist is deemed to be within the control execution distance LC, obstacle information including the position and size of the obstacle is generated. It should be noted that when the second signal processing unit 42 has a segmented neural network function using deep learning, the type of obstacle is also included in the obstacle information.

[0108] The obstacle position calculation unit 43 obtains the distance of the detected obstacle from the vehicle body 19. When the obstacle distance is output from the first signal processing unit 41 and the second signal processing unit 42, the obstacle position calculation unit 43 uses the obstacle distance. If the obstacle position calculation unit 43 determines that the detected obstacle is outside the field 5 based on the obstacle distance, the vehicle position information, and the map information of the field 5 to be driven for operation, the obstacle is ignored and no obstacle information is generated.

[0109] In this embodiment, when moving forward or backward, obstacles are sometimes detected by both the millimeter wave radar 21 and the camera 22 or only by one of them, but obstacle information is generated in either case. However, depending on the situation, a structure that generates obstacle information only when obstacles are detected by both the millimeter wave radar 21 and the camera 22 may be adopted.

[0110] The travel control unit 60 includes a manual travel control unit 61 , an automatic travel control unit 62 , and an avoidance control unit 64 . The automatic travel control unit 62 includes a route selection unit 63 .

[0111] The driving control unit 60 has an engine control function, a steering control function, a vehicle speed control function, etc., generates driving control instructions related to vehicle driving, and distributes them to associated action devices. The manual driving control unit 61 drives the combine harvester based on the driver's operation, which is a widely known technology. The automatic driving control unit 62 drives the combine harvester automatically (for example, unmanned). At this time, the path selection unit 63 selects a target driving path from a plurality of driving paths generated by an automatic driving path generation unit (not shown) based on the vehicle position information. In addition, the driving control unit 60 also has a function of driving the combine harvester based on an operation instruction from a human remote control device (not shown).

[0112] The avoidance control unit 64 executes avoidance control, which is control for avoiding interference between the combine harvester and an obstacle, based on the obstacle information from the sensor signal processing unit 40. The avoidance control includes stopping the combine harvester, slowing down the vehicle speed, making a sharp turn, and the like.

[0113] In the above embodiment, the obstacle position calculation unit 43 is assembled in the sensor signal processing unit 40. In addition, the sensor signal processing unit 40 determines whether to execute the avoidance control or not, and when the avoidance control needs to be executed, the obstacle information is distributed to the driving control unit 60. Alternatively, the function of the obstacle position calculation unit 43 can be incorporated into the avoidance control unit 64, and the avoidance control unit 64 determines the execution of the avoidance control based on the distance to the obstacle. Specifically, when the distance between the body 19 and the obstacle is less than the control execution distance LC, the avoidance control is executed, and when the distance between the body 19 and the obstacle exceeds the control execution distance LC, the avoidance control is not executed.

[0114] Next, use Figure 6 An example of obstacle avoidance control in the above-mentioned electronic control system is described below. Figure 6 The overall control flow shown in Figure 1 is the same as that in Figure 2. In contrast, the “avoidance control” is Figure 6 In the obstacle avoidance control, as an initial setting process, the detectable distance LD and the control execution distance LC of each camera 22 are set to default values, and the detectable distance LD and the control execution distance LC of each millimeter wave radar 21 are set to default values ​​(#01).

[0115] Next, determine the driving state (#11) and check whether the driving state has changed (#12). If the driving state has changed (#12 is a branch), perform distance change processing (#20). If the driving state has not changed (#12 is not a branch), skip the distance change processing. In the distance change processing of step #20, based on the determined current driving state, the detectable distance LD or the control execution distance LC of each camera 22 is changed, or the detectable distance LD or the control execution distance LC of each millimeter-wave radar 21 is changed, or the detectable distance LD or the control execution distance LC of each camera 22 and each millimeter-wave radar 21 is changed (#21, #22).

[0116] After that, the captured images from each camera 22 are obtained (#31), and captured image processing (#32) for obstacle detection using the captured images is performed. When an obstacle is detected (#33 "yes" branch), obstacle information (#51) is generated. Similarly, the ranging signal (#41) from each millimeter-wave radar 21 is obtained, and ranging signal processing (#42) for obstacle detection using the ranging signal is performed. When an obstacle is detected (#43 is a branch), obstacle information (#51) is generated. The generated obstacle information is used in the driving control unit 60, etc. When no obstacle is detected (#33 does not branch, #43 does not branch), return to step #11 and repeat the processing after step #11.

[0117] In the case where obstacle information is generated in step #51, it is determined whether avoidance control for the obstacle should be performed based on the generated obstacle information (#52). If it is determined that avoidance control is necessary (#53 is a branch), avoidance control such as parking, deceleration, left turn, right turn, etc. is performed (#60). After the avoidance control is performed, although not shown in the figure, the obstacle avoidance control is re-performed after appropriate recovery processing (obstacle avoidance processing routine). If it is determined that avoidance control is not necessary (#53 "No" branch), it returns to step #11 and repeats the processing after step #11.

[0118] [Other implementation methods]

[0119] (1) In the above embodiment, the camera 22 as one of the multiple sensors 2 (surrounding monitoring sensor group 2) can be regarded as a monocular camera, but it can also be a stereo camera. In addition, as another distance measuring sensor in the multiple sensors 2 (surrounding monitoring sensor group 2), the millimeter wave radar 21 can be regarded as a LiDAR, sonar, etc. In addition, as the sensor 2, a comprehensive sensor combining a distance measuring sensor and a camera 22 can also be used.

[0120] (2) The number, arrangement, and detection target area of ​​the cameras 22 and the millimeter wave radars 21 in the above-mentioned embodiment are reference examples and are not limited thereto.

[0121] (3) Figure 5 The functional block diagram shown is a schematic structure for the purpose of explanation. The functional blocks shown in the figure can be combined with other functional blocks, or one functional block can be divided.

[0122] In addition, the structure disclosed in the above-mentioned embodiment (including other embodiments, the same below) can be combined with the structure disclosed in other embodiments as long as no contradiction occurs. In addition, the embodiments disclosed in this specification are illustrative, and the embodiments of the present invention are not limited thereto, and can be appropriately changed within the scope of the purpose of the present invention.

[0123] [Industrial Application Possibility]

[0124] The present invention can be used not only in agricultural machines such as combine harvesters, tractors, rice transplanters, and pesticide spreaders, but also in various machines such as civil engineering machines, construction machines, and transport machines.

Claims

1. A control system is a control system for a working machine, characterized in that: have: A plurality of sensors, which respectively sense different directions and detect obstacles to the working machine; a driving state detection unit for detecting the driving state of the working machine; an avoidance control unit that executes avoidance control as control for avoiding interference between the working machine and the obstacle based on the detection results of the plurality of sensors; a distance changing unit configured to change at least one of a detectable distance of each of the plurality of sensors relative to the obstacle and a control execution distance, The avoidance control unit executes the avoidance control when the distance between the working machine and the obstacle is equal to or less than the control execution distance. The distance changing unit changes at least one of the detectable distance and the control execution distance based on the running state detected by the running state detecting unit.

2. The control system according to claim 1, characterized in that: The plurality of sensors include a distance measuring sensor and a camera that generates a captured image.

3. The control system according to claim 2, characterized in that: The detectable distance of the distance measuring sensor is longer than the detectable distance of the camera.

4. The control system according to claim 1, characterized in that: The plurality of sensors include a front detection sensor having a detection range in front of the working machine and a rear detection sensor having a detection range in rear of the working machine. When the working machine is moving forward, at least one of the detectable distance of the rear detection sensor and the control execution distance is made shorter by the distance changing unit than when the working machine is moving backward. During the backward movement, at least one of the detectable distance of the front detection sensor and the control execution distance is made shorter by the distance changing unit than during the forward movement.

5. The control system according to claim 1, characterized in that: The plurality of sensors include a left detection sensor having a detection range to the left of the work machine and a right detection sensor having a detection range to the right of the work machine. When the working machine turns left, at least one of the detectable distance of the right detection sensor and the control execution distance is made shorter by the distance changing unit than when the working machine turns right. During the right turn, the distance changing unit makes at least one of the detectable distance of the left detection sensor and the control execution distance shorter than during the left turn.

6. The control system according to claim 1, characterized in that: A working device is provided at the front of the working machine so as to be liftable. The plurality of sensors include a front detection sensor for detecting the obstacle in front of the working machine, When the travel state detection unit detects the raised state of the working device, obstacle detection by the front detection sensor is stopped.

7. The control system according to any one of claims 1 to 6, characterized in that: An obstacle position calculation unit is provided, which calculates the position of the detected obstacle based on the vehicle position of the work machine calculated by the vehicle position calculation unit and the obstacle detection result of the sensor, and ignores the obstacle when the obstacle is located outside the work site that becomes the driving object of the work machine.

8. A working machine, characterized in that: It has: multiple sensors that sense different directions to detect obstacles; A driving state detection unit, which detects the driving state of the machine body; an avoidance control unit that performs avoidance control as control for avoiding interference between the machine body and the obstacle based on the detection results of the plurality of sensors; a distance changing unit configured to change at least one of a detectable distance of each of the plurality of sensors relative to the obstacle and a control execution distance, The avoidance control unit executes the avoidance control when the distance between the aircraft body and the obstacle is equal to or less than the control execution distance. The distance changing unit changes at least one of the detectable distance and the control execution distance based on the running state detected by the running state detecting unit.

Images