Sensing system, work vehicle, sensing method, and computer program

Abstract

To inhibit an object that is not an obstacle from being detected as the obstacle.SOLUTION: A sensing system of the present disclosure is a sensing system of a work vehicle for performing work using a work machine. The sensing system includes a sensor provided in the work vehicle for sensing the environment around the work vehicle and generating sensor data, and a processing device for detecting an object positioned in a detection region around the work vehicle on the basis of the sensor data. The work machine can change the height with respect to the body of the work vehicle, and the processing device changes a pattern of the detection region for performing detection of the object according to a change in the height of the work machine.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present disclosure relates to a sensing system, a work vehicle, a sensing method, and a computer program.

Background Art

[0002] As next-generation agriculture, research and development of smart agriculture using ICT (Information and Communication Technology) and IoT (Internet of Things) is underway. Research and development are also underway for the automation and unmanning of agricultural machinery such as tractors and harvesters used in fields. For example, agricultural machinery that performs farming operations while automatically driving within a field using a positioning system such as GNSS (Global Navigation Satellite System) capable of precise positioning has been put into practical use.

[0003] Patent Document 1 discloses a harvester that travels automatically while harvesting crops in a field. The harvester can harvest crops by traveling along a preset travel route in the field.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] Development of a technique for searching an area around a work vehicle such as an agricultural machine using a sensor and detecting an obstacle around the work vehicle is also underway.

[0006] In the detection process of an obstacle using a sensor, it is required to suppress detecting an object that is not an obstacle as an obstacle.

Means for Solving the Problem

[0007] A sensing system according to an embodiment of the present disclosure is a sensing system for a work vehicle that performs work using a work machine, and includes a sensor that is provided on the work vehicle, senses the environment around the work vehicle, and generates sensor data, and a processing device that detects an object located in a detection area around the work vehicle based on the sensor data. The work machine can change the height with respect to the main body of the work vehicle, and the processing device changes the pattern of the detection area for detecting the object in response to a change in the height of the work machine.

Advantages of the Invention

[0008] When the height of the work machine changes, the work machine may enter the sensing area sensed by the sensor. According to an embodiment of the present disclosure, by changing the pattern of the detection area for detecting the object in response to a change in the height of the work machine, it is possible to prevent the object detection process from being performed on the part of the sensor data indicating the work machine. Thereby, it is possible to suppress detecting the work machine as an obstacle or the like. Further, by reducing the calculation amount of the processing device, the object detection processing speed can be increased.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

[0010] Hereinafter, embodiments of the present disclosure will be described. However, a more detailed description than necessary may be omitted. For example, detailed descriptions of well-known matters and overlapping descriptions regarding substantially the same configuration may be omitted. This is to avoid making the following description unnecessarily redundant and to facilitate the understanding of those skilled in the art. Note that the inventors provide the accompanying drawings and the following description so that those skilled in the art can fully understand the present disclosure, and do not intend to limit the subject matter described in the claims by these. In the following description, components having the same or similar functions are denoted by the same reference numerals. The reference signs F, Re, L, R, U, and D attached to the drawings represent the front, rear, left, right, top, and bottom, respectively.

[0011] The following embodiments are examples, and the technology of the present disclosure is not limited to the following embodiments. The content of the following embodiments is merely an example, and various modifications are possible as long as there is no technical contradiction. Also, as long as there is no technical contradiction, it is possible to combine one aspect with another aspect.

[0012] [Work Vehicle] The "work vehicle" according to an embodiment of the present disclosure means a mobile agricultural machine or a mobile construction machine used for specific applications such as agriculture or construction. The "agricultural machine" according to this embodiment means a mobile machine used for agricultural purposes. The "construction machine" according to this embodiment means a mobile machine used for civil engineering or construction purposes. "Work" includes, for example, agricultural work, civil engineering work, construction work, debris removal work, snow removal work, and the like. Hereinafter, an example in which the work vehicle is an agricultural machine will be mainly described.

[0013] The agricultural machine according to this embodiment can be a mobile agricultural machine that can harvest crops in the field while moving. The agricultural machine can be, for example, a harvester, a tractor, or an agricultural mobile robot. In some cases, an implement attached to or towed by an agricultural machine such as a tractor and the entire agricultural machine function as one "agricultural machine".

[0014] FIG. 1 is a diagram showing an example of an agricultural machine 100 according to this embodiment. In FIG. 1, a harvester is shown as an example of the agricultural machine 100.

[0015] The harvester 100 can be, for example, a combine harvester. The harvester 100 performs operations such as cutting crops in the field, threshing the cut crops, storing the harvested product after threshing, and discharging the harvested product. The crops in the field can be, but are not limited to, plants such as rice, wheat, corn, and soybeans that can be harvested.

[0016] The harvester 100 is equipped with an automatic driving function. That is, the harvester 100 can travel by the action of the control device without manual operation. The control device in the present embodiment is provided inside the harvester 100 and can control both the speed and steering of the harvester 100. The harvester 100 may automatically travel not only within the field but also outside the field (e.g., on a road). The harvester 100 is equipped with devices used for positioning or self-position estimation, such as a GNSS unit and a LiDAR sensor. The control device of the harvester 100 automatically drives the harvester 100 based on the position of the harvester 100 and the information of the target route.

[0017] The operation of the harvester 100 can be managed by a harvesting management system. The harvesting management system includes the harvester 100, a terminal device, a management device, etc.

[0018] The user terminal device is a computer used by a user who remotely monitors the harvester 100. The management device is a computer managed by an operator who operates the harvesting management system. The harvester 100, the user terminal device, and the management device can communicate with each other via a network. The harvesting management system may include a plurality of harvesters 100. The harvesting management system may include other agricultural machines.

[0019] The management device is a computer that manages the agricultural operations by the harvester 100. The management device can be, for example, a server computer that centrally manages information about the field on the cloud and utilizes the data on the cloud to support agriculture. The management device creates, for example, a work plan for the harvester 100 and causes the harvester 100 to execute agricultural operations according to the work plan.

[0020] The user terminal device is a computer used by a user who is located at a place away from the harvester 100. The user terminal device can be used to remotely monitor the harvester 100 or remotely operate the harvester 100.

[0021] The harvester 100 shown in Fig. 1 includes a vehicle body 101 and a traveling device 102. The illustrated traveling device 102 is a crawler-type traveling device, but it may also be a traveling device equipped with wheels with tires. Above the vehicle body 101, a cabin 114 is provided.

[0022] The harvester 100 includes a prime mover (engine) 111 and a transmission 112. Inside the cabin 114, a driver's seat, an operation lever, an operation terminal, and a group of switches for operation are provided.

[0023] At the front of the harvester 100, a working machine 110 for harvesting crops in the field is arranged. The working machine 110 includes a header 115. The header 115 includes a cutting device 103 for cutting crops and a reel 109 for raising the stem portions of the crops. The header 115 is provided at the front of a conveying device 104. The conveying device 104 conveys the cut crops. The reel 109 is arranged above the cutting device 103. The cutting device 103 and the reel 109 are height-adjustable. The working machine 110 may include the conveying device 104.

[0024] Behind the cabin 114, a threshing device 105 and a tank 106 for storing the harvested product are arranged side by side in the left-right direction. The conveying device 104 is arranged between the cutting device 103 and the threshing device 105. The threshing device 105 threshes the cut crops. The tank 106 stores the harvested product obtained by threshing grains and the like. Behind the threshing device 105, a straw discharging treatment device 108 is provided. The straw discharging treatment device 108 finely cuts the stem portions and the like after the harvested product such as grains has been removed and discharges them to the outside. The tank 106 is provided with a discharging device 107 for discharging the harvested product from the tank 106.

[0025] Since the configurations and operations of various devices performing harvesting operations such as the cutting device 103, the conveying device 104, the threshing device 105, the tank 106, the discharging device 107, the straw discharging treatment device 108, and the reel 109 are well-known, detailed descriptions thereof are omitted here.

[0026] The harvester 100 in this embodiment can operate in both a manual driving mode and an automatic driving mode. In the automatic driving mode, the harvester 100 can travel unmanned. Also, in the automatic driving mode, the harvester 100 can travel unmanned while performing the operation of harvesting the crops in the field.

[0027] The harvester 100 includes a plurality of sensing devices that sense the environment around the harvester 100, and a control device that processes the sensor data output from the plurality of sensing devices. The sensing devices can be a millimeter-wave radar 125, a camera 126, a LiDAR sensor 127, and an obstacle sensor 128.

[0028] The millimeter-wave radar 125 illustrated in FIG. 1 is disposed at the front part of the harvester 100. The millimeter-wave radar 125 may be further provided at the side part and / or the rear part of the harvester 100. The millimeter-wave radar 125 can be a two-dimensional scanning type millimeter-wave radar or a three-dimensional scanning type millimeter-wave radar. The millimeter-wave radar 125 senses the environment around the harvester 100 and outputs sensor data. The millimeter-wave radar 125 repeatedly outputs sensor data indicating the distance to the measurement point corresponding to the object existing in the surrounding environment, the angle of the measurement point, and the speed of the measurement point. The millimeter-wave radar 125 is, for example, a millimeter-wave radar of the FMCW (Frequency Modulated Continuous Wave) method, but is not limited thereto.

[0029] The sensor data output from the millimeter-wave radar 125 is processed by the control device of the harvester 100. The control device can detect an object such as an obstacle existing around the harvester 100 based on the sensor data.

[0030] The camera 126 illustrated in FIG. 1 is provided on the front, rear, left, and right of the harvester 100. The camera 126 captures the environment around the harvester 100 and generates image data. The image acquired by the camera 126 is output to a control device mounted on the harvester 100 and can be transmitted to a user terminal device for remote monitoring. Further, the image can be used to monitor the harvester 100 during unmanned operation.

[0031] The LiDAR sensor 127 illustrated in FIG. 1 is arranged at the front part and the rear part of the harvester 100. The LiDAR sensor 127 may be further provided at the side part of the harvester 100. The harvester 100 may include a plurality of LiDAR sensors arranged at different positions and in different orientations. The LiDAR sensor 127 may be a 3D-LiDAR sensor, or may be a 2D-LiDAR sensor. The LiDAR sensor 127 senses the environment around the harvester 100 and outputs sensor data. The LiDAR sensor 127 repeatedly outputs sensor data indicating the distance and direction to each measurement point of an object existing in the surrounding environment, or the three-dimensional or two-dimensional coordinate values of each measurement point. The sensor data output from the LiDAR sensor 127 is processed by the control device of the harvester 100. The control device can estimate the self-position of the harvester 100 by matching the sensor data with an environmental map. The control device can further detect objects such as obstacles existing around the harvester 100 based on the sensor data. The control device can also generate or edit an environmental map using an algorithm such as SLAM (Simultaneous Localization and Mapping).

[0032] The obstacle sensor 128 illustrated in FIG. 1 is provided on the side portion of the harvester 100. The obstacle sensor 128 can also be arranged at other parts. For example, the obstacle sensor 128 may be provided at the front and rear portions of the harvester 100. The obstacle sensor 128 may include, for example, a laser scanner or an ultrasonic sonar. The obstacle sensor 128 is used to detect surrounding obstacles during automatic driving to stop or detour the harvester 100. The LiDAR sensor 127 may be used as one of the obstacle sensors 128.

[0033] The harvester 100 includes a positioning device 121 that detects the geographical coordinates of the position of the harvester 100. The positioning device 121 is, for example, a GNSS unit. The GNSS unit 121 includes a GNSS receiver. The GNSS receiver may include an antenna that receives signals from GNSS satellites and a processor that calculates the position of the harvester 100 based on the signals received by the antenna. The GNSS unit 121 receives satellite signals transmitted from a plurality of GNSS satellites and performs positioning based on the satellite signals. GNSS is a general term for satellite positioning systems such as GPS (Global Positioning System), QZSS (Quasi-Zenith Satellite System, for example, Michibiki), GLONASS, Galileo, and BeiDou. The GNSS unit 121 in the present embodiment is provided on the upper part of the cabin 114, but it may be provided at other positions.

[0034] In addition to the positioning results obtained by the GNSS unit 121, the control device of the harvester 100 may use sensor data acquired by a sensing device such as the camera 126 and / or the LiDAR sensor 127 for positioning. When there are features in the environment where the harvester 100 travels that function as feature points, based on the data acquired by the camera 126 and / or the LiDAR sensor 127 and the environmental map stored in the storage device in advance, the position and orientation of the harvester 100 can be estimated with high accuracy. By using the data acquired by the camera 126 and / or the LiDAR sensor 127 to correct or complement the position data based on satellite signals, the position of the harvester 100 can be specified with higher accuracy.

[0035] The prime mover 111 can be, for example, a diesel engine. An electric motor may be used instead of the diesel engine. The transmission 112 can change the propulsion force and moving speed of the harvester 100 by shifting gears. The transmission 112 can also switch between forward and reverse of the harvester 100.

[0036] In the form where the harvester 100 is equipped with the crawler-type traveling device 102, the traveling direction of the harvester 100 can be changed by making the rotational speeds of the left and right wheels equipped with endless tracks different from each other or making the rotational directions of those left and right wheels different from each other. In the form where the harvester 100 is equipped with a traveling device having wheels with tires, the harvester 100 is equipped with a power steering device, and by controlling the power steering device to change the steering angle of the steering wheel (also referred to as the "steering angle"), the traveling direction of the harvester 100 can be changed.

[0037] The harvester 100 shown in FIG. 1 can be manned, but may also support only unmanned operation. In that case, components necessary only for manned operation, such as the cabin 114, the steering device, and the driver's seat, may not be provided on the harvester 100. The unmanned harvester 100 can travel by autonomous driving or by remote operation by the user.

[0038] FIG. 2 is a block diagram showing a configuration example of the harvester 100. The harvester 100 can communicate with the above-described user terminal device and management device via a network.

[0039] The harvester 100 illustrated in FIG. 2 includes a GNSS unit 121, an inertial measurement unit (IMU) 122, a millimeter-wave radar 125, a camera 126, a LiDAR sensor 127, an obstacle sensor 128, an operation terminal 131, an operation switch group 132, a drive device 140, a power transmission mechanism 141, a sensor group 150, a control device 160, and a communication device 190. These components are connected to be mutually communicable via a bus.

[0040] The GNSS unit 121 includes, for example, a GNSS receiver and an RTK receiver. The sensor group 150 detects various states of the harvester 100. The sensor group 150 includes an operation lever sensor 151, a rotation sensor 152, and a load sensor 156. The control device 160 includes a processor 161, a RAM (Random Access Memory) 162, a ROM (Read Only Memory) 163, a storage device 164, and a plurality of electronic control units (ECUs) 165 to 167. FIG. 2 shows components that are relatively highly related to the operation of the automatic driving by the harvester 100, and illustration of other components is omitted.

[0041] The GNSS unit 121 receives satellite signals transmitted from a plurality of GNSS satellites and generates GNSS data based on the satellite signals. The GNSS data is generated in a predetermined format such as, for example, the NMEA-0183 format. The GNSS data may include, for example, values indicating the identification number, elevation angle, azimuth angle, and reception intensity of each satellite from which the satellite signal was received.

[0042] The GNSS unit 121 can perform positioning of the harvester 100 using RTK (Real Time Kinematic)-GNSS. In positioning by RTK-GNSS, in addition to satellite signals transmitted from a plurality of GNSS satellites, correction signals transmitted from a reference station are used. The reference station can be installed near the field where the harvester 100 performs work running (for example, at a position within 10 km from the harvester 100). The reference station generates correction signals in, for example, RTCM format based on satellite signals received from a plurality of GNSS satellites and transmits them to the GNSS unit 121. The RTK receiver of the GNSS unit 121 includes an antenna and a modem and receives the correction signals transmitted from the reference station. The GNSS unit 121 corrects the positioning result based on the correction signals. By using RTK-GNSS, it is possible to perform positioning with an accuracy of, for example, several centimeters of error. Position data including latitude, longitude, and altitude information is obtained by high-precision positioning by RTK-GNSS. The GNSS unit 121 calculates the position of the harvester 100, for example, at a frequency of about once to ten times per second.

[0043] Note that the positioning method is not limited to RTK-GNSS, and any positioning method (such as an interferometric positioning method or a relative positioning method) that can obtain position data with the required accuracy can be used. For example, positioning using VRS (Virtual Reference Station) or DGPS (Differential Global Positioning System) may be performed. When position data with the required accuracy can be obtained without using the correction signals transmitted from the reference station, the position data may be generated without using the correction signals. In that case, the GNSS unit 121 may not be equipped with an RTK receiver.

[0044] Even when using RTK-GNSS, in a place where correction signals from the reference station cannot be obtained (for example, on a road far from the field), the position of the harvester 100 is estimated by other methods regardless of the signals from the RTK receiver. For example, the position of the harvester 100 can be estimated by matching the data output from the LiDAR sensor 127 and / or the camera 126 with a high-precision environmental map.

[0045] IMU 122 may include a three-axis acceleration sensor and a three-axis gyroscope. IMU 122 may also include an orientation sensor such as a three-axis geomagnetic sensor. IMU 122 functions as a motion sensor and can output signals indicating various quantities such as the acceleration, velocity, displacement, and attitude of the harvester 100.

[0046] The output signal of IMU 122 can be used to complement the position data. IMU 122 can measure the inclination and minute movements of the harvester 100. By using the data obtained by IMU 122 to complement the satellite signal-based position data, the positioning performance can be improved.

[0047] In addition to the above-described satellite signals and correction signals, based on the signals output from IMU 122, the position and orientation of the harvester 100 can be estimated with higher accuracy. The signals output from IMU 122 can be used for correcting or complementing the position calculated based on the satellite signals and correction signals. IMU 122 outputs signals at a higher frequency than the position detection using satellite signals. Using the high-frequency signals, the position and orientation of the harvester 100 can be measured at a higher frequency (e.g., 10 Hz or more). Instead of IMU 122, a three-axis acceleration sensor and a three-axis gyroscope may be provided separately. IMU 122 may be included in the GNSS unit 121.

[0048] Camera 126 is an imaging device that captures the environment around the harvester 100. The camera 126 includes an image sensor such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor), for example. The camera 126 may also include an optical system including one or more lenses and a signal processing circuit. While the harvester 100 is in motion, the camera 126 captures the environment around the harvester 100 and generates image (e.g., video) data. The camera 126 can capture video at a frame rate of, for example, 3 frames per second (fps) or higher. The image generated by the camera 126 can be used, for example, when a remote monitor uses a user terminal device to check the environment around the harvester 100. The image generated by the camera 126 may also be used for positioning or obstacle detection. A plurality of cameras 126 may be provided at different positions of the harvester 100, or a single camera may be provided. A visible camera that generates a visible light image and an infrared camera that generates an infrared image may be provided separately. Both the visible camera and the infrared camera may be provided as cameras that generate images for monitoring. The infrared camera can also be used for detecting obstacles at night.

[0049] The obstacle sensor 128 detects an object present around the harvester 100. The obstacle sensor 128 may include, for example, a laser scanner or an ultrasonic sonar. The obstacle sensor 128 outputs a signal indicating the presence of an obstacle when an object is present closer than a predetermined distance from the obstacle sensor 128. A plurality of obstacle sensors 128 may be provided at different positions of the harvester 100. For example, a plurality of laser scanners and a plurality of ultrasonic sonars may be arranged at different positions of the harvester 100. By providing a plurality of obstacle sensors 128, it is possible to reduce dead spots in monitoring obstacles around the harvester 100.

[0050] The operation lever sensor 151 detects the operation of the operation lever by the user within the cabin 114. The output signal of the operation lever sensor 151 is used for operation control by the control device 160. The rotation sensor 152 measures the rotational speed of the axle of the traveling device 102, that is, the number of rotations per unit time. The rotation sensor 152 can be, for example, a sensor using a magnetoresistive element (MR), a Hall element, or an electromagnetic pickup. The rotation sensor 152 outputs a numerical value indicating, for example, the number of rotations per minute (unit: rpm) of the axle. The rotation sensor 152 is used, for example, to measure the speed of the harvester 100.

[0051] The load sensor 156 is provided at the lower part of the tank 106 and detects the weight of the harvested product within the tank 106. By detecting the weight of the harvested product within the tank 106, the control device 160 can recognize the storage state of the harvested product within the tank 106. A yield sensor and a taste sensor may be provided inside or around the tank 106. Data such as the moisture value and protein value of the harvested product are output from the taste sensor as quality data.

[0052] The drive device 140 includes various devices necessary for driving the harvester 100 to travel, such as the prime mover 111 and the transmission 112. The prime mover 111 includes, for example, an internal combustion engine such as a diesel engine. The drive device 140 may be provided with a traction electric motor instead of or together with the internal combustion engine.

[0053] The power transmission mechanism 141 transmits the power generated by the prime mover 111 to various devices that perform the harvesting operation. The devices that perform the harvesting operation are the cutting device 103, the conveying device 104, the threshing device 105, the tank 106, the discharging device 107, the straw treatment device 108, the reel 109, etc. The harvester 100 may be provided with a power source (such as an electric motor) that supplies power to at least one of these devices that perform the harvesting operation separately from the prime mover 111.

[0054] The processor 161 may be a semiconductor integrated circuit including, for example, a central processing unit (CPU). The processor 161 may be implemented by a microprocessor or a microcontroller. Alternatively, the processor 161 may also be implemented by an FPGA (Field Programmable Gate Array) equipped with a CPU, a GPU (Graphics Processing Unit), an ASIC (Application Specific Integrated Circuit), an ASSP (Application Specific Standard Product), or a combination of two or more circuits selected from these circuits. The processor 161 sequentially executes a computer program describing a group of instructions for executing at least one process stored in the ROM 163 to realize a desired process.

[0055] The ROM 163 is, for example, a writable memory (e.g., PROM), a rewritable memory (e.g., flash memory), or a read-only memory. The ROM 163 stores a program for controlling the operation of the processor 161. The ROM 163 does not necessarily have to be a single storage medium and may be an aggregate of a plurality of storage media. A part of the aggregate of the plurality of storage media may be a removable memory.

[0056] The RAM 162 provides a work area for temporarily expanding the control program stored in the ROM 163 at the time of booting. The RAM 162 does not necessarily have to be a single storage medium and may be an aggregate of a plurality of storage media.

[0057] The storage device 164 includes one or more storage media such as a flash memory or a magnetic disk. The storage device 164 stores various data generated by the GNSS unit 121, the millimeter-wave radar 125, the camera 126, the LiDAR sensor 127, the obstacle sensor 128, the sensor group 150, and the control device 160. The data stored in the storage device 164 may include map data (environmental map) of the environment in which the harvester 100 travels and data of a target route for autonomous driving. The environmental map includes information on a plurality of fields where the harvester 100 performs farming operations and the roads around them. The environmental map and the target route may be generated by a processor of the management device. Note that the control device 160 may have a function of generating or editing the environmental map and the target route. The control device 160 can edit the environmental map and the target route acquired from the management device according to the driving environment of the harvester 100. The storage device 164 also stores the data of the work plan received by the communication device 190 from the management device.

[0058] The storage device 164 also stores a computer program that causes the processor 161 and the ECUs 165 - 167 to execute various operations described later. Such a computer program can be provided to the harvester 100 via a storage medium (e.g., a semiconductor memory or an optical disk, etc.) or a telecommunication line (e.g., the Internet). Such a computer program may be sold as commercial software.

[0059] The control device 160 includes a plurality of ECUs 165 - 167. The ECU 165 controls the traveling speed and turning operation of the harvester 100 by controlling the prime mover 111, the transmission 112, and the traveling device 102 included in the driving device 140.

[0060] Based on the data output from the GNSS unit 121, millimeter-wave radar 125, camera 126, LiDAR sensor 127, obstacle sensor 128, sensor group 150, and processor 161, the ECU 165 performs calculations and controls for realizing autonomous driving. For example, the ECU 165 identifies the position of the harvester 100 based on the data output from at least one of the GNSS unit 121, camera 126, and LiDAR sensor 127. In the field, the ECU 165 may determine the position of the harvester 100 based only on the data output from the GNSS unit 121. The ECU 165 may estimate or correct the position of the harvester 100 based on the data acquired by the camera 126 and / or LiDAR sensor 127. By using the data acquired by the camera 126 and / or LiDAR sensor 127, the positioning accuracy can be further improved. For example, the ECU 165 may estimate the position of the harvester 100 by matching the data output from the LiDAR sensor 127 and / or camera 126 with the environmental map. During autonomous driving, the ECU 165 performs the calculations necessary for the harvester 100 to travel along the target route based on the estimated position of the harvester 100.

[0061] Based on the work plan stored in the storage device 164, the ECU 166 can determine the destination of the harvester 100 and determine the target route from the starting point to the destination point of the movement of the harvester 100. The ECU 166 may perform a process of detecting an object located around the harvester 100 based on the data output from the millimeter-wave radar 125, camera 126, obstacle sensor 128, and LiDAR sensor 127.

[0062] The ECU 167 controls the operations of the power transmission mechanism 141 and the like in order to cause various devices that perform the above-described harvesting operations to execute desired operations.

[0063] By the functions of these ECUs, the control device 160 realizes the automatic driving and the crop harvesting operation. During the automatic driving, the control device 160 controls the driving device 140 based on the measured or estimated position of the harvester 100 and the target path. Thereby, the control device 160 can make the harvester 100 travel along the target path.

[0064] The plurality of ECUs included in the control device 160 can communicate with each other according to a vehicle bus standard such as CAN (Controller Area Network). Instead of CAN, a faster communication method such as in-vehicle Ethernet (registered trademark) may be used. In FIG. 2, each of the ECUs 165 to 167 is shown as an individual block, but the respective functions of these may be realized by a plurality of ECUs. An in-vehicle computer integrating at least some of the functions of the ECUs 165 to 167 may be provided. The control device 160 may include ECUs other than the ECUs 165 to 167, and any number of ECUs can be provided according to the functions. Each ECU includes a processing circuit including one or more processors. The processor 161 may be integrated with any one of the ECUs included in the control device 160.

[0065] The communication device 190 is a device including a circuit for communicating with the user terminal device and the management device. The communication device 190 may include an antenna and a communication circuit for transmitting and receiving signals via a network between the respective communication devices of the user terminal device and the management device. The network may include, for example, a cellular mobile communication network such as 3G, 4G or 5G and the Internet. The communication device 190 may be provided with a function of communicating with a mobile terminal used by a monitor near the harvester 100. Between such a mobile terminal, communication compliant with any wireless communication standard such as Wi-Fi (registered trademark), cellular mobile communication such as 3G, 4G or 5G, or Bluetooth (registered trademark) can be performed.

[0066] The operation terminal 131 is a terminal for a user to perform operations related to the travel of the harvester 100, and is also referred to as a virtual terminal (VT). The operation terminal 131 may include a display device such as a touch screen and / or one or more buttons. The display device may be a display such as a liquid crystal or an organic light emitting diode (OLED). By operating the operation terminal 131, the user can perform various operations such as switching on / off the automatic driving mode, recording or editing the environmental map, and setting the target route. At least a part of these operations may also be realized by operating the operation switch group 132. The operation terminal 131 may be configured to be removable from the harvester 100. A user located at a place away from the harvester 100 may operate the removed operation terminal 131 to control the operation of the harvester 100. Instead of the operation terminal 131, the user may operate a computer installed with necessary application software such as a user terminal device to control the operation of the harvester 100.

[0067] [Setting of Detection Region According to Height of Working Machine] Next, a process of setting a detection region according to a change in the height of the working machine will be described.

[0068] As described above, sensing devices such as the millimeter wave radar 125, the camera 126, the LiDAR sensor 127, and the obstacle sensor 128 sense the environment around the harvester 100 and output sensor data. The processor 161 detects an object located in the detection region around the harvester 100 based on the sensor data. The detection region is a region in the area around the harvester 100 sensed by the sensing device where an object is detected. The detection region may be the same size as the sensing region sensed by the sensing device, or may be smaller than the sensing region. The detection region may also be referred to as a search region or a region of interest (ROI).

[0069] In the embodiment illustrated below, in the process of detecting an object using the sensor data output by the millimeter-wave radar 125, the pattern of the detection area is changed according to the change in the height of the working machine 110. Changing the pattern of the detection area means, for example, changing at least one of the shape, size, and relative position with respect to the harvester 100 of the detection area.

[0070] The harvester 100 of the present embodiment includes a sensing system 10 (FIG. 2) that detects an object located around the harvester 100 using the sensor data output by the millimeter-wave radar 125. The sensing system 10 includes a control device 160 and a millimeter-wave radar 125.

[0071] The millimeter-wave radar 125 outputs radio waves in the millimeter-wave band and receives reflected waves obtained by reflecting the radio waves from an object existing in the surrounding environment. The millimeter-wave radar 125 outputs sensor data indicating the distance to a measurement point corresponding to an object existing in the surrounding environment, the angle of the measurement point, and the speed of the measurement point.

[0072] FIG. 3 is a top view showing an example of a sensing area 200 sensed by the millimeter-wave radar 125. FIG. 4 is a side view showing an example of the sensing area 200 sensed by the millimeter-wave radar 125. The sensing area 200 has a shape that extends in front of the harvester 100 and spreads in the vertical, horizontal, and lateral directions. The processor 161 performs object detection using, for example, the sensing area 200 as the detection area.

[0073] The working machine 110 of the harvester 100 can change its relative height with respect to the vehicle body 101. In the harvester 100 of the present embodiment, the actuator 116 is operated to rotate the conveying device 104 about the rotation axis 117. By rotating the conveying device 104 about the rotation axis 117, the height of the header 115 disposed at the front portion of the conveying device 104 can be changed.

[0074] A height sensor 118 for detecting the relative height of the header 115 with respect to the vehicle body 101 is provided on the rotation axis 117. The sensor 118 is, for example, an angle sensor, and detects the angle of the transfer device 104 with respect to the vehicle body 101. The height of the header 115 can be calculated based on the angle of the transfer device 104 with respect to the vehicle body 101.

[0075] For example, height information indicating the relationship between the angle of the transfer device 104 with respect to the vehicle body 101 and the height of the header 115 (for example, the height of the top of the reel 109) is stored in advance in the ROM 163 or the storage device 164. The processor 161 can calculate the height of the header 115 based on the output signal of the sensor 118 and the height information.

[0076] The height of the header 115 may be changed by other methods. For example, the header 115 may be linearly moved up and down to change the height.

[0077] The height sensor for detecting the height of the header 115 is not limited to the above form, and the height of the header 115 can be detected by providing an arbitrary sensor at an arbitrary position.

[0078] FIG. 5 is a side view showing the sensing area 200 and the header 115 displaced upward. When the header 115 is displaced upward and its position becomes higher, as shown in FIG. 5, a part of the header 115 may enter the sensing area 200. In the example shown in FIG. 5, a part of the reel 109 of the header 115 enters the sensing area 200.

[0079] In the present embodiment, in order to suppress the header 115 located in the sensing area 200 from being detected as an obstacle, the pattern of the detection area for detecting an object is changed according to the change in the height of the header 115.

[0080] FIG. 6 is a flowchart showing an example of a process of changing the pattern of the detection area according to the change in the height of the header 115.

[0081] The processor 161 detects the height of the header 115 based on the output signal of the height sensor 118 (step S201). The height of the header 115 to be detected can be the relative height with respect to the vehicle body 101. For example, the height of the header 115 with respect to the vehicle body 101 when the header 115 is in the lowest state is set to zero.

[0082] The processor 161 determines whether the detected height of the header 115 is equal to or greater than a predetermined height (step S202). The predetermined height is the height at which a part of the header 115 (for example, a part of the reel 109) enters the sensing area 200.

[0083] For example, information indicating the value of the predetermined height is stored in advance in the ROM 163 or the storage device 164, and the processor 161 can determine whether the height of the header 115 is equal to or greater than the predetermined height using that information.

[0084] If the processor 161 determines that the height of the header 115 is less than the predetermined height, it sets the first detection area 201 as the detection area (step S204). The processor 161 sets the first detection area 201, for example, to the same size as the sensing area 200 sensed by the millimeter wave radar 125. The first detection area 201 may be smaller than the sensing area 200. The processor 161 performs a process of detecting an object using the millimeter wave radar 125 on the first detection area 201 (step S205).

[0085] If the processor 161 determines in step S202 that the height of the header 115 is equal to or greater than the predetermined height, it sets the second detection area 202 as the detection area (step S203). The processor 161 sets, as the detection area 202, the area obtained by removing from the sensing area 200 the portion where the header 115 within the sensing area 200 is located. For example, the processor 161 sets, as the second detection area 202, the portion obtained by removing at least a part of the portion of the sensing area 200 where the distance from the millimeter wave radar 125 is equal to or less than a predetermined distance.

[0086] FIG. 7 is a top view showing an example of a portion 210 to be removed in the setting of the second detection area 202. FIG. 8 is a top view showing an example of the second detection area 202.

[0087] The processor 161 sets, for example, a portion of the sensing area 200 excluding a portion 210 where the distance from the millimeter-wave radar 125 is equal to or less than a predetermined distance D1 as the second detection area 202. The predetermined distance D1 is, for example, 2 m or more and 5 m or less, but is not limited to that value. The predetermined distance D1 can be set to an arbitrary length according to the size of the work machine 110.

[0088] FIG. 8 shows the second detection area 202 set by excluding the portion 210 from the sensing area 200.

[0089] The sensor data output by the millimeter-wave radar 125 includes data on the distance between each measurement point and the millimeter-wave radar 125, data on the angle of each measurement point with respect to the millimeter-wave radar 125, and the like. The processor 161 can prevent the detection of an object in the portion 210 by not using a measurement point where the distance from the millimeter-wave radar 125 is equal to or less than the predetermined distance D1 for object detection. The processor 161 performs a process of detecting an object using the millimeter-wave radar 125 on the second detection area 202 obtained by excluding the portion 210 from the sensing area 200 (step S205).

[0090] When the operation of the harvester 100 ends, the process of detecting an object ends (step S206).

[0091] When the height of the header 115 changes, the header 115 may enter the sensing area 200. According to the present embodiment, by changing the pattern of the detection areas 201 and 202 for performing object detection in response to a change in the height of the header 115, it is possible to prevent an object detection process from being performed on a portion indicating the header 115 in the sensor data. Thereby, it is possible to suppress the detection of the header 115 as an obstacle or the like. In addition, by reducing the calculation amount of the processor 161, the object detection processing speed can be increased.

[0092] The magnitude of the predetermined distance D1 may be changed according to the change in the height of the header 115. FIG. 9 is a top view showing a plurality of types of portions 210 having different magnitudes.

[0093] For example, when the height of the header 115 is relatively high, the processor 161 makes the predetermined distance D1 larger than when it is low. For example, as the height of the header 115 increases, in a form where the header 115 enters a more distant portion of the sensing region 200, the predetermined distance D1 is increased as the height of the header 115 increases. Thereby, while suppressing the detection of the header 115 as an obstacle or the like, a wide detection region can be secured when the header 115 is at a relatively low position.

[0094] In the examples shown in FIGS. 7 and 9, the shape of the excluded portion 210 was substantially fan-shaped, but it is not limited thereto, and it may have another shape. FIG. 10 is a top view showing another example of the excluded portion 210. In the example shown in FIG. 10, the shape of the excluded portion 210 is substantially triangular. FIG. 11 is a diagram showing an example of the second detection region 202 set by excluding the substantially triangular portion 210 from the sensing region 200. Depending on the form of the harvesting machine 100 and the working machine 110, the excluded portion 210 can take any shape.

[0095] In the description of the above embodiment, a harvesting machine was exemplified as an example of the agricultural machine 100, but as described above, the agricultural machine 100 is not limited to a harvesting machine. For example, the agricultural machine 100 may be a tractor to which a working machine is connected.

[0096] FIG. 12 is a diagram showing an example of a tractor 100 to which a work machine 110 is connected. In the example shown in FIG. 12, as an example of the work machine 110, a front loader is connected to the tractor 100. Even in a form in which the work machine 110 is connected to the tractor 100, as described above, in accordance with a change in the height of the work machine 110, the pattern of the detection area for detecting an object using the millimeter-wave radar 125 is changed. Thereby, it is possible to suppress detecting the work machine 110 as an obstacle or the like. Further, by reducing the amount of computation of the processor 161, the object detection processing speed can be increased.

[0097] As described above, the work vehicle 100 may be a construction machine. FIG. 13 is a diagram showing an example of the construction machine 100 of the present embodiment. In the example shown in FIG. 13, the construction machine 100 is a loader. The construction machine 100 can be a wheel loader or a crawler loader. The construction machine 100 may be a compact track loader (CTL) or a skid steer loader (SSL). The work machine 110 of the construction machine 100 shown in FIG. 13 includes an arm and a bucket. The type of the work machine 110 is not limited to the above and is arbitrary. For example, the work machine 110 may include pallet forks.

[0098] Even in a form of the construction machine 100 provided with the work machine 110, as described above, in accordance with a change in the height of the work machine 110, the pattern of the detection area for detecting an object using the millimeter-wave radar 125 is changed. Thereby, it is possible to suppress detecting the work machine 110 as an obstacle or the like. Further, by reducing the amount of computation of the processor 161, the object detection processing speed can be increased.

[0099] In the description of the above embodiment, although the pattern of the detection area for detecting an object using the millimeter-wave radar 125 is changed according to the change in the height of the working machine 110, the present invention is not limited thereto. The pattern of the detection area for detecting an object using a sensor other than the millimeter-wave radar 125 may be changed according to the change in the height of the working machine 110. For example, the pattern of the detection area for detecting an object using the LiDAR sensor 127 may be changed according to the change in the height of the working machine 110. Thereby, it is possible to suppress the working machine 110 from being detected as an obstacle or the like. Further, by reducing the calculation amount of the processor 161, the object detection processing speed can be increased.

[0100] The sensing system 10 of the present embodiment can also be retrofitted to a work vehicle that does not have those functions. Such a system can be manufactured and sold independently of the work vehicle. A computer program used in such a system can also be manufactured and sold independently of the work vehicle. The computer program can be provided, for example, stored in a non-transitory computer-readable storage medium. The computer program can also be provided by downloading via a telecommunication line (for example, the Internet).

[0101] Part or all of the processing executed by the processor 161 in the sensing system 10 may be executed by another device. Such another device may be at least one of the signal processing circuits of the ECUs 165-167 and the millimeter-wave radar 125. In that case, such another device and the processor 161 function as a processing device of the sensing system 10, or such another device functions as a processing device of the sensing system 10. For example, when a part of the processing executed by the processor 161 is executed by the signal processing circuit of the millimeter-wave radar 125, the processor 161 and the signal processing circuit of the millimeter-wave radar 125 function as a processing device of the sensing system 10. The control device 160 may function as a processing device of the sensing system 10.

[0102] As described above, the present disclosure includes the sensing system, work vehicle, sensing method, and computer program described below.

[0103] [Item 1] A sensing system for a work vehicle that performs work using a work machine, a sensor provided on the work vehicle that senses the environment around the work vehicle and generates sensor data, and a processing device that detects an object located in a detection area around the work vehicle based on the sensor data, comprising: the work machine is capable of changing its height relative to the body of the work vehicle, and the processing device changes the pattern of the detection area for detecting the object in accordance with a change in the height of the work machine.

[0104] [Item 2] The sensing system according to Item 1, wherein the processing device sets, as the detection area, a portion of the sensing area sensed by the sensor excluding a portion where the work machine enters the sensing area.

[0105] [Item 3] The sensing system according to Item 1 or 2, wherein when the height of the work machine is equal to or greater than a predetermined height, the processing device sets, as the detection area, a portion of the sensing area sensed by the sensor excluding at least a part of a portion where the distance from the sensor is equal to or less than a predetermined distance.

[0106] [Item 4] The sensing system according to Item 3, wherein the processing device changes the magnitude of the predetermined distance in accordance with a change in the height of the work machine.

[0107] [Item 5] The sensing system according to Item 4, wherein the processing device increases the predetermined distance when the height of the work machine is high as compared to when it is low.

[0108] [Item 6] The sensing system according to any one of Items 3 to 5, wherein the predetermined distance is 2 m or more and 5 m or less.

[0109] [Item 7] The sensing system according to any one of Items 1 to 6, wherein the processing device acquires data on the height of the working machine based on an output signal of a sensor that detects the height of the working machine with respect to the main body of the work vehicle.

[0110] [Item 8] The sensing system according to any one of Items 1 to 7, wherein the sensor is a millimeter-wave radar.

[0111] [Item 9] The sensing system according to any one of Items 1 to 8, wherein the work vehicle is a harvester, and the working machine includes a header that cuts crops in a field.

[0112] [Item 10] The sensing system according to any one of Items 1 to 8, wherein the work vehicle is a tractor, and the working machine includes an implement connected to the tractor.

[0113] [Item 11] The sensing system according to any one of Items 1 to 8, wherein the work vehicle is a mobile construction machine.

[0114] [Item 12] The sensing system according to Item 11, wherein the construction machine is a loader.

[0115] [Item 13] A work vehicle including the sensing system according to any one of Items 1 to 12.

[0116] [Item 14] A traveling device that causes the work vehicle to travel, A control device that controls the operation of the traveling device and automatically drives the work vehicle, The work vehicle according to item 13, further comprising

[0117] [Item 15] A sensing method for detecting an object located around a work vehicle that performs work using a work machine, which is executed by a computer, comprising: The work machine is capable of changing its height relative to the main body of the work vehicle. The sensing method includes: Detecting an object located in a detection area around the work vehicle based on sensor data obtained by a sensor sensing the environment around the work vehicle; Changing a pattern of the detection area for detecting the object in accordance with a change in the height of the work machine. A sensing method including the above.

[0118] [Item 16] A computer program for causing a computer to execute a process of detecting an object located around a work vehicle that performs work using a work machine, comprising: The work machine is capable of changing its height relative to the main body of the work vehicle. The computer program causes the computer to: Detect an object located in a detection area around the work vehicle based on sensor data obtained by a sensor sensing the environment around the work vehicle; Change a pattern of the detection area for detecting the object in accordance with a change in the height of the work machine. A computer program for causing the computer to execute the above.

Industrial Applicability

[0119] The technology of the present disclosure is particularly useful in the fields of mobile agricultural machines and mobile construction machines.

Explanation of Signs

[0120] 10: Sensing system, 100: Work vehicle (harvester), 101: Vehicle body, 102: Travel device, 103: Mowing device, 104: Conveyor device, 105: Threshing device, 106: Tank, 107: Discharge device, 108: Straw disposal device, 109: Reel, 110: Working machine, 111: Prime mover (engine), 112: Transmission, 114: Cabin, 115: Header, 116: Actuator, 117: Rotating shaft, 118: Height sensor, 121: Positioning device (GNSS unit), 122: Inertial measurement unit (IMU), 125: Millimeter-wave radar, 126: Camera, 127: LiDAR sensor, 128: Obstacle sensor, 131: Operation terminal, 132: Operation switch group, 140: Driving device, 141: Power transmission mechanism, 150: Sensor group, 151: Operation lever sensor, 152: Rotation sensor, 156: Load sensor, 160: Control device, 161: Processor, 162: RAM, 163: ROM, 164: Storage device, 165 - 167: ECU, 190: Communication device, 200: Sensing range, 201: First detection area, 202: Second detection area, 210: Part where object detection is not performed, D1: Distance

Claims

1. A sensing system for a work vehicle that performs work using a work machine, comprising: a sensor provided on the work vehicle that senses the environment around the work vehicle and generates sensor data; a processing device that detects an object located in a detection area around the work vehicle based on the sensor data; wherein the work machine is capable of changing its height relative to the body of the work vehicle; the processing device changes a pattern of the detection area for detecting the object in accordance with a change in the height of the work machine, the sensing system.

2. The sensing system according to claim 1, wherein the processing device sets, as the detection area, a portion of a sensing area sensed by the sensor excluding a portion where the work machine enters the sensing area.

3. The sensing system according to claim 1 or 2, wherein when the height of the work machine is equal to or greater than a predetermined height, the processing device sets, as the detection area, a portion of the sensing area sensed by the sensor excluding at least a part of a portion having a distance from the sensor equal to or less than a predetermined distance.

4. The sensing system according to claim 3, wherein the processing device changes a magnitude of the predetermined distance in accordance with a change in the height of the work machine.

5. The sensing system according to claim 4, wherein the processing device increases the predetermined distance when the height of the work machine is high as compared to when it is low.

6. The sensing system according to claim 3, wherein the predetermined distance is 2 m or more and 5 m or less.

7. The sensing system according to claim 1 or 2, wherein the processing device acquires height data of the work machine based on an output signal of a sensor that detects the height of the work machine relative to the body of the work vehicle.

8. The sensing system according to claim 1 or 2, wherein the sensor is a millimeter-wave radar.

9. The sensing system according to claim 1 or 2, wherein the work vehicle is a harvester, and the work machine includes a header for harvesting crops in a field.

10. The sensing system according to claim 1 or 2, wherein the work vehicle is a tractor, and the work machine includes an implement connected to the tractor.

11. The sensing system according to claim 1 or 2, wherein the work vehicle is a mobile construction machine.

12. The construction machine is a loader, and the sensing system according to claim 11.

13. A work vehicle including the sensing system according to claim 1 or 2.

14. A traveling device for traveling the work vehicle, A control device that controls the operation of the traveling device and automatically drives the work vehicle, The work vehicle according to claim 13, further including the above.

15. A sensing method for detecting an object located around a work vehicle that performs work using a work machine, which is executed by a computer, The work machine is capable of changing the height with respect to the main body of the work vehicle, The sensing method includes: Detecting an object located in a detection area around the work vehicle based on sensor data obtained by a sensor sensing the environment around the work vehicle; Changing a pattern of the detection area for detecting the object according to a change in the height of the work machine; A sensing method including the above.

16. A computer program for causing a computer to execute a process of detecting an object located around a work vehicle that performs work using a work machine, The work machine is capable of changing the height with respect to the main body of the work vehicle, The computer program includes: Detecting an object located in a detection area around the work vehicle based on sensor data obtained by a sensor sensing the environment around the work vehicle; Changing a pattern of the detection area for detecting the object according to a change in the height of the work machine; A computer program for causing the computer to execute the above.

Images