Area recognition system and work vehicle

Abstract

To improve accuracy in the discrimination of a road and a place other than a road.SOLUTION: A region recognition system includes a first discrimination processing unit for discriminating whether or not a place is a road on the basis of first information acquired with the periphery of a work vehicle 10 as a detection target, and a second discrimination processing unit for discriminating a road and a place other than a road using specific information on the height or the width based on the second information acquired for a range overlapping with the range discriminated by using the first information.SELECTED DRAWING: Figure 6

Description

Technical Field

[0001] The present invention relates to an area recognition system and a work vehicle.

Background Art

[0002] Patent Document 1 discloses a work vehicle that acquires surrounding information and performs work using that information. The work vehicle has a three-dimensional scanner, a camera, and a control device. The control device processes the information acquired from the three-dimensional scanner and the camera, and the work vehicle performs various works at the work site.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Regarding work vehicles, technological development for autonomous driving is underway. Autonomous driving is also executed on the road leading to the work site in addition to the work site. For the control of autonomous driving, it is necessary to distinguish between the road and areas other than the road. This distinction is made by processing image information such as an in-vehicle camera. Areas other than the road include, for example, land such as a farm field (work site).

[0005] For example, in the case of a work vehicle performing agricultural work, it may be difficult to distinguish between a road with weeds and an uncultivated farm field (area other than the road) only by information processing using image information from an in-vehicle camera or the like. When the work vehicle is traveling on the road, for example, if it wobbles due to unevenness of the road surface and suddenly the in-vehicle camera faces the farm field, there is a possibility of misrecognizing the road and the farm field (area other than the road) at that timing. In this case, it is also conceivable that autonomous driving will be interrupted.

[0006] Therefore, this disclosure provides a region recognition system that can improve the accuracy of distinguishing between roads and non-road areas, and a work vehicle equipped with the region recognition system. [Means for solving the problem]

[0007] The area recognition system of this disclosure includes a first discrimination processing unit that determines whether or not the area is a road based on first information obtained as a detection target around a work vehicle, and a second discrimination processing unit that determines whether or not an area is a road based on specific information relating to height or width obtained from second information relating to an area that overlaps with the area determined using the first information.

[0008] The work vehicle of this disclosure comprises a vehicle body, a detection device that detects the area around the vehicle body, and the area recognition system. [Effects of the Invention]

[0009] According to the invention disclosed herein, it is possible to improve the accuracy of distinguishing between roads and non-road areas. [Brief explanation of the drawing]

[0010] [Figure 1] Figure 1 is a side view showing one configuration of the work vehicle. [Figure 2] Figure 2 is a block diagram showing the system configuration of the work vehicle. [Figure 3] Figure 3 is an illustrative diagram showing an example of image information from a camera. [Figure 4] Figure 4 is an illustrative diagram showing an example of a three-dimensional point cloud dataset obtained from a LiDAR sensor. [Figure 5] Figure 5 is a flowchart illustrating an example of the processing performed by the region recognition system. [Figure 6] Figure 6 is an explanatory diagram of the discrimination process performed by the second discrimination processing unit. [Figure 7] Figure 7 is an explanatory diagram of the discrimination process performed by the second discrimination processing unit. [Figure 8] Figure 8 is an explanatory diagram of the discrimination process performed by the second discrimination processing unit. [Figure 9] Figure 9 is an explanatory diagram of the discrimination process performed by the second discrimination processing unit. [Modes for carrying out the invention]

[0011] <Summary of the embodiments of this disclosure> The embodiments of this disclosure are outlined below. (1) The area recognition system of this embodiment includes a first discrimination processing unit that determines whether or not the area is a road based on first information obtained as a detection target around a work vehicle, and a second discrimination processing unit that determines whether or not an area is a road based on specific information relating to height or width obtained from second information relating to the area that overlaps with the area determined using the first information. According to the aforementioned region recognition system, for example, even if the first discrimination processing unit determines that it is a road, the second discrimination processing unit can re-determine that it is not a road.

[0012] (2) Preferably, the first discrimination processing unit uses a trained model based on the first information to determine whether or not it is a road. The first discrimination processing unit can determine whether or not the area around the work vehicle is a road by using a trained model based on the first information obtained as the detection target.

[0013] (3) The first information and the second information may be the same information, or the first information and the second information may be different information.

[0014] (4) In any one of the area recognition systems described in (1) to (3) above, preferably the area recognition system has an information acquisition unit that acquires the specific information, and the second discrimination processing unit determines that the portion that the first discrimination processing unit has determined to be a road is not a road when the specific information satisfies the discrimination conditions relating to height or width. According to this configuration, even if the first discrimination processing unit discriminates that it is a road, the second discrimination processing unit can re-discriminate that it is not a road for the part discriminated by the first discrimination processing unit as a road when the specific information satisfies the discrimination condition regarding height or width.

[0015] (5) For example, in the relationship between a farmland and the adjacent road, the ground surface of the land other than the road such as the farmland is often one level lower than the ground surface (road surface) of the road. Therefore, preferably, the area recognition system in (4) above has a confirmation processing unit that can confirm that the position where the work vehicle exists is a road, the specific information includes height information of the part discriminated by the first discrimination processing unit as a road, and the discrimination condition includes the condition that the height of the part discriminated by the first discrimination processing unit as a road is lower than the first threshold value beyond the reference position of the work vehicle.

[0016] According to this configuration, even if the first discrimination processing unit discriminates that it is a road, the second discrimination processing unit can re-discriminate that it is not a road. In this case, for the part discriminated by the first discrimination processing unit as a road, for example, the following two cases can be considered. <1> The ground surface of the land other than the road where no plants are growing. <2> A virtual surface including the upper ends of these plants where many relatively short plants are growing.

[0017] (6) Even if the actual ground surface of the land other than the road is substantially the same height as the actual ground surface of the road (or there are small height differences between these ground surfaces), if many relatively tall plants grow on one side of the land other than the road and the virtual surface including the upper ends of these plants becomes the part discriminated as the road, the first discrimination processing unit may discriminate that it is a road.

[0018] Preferably, the area recognition system in (4) has a confirmation processing unit capable of confirming that the location of the work vehicle is a road, the identification information includes height information of the portion that the first discrimination processing unit has determined to be a road, and the discrimination condition includes the condition that the height of the portion that the first discrimination processing unit has determined to be a road is higher than the reference position of the work vehicle by a second threshold. With this configuration, even if the first discrimination processing unit determines that it is a road, the second discrimination processing unit can re-determine that it is not a road.

[0019] (7) In the area recognition system of (5) or (6) above, preferably, the confirmation processing unit can confirm that the location of the work vehicle is a road by using information other than the first information and the second information, or by using an algorithm different from the determination of the first determination processing unit.

[0020] (8) Even if the actual ground surface of land other than the road is at approximately the same height as the actual ground surface of the road (or even if there are small differences in elevation between these ground surfaces), if many relatively tall plants grow all over the land other than the road, and the virtual surface including the tops of these plants becomes the part that the first discrimination processing unit identifies as the road, the first discrimination processing unit may identify it as the road. However, the upper ends (virtual planes) of these plants are not strictly coplanar. Therefore, when a region is detected by a three-dimensional range sensor such as a LiDAR sensor, variations occur in the height component of the three-dimensional point cloud dataset in the portion of that region that is identified as a road.

[0021] Therefore, in the region recognition system of (4) above, preferably, the second information includes a three-dimensional point cloud dataset in the portion that the first discrimination processing unit has determined to be a road, the specific information includes information on values ​​indicating the variation of the height component of the three-dimensional point cloud dataset, and the discrimination condition includes the condition that the value indicating the variation exceeds the range of a third threshold. With this configuration, even if the first discrimination processing unit determines that it is a road, the second discrimination processing unit can re-determine that it is not a road.

[0022] (9) There is a certain upper limit to the width of roads, as exemplified by roads on which vehicles can travel. In contrast, land other than roads, such as fields, is often much larger than roads. Therefore, in the area recognition system of (4) above, preferably, the specific information includes information on the width dimension of the portion that the first discrimination processing unit has determined to be a road, and the discrimination condition includes the condition that the width dimension exceeds the reference dimension. With this configuration, even if the first discrimination processing unit determines that it is a road, the second discrimination processing unit can re-determine that it is not a road.

[0023] (10) In any one of the area recognition systems described in (1) to (9) above, preferably, the first discrimination processing unit uses the image information acquired by the camera as the first information to determine whether or not it is a road. According to the above configuration, it becomes easy to determine whether or not something is a road. The first discrimination processing unit performs, for example, image processing such as segmentation, and makes the determination using the result. The segmentation processing is performed, for example, using a trained model.

[0024] (11) In the region recognition system described in any one of (4) to (9) above, preferably, the information acquisition unit acquires a three-dimensional point cloud dataset of the target region using a three-dimensional range sensor, and uses the three-dimensional point cloud dataset to acquire the height of the portion that the first discrimination processing unit has determined to be a road as the specific information. According to the above configuration, the height of the portion that the first discrimination processing unit has determined to be a road can be determined with high accuracy.

[0025] (12) In the region recognition system described in any one of (1) to (11) above, preferably, the first discrimination processing unit uses the three-dimensional point cloud dataset acquired by the three-dimensional range sensor as the first information to determine whether or not it is a road. According to the above configuration, it becomes possible to determine whether or not something is a road without using image information from a camera. The three-dimensional point cloud dataset allows the information acquisition unit to easily obtain information about the height of the portion that the first discrimination processing unit has determined to be a road.

[0026] (13) Preferably, the area recognition system described in any one of (5) to (12) above has a position detection unit that detects the height of the reference position of the work vehicle. With the above configuration, it is possible to determine the height of the reference position of the work vehicle. The position detection unit detects the height position of the work vehicle based on, for example, the position information of the work vehicle from GNSS (Global Navigation Satellite System) or information from a barometer.

[0027] (14) In the area recognition system described in any one of (5) to (13) above, preferably, the confirmation processing unit uses at least one of the following: output information from an inertial measuring device mounted on the vehicle, information combining GNSS position information and map information for the vehicle, and work plan information for the vehicle in which work content and time are associated. According to the above configuration, the verification processing unit can confirm that the location of the work vehicle is a road.

[0028] (15) The work vehicle of this embodiment comprises a vehicle body, a detection device that detects the area around the vehicle body, and an area recognition system according to any one of (1) to (14) above. According to the aforementioned work vehicle, for example, even if the first discrimination processing unit determines that it is a road, the second discrimination processing unit can re-determine that it is not a road.

[0029] <Details of the embodiments of this disclosure> The embodiments of this disclosure will be described in detail below with reference to the drawings. At least some of the embodiments described below may be combined in any way. The technology disclosed herein relates to a region recognition system in which a detection device such as a camera mounted on a work vehicle detects the area around the work vehicle and recognizes whether the target area included in the detected area is a road or not, that is, whether it is a "road" or "land other than a road". The work vehicle may have the region recognition system, or a management device other than the work vehicle (management computer) that can communicate with the work vehicle may have the region recognition system.

[0030] In the following embodiment, the work vehicle is a vehicle that performs work, and the case where the work vehicle has an area recognition system will be described. Figure 1 is a side view showing one configuration of a work vehicle. The work vehicle 10 shown in Figure 1 is a tractor, a vehicle used for agricultural work. Figure 1 shows the tractor with an implement 50 attached.

[0031] In this embodiment, with respect to the target area for recognizing whether it is a road or land other than a road, "road" is described as a "vehicle road" on which vehicles can travel, and "land other than a road" is described as a "field." Note that if the work vehicle 10 is a farming rover smaller than a tractor, it will travel on narrow paths (field paths) not intended for vehicles. Therefore, "road" is not limited to "vehicle roads." Similarly, "land other than a road" is not limited to "fields." For example, "land other than a road" could be vacant land or abandoned land.

[0032] [Regarding work vehicle 10] The work vehicle 10 of this embodiment is equipped with functions to perform both a manual driving mode operated by a driver and an automatic driving mode that does not require driver intervention. The work vehicle 10 can be driven automatically and manually both inside the field and on roads outside the field.

[0033] Manual operation refers to operation in which the work vehicle 10 is operated (including driving) by a driver seated in the driver's seat 20 of the work vehicle 10. Automated driving is a type of driving in which the operation (including driving) of the work vehicle 10 is performed by the functions of the control device 70 that the work vehicle 10 has, without manual operation by the driver.

[0034] Autonomous driving can be performed not only when the driver is not seated in the driver's seat 20, but also when the driver is seated in the driver's seat 20. Automated driving is achieved through the functions of the control device 70. The control device 70 can control at least one of the following necessary actions for the movement of the work vehicle 10: steering, adjusting the speed of movement, and starting and stopping movement. In the case of autonomous driving, in addition to controlling the movement of the work vehicle 10, the operation of the work machine 50 is also controlled without the driver's intervention. In other words, the work vehicle 10 moves automatically while the work machine 50 performs the work automatically.

[0035] The work vehicle 10 has a positioning device 37, which includes a GNSS receiver, as will be explained later. The control device 70 automatically drives the work vehicle 10 based on the position of the work vehicle 10 identified by the positioning device 37 and the target route pre-stored in the storage device 79 (see Figure 2).

[0036] Autonomous driving also includes cases where the work vehicle 10 moves autonomously without human intervention in controlling its movement, while sensing the surrounding environment using detection devices described later. Autonomous driving includes not only the movement of the work vehicle 10 toward its destination along a predetermined route (the aforementioned target route), but also movement that follows a target being tracked. During this type of autonomous driving, as will be explained later, the surroundings are detected by detection devices such as cameras 35, and the system recognizes whether the target area is a road or a field.

[0037] The work vehicle 10 can also be driven remotely by a person other than the driver seated in the driver's seat 20. For this purpose, the work vehicle 10 is equipped with a communication device 16 (see Figure 2). The work vehicle 10 is remotely controlled by wireless communication between the work vehicle 10 and the control center's management computer (not shown). During remote operation, the driver's seat 20 may be unoccupied or occupied. Remote operation may take precedence over manual operation. If the management computer has a region recognition system, information communication for the operation of the region recognition system is performed through the communication device 16.

[0038] The direction of the work vehicle 10 is defined. The front-to-back, left-to-right, and up-and-down directions of the work vehicle 10 are defined based on the driver seated in the seat 20c of the driver's seat 20. In other words, the direction forward for the driver is "forward," and the direction behind is "rear." The direction to the right for the driver is "right," and the direction to the left is "left." The front-to-back and left-to-right directions are parallel to the ground, and the front-to-back and left-to-right directions are perpendicular to each other. The up-and-down direction is perpendicular to both the front-to-back and left-to-right directions. The up-and-down direction may be described as the "height direction." The left-to-right direction may be described as the "vehicle width direction." The front direction is the "direction of travel" of the work vehicle 10. Note that the work vehicle 10 does not necessarily have a driver's seat 20. In this case, the direction of work travel of the work vehicle 10 is "forward," and the opposite direction is "rearward." With respect to the direction of work travel, the right side of the work vehicle 10 is "right," and the left side is "left."

[0039] The work vehicle 10 comprises a vehicle body 11, a prime mover 12, a transmission 13, a running gear 14, a steering gear 15, a coupling device 40, and a control device 70. The vehicle body 11 comprises a chassis 21 which forms the vehicle's frame, a body 22 which forms the exterior, and a driver's seat 20. The driver's seat 20 is equipped with a steering wheel 30 operated by the driver, and an operating unit (operating interface) 31 which includes an operating terminal and a group of operating switches operated by the driver.

[0040] The prime mover 12 is an engine or a motor, and in this embodiment, it is a diesel engine. The running gear 14 has front wheels 14a and rear wheels 14b. The rotational force of the prime mover 12 is shifted by the transmission 13, and this rotational force is transmitted to the wheels, causing the work vehicle 10 to move. When the work vehicle 10 is traveling within a field to perform work, the running gear 14 may have crawler tracks as one or both of the front and rear wheels.

[0041] The steering system 15 has a steering shaft 25 that is rotated by the steering wheel 30. The steering system 15 changes the direction of travel of the work vehicle 10 by changing the direction of rolling of the wheels (front wheels 14a). The steering system 15 has an auxiliary mechanism (power steering system). The auxiliary mechanism assists the driver's operating force on the steering wheel 30 by hydraulics or electric power.

[0042] The transmission 13 is composed of multiple gears, etc. The transmission 13 changes the propulsion force and travel speed of the work vehicle 10. The transmission 13 can also switch the work vehicle 10 between forward and reverse. The work vehicle 10 has a power take-off mechanism (hereinafter referred to as the "PTO mechanism"). In this embodiment, the transmission 13 has the PTO mechanism. The PTO mechanism has a PTO shaft 17, which is one of the output shafts of the transmission 13. The PTO shaft 17 rotates due to the power of the prime mover 12. The rotational force of the PTO shaft 17 operates the various drive units of the work implement 50. The PTO shaft 17 is the output shaft for operating the work implement 50.

[0043] The coupling device 40 connects the implement 50 to the vehicle body 11. The coupling device 40 is mounted at the rear of the vehicle body 11 (chassis 21). The coupling device 40 has a lifting link mechanism that supports the implement 50 so that it can be raised and lowered. The lifting link mechanism is composed of, for example, a three-point link mechanism. The implement 50 can be attached to and detached from the work vehicle 10 by the coupling device 40. The lifting link mechanism changes the height position of the implement 50 or changes the posture of the implement 50 by means of an actuator, for example, a hydraulic system.

[0044] The implement 50 shown in Figure 1 is a rotary tiller. However, the implement 50 is not limited to this and may be, for example, a seed planter, fertilizer spreader, transplanter, mower, rake, grass collector, harvester, etc. The coupling device 40 connects the desired implement 50 to the work vehicle 10. The work vehicle 10 tows the implement 50 and allows the implement 50 to perform a predetermined task. The coupling device 40 may be located at the front of the vehicle body 11.

[0045] The work vehicle 10 has a camera. The camera in this embodiment is a camera 35. The camera 35 is installed, for example, on the front, rear, left, and right sides of the work vehicle 10 and captures images of the environment around the work vehicle 10. The camera 35 is, for example, a CCD camera equipped with a CCD image sensor, or a CMOS camera equipped with a CMOS image sensor. The camera 35 has a processing circuit that processes the signal output from the image sensor, and the processing circuit acquires image information of the surroundings.

[0046] Camera 35 is used not only when the work vehicle 10 is traveling on a road or field to recognize white lines, signs, markings, etc., but also when it is recognizing obstacles in the surrounding area. Camera 35 may be either a visible light camera that generates a visible light image, an infrared camera that generates an infrared image, or both. In the case of an infrared camera, detection of the surroundings at night becomes easier.

[0047] Image information acquired by camera 35 is transmitted to control device 70. This image information is used for controlling automatic driving as well as manual driving. As will be explained later, the control device 70 uses the image information in conjunction with other information (sensor data described later) to distinguish between roadways and fields, and to detect the road surface.

[0048] The work vehicle 10 has a three-dimensional range sensor. The three-dimensional range sensor in this embodiment is a LiDAR (Light Detection And Ranging) sensor 36. The LiDAR sensor 36 is positioned, for example, at the lower front of the vehicle body 11. The LiDAR sensor 36 may be provided at other locations. The LiDAR sensor 36 acquires and outputs sensor data indicating the distance and direction of each measurement point in surrounding objects, and sensor data indicating the two-dimensional or three-dimensional coordinate values ​​of each measurement point in surrounding objects. The sensor data acquired by the LiDAR sensor 36 is transmitted to the control device 70.

[0049] The sensor data from the LiDAR sensor 36 is used to detect the surrounding area. The control device 70 uses the sensor data in conjunction with other information (the image information) to distinguish between roadways and fields, and to detect the roadway plane, etc.

[0050] To determine the type of area surrounding the work vehicle 10, the camera 35 and LiDAR sensor 36 function as sensors (detection devices) that detect the surrounding environment. The image information output from the camera 35 and the three-dimensional point cloud dataset output from the LiDAR sensor 36 are information (detection information) obtained by detecting the area around the work vehicle 10.

[0051] The sensor data from the LiDAR sensor 36 can also be used for other purposes. Based on the sensor data, the control device 70 can perform environmental map generation processing using algorithms such as SLAM (Simultaneous Localization and Mapping). The environmental map generation processing may also be performed on an external computer, such as a management device, that can communicate with the work vehicle 10.

[0052] The work vehicle 10 is equipped with a positioning device 37. The positioning device 37 receives satellite signals transmitted from multiple GNSS satellites and performs positioning based on the satellite signals. GNSS is a general term for satellite positioning systems such as GPS (Global Positioning System), QZSS (Quasi-Zenith Satellite System: e.g., "Michibiki"), GLONASS (Russia), Galileo (Europe), and BeiDou (China).

[0053] The positioning device 37 includes a receiver 37a that receives satellite signals and a processor (arithmetic processing unit) 37b. The receiver 37a has an antenna that receives signals from GNSS satellites. The processor 37b calculates and determines the position (coordinates) of the work vehicle 10 based on the signals received by the antenna. The receiver 37a is, for example, placed above the driver's seat 20. Information indicating the position of the work vehicle 10 is transmitted to the control device 70 and used for automated driving, etc. The positioning device 37 uses data acquired by the camera 35 and LiDAR sensor 36 to correct or supplement the position information of the work vehicle 10 based on satellite signals. The position of the work vehicle 10 is determined with higher accuracy.

[0054] The work vehicle 10 has an inertial measuring device 38. The inertial measuring device 38 has a three-axis gyro sensor and a three-directional accelerometer. The inertial measuring device 38 detects the tilt and movement of the work vehicle 10. The signals acquired by the inertial measuring device 38 are transmitted to the control device 70. The detection signals from the inertial measuring device 38 are used to supplement the position information of the work vehicle 10. This improves positioning accuracy.

[0055] [Regarding the system configuration of work vehicle 10] Figure 2 is a block diagram showing the system configuration of the work vehicle 10. The control device 70 is composed of a processor (arithmetic processing unit) and a control unit (computer) that includes memory such as RAM and ROM. The processor reads and executes computer programs from memory, thereby executing each function of the control device 70. The control device 70 may be composed of one control unit (ECU: Electronic Control Unit) or of multiple control units. When the control device 70 is composed of multiple control units, information communication is possible between these control units.

[0056] In this embodiment, the control device 70 includes a control unit 71 for speed control, a control unit 72 for steering, a control unit 73 for first discrimination processing, a control unit 74 for information acquisition, a control unit 75 for second discrimination processing, a control unit 76 for confirmation processing, and a control unit 77 for position detection.

[0057] The control device 70 includes a storage device (storage unit) 79 consisting of a non-volatile memory for storing various types of information. Various computer programs for operating the control unit are stored in the storage device 79. Map information that can be used for autonomous driving, a trained model described later, and a database are also stored in the storage device 79.

[0058] The speed control unit 71 provides the generated drive signals to the prime mover 12, the transmission 13, and the brake system, and controls the travel speed and stopping of the work vehicle 10. The steering control unit 72 provides the steering signal it generates to the steering device 15. Based on the measurement value from the rotation sensor of the steering shaft 25, the control unit 72 controls the steering of the work vehicle 10 by controlling the hydraulic system or electric motor of the auxiliary mechanism of the steering device 15.

[0059] In the following description, the control unit 73 for the first discrimination process will be referred to as the first discrimination processing unit 73. The control unit 74 for information acquisition will be referred to as the information acquisition unit 74. The control unit 75 for the second discrimination process will be referred to as the second discrimination processing unit 75. The control unit 76 for confirmation processing will be referred to as the confirmation processing unit 76. The control unit 77 for position detection will be referred to as the position detection unit 77.

[0060] [Regarding Area Recognition System 7] In this embodiment, as described above, the work vehicle 10 has an area recognition system 7. Detection devices such as a camera 35 mounted on the vehicle body 11 detect the surroundings. The area recognition system 7 uses the detection information from the detection devices to determine whether the vehicle's position and its surroundings are on a roadway or not. The functions of the area recognition system 7 will be explained assuming that the work vehicle 10 is located on a roadway or farm road.

[0061] The area recognition system 7 comprises a first discrimination processing unit 73, an information acquisition unit 74, a second discrimination processing unit 75, a confirmation processing unit 76, and a position detection unit 77. The functions of each unit are described below.

[0062] [First discrimination processing unit 73] Figure 3 is an illustrative diagram showing an example of image information i1 from camera 35. The first discrimination processing unit 73 uses the image information i1 as primary information to determine whether the area contained in the image information i1 is a roadway or not. To this end, the first discrimination processing unit 73 first performs image processing. As part of the image processing, it performs segmentation processing on the image information i1.

[0063] Segmentation is performed using semantic segmentation algorithms, which are deep learning algorithms that associate a label (or category) with every pixel in an image. This allows for the recognition of sets of pixels that form characteristic labels (or categories).

[0064] In this embodiment, a pre-trained model for discriminating roadways is used. For this purpose, machine learning is performed in advance using image information of regions and the type (label) of those regions as the dataset. A pre-trained model is generated that takes the image information as input data and the type of region as output data. This pre-trained model is stored in the memory device 79.

[0065] Figure 4 is an illustrative diagram showing an example of a three-dimensional point cloud dataset i2 obtained from the LiDAR sensor 36. The first discrimination processing unit 73 may use the three-dimensional point cloud dataset i2 as primary information to determine whether or not the region included in the three-dimensional point cloud dataset i2 is a roadway. In this case as well, segmentation processing is performed using the three-dimensional point cloud dataset i2. Whether or not it is a roadway is determined using a pre-trained model obtained through machine learning.

[0066] The determination of whether or not an area is a road may be made using both image information i1 and a three-dimensional point cloud dataset i2. In this case, the one with the higher confidence level may be adopted. Thus, the first discrimination processing unit 73 has the function of a first discrimination processing unit that determines whether or not the detected location within the detection target is a roadway, based on first information obtained with the surrounding area of ​​the work vehicle 10 as the detection target.

[0067] [Information Acquisition Unit 74] The information acquisition unit 74 acquires specific information i3 regarding the height or width of the portion that the first discrimination processing unit 73 has determined to be a roadway. In this embodiment, the information acquisition unit 74 uses a three-dimensional point cloud dataset i2 as second information to acquire the specific information i3. In other words, the information acquisition unit 74 uses the three-dimensional point cloud dataset i2 to acquire specific information i3 regarding the height or width of the portion that the first discrimination processing unit 73 has determined to be a roadway.

[0068] The "part that the first discrimination processing unit 73 has determined to be a roadway" is, for example, the area A2 of the image information i1 shown in Figure 3. The information acquisition unit 74 uses the three-dimensional point cloud dataset i2 contained in that area A2 to acquire specific information i3 regarding the height or width of that part. Hereinafter, the "part that the first discrimination processing unit 73 has determined to be a roadway" will be referred to as the "discriminated part".

[0069] The aforementioned "information regarding the height of the discriminant portion" includes the following (A) and (B). (A) The height of the discriminant portion. In other words, the coordinate value in the height direction of the discriminant portion. For example, the average height of the discriminant portion is used as the height. (B) A value indicating the variability of the height component of the three-dimensional point cloud dataset included in the discrimination portion. Standard deviation, for example, is used as a value to indicate variability.

[0070] The information acquisition unit 74 may acquire specific information i3 by adding a three-dimensional point cloud dataset i2, which is three-dimensional information, to the two-dimensional image information i1, and then detecting the plane of the region. For this purpose, a plane estimation algorithm using RANSAC (Random Sample Consensus) is used. An example of such plane estimation is "https: / / tech-deliberate-jiro.com / python-ransac / ". By this means, the discrimination portion is detected as a plane, and that plane is designated as the "detected surface". In the following embodiments, the discrimination portion will be described as the "detected surface" obtained by the plane estimation.

[0071] In the case of (A) described above, the height value of the detected surface is a value based on the three-dimensional coordinate system of the LiDAR sensor 36. The LiDAR sensor 36 is mounted on the work vehicle 10. The LiDAR sensor 36 has a receiving unit that includes multiple optical sensors. For example, the mounting position of the receiving unit (center position) on the vehicle body 11 becomes the origin of the three-dimensional coordinate system for the LiDAR sensor 36.

[0072] In the case of (B) above, the value indicating the variation in the height component is calculated from the height component of the coordinate values ​​of the three-dimensional point cloud dataset i2. As shown in Figure 4, the three-dimensional point cloud dataset i2 acquired from the roadway in the central region A1 shows little variation in its height component. This is because the height of the road surface is approximately uniform. In contrast, as shown in Figure 4, the three-dimensional point cloud dataset i2 acquired from regions A2 on both sides (especially region A2 on the right side of Figure 4) shows a large variation in its height component. This is because the ground in the field is often uneven, and many crops (plants) grow across the field, causing a virtual surface including the tops of these crops to be detected as the detection surface. In other words, the tops of the crops growing across the field (virtual surfaces) are not strictly on the same plane.

[0073] The means for obtaining information regarding the height of the discriminant portion may be other than those based on plane estimation by RANSAC. For example, several points may be sampled from the three-dimensional point cloud dataset i2 belonging to the discriminant portion. The height values ​​(coordinate values) of each sampled point may be obtained. These statistical values ​​(e.g., mean, variability) may be used as the height information for the discriminant portion.

[0074] The information acquisition unit 74 can also acquire specific information i3, including the width dimension (horizontal dimension) of the detected surface. For example, as specific information i3 regarding the width dimension, the width dimension of the detected surface at a distance of 10 meters from the work vehicle 10 is obtained from the three-dimensional point cloud dataset i2. In the example shown in Figure 4, both sides of region A2 are large fields, and the detection range of the LiDAR sensor 36 is wide, so the width dimension of each side of region A2 is acquired as a large value of several tens of meters or more. The second piece of information used to obtain specific information i3 is a three-dimensional point cloud dataset i2, which is different from the first piece of information (image information i1) used by the first discrimination processing unit 73.

[0075] Furthermore, as specific information i3, the width dimension of the detected surface may be obtained based on image information i1 from the camera 35. For example, the width dimension of a portion 10 meters away from the work vehicle 10 may be determined based on the image information i1. In other words, the second information used to obtain specific information i3 is image information i1, and may be the same information as the first information (image information i1) used by the first discrimination processing unit 73.

[0076] Thus, the information acquisition unit 74 functions as an information acquisition processing unit that acquires specific information i3 regarding the height of the detected surface and specific information i3 regarding the width of the detected surface.

[0077] [Verification Processing Unit 76] The verification processing unit 76 confirms, for example, that the location of the work vehicle 10 is on a roadway, using information separate from the detection information from the camera 35 and the LiDAR sensor 36. The verification processing unit 76 may use at least one of the information described in (1) to (3) below as the separate information, but may also use information other than (1) to (3). (1) Output information from the inertial measuring device 38 mounted on the work vehicle 10 (2) Information combining position information from the positioning device 37 (GNSS) for the work vehicle 10 and map information (3) Information on the work plan for the work vehicle 10, with the work content and time associated with it.

[0078] A specific example of using the information in (1) above will be explained. When the work vehicle 10 performs autonomous driving, the garage (barn) becomes the starting position of the work vehicle 10. The garage is located adjacent to a road such as a farm road or a roadway. Therefore, the confirmation processing unit 76 acquires information output by the inertial measuring device 38 moment by moment from the starting position.

[0079] Generally, fields are located at a lower elevation than roadways, and the access roads from roadways to fields are relatively steep. Therefore, when the inertial measurement device 38 acquires information indicating a steep gradient (especially a downhill gradient), it is estimated that the work vehicle 10 proceeded along the access road from the roadway to the field. In this case, it is confirmed that the area where the work vehicle 10 is located is a field. On the other hand, if information indicating a steep gradient is not acquired, it is confirmed that the work vehicle 10 continued to travel along the roadway, and the area where the work vehicle 10 is located is the roadway.

[0080] A specific example of using the information in (2) above will be explained. The map information includes latitude and longitude information for each location, as well as information indicating the area of ​​the field and information indicating the area of ​​the road. Therefore, the confirmation processing unit 76 compares the location information (latitude and longitude information) of the work vehicle 10 obtained by the positioning device 37 with the map information. Through this comparison, it is confirmed that the area where the work vehicle 10 is located is a road.

[0081] A specific example of using the information in (3) above will be explained. "Work plan" refers to a plan for work to be performed by the work vehicle 10 in and outside the field. The plan for work in the field includes the type of agricultural work (e.g., soil preparation, levee construction, tilling, rice planting, etc.), the start and end times of that work, etc. The work plan also includes a plan for work outside the field, for example, a plan for traveling on the road (vehicle road) to the field. In this case, the work plan information includes the road to be traveled and the time of travel on that road.

[0082] Therefore, the verification processing unit 76 refers to the work plan information. The work plan information is stored in the storage device 79. For example, the work plan is set so that the time from 9:00 AM to 9:30 AM is the time to travel from the garage to the first roadway and move to the first field. If the time of verification by the verification processing unit 76 is 9:10 AM, it confirms that the area where the work vehicle 10 is located is the roadway.

[0083] The confirmation processing unit 76 may use the same information as the detection information from the camera 35 or LiDAR sensor 36 to confirm that the location of the work vehicle 10 is on a roadway. In this case, the confirmation processing unit 74 uses a different algorithm than the one used for determination by the first determination processing unit 73. For example, the location of the work vehicle 10 is confirmed to be on a roadway by recognizing, for example, lanes or signs from the image from the camera 35 using image processing.

[0084] Thus, the confirmation processing unit 76 has the function of a confirmation processing unit that confirms that the location of the work vehicle 10 is on a roadway.

[0085] [Position detection unit 77] The position detection unit 77 detects the height of the reference position of the work vehicle 10. Any position on the work vehicle 10 can be set as the reference position. In this embodiment, the reference position is set to the position indicating the vehicle height of the work vehicle 10, that is, the highest point of the work vehicle 10 (the upper edge of the roof of the driver's seat 20). The position detection unit 77 can detect the height position (coordinates) of the reference position of the work vehicle 10 based on the position information of the work vehicle 10 obtained by the positioning device 37 (GNSS).

[0086] The position (coordinates) obtained by the positioning device 37 is the mounting position of the satellite signal receiver 37a. The mounting position of the receiver 37a, the position of the highest point of the work vehicle 10, and the difference between these positions are known. Therefore, if the position of the receiver 37a does not coincide with the position of the highest point of the work vehicle 10, the position detection unit 77 uses the position information from the positioning device 37 and the known value to determine the height of the reference position of the work vehicle 10. Alternatively, the work vehicle 10 may have a barometer. In this case, the position detection unit 77 may detect the height position of the work vehicle 10 based on the information measured by the barometer.

[0087] The aforementioned "reference position" can be set (changed) to any other position, for example, it may be the position where the work vehicle 10 is in contact with the ground (road surface), or it may be the mounting position of the receiver 37a of the positioning device 37 which acts as a detector.

[0088] In this embodiment, to detect the height of the reference position, information from the positioning device 37, etc., is used as information separate from the detection information from the camera 35 and LiDAR sensor 36. However, the detection information from the camera 35 and LiDAR sensor 36 may also be used to detect the height of the reference position.

[0089] Thus, the position detection unit 77 functions as a position detection unit that detects the height of the reference position of the work vehicle 10.

[0090] [Second discrimination processing unit 75] As described above, the first discrimination processing unit 73 determines whether or not the area around the work vehicle 10 is a roadway (primary discrimination) based on the first information obtained by detecting the area around the work vehicle 10. In the following explanation, it is assumed that, contrary to reality, the right-hand region A2 shown in Figures 3 and 4 is determined to be a roadway as a primary discrimination. The information acquisition unit 74 acquires specific information i3 regarding the height of the detection surface of region A2, and specific information i3 regarding the width of the detection surface of region A2.

[0091] Although area A2 has been determined to be a roadway, the second discrimination processing unit 75, if the specific information i3 satisfies predetermined discrimination conditions, determines that the portion that the first discrimination processing unit 73 determined to be a roadway is not a roadway (secondary discrimination). The aforementioned discrimination conditions are conditions relating to height or conditions relating to width, and specific examples will be explained later. The discrimination conditions are set in advance, and information indicating these discrimination conditions (hereinafter referred to as "discrimination condition information") is stored in the storage device 79. The second discrimination processing unit 75 compares the specific information i3 with the discrimination condition information, and according to the result, it determines that the portion that the first discrimination processing unit 73 has determined to be a roadway is not a roadway.

[0092] The discrimination criteria (discrimination criteria information) include at least one of the first to fourth discrimination criteria described later. In this embodiment, the discrimination criteria include all of the first to fourth discrimination criteria described later and are used selectively. The second discrimination processing unit 75 performs a process of comparing the specific information i3 with each of the discrimination criteria information. In the following explanation, we will focus on area A2, which is the field on the right side as shown in Figures 3 and 4.

[0093] In the case of the first and second discrimination conditions, the height of the detected surface of area A2, which is the field, is compared with the reference position of the work vehicle 10. The height of the detected surface of area A2, which is the field, is determined using the three-dimensional point cloud dataset i2 obtained by the LiDAR sensor 36. The reference position of the work vehicle 10 is determined using positioning information obtained by the positioning device 37. The height of the detected surface of area A2, which is the field, and the reference position of the work vehicle 10 are based on different coordinate systems. In order to compare the height of the detected surface of area A2, which is the field, and the reference position of the work vehicle 10, the coordinate system transformation process described below is performed.

[0094] A basic coordinate system is defined with an arbitrary position on the work vehicle 10 as the reference (origin). The mounting position of the positioning device 37 (receiver 37a) on the work vehicle 10 is known. The coordinates indicated by the position information acquired by the positioning device 37 are applied to (converted) the basic coordinate system. The mounting position of the LiDAR sensor 36 (receiver unit including multiple optical sensors) on the work vehicle 10 is known. The coordinates of the three-dimensional point cloud dataset acquired by the LiDAR sensor 36 are applied to (converted) the basic coordinate system. As a result, the second discrimination processing unit 75 can compare heights using the basic coordinate system. The above conversion process may be performed by any of the information acquisition unit 74, position detection unit 77, and second discrimination processing unit 75, or by any other control unit.

[0095] As described above, the second discrimination processing unit 75 functions as a second discrimination processing unit that distinguishes between roadways and non-roadways using specific information i3 relating to height or width. This specific information i3 is information based on second information such as a three-dimensional point cloud dataset i2, obtained for the area that overlaps with the area discriminated by the first discrimination processing unit 73 using first information such as image information i1. The "area that the first discrimination processing unit 73 has determined to be a roadway using first information such as image information i1" is, for example, the area of ​​region A2 in the image information i1 shown in Figure 3. The "second information (three-dimensional point cloud dataset i2) obtained for the area overlapping with the first discrimination processing unit 73" is information for all or part of the area included in region A2. Based on this second information, specific information i3 is obtained regarding the height or width of the discriminated portion (detected plane) that the first discrimination processing unit 73 has determined to be a roadway. Then, this specific information i3 is used to distinguish between roadways and non-roadways.

[0096] In the following, we will explain the first to fourth discrimination conditions, and how the first discrimination processing unit 73 incorrectly identifies area A2, which is a field, as a roadway (primary discrimination), and how the second discrimination processing unit 75 identifies whether area A2 is a roadway or not (secondary discrimination).

[0097] [Specific examples of discrimination criteria and second discrimination process] Figure 5 is a flowchart showing an example of the processing performed by the area recognition system 7. The work vehicle 10 located on the roadway detects its surrounding environment using a camera 35 and a LiDAR sensor 36 (step ST1 in Figure 5). When the first discrimination processing unit 73 acquires detection information from the camera 35 (Figure 3), it uses a trained model that takes the detection information from the camera 35 as input data to determine whether each position (area of ​​pixel group) of the image information i1 from the camera 35 is a roadway or not (step ST2 in Figure 5). In this embodiment, the position corresponding to area A2, which is a field, is incorrectly determined to be a roadway (step ST3 in Figure 5).

[0098] The information acquisition unit 74 determines the detected surface, for example, by a plane estimation algorithm using RANSAC (step ST4 in Figure 5). The three-dimensional point cloud dataset i2 is used to determine the detected surface for the portion that the first discrimination processing unit 73 has determined to be a roadway. The information acquisition unit 74 acquires specific information i3 regarding the height and width of the detected surface using the three-dimensional point cloud dataset i2. In other words, specific information i3 regarding the height of the detected surface and specific information i3 regarding the width of the detected surface are acquired for region A2, which should be a field but has been determined to be a roadway (step ST5 in Figure 5). The specific information i3 regarding the height of the detected surface includes (A) and (B) above.

[0099] The position detection unit 77 uses the position information acquired by the positioning device 37 to detect the height of the reference position of the work vehicle 10 (step ST6 in Figure 5). The verification processing unit 76 performs a process to confirm that the location of the work vehicle 10 is on a roadway (step ST7 in Figure 5). The order of steps ST5, ST6, and ST7 is arbitrary, and one of these steps may be performed simultaneously with the others.

[0100] In step ST3, area A2, which is actually a field, is determined to be a roadway. Even though the field is determined to be a roadway in this way, the second discrimination processing unit 75 determines that the portion (area A2) that the first discrimination processing unit 73 determined to be a roadway is not a roadway if the specific information i3 satisfies any of the first to fourth discrimination conditions. In other words, the portion (area A2) that the first discrimination processing unit 73 determined to be a roadway is determined to be not a roadway (field) (step ST8 in Figure 5).

[0101] [(1) When the first discrimination criterion applies] As shown in Figure 6(A), the area A2 adjacent to area A1 where the work vehicle 10 is located, and which is identified as a roadway, is a field. However, as shown in Figure 6(B), it is initially incorrectly identified as a roadway. Therefore, the second discrimination processing unit 75 compares the specific information i3 indicating the height of the detected surface K in region A2 with the following first discrimination condition (1).

[0102] First discrimination condition: The height of the detection surface K in region A2 is lower than the reference position P of the work vehicle 10 by more than the first threshold. The first threshold is an arbitrary value and is set in advance. If the reference position P is the highest point of the work vehicle 10, the first threshold is a value that takes into account the height of the work vehicle 10 and is changed depending on the type of work vehicle 10. If the reference position P is the point where the work vehicle 10 touches the ground (road surface), the first threshold is, for example, 0.3 meters. Alternatively, the first threshold is 1 / 3 of the diameter of the work vehicle 10's tires.

[0103] The first criterion for identification is based on the fact that, generally, the ground surface of a field is often one level lower than the ground surface (road surface) of a road. Furthermore, when the first discrimination criterion is applied, the detection surface K can be considered in two possible cases, for example: <1> The soil surface of a field where no plants are growing. <2> A hypothetical area where many relatively short crops (plants) are growing, including the tops of these crops.

[0104] If the specific information i3 satisfies the first discrimination condition, then, as shown in Figure 6(C), the area A2 is determined to be something other than a roadway (a field), not a roadway.

[0105] [(2) When the second discrimination criterion applies] As shown in Figure 7(A), the area A2 adjacent to area A1 where the work vehicle 10 is located is a field, and the ground level of that field is lower than that of area A1, which is a road. However, many relatively tall crops are growing across that field. A virtual surface including the tops of these crops becomes the detection surface K of area A2, and the first discrimination processing unit 73 may mistakenly determine that area A2 is a road (see Figure 7(B)). Therefore, the second discrimination processing unit 75 compares the specific information i3 indicating the height of the detection surface K of area A2 with the following second discrimination condition.

[0106] Second discrimination criterion: The height of the detection surface K in region A2 is higher than the reference position P of the work vehicle 10, exceeding the second threshold. The second threshold is an arbitrary value and is set in advance. If the reference position P is the highest point of the work vehicle 10, the second threshold is a value that takes into account the height of the work vehicle 10 and is changed depending on the type of work vehicle 10. If the reference position P is the point where the work vehicle 10 touches the ground (road surface), the second threshold is, for example, 0.5 meters. Alternatively, the second threshold is 1 / 3 of the diameter of the work vehicle 10's tires.

[0107] If the specific information i3 satisfies the second discrimination condition, then, as shown in Figure 7(C), the area A2 is determined to be something other than a roadway (a field), not a roadway.

[0108] [(3) When the third discrimination condition applies] As shown in Figure 8(A), if the roadway (area A1) and the field (area A2) are at approximately the same height, the first and second discrimination conditions are not met. As shown in Figures 8(B) and 8(C), a virtual surface including the tops of crops growing across the field (region A2) becomes the detection surface K of region A2. If this detection surface K and the road (region A1) are at approximately the same height, then, as in Figure 8(A), the first and second discrimination conditions are not met. Therefore, even if the second area A2 is initially incorrectly identified as a roadway, this is not corrected.

[0109] Even in such cases, it may be possible to correct the incorrect primary discrimination by using a third discrimination criterion. The second discrimination processing unit 75 compares the specific information i3 indicating the width dimension of area A2 with the following third discrimination criterion. The third criterion for distinction is based on the following findings: As exemplified by roads where vehicles can travel, there is a certain upper limit on the width of roads, and this limit is set. In contrast, land other than roads, such as fields, is often much larger than roads (roads).

[0110] Third discrimination criterion: The width dimension of area A2 exceeds the standard dimension. The aforementioned standard dimension is, for example, 15 meters. If the width dimension of the area exceeds 15 meters, the area is considered to be a field rather than a roadway. Alternatively, the aforementioned standard dimension may be the sum of the widths of four work vehicles 10.

[0111] As shown in Figure 8(D), even if area A2 is initially incorrectly identified as a roadway, if the specific information i3 satisfies the third identification condition, then, as shown in Figure 8(E), area A2 is identified as something other than a roadway (a field).

[0112] [(4) When the fourth discrimination criterion applies] The second discrimination processing unit 75 can perform secondary discrimination processing using a fourth discrimination condition instead of the third discrimination condition, or in combination with the third discrimination condition. As shown in Figures 9(B) and 9(C), a virtual surface including the tops of crops growing across the field (region A2) becomes the detection surface K of the second region A2, and there are cases where this detection surface K and the road (region A1) are at approximately the same height. In such cases, region A2 may be mistakenly classified as a road in the primary discrimination (see Figure 9(D)).

[0113] However, the upper edges of these crops (virtual plane K) are not strictly coplanar. Therefore, as shown in Figure 4, when region A2 is detected by the LiDAR sensor 36, there is variation in the height component of the three-dimensional point cloud dataset on the detected surface of region A2. Therefore, in this case, the second discrimination processing unit 75 uses a value indicating the variation in the height component of the (B) three-dimensional point cloud dataset as specific information i3 regarding the height of the detected surface of region A2. This specific information i3 is then compared with the following fourth discrimination criterion.

[0114] Fourth discrimination criterion: The value indicating the variability of the height component of the three-dimensional point cloud dataset in region A2 exceeds the range of the third threshold. The range of the third threshold can be set arbitrarily. In other words, if the variability of the height component of the three-dimensional point cloud dataset in the second region A2 is large, the fourth discrimination condition is met.

[0115] When the specific information i3 satisfies the fourth discrimination condition, the area A2 is determined to be something other than a roadway (a field), as shown in Figure 9(E).

[0116] Furthermore, as shown in Figure 7(A), in reality, the roadway (area A1) and the field (area A2) are at approximately the same height. However, a virtual surface including the tops of the crops growing across the field (area A2) becomes the detected surface K of area A2, and area A2 may be mistakenly identified as a roadway in the primary classification. Even in such cases, the primary classification can be corrected by performing a secondary classification process using the fourth classification criterion.

[0117] [Regarding each discrimination criterion] As described above, even if the first discrimination processing unit 73 makes an incorrect initial determination that area A2 is a roadway, the second discrimination processing unit 75 can correct the initial determination by determining that area A2 is something other than a roadway (a field) if the specific information i3 satisfies the determination conditions.

[0118] [Regarding the area recognition system of this embodiment] The area recognition system of this embodiment has a control device (computer) 70 that includes one or more control units that use detection information obtained by detecting the area around the work vehicle 10 as the detection target to determine whether or not it is a road (vehicleway).

[0119] The control device 70 includes a first discrimination processing unit 73 and a second discrimination processing unit 75. The first discrimination processing unit 73 determines whether or not the area around the work vehicle 10 is a roadway based on first information (image information i1 from the camera 35) obtained as the detection target. The second discrimination processing unit 75 determines whether or not the area is a roadway using specific information i3 relating to height or width. This specific information i3 is based on second information (three-dimensional point cloud dataset i2) obtained for the area that overlaps with the area determined to be a roadway using the first information.

[0120] The "area identified as a roadway using the first information" is, for example, the area of ​​region A2 in the image information i1 shown in Figure 3. The "second information (three-dimensional point cloud dataset i2) obtained for the area overlapping with the first information" is information for all or part of the area A2. Based on this second information, the first discrimination processing unit 73 obtains specific information i3 regarding the height or width of the part (detection plane) that it has identified as a roadway. Then, this specific information i3 is used to distinguish between roadways and non-roadways.

[0121] According to the area recognition system of this embodiment, if a portion is initially identified as a roadway, and the specific information i3 relating to that portion satisfies the identification conditions, it becomes possible to identify that portion as something other than a roadway. Therefore, it becomes possible to suppress the misidentification of a field as a roadway. As a result, it becomes possible to prevent the automatic operation of the work vehicle 10 from being interrupted due to misidentification of the roadway.

[0122] 〔others〕 The technology disclosed herein is applicable not only to tractors (working vehicles 10) but also to other types of vehicles. For example, the vehicles may be harvesters, rice transplanters, riding cultivators, vegetable transplanters, lawnmowers, seeders, and fertilizer spreaders. It is also possible to apply the area recognition system to agricultural machinery other than vehicles.

[0123] The embodiments described above are illustrative and not restrictive in all respects. The scope of the present invention is indicated by the claims rather than by the embodiments, and includes all modifications within the scope equivalent to the configurations described in the claims. [Explanation of symbols]

[0124] 7. Area Recognition System 10 Work Vehicles 11 Vehicle body 35. Camera (detection device) 36. LiDAR sensor (three-dimensional range sensor, detection device) 73 Control unit for the first discrimination process (first discrimination processing unit) 74. Control unit for information acquisition (information acquisition unit) 75 Control unit for the second discrimination process (second discrimination processing unit) 76 Control unit for verification processing (verification processing unit) 77. Control unit for position detection (position detection unit) i1 Image Information i2 3D point cloud dataset i3 Specific Information K detection surface

Claims

1. A first discrimination processing unit determines whether or not the area around the work vehicle is a road based on the first information obtained by detecting the area around the work vehicle, A second discrimination processing unit that distinguishes between roads and non-road areas using specific information regarding height or width based on second information obtained for the area overlapping with the area determined using the first information, It has, The second discrimination processing unit, with respect to the portion that the first discrimination processing unit has determined to be a road, determines that the specific information is not a road if it satisfies predetermined discrimination conditions. Area recognition system.

2. The first discrimination processing unit uses a trained model based on the first information to determine whether or not it is a road. The region recognition system according to claim 1.

3. The first piece of information and the second piece of information are different pieces of information. The region recognition system according to claim 1.

4. It has an information acquisition unit that acquires the aforementioned specific information, The second discrimination processing unit, when the specific information satisfies the discrimination conditions regarding height or width, determines that the portion that the first discrimination processing unit has determined to be a road is not a road. The region recognition system according to claim 1.

5. The system includes a confirmation processing unit capable of confirming that the location of the work vehicle is a road. The specified information includes height information of the portion that the first discrimination processing unit has determined to be a road, The aforementioned determination conditions include the condition that the height of the portion determined by the first determination processing unit to be a road is lower than the reference position of the work vehicle by a first threshold, The region recognition system according to claim 4.

6. The system includes a confirmation processing unit capable of confirming that the location of the work vehicle is a road. The specified information includes height information of the portion that the first discrimination processing unit has determined to be a road, The aforementioned determination conditions include the condition that the height of the portion that the first determination processing unit has determined to be a road is higher than the reference position of the work vehicle by a second threshold, The region recognition system according to claim 4.

7. The verification processing unit can confirm that the location of the work vehicle is a road, using information other than the first information and the second information, or using an algorithm different from the determination made by the first determination processing unit. The region recognition system according to claim 5 or claim 6.

8. The second information includes a three-dimensional point cloud dataset of the portion that the first discriminant processing unit has determined to be a road, The aforementioned specific information includes information on values ​​indicating the variability of the height component of a three-dimensional point cloud dataset. The aforementioned discrimination criterion includes the condition that the value indicating the variation exceeds the range of the third threshold, The region recognition system according to claim 4.

9. The specified information includes information on the width dimension of the portion that the first discrimination processing unit has determined to be a road, The aforementioned determination conditions include the condition that the width dimension exceeds the standard dimension. The region recognition system according to claim 4.

10. The first discrimination processing unit uses the image information acquired by the camera as the first information to determine whether or not it is a road. The region recognition system according to any one of claims 1 to 3.

11. The aforementioned information acquisition unit, A three-dimensional point cloud dataset of the target area is obtained using a three-dimensional range sensor. Using the aforementioned three-dimensional point cloud dataset, the height of the portion that the first discrimination processing unit has determined to be a road is obtained as the specified information. The region recognition system according to claim 4.

12. The first discrimination processing unit uses the three-dimensional point cloud dataset acquired by the three-dimensional range sensor as the first information to determine whether or not it is a road. The region recognition system according to any one of claims 1 to 3.

13. The work vehicle has a position detection unit that detects the height of the reference position of the work vehicle, The region recognition system according to claim 5 or claim 6.

14. The aforementioned verification processing unit, Using at least one of the following pieces of information: output information from an inertial measuring device mounted on the vehicle, information combining position information and map information from GNSS for the vehicle, and work plan information for the vehicle in which work content and time are associated; The region recognition system according to claim 5 or claim 6.

15. The vehicle body and A detection device that detects the area around the vehicle body, A work vehicle comprising the area recognition system described in claim 1.

Images