Agricultural machinery, detection systems used in agricultural machinery, and detection methods.

Abstract

Invention details;

Description

Agricultural machinery, sensing system for use in agricultural machinery, and sensing method

[0001] The present disclosure relates to agricultural machinery, and sensing systems and sensing methods for use in agricultural machinery.

[0002] Research and development is underway on smart agriculture, which utilizes ICT (Information and Communication Technology) and IoT (Internet of Things) as the next generation of agriculture. Research and development is also underway to automate and unmanned farm vehicles such as tractors used in farm fields. For example, farm vehicles that can run with automatic steering using positioning systems such as the Global Navigation Satellite System (GNSS), which enables precise positioning, are now being put into practical use.

[0003] Furthermore, technology is being developed that uses obstacle sensors to search the area around a work vehicle and detect obstacles around the work vehicle. For example, Patent Document 1 discloses technology that uses a LiDAR (Light Detection and Ranging) sensor to detect obstacles around an autonomous tractor.

[0004] Japanese Patent Application Laid-Open No. 2019-175059

[0005] The present disclosure provides techniques for performing a search of the environment around an agricultural machine that is appropriate for the area in which the agricultural machine is located.

[0006] A sensing system according to one embodiment of the present disclosure is a sensing system for a mobile agricultural machine, comprising one or more sensors mounted on the agricultural machine that sense the environment around the agricultural machine and output sensing data, and a processing device that detects objects located in a search area around the agricultural machine based on the sensing data, and the processing device changes the pattern of the search area in which the object is detected depending on the area in which the agricultural machine is located.

[0007] A sensing method according to one embodiment of the present disclosure is a sensing method for a mobile agricultural machine, and includes sensing the environment around the agricultural machine using one or more sensors and outputting sensing data, detecting an object located in a search area around the agricultural machine based on the sensing data, and changing the pattern of the search area for detecting the object depending on the area in which the agricultural machine is located.

[0008] A general or specific aspect of the present disclosure may be realized by an apparatus, a system, a method, an integrated circuit, a computer program, or a computer-readable non-transitory storage medium, or any combination thereof. The computer-readable storage medium may include a volatile storage medium or a non-volatile storage medium. An apparatus may be composed of multiple devices. When an apparatus is composed of two or more devices, the two or more devices may be located in a single device or may be located separately in two or more separate devices.

[0009] According to an embodiment of the present disclosure, the pattern of the search area for detecting an object is changed depending on the area the agricultural machine is located in. This allows a search appropriate for the area in which the agricultural machine is located to be performed.

[0010] 1 is a diagram for explaining an overview of an agricultural management system according to an exemplary embodiment of the present disclosure.

[0022] FIG. 1 is a side view schematically illustrating an example of a work vehicle and a work implement coupled to the work vehicle.

[0023] FIG. 2 is a block diagram illustrating an example configuration of a work vehicle and a work implement.

[0024] FIG. 3 is a conceptual diagram illustrating an example of a work vehicle that performs positioning using RTK-GNSS.

[0025] FIG. 4 is a diagram illustrating an example of an operation terminal and a group of operation switches provided inside the cabin.

[0026] FIG. 5 is a block diagram illustrating an example of the hardware configuration of a management device and a terminal device.

[0027] FIG. 6 is a diagram illustrating an example of a work vehicle that automatically travels along a target route in a field.

[0028] FIG. 7 is a flowchart illustrating an example of steering control operation during autonomous driving.

[0029] FIG. 8 is a diagram illustrating an example of a work vehicle traveling along a target route P.

[0029] FIG. 9 is a diagram illustrating an example of a work vehicle that is shifted to the right from the target route P.

[0030] FIG. 10 is a diagram illustrating an example of a work vehicle that is facing in an inclined direction with respect to the target route P.

[0031] FIG. 11 is a diagram illustrating an example of a situation in which multiple work vehicles are automatically traveling inside a field and on roads outside the field.

[0032] FIG. 12 is a flowchart illustrating an example of a process for changing the pattern of a search area depending on the area in which the agricultural machine is located.

[0033] FIG. 13 is a diagram illustrating an example of an area in which the pattern of the search area is changed. 1 is a diagram showing examples of a first search area and a second search area. FIG. 1 is a diagram showing the relationship between a sensing area sensed by a LiDAR sensor and a search area in which an object is searched. FIG. 2 is a diagram showing the relationship between a sensing area sensed by a LiDAR sensor and a search area in which an object is searched. FIG. 3 is a diagram showing another example of an area in which the pattern of the search area is changed. A flowchart showing an example of processing when an obstacle is detected. FIG. 1 is a diagram showing another example of a first search area and a second search area. FIG. 2 is a diagram showing yet another example of a first search area and a second search area. FIG. 3 is a diagram showing yet another example of a first search area and a second search area. FIG. 4 is a diagram showing yet another example of an area in which the pattern of the search area is changed. FIG. 4 is a diagram showing yet another example of a first search area and a second search area. FIG. 5 is a diagram showing yet another example of an area in which the pattern of the search area is changed.1 is a diagram showing an example of a search area that is set according to the size of an implement connected to a work vehicle, and FIG. 2 is a diagram showing an example of a search area that is set according to the positional relationship between the work vehicle and the implement.

[0011] (Definition of Terms) In this disclosure, "agricultural machine" refers to a machine used for agricultural purposes. The agricultural machine of this disclosure may be a mobile agricultural machine capable of performing agricultural work while moving. Examples of agricultural machines include tractors, harvesters, rice transplanters, riding cultivators, vegetable transplanters, mowers, seed sowing machines, fertilizer applicators, and agricultural mobile robots. Not only can a work vehicle such as a tractor function alone as an "agricultural machine," but the entire work vehicle and an implement attached to or towed by the work vehicle can also function as a single "agricultural machine." Agricultural machines perform agricultural work on the ground in a field, such as plowing, sowing seeds, pest control, fertilizing, planting crops, or harvesting. These agricultural works are sometimes referred to as "ground work" or simply "work." Traveling while performing agricultural work by a vehicle-type agricultural machine is sometimes referred to as "work driving."

[0012] "Autonomous driving" refers to controlling the movement of an agricultural machine through the action of a control device, without manual operation by a driver. Agricultural machines that perform autonomous driving are sometimes called "autonomous agricultural machines" or "robotic agricultural machines." During autonomous driving, not only the movement of the agricultural machine but also the agricultural work operations (e.g., the operation of the implement) may be automatically controlled. When the agricultural machine is a vehicle-type machine, the movement of the agricultural machine through autonomous driving is referred to as "autonomous driving." The control device may control at least one of the steering, speed adjustment, and start and stop of movement required for the movement of the agricultural machine. When controlling a work vehicle equipped with implements, the control device may control operations such as raising and lowering the implement and starting and stopping its operation. Autonomous driving movement may include not only movement of the agricultural machine toward a destination along a predetermined route, but also movement of the agricultural machine following a tracking target. An autonomously driving agricultural machine may move partially based on user instructions. Furthermore, an autonomously driving agricultural machine may operate in a manual driving mode, in which movement is performed by manual operation by the driver, in addition to an autonomous driving mode. Steering an agricultural machine by the action of a control device, without manual operation, is called "automatic steering." Part or all of the control device may be external to the agricultural machine. Communication of control signals, commands, data, etc. may take place between the agricultural machine and a control device external to the agricultural machine. An agricultural machine that performs automatic driving may move autonomously while sensing the surrounding environment, without a human being being involved in controlling the movement of the agricultural machine. An agricultural machine capable of autonomous movement can travel unmanned within or outside a field (e.g., on a road). During autonomous movement, it may detect obstacles and take action to avoid the obstacles.

[0013] A "work plan" is data that schedules one or more agricultural tasks to be performed by an agricultural machine. The work plan may include, for example, information indicating the order of agricultural tasks to be performed by the agricultural machine and the field on which each task will be performed. The work plan may also include information on the scheduled date and time for each task to be performed. The work plan may be created by a processing device that communicates with the agricultural machine to manage the agricultural tasks, or a processing device mounted on the agricultural machine. The processing device may create the work plan based on information entered by a user (such as a farm manager or farm worker) operating a terminal device, for example. In this specification, a processing device that communicates with the agricultural machine to manage the agricultural tasks is referred to as a "management device." The management device may manage the agricultural tasks of multiple agricultural machines. In this case, the management device may create a work plan that includes information on each agricultural task to be performed by each of the multiple agricultural machines. The work plan may be downloaded by each agricultural machine and stored in a storage device. Each agricultural machine can automatically head to the field and perform the scheduled agricultural tasks according to the work plan.

[0014] An "environmental map" is data that represents the positions or areas of objects in the environment in which the agricultural machine moves using a specified coordinate system. An environmental map may be simply referred to as a "map" or "map data." The coordinate system that defines the environmental map may be, for example, a world coordinate system such as a geographic coordinate system fixed relative to the Earth. An environmental map may also include information other than the positions of objects in the environment (for example, attribute information and other information). Environmental maps include maps in various formats, such as point cloud maps or grid maps. Data for local maps or partial maps that are generated or processed in the process of constructing an environmental map are also referred to as a "map" or "map data."

[0015] "Farm road" means a road that is primarily used for agricultural purposes. Farm roads are not limited to roads paved with asphalt, but also include unpaved roads covered with dirt or gravel. Farm roads include roads (including private roads) that are exclusively accessible to vehicle-type agricultural machinery (e.g., work vehicles such as tractors) and roads that are also accessible to general vehicles (passenger cars, trucks, buses, etc.). Work vehicles may automatically travel on public roads in addition to farm roads. Public roads are roads that have been developed for the traffic of general vehicles.

[0016] (Embodiments) Hereinafter, embodiments of the present disclosure will be described. However, more detailed descriptions than necessary may be omitted. For example, detailed descriptions of already well-known matters and redundant descriptions of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. Note that the inventors provide the accompanying drawings and the following description to enable those skilled in the art to fully understand the present disclosure, and do not intend for them to limit the subject matter described in the claims. In the following description, components having the same or similar functions are designated by the same reference numerals.

[0017] The following embodiments are examples, and the technology of the present disclosure is not limited to the following embodiments. For example, the numerical values, shapes, materials, steps, step order, display screen layout, etc. shown in the following embodiments are merely examples, and various modifications are possible as long as no technical contradiction occurs. Furthermore, one aspect can be combined with another aspect as long as no technical contradiction occurs.

[0018] The following mainly describes an embodiment in which the technology of the present disclosure is applied to a work vehicle such as a tractor, which is an example of agricultural machinery. The technology of the present disclosure is not limited to work vehicles such as tractors, but can also be applied to other types of agricultural machinery.

[0019] FIG. 1 is a diagram illustrating an overview of an agricultural management system 1 according to an exemplary embodiment of the present disclosure. The agricultural management system 1 shown in FIG. 1 includes a work vehicle 100, a terminal device 400, and a management device 600. The terminal device 400 is a computer used by a user to remotely monitor the work vehicle 100. The management device 600 is a computer managed by the business operator that operates the agricultural management system 1. The work vehicle 100, the terminal device 400, and the management device 600 can communicate with each other via a network 80. Although FIG. 1 illustrates one work vehicle 100, the agricultural management system 1 may include multiple work vehicles or other agricultural machinery.

[0020] The work vehicle 100 in this embodiment is a tractor. The work vehicle 100 can be fitted with a work implement at either the rear or the front, or both. The work vehicle 100 can travel within a field while performing agricultural work according to the type of work implement. The work vehicle 100 may also travel within or outside a field without a work implement attached.

[0021] The work vehicle 100 has an automatic driving function. That is, the work vehicle 100 can travel by the operation of a control device, without manual operation. The control device in this embodiment is provided inside the work vehicle 100, and can control both the speed and steering of the work vehicle 100. The work vehicle 100 can travel automatically not only within a field, but also outside the field (for example, on a road).

[0022] The work vehicle 100 is equipped with devices used for positioning or self-location estimation, such as a GNSS receiver and a LiDAR sensor. The control device of the work vehicle 100 causes the work vehicle 100 to travel automatically based on the position of the work vehicle 100 and information about a target route. In addition to controlling the travel of the work vehicle 100, the control device also controls the operation of the work implement. This allows the work vehicle 100 to perform agricultural work using the work implement while traveling automatically within a field. Furthermore, the work vehicle 100 can automatically travel along roads outside the field (e.g., farm roads or public roads) along a target route. The work vehicle 100 automatically travels along roads outside the field while utilizing data output from sensing devices such as a camera 120, an obstacle sensor 130, and a LiDAR sensor 140.

[0023] The management device 600 is a computer that manages agricultural work performed by the work vehicle 100. The management device 600 may be, for example, a server computer that centrally manages information about a field on the cloud and supports agriculture by utilizing data on the cloud. The management device 600, for example, creates a work plan for the work vehicle 100 and causes the work vehicle 100 to perform agricultural work in accordance with the work plan. The management device 600 may generate a target route within the field based on information input by a user using the terminal device 400 or another device. The management device 600 may also generate and edit an environmental map based on data collected by the work vehicle 100 or other moving objects using a sensing device such as a LiDAR sensor. The management device 600 transmits the generated work plan, target route, and environmental map data to the work vehicle 100. The work vehicle 100 automatically moves and performs agricultural work based on this data.

[0024] The terminal device 400 is a computer used by a user located remotely from the work vehicle 100. While the terminal device 400 shown in FIG. 1 is a laptop computer, the present invention is not limited to this. The terminal device 400 may be a stationary computer such as a desktop personal computer (PC), or a mobile terminal such as a smartphone or tablet computer. The terminal device 400 may be used to remotely monitor or remotely operate the work vehicle 100. For example, the terminal device 400 can display images captured by one or more cameras (imaging devices) equipped on the work vehicle 100 on a display. The terminal device 400 can also display a setting screen on the display that allows the user to input information necessary to create a work plan for the work vehicle 100 (e.g., a schedule for each agricultural work). When the user inputs the necessary information on the setting screen and performs a send operation, the terminal device 400 transmits the input information to the management device 600. The management device 600 creates a work plan based on the information. The terminal device 400 may further have a function of displaying a setting screen on the display for the user to input information necessary for setting a target route.

[0025] The configuration and operation of the system in this embodiment will be described in more detail below.

[0026] 2 is a side view that schematically shows an example of a work vehicle 100 and a work implement 300 coupled to the work vehicle 100. The work vehicle 100 in this embodiment can operate in both a manual driving mode and an automatic driving mode. In the automatic driving mode, the work vehicle 100 can travel unmanned. The work vehicle 100 can be driven automatically both inside and outside a field.

[0027] As shown in Fig. 2, work vehicle 100 includes a vehicle body 101, a prime mover (engine) 102, and a transmission 103. Vehicle body 101 is provided with wheels 104 with tires and a cabin 105. Wheels 104 include a pair of front wheels 104F and a pair of rear wheels 104R. Inside cabin 105, a driver's seat 107, a steering device 106, an operation terminal 200, and a group of switches for operation are provided. When work vehicle 100 travels through a field, one or both of front wheels 104F and rear wheels 104R may be multiple wheels (crawlers) equipped with tracks instead of wheels with tires.

[0028] Work vehicle 100 may be equipped with at least one sensing device that senses the environment around work vehicle 100, and a processing device that processes sensing data output from the at least one sensing device. In the example shown in Figure 2, work vehicle 100 is equipped with multiple sensing devices. The sensing devices include multiple cameras 120, a LiDAR sensor 140, and multiple obstacle sensors 130.

[0029] Cameras 120 may be installed, for example, on the front, rear, left and right sides of work vehicle 100. Cameras 120 capture images of the environment around work vehicle 100 and generate image data. Images captured by cameras 120 may be output to a processing device mounted on work vehicle 100 and transmitted to terminal device 400 for remote monitoring. These images may also be used to monitor work vehicle 100 during unmanned operation. Cameras 120 may also be used to generate images for recognizing surrounding features or obstacles, white lines, signs, or indications when work vehicle 100 travels on roads outside of fields (farm roads or public roads).

[0030] In the example of FIG. 2 , the LiDAR sensor 140 is disposed at the lower front portion of the vehicle body 101. The LiDAR sensor 140 may be disposed at another location. For example, the LiDAR sensor 140 may be disposed at the top of the cabin 105. The LiDAR sensor 140 may be a 3D-LiDAR sensor, but may also be a 2D-LiDAR sensor. The LiDAR sensor 140 senses the environment surrounding the work vehicle 100 and outputs sensing data. While the work vehicle 100 is traveling mainly outside the field, the LiDAR sensor 140 repeatedly outputs sensor data indicating the distance and direction to each measurement point of an object present in the surrounding environment, or the three-dimensional or two-dimensional coordinate values ​​of each measurement point. The sensor data output from the LiDAR sensor 140 is processed by a control device of the work vehicle 100. The control device can estimate the self-position of the work vehicle 100 by matching the sensor data with an environmental map. The control device can further detect objects such as obstacles present around the work vehicle 100 based on the sensor data. The control device can also generate or edit an environmental map using an algorithm such as SLAM (Simultaneous Localization and Mapping). The work vehicle 100 may be equipped with multiple LiDAR sensors arranged in different positions and with different orientations.

[0031] The multiple obstacle sensors 130 shown in FIG. 2 are provided at the front and rear of the cabin 105. The obstacle sensors 130 may also be located in other locations. For example, one or more obstacle sensors 130 may be provided at any position on the side, front, or rear of the vehicle body 101. The obstacle sensors 130 may include, for example, a laser scanner or ultrasonic sonar. The obstacle sensors 130 are used to detect surrounding obstacles during autonomous driving and to stop or detour the work vehicle 100. A LiDAR sensor 140 may be used as one of the obstacle sensors 130.

[0032] The work vehicle 100 further includes a GNSS unit 110. The GNSS unit 110 includes a GNSS receiver. The GNSS receiver may include an antenna that receives signals from GNSS satellites and a processor that calculates the position of the work vehicle 100 based on the signals received by the antenna. The GNSS unit 110 receives satellite signals transmitted from multiple GNSS satellites and performs positioning based on the satellite signals. GNSS is a general term for satellite positioning systems such as GPS (Global Positioning System), QZSS (Quasi-Zenith Satellite System, e.g., Michibiki), GLONASS, Galileo, and BeiDou. In this embodiment, the GNSS unit 110 is provided on top of the cabin 105, but may be provided in another location.

[0033] The GNSS unit 110 may include an inertial measurement unit (IMU). Signals from the IMU can be used to supplement position data. The IMU can measure the tilt and minute movements of the work vehicle 100. By using data acquired by the IMU to supplement position data based on satellite signals, positioning performance can be improved.

[0034] The control device of the work vehicle 100 may use, for positioning, sensing data acquired by sensing devices such as the camera 120 and / or the LiDAR sensor 140, in addition to the positioning results from the GNSS unit 110. If there are features that function as characteristic points in the environment in which the work vehicle 100 travels, such as farm roads, forest roads, public roads, or orchards, the position and orientation of the work vehicle 100 can be estimated with high accuracy based on the data acquired by the camera 120 and / or the LiDAR sensor 140 and an environmental map that has been stored in advance in a storage device. By using the data acquired by the camera 120 and / or the LiDAR sensor 140 to correct or complement position data based on satellite signals, the position of the work vehicle 100 can be identified with higher accuracy.

[0035] The prime mover 102 may be, for example, a diesel engine. An electric motor may be used instead of a diesel engine. The transmission 103 can change the propulsive force and travel speed of the work vehicle 100 by changing gears. The transmission 103 can also switch the work vehicle 100 between forward and reverse travel.

[0036] The steering device 106 includes a steering wheel, a steering shaft connected to the steering wheel, and a power steering device that assists steering by the steering wheel. The front wheels 104F are steerable wheels, and the traveling direction of the work vehicle 100 can be changed by changing the turning angle (also referred to as the "steering angle"). The steering angle of the front wheels 104F can be changed by operating the steering wheel. The power steering device includes a hydraulic device or an electric motor that supplies an assisting force for changing the steering angle of the front wheels 104F. When automatic steering is performed, the steering angle is automatically adjusted by the force of the hydraulic device or electric motor under control of a control device arranged in the work vehicle 100.

[0037] A coupling device 108 is provided at the rear of the vehicle body 101. The coupling device 108 includes, for example, a three-point support device (also referred to as a "three-point link" or "three-point hitch"), a PTO (Power Take Off) axle, a universal joint, and a communication cable. The coupling device 108 allows the work implement 300 to be attached to and detached from the work vehicle 100. The coupling device 108 can raise and lower the three-point link using, for example, a hydraulic device, thereby changing the position or attitude of the work implement 300. Power can also be transmitted from the work vehicle 100 to the work implement 300 via the universal joint. The work vehicle 100 can cause the work implement 300 to perform a predetermined task while towing the work implement 300. The coupling device may be provided at the front of the vehicle body 101. In this case, the work implement 300 can be connected to the front of the work vehicle 100.

[0038] 2 is a rotary tiller, but the work machine 300 is not limited to a rotary tiller. For example, any work machine such as a seeder (seed sowing machine), a spreader (fertilizer applicator), a transplanter, a mower (grass cutting machine), a rake, a baler (grass collector), a harvester (harvesting machine), a sprayer, or a harrow can be connected to the work vehicle 100 and used.

[0039] 2 is capable of being driven by a driver, but may also be capable of being driven only unmanned. In that case, components required only for driven operation, such as the cabin 105, steering device 106, and driver's seat 107, may not be provided in the work vehicle 100. The unmanned work vehicle 100 can travel autonomously or by remote control by a user.

[0040] 3 is a block diagram showing an example configuration of the work vehicle 100 and the work implement 300. The work vehicle 100 and the work implement 300 can communicate with each other via a communication cable included in the coupling device 108. The work vehicle 100 can communicate with the terminal device 400 and the management device 600 via the network 80.

[0041] In the example of FIG. 3 , the work vehicle 100 includes a GNSS unit 110, a camera 120, an obstacle sensor 130, a LiDAR sensor 140, and an operation terminal 200, as well as a group of sensors 150 that detect the operating state of the work vehicle 100, a control system 160, a communication device 190, a group of operation switches 210, a buzzer 220, and a drive unit 240. These components are communicatively connected to each other via a bus. The GNSS unit 110 includes a GNSS receiver 111, an RTK receiver 112, an inertial measurement unit (IMU) 115, and a processing circuit 116. The group of sensors 150 includes a steering wheel sensor 152, a turning angle sensor 154, and an axle sensor 156. The control system 160 includes a processing device 161, a storage device 170, and a control device 180. The control device 180 includes multiple electronic control units (ECUs) 181 to 185. The work machine 300 includes a drive unit 340, a control unit 380, and a communication unit 390. Note that Fig. 3 shows components that are relatively highly related to the operation of the autonomous driving by the work vehicle 100, and does not show other components.

[0042] The GNSS receiver 111 in the GNSS unit 110 receives satellite signals transmitted from multiple GNSS satellites and generates GNSS data based on the satellite signals. The GNSS data is generated in a predetermined format, such as the NMEA-0183 format. The GNSS data may include, for example, values ​​indicating the identification number, elevation angle, azimuth angle, and reception strength of each satellite from which the satellite signal is received.

[0043] The GNSS unit 110 shown in FIG. 3 performs positioning of the work vehicle 100 using RTK (Real Time Kinematic)-GNSS. FIG. 4 is a conceptual diagram showing an example of a work vehicle 100 performing positioning using RTK-GNSS. Positioning using RTK-GNSS uses correction signals transmitted from a reference station 60 in addition to satellite signals transmitted from multiple GNSS satellites 50. The reference station 60 can be installed near the field where the work vehicle 100 will be traveling (for example, within 10 km of the work vehicle 100). The reference station 60 generates correction signals, for example in RTCM format, based on the satellite signals received from the multiple GNSS satellites 50 and transmits them to the GNSS unit 110. The RTK receiver 112 includes an antenna and a modem, and receives the correction signals transmitted from the reference station 60. The processing circuit 116 of the GNSS unit 110 corrects the positioning results obtained by the GNSS receiver 111 based on the correction signal. By using RTK-GNSS, it is possible to perform positioning with an accuracy of, for example, a few centimeters. Position data including latitude, longitude, and altitude information is obtained through highly accurate positioning using RTK-GNSS. The GNSS unit 110 calculates the position of the work vehicle 100, for example, at a frequency of approximately 1 to 10 times per second.

[0044] The positioning method is not limited to RTK-GNSS, and any positioning method (such as interferometric positioning or relative positioning) that can obtain position data with the required accuracy can be used. For example, positioning may be performed using a virtual reference station (VRS) or a differential global positioning system (DGPS). If position data with the required accuracy can be obtained without using a correction signal transmitted from the reference station 60, the position data may be generated without using a correction signal. In this case, the GNSS unit 110 does not need to be equipped with an RTK receiver 112.

[0045] Even when RTK-GNSS is used, in locations where correction signals from the reference station 60 cannot be obtained (for example, on a road far from a field), the position of the work vehicle 100 is estimated by other methods without relying on signals from the RTK receiver 112. For example, the position of the work vehicle 100 can be estimated by matching data output from the LiDAR sensor 140 and / or camera 120 with a highly accurate environmental map.

[0046] The GNSS unit 110 in this embodiment further includes an IMU 115. The IMU 115 may include a three-axis acceleration sensor and a three-axis gyroscope. The IMU 115 may also include a direction sensor such as a three-axis geomagnetic sensor. The IMU 115 functions as a motion sensor and can output signals indicating various quantities such as the acceleration, velocity, displacement, and attitude of the work vehicle 100. The processing circuit 116 can estimate the position and orientation of the work vehicle 100 with higher accuracy based on the signals output from the IMU 115 in addition to the satellite signals and correction signals. The signals output from the IMU 115 can be used to correct or complement the position calculated based on the satellite signals and correction signals. The IMU 115 outputs signals at a higher frequency than the GNSS receiver 111. Using these high-frequency signals, the processing circuit 116 can measure the position and orientation of the work vehicle 100 at a higher frequency (e.g., 10 Hz or higher). A three-axis acceleration sensor and a three-axis gyroscope may be separately provided instead of the IMU 115. The IMU 115 may be provided as a device separate from the GNSS unit 110.

[0047] The camera 120 is an imaging device that captures images of the environment surrounding the work vehicle 100. The camera 120 includes an image sensor, such as a charge-coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). The camera 120 may also include an optical system including one or more lenses and a signal processing circuit. The camera 120 captures images of the environment surrounding the work vehicle 100 while the work vehicle 100 is traveling and generates image (e.g., video) data. The camera 120 can capture video at a frame rate of, for example, 3 frames per second (fps) or higher. The images generated by the camera 120 can be used, for example, when a remote monitor uses the terminal device 400 to check the environment surrounding the work vehicle 100. The images generated by the camera 120 may be used for positioning or obstacle detection. As shown in FIG. 2 , multiple cameras 120 may be installed at different positions on the work vehicle 100, or a single camera may be installed. A visible light camera that generates a visible light image and an infrared camera that generates an infrared image may be provided separately. Both a visible light camera and an infrared camera may be provided as cameras that generate images for surveillance. The infrared camera can also be used to detect obstacles at night.

[0048] The obstacle sensor 130 detects objects present in the vicinity of the work vehicle 100. The obstacle sensor 130 may include, for example, a laser scanner or an ultrasonic sonar. The obstacle sensor 130 outputs a signal indicating the presence of an obstacle when an object is present closer than a predetermined distance from the obstacle sensor 130. Multiple obstacle sensors 130 may be provided at different positions on the work vehicle 100. For example, multiple laser scanners and multiple ultrasonic sonars may be arranged at different positions on the work vehicle 100. By providing such a large number of obstacle sensors 130, blind spots in monitoring obstacles around the work vehicle 100 can be reduced.

[0049] The steering wheel sensor 152 measures the rotation angle of the steering wheel of the work vehicle 100. The turning angle sensor 154 measures the turning angle of the front wheels 104F, which are the steered wheels. The measurement values ​​from the steering wheel sensor 152 and the turning angle sensor 154 are used for steering control by the control device 180.

[0050] The axle sensor 156 measures the rotational speed of the axle connected to the wheel 104, i.e., the number of rotations per unit time. The axle sensor 156 may be a sensor that uses, for example, a magnetoresistive element (MR), a Hall element, or an electromagnetic pickup. The axle sensor 156 outputs a numerical value that indicates, for example, the number of rotations per minute (unit: rpm) of the axle. The axle sensor 156 is used to measure the speed of the work vehicle 100.

[0051] The drive device 240 includes various devices necessary for the travel of the work vehicle 100 and the drive of the work implement 300, such as the prime mover 102, transmission 103, steering device 106, and coupling device 108 described above. The prime mover 102 may be equipped with an internal combustion engine such as a diesel engine. The drive device 240 may be equipped with an electric motor for traction instead of or in addition to the internal combustion engine.

[0052] The buzzer 220 is an audio output device that emits a warning sound to notify of an abnormality. For example, the buzzer 220 emits the warning sound when an obstacle is detected during autonomous driving. The buzzer 220 is controlled by the control device 180.

[0053] The processing device 161 is, for example, a microprocessor or a microcontroller. The processing device 161 processes sensing data output from sensing devices such as the camera 120, the obstacle sensor 130, and the LiDAR sensor 140. For example, the processing device 161 detects objects located around the work vehicle 100 based on the data output from the camera 120, the obstacle sensor 130, and the LiDAR sensor 140.

[0054] The storage device 170 includes one or more storage media, such as a flash memory or a magnetic disk. The storage device 170 stores various data generated by the GNSS unit 110, the camera 120, the obstacle sensor 130, the LiDAR sensor 140, the sensor group 150, and the control device 180. The data stored in the storage device 170 may include map data (environmental map) of the environment in which the work vehicle 100 travels and target route data for autonomous driving. The environmental map includes information on multiple fields in which the work vehicle 100 will perform agricultural work and the roads in their surrounding areas. The environmental map and target route may be generated by a processor in the management device 600. The control device 180 may also have a function for generating or editing the environmental map and target route. The control device 180 can edit the environmental map and target route obtained from the management device 600 according to the travel environment of the work vehicle 100. The storage device 170 also stores work plan data received by the communication device 190 from the management device 600.

[0055] The storage device 170 also stores computer programs that cause the processing device 161 and each ECU in the control device 180 to execute various operations, which will be described later. Such computer programs may be provided to the work vehicle 100 via a storage medium (e.g., a semiconductor memory or an optical disk) or an electric communication line (e.g., the Internet). Such computer programs may also be sold as commercial software.

[0056] The control device 180 includes a plurality of ECUs, such as an ECU 181 for speed control, an ECU 182 for steering control, an ECU 183 for work machine control, an ECU 184 for automatic driving control, and an ECU 185 for path generation.

[0057] The ECU 181 controls the speed of the work vehicle 100 by controlling the prime mover 102 , the transmission 103 , and the brakes included in the drive device 240 .

[0058] The ECU 182 controls the steering of the work vehicle 100 by controlling the hydraulic device or electric motor included in the steering device 106 based on the measurement value of the steering wheel sensor 152 .

[0059] The ECU 183 controls the operation of the three-point link and the PTO shaft included in the coupling device 108, etc., in order to cause the work machine 300 to perform a desired operation. The ECU 183 also generates signals that control the operation of the work machine 300, and transmits these signals from the communication device 190 to the work machine 300.

[0060] The ECU 184 performs calculations and control to achieve autonomous driving based on data output from the GNSS unit 110, the camera 120, the obstacle sensor 130, the LiDAR sensor 140, the sensor group 150, and the processing device 161. For example, the ECU 184 identifies the position of the work vehicle 100 based on data output from at least one of the GNSS unit 110, the camera 120, and the LiDAR sensor 140. Within a farm field, the ECU 184 may determine the position of the work vehicle 100 based solely on data output from the GNSS unit 110. The ECU 184 may estimate or correct the position of the work vehicle 100 based on data acquired by the camera 120 and / or the LiDAR sensor 140. By utilizing the data acquired by the camera 120 and / or the LiDAR sensor 140, the accuracy of positioning can be further improved. Furthermore, outside the field, ECU 184 estimates the position of work vehicle 100 using data output from LiDAR sensor 140 and / or camera 120. For example, ECU 184 may estimate the position of work vehicle 100 by matching the data output from LiDAR sensor 140 and / or camera 120 with an environmental map. During autonomous driving, ECU 184 performs calculations necessary for work vehicle 100 to travel along a target route based on the estimated position of work vehicle 100. ECU 184 sends a speed change command to ECU 181 and a steering angle change command to ECU 182. In response to the speed change command, ECU 181 changes the speed of work vehicle 100 by controlling prime mover 102, transmission 103, or brakes. In response to the steering angle change command, ECU 182 changes the steering angle by controlling steering device 106.

[0061] ECU 185 can determine the destination of work vehicle 100 based on the work plan stored in storage device 170, and can determine a target route from the start point to the destination point of work vehicle 100. ECU 185 can perform processing to detect objects located around work vehicle 100 based on data output from camera 120, obstacle sensor 130, and LiDAR sensor 140.

[0062] Through the actions of these ECUs, control device 180 realizes autonomous driving. During autonomous driving, control device 180 controls drive device 240 based on the measured or estimated position of work vehicle 100 and the target route. In this way, control device 180 can cause work vehicle 100 to travel along the target route.

[0063] The multiple ECUs included in the control device 180 can communicate with each other in accordance with a vehicle bus standard such as CAN (Controller Area Network). Instead of CAN, a faster communication method such as Automotive Ethernet (registered trademark) may be used. In FIG. 3 , each of the ECUs 181 to 185 is shown as an individual block, but the functions of each of these may be realized by multiple ECUs. An on-board computer that integrates at least some of the functions of the ECUs 181 to 185 may be provided. The control device 180 may include ECUs other than the ECUs 181 to 185, and any number of ECUs may be provided depending on the functions. Each ECU includes a processing circuit including one or more processors. The control device 180 may include a processing device 161. The processing device 161 may be integrated with one of the ECUs included in the control device 180.

[0064] The communication device 190 includes circuits for communicating with the work machine 300, the terminal device 400, and the management device 600. The communication device 190 includes circuits for transmitting and receiving signals compliant with ISOBUS standards, such as ISOBUS-TIM, between the communication device 390 of the work machine 300. This allows the work machine 300 to perform desired operations and acquire information from the work machine 300. The communication device 190 may further include an antenna and communication circuits for transmitting and receiving signals via the network 80 between the communication devices of the terminal device 400 and the management device 600. The network 80 may include, for example, a cellular mobile communication network such as 3G, 4G, or 5G, and the Internet. The communication device 190 may also have a function for communicating with a mobile device used by a supervisor near the work vehicle 100. Communication between such mobile terminals can be performed in accordance with any wireless communication standard, such as Wi-Fi (registered trademark), cellular mobile communication such as 3G, 4G or 5G, or Bluetooth (registered trademark).

[0065] The operation terminal 200 is a terminal through which a user performs operations related to the travel of the work vehicle 100 and the operation of the work implement 300, and is also referred to as a virtual terminal (VT). The operation terminal 200 may include a display device such as a touch screen and / or one or more buttons. The display device may be, for example, a liquid crystal display or an organic light-emitting diode (OLED) display. By operating the operation terminal 200, a user can perform various operations, such as switching the autonomous driving mode on / off, recording or editing an environmental map, setting a target route, and switching the work implement 300 on / off. At least some of these operations can also be achieved by operating the operation switch group 210. The operation terminal 200 may be configured to be detachable from the work vehicle 100. A user located remote from the work vehicle 100 may operate the detached operation terminal 200 to control the operation of the work vehicle 100. Instead of the operation terminal 200, the user may control the operation of the work vehicle 100 by operating a computer, such as a terminal device 400, on which necessary application software is installed.

[0066] 5 is a diagram showing an example of an operation terminal 200 and an operation switch group 210 provided inside the cabin 105. An operation switch group 210 including a plurality of switches that can be operated by the user is arranged inside the cabin 105. The operation switch group 210 may include, for example, a switch for selecting a gear position of the main transmission or the auxiliary transmission, a switch for switching between automatic driving mode and manual driving mode, a switch for switching between forward and reverse, and a switch for raising and lowering the work implement 300. Note that if the work vehicle 100 only performs unmanned operation and does not have a function for manned operation, the work vehicle 100 does not need to be equipped with the operation switch group 210.

[0067] The drive unit 340 in the work machine 300 shown in Figure 3 performs the operations required for the work machine 300 to perform a predetermined task. The drive unit 340 includes devices appropriate for the intended use of the work machine 300, such as a hydraulic system, an electric motor, or a pump. The control device 380 controls the operation of the drive unit 340. The control device 380 causes the drive unit 340 to perform various operations in response to signals transmitted from the work vehicle 100 via the communication device 390. In addition, a signal appropriate for the state of the work machine 300 can also be transmitted from the communication device 390 to the work vehicle 100.

[0068] Next, the configurations of the management device 600 and the terminal device 400 will be described with reference to Fig. 6. Fig. 6 is a block diagram illustrating a schematic hardware configuration of the management device 600 and the terminal device 400.

[0069] The management device 600 includes a storage device 650, a processor 660, a read-only memory (ROM) 670, a random access memory (RAM) 680, and a communication device 690. These components are communicatively connected to each other via a bus. The management device 600 manages the schedule of agricultural work performed in the field by the work vehicle 100 and can function as a cloud server that supports agriculture by utilizing the managed data. A user can input information necessary for creating a work plan using the terminal device 400 and upload that information to the management device 600 via the network 80. The management device 600 can create a schedule of agricultural work, i.e., a work plan, based on that information. The management device 600 can also generate or edit an environmental map. The environmental map may be distributed from a computer external to the management device 600.

[0070] The communication device 690 is a communication module for communicating with the work vehicle 100 and the terminal device 400 via the network 80. The communication device 690 can perform wired communication in accordance with communication standards such as IEEE 1394 (registered trademark) or Ethernet (registered trademark). The communication device 690 may also perform wireless communication in accordance with the Bluetooth (registered trademark) standard or the Wi-Fi standard, or cellular mobile communication such as 3G, 4G, or 5G.

[0071] The processor 660 may be, for example, a semiconductor integrated circuit including a central processing unit (CPU). The processor 660 may be implemented by a microprocessor or a microcontroller. Alternatively, the processor 660 may be implemented by a field programmable gate array (FPGA) equipped with a CPU, a graphics processing unit (GPU), an application specific integrated circuit (ASIC), an application specific standard product (ASSP), or a combination of two or more circuits selected from these circuits. The processor 660 sequentially executes a computer program stored in the ROM 670, which describes a set of instructions for executing at least one process, to achieve the desired process.

[0072] The ROM 670 is, for example, a writable memory (e.g., a PROM), a rewritable memory (e.g., a flash memory), or a read-only memory. The ROM 670 stores a program that controls the operation of the processor 660. The ROM 670 does not have to be a single storage medium, but may be a collection of multiple storage media. Part of the collection of multiple storage media may be removable memory.

[0073] The RAM 680 provides a working area for temporarily loading the control program stored in the ROM 670 at boot time. The RAM 680 does not have to be a single storage medium, but may be a collection of multiple storage media.

[0074] The storage device 650 mainly functions as database storage. The storage device 650 may be, for example, a magnetic storage device or a semiconductor storage device. An example of a magnetic storage device is a hard disk drive (HDD). An example of a semiconductor storage device is a solid state drive (SSD). The storage device 650 may be a device independent of the management device 600. For example, the storage device 650 may be a storage device connected to the management device 600 via the network 80, such as cloud storage.

[0075] The terminal device 400 includes an input device 420, a display device 430, a storage device 450, a processor 460, a ROM 470, a RAM 480, and a communication device 490. These components are communicatively connected to one another via a bus. The input device 420 is a device for converting user instructions into data and inputting the data to a computer. The input device 420 may be, for example, a keyboard, a mouse, or a touch panel. The display device 430 may be, for example, a liquid crystal display or an organic EL display. The processor 460, ROM 470, RAM 480, storage device 450, and communication device 490 are described in the hardware configuration example of the management device 600, and therefore their description will be omitted.

[0076] 2. Operation Next, the operations of work vehicle 100, terminal device 400, and management device 600 will be described.

[0077] [2-1. Autonomous Driving Operation] First, an example of the operation of the automatic driving by the work vehicle 100 will be described. The work vehicle 100 in this embodiment can travel automatically both inside and outside the field. In the field, the work vehicle 100 drives the work implement 300 to perform predetermined agricultural work while traveling along a predetermined target route. If the work vehicle 100 detects an obstacle while traveling in the field, it stops traveling, issues a warning sound from the buzzer 220, and transmits a warning signal to the terminal device 400. In the field, the positioning of the work vehicle 100 is performed mainly based on data output from the GNSS unit 110. On the other hand, outside the field, the work vehicle 100 travels automatically along a target route set on a farm road or public road outside the field. When work vehicle 100 is traveling outside the field, it travels using data acquired by camera 120 and / or LiDAR sensor 140. When work vehicle 100 detects an obstacle outside the field, it either avoids the obstacle or stops on the spot. Outside the field, the position of work vehicle 100 is estimated based on the data output from LiDAR sensor 140 and / or camera 120 in addition to the positioning data output from GNSS unit 110.

[0078] An example of the operation of the work vehicle 100 when it travels automatically within a farm field will now be described.

[0079] FIG. 7 is a schematic diagram illustrating an example of a work vehicle 100 that automatically travels through a field along a target route. In this example, the field 70 includes a work area 72 where the work vehicle 100 performs work using the work implement 300 and a headland 74 located near the outer periphery of the field 70. The user can set in advance which areas of the field 70 on the map correspond to the work area 72 or the headland 74. The target route in this example includes multiple parallel main routes P1 and multiple turning routes P2 connecting the multiple main routes P1. The main routes P1 are located within the work area 72, and the turning routes P2 are located within the headland 74. Although each main route P1 shown in FIG. 7 is a straight route, each main route P1 may also include a curved portion. The dashed line in FIG. 7 represents the working width of the work implement 300. The working width is set in advance and recorded in the storage device 170. The working width may be set and recorded by the user operating the operation terminal 200 or the terminal device 400. Alternatively, the working width may be automatically recognized and recorded when the work implement 300 is connected to the work vehicle 100. The spacing between the multiple main paths P1 may be set to match the working width. The target route may be created based on user operation before the start of automated driving. The target route may be created to cover the entire working area 72 within the field 70, for example. The work vehicle 100 automatically travels back and forth from the start point of the work to the end point of the work, along the target route as shown in FIG. 7. Note that the target route shown in FIG. 7 is merely an example, and the target route may be defined in any manner.

[0080] Next, an example of control by the control device 180 during automatic operation in a farm field will be described.

[0081] FIG. 8 is a flowchart showing an example of the operation of steering control during automatic driving executed by the control device 180. The control device 180 performs automatic steering by executing the operations of steps S121 to S125 shown in FIG. 8 while the work vehicle 100 is traveling. The speed is maintained at, for example, a preset speed. While the work vehicle 100 is traveling, the control device 180 acquires data indicating the position of the work vehicle 100 generated by the GNSS unit 110 (step S121). Next, the control device 180 calculates the deviation between the position of the work vehicle 100 and the target route (step S122). The deviation represents the distance between the position of the work vehicle 100 at that time and the target route. The control device 180 determines whether the calculated position deviation exceeds a preset threshold (step S123). If the deviation exceeds the threshold, the control device 180 changes the steering angle by changing the control parameters of the steering device included in the drive device 240 so as to reduce the deviation. If the deviation does not exceed the threshold value in step S123, the operation of step S124 is skipped. In the following step S125, the control device 180 determines whether or not a command to end the operation has been received. A command to end the operation may be issued, for example, when a user remotely instructs the work vehicle 100 to stop autonomous driving, or when the work vehicle 100 reaches its destination. If a command to end the operation has not been issued, the process returns to step S121, and the same operation is performed based on the newly measured position of the work vehicle 100. The control device 180 repeats the operations of steps S121 to S125 until a command to end the operation is issued. The above operations are performed by the ECUs 182, 184 in the control device 180.

[0082] 8, the control device 180 controls the drive device 240 based only on the deviation between the position of the work vehicle 100 identified by the GNSS unit 110 and the target route, but the control may also take into consideration the deviation in heading. For example, when the heading deviation, which is the angular difference between the orientation of the work vehicle 100 identified by the GNSS unit 110 and the direction of the target route, exceeds a preset threshold, the control device 180 may change the control parameters (e.g., steering angle) of the steering device of the drive device 240 in accordance with the deviation.

[0083] An example of steering control by the control device 180 will be described in more detail below with reference to FIGS. 9A to 9D.

[0084] FIG. 9A is a diagram showing an example of a work vehicle 100 traveling along a target route P. FIG. 9B is a diagram showing an example of a work vehicle 100 shifted to the right from the target route P. FIG. 9C is a diagram showing an example of a work vehicle 100 shifted to the left from the target route P. FIG. 9D is a diagram showing an example of a work vehicle 100 facing in an inclined direction with respect to the target route P. In these figures, the pose indicating the position and orientation of the work vehicle 100 measured by the GNSS unit 110 is expressed as r(x, y, θ). (x, y) are coordinates representing the position of the reference point of the work vehicle 100 in the XY coordinate system, which is a two-dimensional coordinate system fixed to the Earth. In the examples shown in FIGS. 9A to 9D , the reference point of the work vehicle 100 is located at the position where the GNSS antenna on the cabin is installed, but the position of the reference point is arbitrary. θ is an angle representing the measured orientation of the work vehicle 100. In the illustrated example, the target path P is parallel to the Y axis, but in general, the target path P is not necessarily parallel to the Y axis.

[0085] As shown in FIG. 9A, if the position and orientation of the work vehicle 100 do not deviate from the target route P, the control device 180 maintains the steering angle and speed of the work vehicle 100 without changing them.

[0086] 9B , when the position of work vehicle 100 has shifted to the right from target route P, control device 180 changes the steering angle so that the traveling direction of work vehicle 100 tilts to the left and approaches route P. At this time, the speed may also be changed in addition to the steering angle. The magnitude of the steering angle can be adjusted, for example, according to the magnitude of position deviation Δx.

[0087] 9C , when the position of work vehicle 100 has shifted to the left from target route P, control device 180 changes the steering angle so that the traveling direction of work vehicle 100 tilts to the right and approaches route P. In this case, too, the speed may be changed in addition to the steering angle. The amount of change in the steering angle may be adjusted, for example, according to the magnitude of position deviation Δx.

[0088] As shown in FIG. 9D , when the position of the work vehicle 100 is not significantly deviated from the target route P but the heading is different from the direction of the target route P, the control device 180 changes the steering angle to reduce the azimuth deviation Δθ. In this case, the speed may also be changed in addition to the steering angle. The magnitude of the steering angle may be adjusted, for example, according to the magnitudes of the position deviation Δx and the azimuth deviation Δθ. For example, the smaller the absolute value of the position deviation Δx, the greater the amount of change in the steering angle according to the azimuth deviation Δθ. When the absolute value of the position deviation Δx is large, the steering angle will be changed significantly to return to the route P, which inevitably increases the absolute value of the azimuth deviation Δθ. Conversely, when the absolute value of the position deviation Δx is small, it is necessary to bring the azimuth deviation Δθ closer to zero. For this reason, it is appropriate to relatively increase the weight of the azimuth deviation Δθ (i.e., the control gain) used to determine the steering angle.

[0089] Control techniques such as PID control or MPC control (model predictive control) can be applied to the steering control and speed control of the work vehicle 100. By applying these control techniques, it is possible to smooth the control that brings the work vehicle 100 closer to the target path P.

[0090] If an obstacle is detected by sensing devices such as camera 120, obstacle sensor 130, and LiDAR sensor 140 while the work vehicle 100 is traveling, control device 180 stops the work vehicle 100. At this time, the buzzer 220 may be caused to emit a warning sound or a warning signal may be sent to terminal device 400. If it is possible to avoid the obstacle, control device 180 may control drive device 240 to avoid the obstacle.

[0091] The work vehicle 100 in this embodiment is capable of autonomous driving not only within a field but also outside the field. Outside the field, the processing device 161 and / or the control device 180 can detect objects (e.g., other vehicles or pedestrians) present around the work vehicle 100 based on data output from sensing devices such as the camera 120, the obstacle sensor 130, and the LiDAR sensor 140. By using the camera 120 and the LiDAR sensor 140, it is possible to detect objects present at a relatively large distance from the work vehicle 100. The control device 180 can realize autonomous driving on roads outside the field by performing speed control and steering control so as to avoid detected objects.

[0092] In this way, the work vehicle 100 in this embodiment can travel autonomously within and outside a field without a driver. FIG. 10 is a diagram schematically illustrating an example of a situation in which multiple work vehicles 100 are traveling autonomously inside a field 70 and on a road 76 outside the field 70. An environmental map and a target route of an area including multiple fields 70 and their surrounding roads are recorded in the storage device 170. The environmental map and the target route can be generated by the management device 600 or the ECU 185. When the work vehicle 100 travels on a road, the work vehicle 100 travels along the target route with the work implement 300 raised, while sensing the surroundings using sensing devices such as the camera 120, obstacle sensor 130, and LiDAR sensor 140.

[0093] 2-2. Setting of search area according to area where agricultural machine is located Next, a process of setting a search area according to area where agricultural machine is located will be described.

[0094] As described above, sensing devices such as camera 120, obstacle sensor 130, and LiDAR sensor 140 sense the environment around work vehicle 100 and output sensing data. Processing device 161 ( FIG. 3 ) detects objects located in a search area around work vehicle 100 based on the sensing data. The search area is an area within the area around work vehicle 100 sensed by the sensing device that is searched for objects. The search area may be the same size as the sensing area sensed by the sensing device, or may be smaller than the sensing area. The search area may also be referred to as a region of interest (ROI).

[0095] In the embodiment exemplified below, the pattern of the search area in the process of detecting an object using sensing data output by the LiDAR sensor 140 is changed according to the area in which the work vehicle 100 is located. Changing the pattern of the search area means, for example, changing at least one of the shape, size, and relative position of the search area with respect to the work vehicle 100.

[0096] The work vehicle 100 of this embodiment is equipped with a sensing system 10 ( FIG. 3 ) that uses sensing data output by a LiDAR sensor 140 to detect objects located around the work vehicle 100. The sensing system 10 includes a processing device 161 and a LiDAR sensor 140. When the area in which the work vehicle 100 is located is detected using position data and map data generated by the GNSS unit 110, the sensing system 10 may include the GNSS unit 110 and a storage device 170. When the area in which the work vehicle 100 is located is estimated by matching the data output from the camera 120 with an environmental map, the sensing system 10 may include the camera 120 and a storage device 170.

[0097] The LiDAR sensor 140 emits pulses of a laser beam (hereinafter referred to as "laser pulses") one after another while changing the emission direction, and can measure the distance to each reflection point from the time difference between the emission time and the time when the reflected light of each laser pulse is acquired. The "reflection points" may be objects located in the environment surrounding the work vehicle 100.

[0098] The LiDAR sensor 140 can measure the distance from the LiDAR sensor 140 to an object using any method. Measurement methods for the LiDAR sensor 140 include, for example, mechanical rotation, MEMS, and phased array methods. These measurement methods differ in the way they emit laser pulses (scanning methods). For example, a mechanical rotation LiDAR sensor rotates a cylindrical head that emits laser pulses and detects the reflected light of the laser pulses to scan the surrounding environment in all directions 360 degrees around the rotation axis. A MEMS LiDAR sensor uses a MEMS mirror to oscillate the emission direction of the laser pulses and scan the surrounding environment within a predetermined angular range centered on the oscillation axis. A phased array LiDAR sensor controls the phase of light to oscillate the emission direction of light and scan the surrounding environment within a predetermined angular range centered on the oscillation axis.

[0099] FIG. 11 is a flowchart showing an example of a process for changing the pattern of a search region depending on the area where the agricultural machine is located.

[0100] Similar to the processing of step S121 (FIG. 8) described above, the control device 180 (FIG. 3) acquires position data indicating the position of the work vehicle 100 generated by the GNSS unit 110 while the work vehicle 100 is traveling (step S201). The position data includes information on the geographic coordinates of the position of the work vehicle 100. The storage device 170 stores map data of the area in which the work vehicle 100 travels. The map data includes information on the geographic coordinates of the area indicated by the map.

[0101] The processing device 161 uses the map data to determine the area corresponding to the geographic coordinates indicated by the position data (step S202). The area corresponding to the geographic coordinates indicated by the position data corresponds to the area where the work vehicle 100 is located. The processing device 161 determines whether the area corresponding to the geographic coordinates indicated by the position data is a predetermined area (step S203). The predetermined area is registered in advance in the map data.

[0102] If the area corresponding to the geographic coordinates indicated by the position data is not a predetermined area, the processing device 161 sets a first search area as the search area (step S205).If the area corresponding to the geographic coordinates indicated by the position data is a predetermined area, the processing device 161 sets a second search area as the search area (step S204).

[0103] Figure 12 is a diagram showing an example of an area for which the pattern of the search region is changed. In the example shown in Figure 12, the predetermined area is area 712 close to the outer periphery of the field 70. In Figure 12, area 712 is shown with diagonal hatching. In the example shown in Figure 12, a ridge 710 is formed on the outer periphery of the field 70, and in this case, area 712 may be a ridge-edge area. Note that the field on which the ridge is formed is not limited to a rice paddy.

[0104] 13 is a diagram showing examples of the first search area and the second search area. The first search area 810 is a search area that is set when the work vehicle 100 is not located in a predetermined area. The second search area 820 is a search area that is set when the work vehicle 100 is located within a predetermined area.

[0105] The first search area 810 includes a forward search area 810F, a rearward search area 810Re, a left lateral search area 810L, and a right lateral search area 810R. The second search area 820 includes a forward search area 820F, a rearward search area 820Re, a left lateral search area 820L, and a right lateral search area 820R. Figure 13 shows the search area in a planar view seen from the vertical direction when the work vehicle 100 is located on level ground. In this embodiment, the pattern of the search area in a planar view seen from the vertical direction is changed.

[0106] In this example, work vehicle 100 is provided with four LiDAR sensors 140F, 140Re, 140L, and 140R. LiDAR sensor 140F is disposed at the front of work vehicle 100 and mainly senses the surrounding environment extending in front of work vehicle 100. LiDAR sensor 140Re is disposed at the rear of work vehicle 100 and mainly senses the surrounding environment extending behind work vehicle 100. LiDAR sensor 140L is disposed at the left side of work vehicle 100 and mainly senses the surrounding environment extending to the left side of work vehicle 100. LiDAR sensor 140R is disposed at the right side of work vehicle 100 and mainly senses the surrounding environment extending to the right side of work vehicle 100. LiDAR sensors 140Re, 140L, and 140R may be provided, for example, in cabin 105 ( FIG. 2 ) of work vehicle 100. The LiDAR sensor 140Re may be provided on the implement 300.

[0107] 14 and 15 are diagrams showing the relationship between the sensing area sensed by the LiDAR sensor and the search area searched for an object. The shape, size, and position of the search area can be realized, for example, by changing the data portion used for object search among the 3D point cloud data output by the LiDAR sensor.

[0108] The 3D point cloud data output by the LiDAR sensor includes information about the positions of multiple points and information (attribute information) such as the reception intensity of the photodetector. The information about the positions of the multiple points is, for example, information about the emission direction of the laser pulse corresponding to the point and the distance between the LiDAR sensor and the point. Furthermore, for example, the information about the positions of the multiple points is information about the coordinates of the points in a local coordinate system. The local coordinate system is a coordinate system that moves along with the work vehicle 100 and is also referred to as a sensor coordinate system. The coordinates of each point can be calculated from the emission direction of the laser pulse corresponding to the point and the distance between the LiDAR sensor and the point.

[0109] For example, a search region can be set based on the coordinates of each point. By selecting points located within a desired shape in the local coordinate system as points to be used for searching the object, a search region of the desired shape can be set.

[0110] FIG. 14 shows a sensing area 830L sensed by LiDAR sensor 140L and a sensing area 830R sensed by LiDAR sensor 140R.

[0111] A search region 810L can be set by selecting points located within a predetermined shape in the local coordinate system from among the multiple points indicated by the 3D point cloud data output by the LiDAR sensor 140L, or a search region 820L can be set by selecting points located within another shape in the local coordinate system.

[0112] Search region 810R can be set by selecting points located within a predetermined shape in the local coordinate system from among the multiple points indicated by the 3D point cloud data output by LiDAR sensor 140R, and search region 820R can be set by selecting points located within another shape in the local coordinate system.

[0113] In the example shown in FIG. 14, the search regions 810L, 810R, 820L, and 820R are substantially rectangular in shape, but are not limited to this and may have other shapes.

[0114] Figure 15 shows sensing area 830F sensed by LiDAR sensor 140F and sensing area 830Re sensed by LiDAR sensor 140Re.

[0115] A search region 810F can be set by selecting points located within a predetermined shape in the local coordinate system from among the multiple points indicated by the 3D point cloud data output by LiDAR sensor 140F, or a search region 820F can be set by selecting points located within another shape in the local coordinate system.

[0116] The search region 810Re can be set by selecting points located within a predetermined shape in the local coordinate system from among the multiple points indicated by the 3D point cloud data output by the LiDAR sensor 140Re, and the search region 820Re can be set by selecting points located within another shape in the local coordinate system.

[0117] In the example shown in Figure 15, the shapes of the search regions 810F, 810Re, 820F, and 820Re are approximately sector-shaped, but are not limited to such shapes and may be other shapes. In the example shown in Figure 15, the search regions 810F and 820F may be the same as each other, but may also be different from each other. Similarly, in the example shown in Figure 15, the search regions 810Re and 820Re may be the same as each other, but may also be different from each other.

[0118] As shown in FIG. 13, the length L of the side search areas 820L and 820R in the front-rear direction is ２ is the length L of the side search areas 810L and 810R in the front-rear direction. １ It is desirable to be able to detect the conditions on the sides of work vehicle 100 early when traveling in ridge area 712 ( FIG. 12 ) within field 70. By making the length of side search areas 820L and 820R in the front-to-rear direction larger when traveling in ridge area 712 than when traveling in areas other than ridge area 712 (for example, areas relatively far from the outer periphery within field 70), it is possible to detect the conditions on the sides of work vehicle 100 early.

[0119] In addition, the points to be searched within the lateral search area may include points indicated by the three-dimensional point cloud data output by LiDAR sensors 140F and / or 140Re.

[0120] FIG. 16 is a diagram showing another example of an area where the pattern of the search area is changed. In the example shown in FIG. 16 , the predetermined area where the pattern of the search area is changed is an area 722 adjacent to a waterway 720 on a road 76 (a farm road or a public road) outside the field. In FIG. 16 , the area 722 is indicated by diagonal hatching. When traveling in the waterway-adjacent area 722, it is desirable to be able to detect the conditions on the sides of the work vehicle 100 early. When traveling in the waterway-adjacent area 722, the longitudinal lengths of the side search areas 820L and 820R can be made longer than when traveling on a road 76 other than the waterway-adjacent area 722, thereby enabling early detection of the conditions on the sides of the work vehicle 100.

[0121] 11 , the processing device 161 uses output data from the LiDAR sensor 140 corresponding to the set search area to detect objects around the work vehicle 100 (step S206). The processing device 161 repeats the operations of steps S201 to S206 until a command to end the operation is issued (step S207).

[0122] When an obstacle is detected in the process of detecting objects in the vicinity of the work vehicle 100, an operation to avoid the obstacle may be performed or the travel of the work vehicle 100 may be stopped. Figure 17 is a flowchart showing an example of the process when an obstacle is detected.

[0123] For example, when an object such as a human, animal, or vehicle is detected on a predetermined target route, the processing device 161 determines that an obstacle is present (step S301). For example, when an object that is not included in a pre-generated "environmental map" and is equal to or higher than a predetermined height is detected on the target route, the processing device 161 determines that an obstacle is present.

[0124] If an obstacle is detected, the ECU 185 determines whether a detour route that can avoid the obstacle can be generated (step S302). For example, if there is sufficient space on the road 76 to allow for a detour, it is determined that a detour route can be generated. Within the field 70, for example, if a detour route that does not affect agricultural work or crops can be generated, it is determined that a detour route can be generated. For example, if the agricultural work being performed is one for which generation of a detour route is prohibited in advance, or if it is determined that detouring would cause the work vehicle 100 to come into contact with crops, it is determined that a detour route cannot be generated. Furthermore, for example, if a detour route that does not enter an area of ​​the field 70 that has already been worked on can be generated, it is determined that a detour route can be generated.

[0125] If it is determined that a detour route can be generated, ECU 185 generates the detour route, and control device 180 controls work vehicle 100 to travel along the detour route (step S303). After traveling along the detour route, control device 180 returns work vehicle 100 to the target route, and returns to the processing of step S207 shown in FIG.

[0126] If it is determined that a detour route cannot be generated, the control device 180 performs control to stop the work vehicle 100 (step S304). In parallel, operations such as issuing a warning sound from the buzzer 220 and transmitting a warning signal to the terminal device 400 are performed.

[0127] If it is determined that the object detected as an obstacle has moved or the worker has removed the obstacle, the control device 180 causes the work vehicle 100 to resume traveling (steps S305 and S306) and returns to the processing of step S207 shown in Figure 11.

[0128] In the above description, the area in which the work vehicle 100 is located is detected using the position data generated by the GNSS unit 110, but this is not limiting. For example, the area in which the work vehicle 100 is located may be estimated by matching the data output from the LiDAR sensor 140 and / or the camera 120 with an environmental map.

[0129] Figure 18 is a diagram showing another example of the first search area 810 and the second search area 820. Compared to the search areas 810 and 820 shown in Figure 13, the search areas 810 and 820 shown in Figure 18 have a larger longitudinal length of the lateral search area relative to the forward search area and the rearward search area. By increasing the longitudinal length of the lateral search area, the conditions to the sides of the work vehicle 100 can be detected earlier.

[0130] 19 is a diagram showing yet another example of the first search region 810 and the second search region 820. In the example described above, the shapes of the lateral search regions 810L, 810R, 820L, and 820R were approximately rectangular, but they may have other shapes. For example, as shown in FIG. 19, the shapes of the lateral search regions 810L, 810R, 820L, and 820R may be approximately sector-shaped.

[0131] As described above, the LiDAR sensor can sense the surrounding environment within a predetermined angular range centered on the swing axis. When the search area is approximately fan-shaped, points to be used in the object search among the multiple points indicated by the 3D point cloud data output by the LiDAR sensor may be selected based on the emission angle of the corresponding laser pulse. The pattern of the search area can be changed by changing the angle range that serves as the basis for this selection. Furthermore, when the search area is approximately fan-shaped, points to be used in the object search among the multiple points indicated by the 3D point cloud data may be selected based on the distance between the LiDAR sensor and the points. The pattern of the search area can be changed by changing the magnitude of the distance that serves as the basis for this selection.

[0132] The pattern of the search area may also be changed by changing the range sensed by the LiDAR sensor. For example, the size of the search area may be changed by changing the output of the laser pulse emitted from the LiDAR sensor. For example, the angular range of the search area may be changed by changing the angular range over which the LiDAR sensor emits the laser pulse. For example, the angular range of the search area may be changed by changing the angular range over which the emission direction of the laser pulse is swung.

[0133] Fig. 20 is a diagram showing yet another example of the first search area 810 and the second search area 820. In the example shown in Fig. 20, compared to the lateral search areas 810L and 810R, the lateral search areas 820L and 820R have shapes that include areas closer to the front wheel 104F and the rear wheel 104R. The lateral search areas 820L and 820R are set to include areas particularly close to the front outer ends of the front wheel 104F.

[0134] Figure 21 is a diagram showing yet another example of an area where the search area pattern is changed. Figure 21 shows a bridge area 732 where the search area pattern is changed. The bridge area 732 can include an area on a bridge 730 spanning a river or waterway 734 and an area of ​​the road 76 near the bridge 730. When traveling in the bridge area 732, it is desirable to be able to detect the conditions near the wheels of the work vehicle 100, particularly near the front wheels. When traveling in the bridge area 732, a search appropriate for the bridge area 732 can be performed by setting the patterns of side search areas 820L, 820R shown in Figure 20.

[0135] Fig. 22 is a diagram showing yet another example of the first search region 810 and the second search region 820. In the example shown in Fig. 20, the shapes of the lateral search regions 810L, 810R, 820L, and 820R are approximately rectangular, but they may have other shapes. For example, as shown in Fig. 22, the shapes of the lateral search regions 810L, 810R, 820L, and 820R may be approximately sector-shaped.

[0136] Figure 21 also shows a barn entrance / exit area 742 where the pattern of the search area is changed. At the entrance / exit 741 of the barn 740 where the work vehicle 100 is stored, it is desirable to be able to detect the conditions on the sides of the work vehicle 100 early. When traveling through the entrance / exit area 742, the search area 820 shown in Figures 13, 18, and 19 is set. When traveling through the entrance / exit area 742, the length of the side search areas 820L and 820R in the front-to-rear direction is made longer than when traveling in areas other than the entrance / exit area 742, so that the conditions on the sides of the work vehicle 100 can be detected early.

[0137] Figure 23 is a diagram showing yet another example of an area where the pattern of the search area is changed. In the example shown in Figure 23, the predetermined area where the pattern of the search area is changed is an area 772 in a work site 78 where work vehicles 100 are loaded onto transporters 770. Area 772 may be located on a road 76.

[0138] Loading of the work vehicle 100 onto the transporter 770 is performed by having the work vehicle 100 travel along ladder rails 771 provided on the transporter 770. A loading operation area 772 includes the area near the transporter 770 and ladder rails 771. When loading the work vehicle 100, it is desirable to be able to detect the condition near the wheels of the work vehicle 100, particularly near the front wheels. When traveling in the bridge area 732, a search suitable for loading operations can be performed by setting the patterns of the side search areas 820L, 820R shown in Figures 20 and 22.

[0139] Figure 24 is a diagram showing yet another example of an area where the pattern of the search area is changed. Figure 25 is a diagram showing yet another example of a first search area 810 and a second search area 820. In the example shown in Figure 24 , the predetermined area where the pattern of the search area is changed is a ridge area 752 where ridges 750 are provided in field 70. In ridge area 752, it is desirable to be able to detect the condition of a wider area around work vehicle 100.

[0140] In the example shown in Figure 25, the size of search area 820 is larger than search area 810. In ridge area 752, search area 820 shown in Figure 25 is set. By making the size of search area 820 larger when traveling in ridge area 752 than when traveling in areas of field 70 other than ridge area 752, it is possible to detect the condition of a wider area around work vehicle 100.

[0141] Figure 26 is a diagram showing yet another example of an area in which the search area pattern is changed. In the example shown in Figure 26, the specified area in which the search area pattern is changed is a crop row area 762 in which a crop row 760 in the field 70 is located. In the crop row area 762, it is desirable to be able to detect the condition of a wider area around the work vehicle 100. In the crop row area 762, a search area 820 shown in Figure 25 is set. When traveling in the crop row area 762, the size of the search area 820 is made larger than when traveling in the field 70 other than the crop row area 762, making it possible to detect the condition of a wider area around the work vehicle 100.

[0142] 2-3. Setting of Search Area According to Implement Next, a process for setting a search area according to the implement 300 connected to the work vehicle 100 will be described.

[0143] 27 is a diagram showing an example of a search area 810 that is set according to the size of the implement 300 connected to the work vehicle 100. A plurality of types of implements 300 of different sizes can be connected to the work vehicle 100, and the processing device 161 changes the pattern of the search area 810 according to the size of the implement 300 connected to the work vehicle 100.

[0144] The storage device 170 ( FIG. 3 ) pre-stores information regarding the sizes of multiple types of implements 300. The processing device 161 communicates with the implement 300 and acquires unique information that can identify the model of the implement 300. The unique information includes, for example, the model number of the implement 300. The processing device 161 can determine the size of the implement 300 connected to the work vehicle 100 by reading, from the storage device 170, information regarding the size that corresponds to the unique information of the implement 300. Note that the size information of the implement 300 may also be input into the work vehicle 100 by the worker.

[0145] The size of the implement 300b connected to the work vehicle 100 on the right side in FIG. 27 is larger than the size of the implement 300a connected to the work vehicle 100 on the left side. For example, compared to the implement 300a, the implement 300b is larger in at least one of the front-to-rear and left-to-right directions. In the example shown in FIG. 27 , compared to the side search areas 810L and 810R shown on the left side, the side search areas 810L and 810R shown on the right side are larger in left-to-right dimensions. Also, compared to the rear search area 810Re shown on the left side, the rear search area 810Re shown on the right side is larger in both the front-to-rear and left-to-right dimensions. By changing the pattern of the search area 810 according to the size of the implement 300, a search appropriate for the size of the implement 300 can be performed.

[0146] Next, a description will be given of a process for changing the pattern of the search region 810 in accordance with a change in the positional relationship between the work vehicle 100 and the implement 300. Figure 28 is a diagram showing an example of a search region 810 that is set in accordance with the positional relationship between the work vehicle 100 and the implement 300.

[0147] 28 is an implement whose positional relationship with the work vehicle 100 can be changed. Such an implement may also be referred to as an offset-type implement. The processing device 161 can determine the position of the implement 300c based on, for example, a control signal output from the work vehicle 100 to the implement 300c.

[0148] In the example shown in Figure 28, compared to the implement 300c shown on the left, the implement 300c shown on the right is positioned further to the right of the work vehicle 100. In the example shown in Figure 28, compared to the side search region 810R shown on the left, the side search region 810R shown on the right is larger in size in the front-to-rear and left-to-right directions. Also, compared to the rear search region 810Re shown on the left, the rear search region 810Re shown on the right is larger in size in the front-to-rear and left-to-right directions. By changing the pattern of the search region 810 in response to changes in the positional relationship between the work vehicle 100 and the implement 300, a search appropriate to the position of the implement 300 can be performed.

[0149] Furthermore, compared to the implement 300a shown on the left side of Fig. 27, the implement 300c shown on the left side of Fig. 28 is positioned extending further to the right with respect to the work vehicle 100. By increasing the size in the front-to-rear and left-to-right directions of the side search area 810R shown on the left side of Fig. 28 compared to the side search area 810R shown on the left side of Fig. 27, it is possible to perform a search that is appropriate for the position of the implement 300.

[0150] In the above-described embodiment, the pattern of the search area in the process of detecting an object using sensing data output by a LiDAR sensor was changed. However, the pattern of the search area in the process of detecting an object using sensing data output by a sensor other than a LiDAR sensor (e.g., a camera, an ultrasonic sonar, etc.) may also be changed. For example, the pattern of the search area may be changed by changing the portion of the image data captured by the camera that is used to detect the object. Furthermore, for example, the pattern of the search area may be changed by changing the output of the ultrasonic sonar or changing the sensing angle range.

[0151] The sensing system 10 of the present embodiment can also be retrofitted to agricultural machinery that does not have these functions. Such a system can be manufactured and sold independently of the agricultural machinery. The computer program used in such a system can also be manufactured and sold independently of the agricultural machinery. The computer program can be provided, for example, by being stored on a computer-readable non-transitory storage medium. The computer program can also be provided by downloading via a telecommunications line (for example, the Internet).

[0152] A part or all of the processing executed by the processing device 161 in the sensing system 10 may be executed by another device. Such another device may be at least one of the processor 660 of the management device 600, the processor 460 of the terminal device 400, and the operation terminal 200. In this case, such another device and the processing device 161 function as the processing device of the sensing system 10, or such another device functions as the processing device of the sensing system 10. For example, when a part of the processing executed by the processing device 161 is executed by the processor 660 of the management device 600, the processing device 161 and the processor 660 function as the processing device of the sensing system 10.

[0153] A part or all of the processing performed by the processing device 161 may be performed by the control device 180. In that case, the control device 180 and the processing device 161 function as processing devices of the sensing system 10, or the control device 180 functions as processing devices of the sensing system 10.

[0154] As described above, the present disclosure includes the agricultural machine, sensing system used in the agricultural machine, and sensing method described below.

[0155] A sensing system 10 according to an embodiment of the present disclosure is a sensing system 10 for a mobile agricultural machine 100, and includes one or more sensors 140 that are provided on the agricultural machine 100 and sense the environment around the agricultural machine 100 and output sensing data, and a processing device 161 that detects objects located in search areas 810, 820 around the agricultural machine 100 based on the sensing data, and the processing device 161 changes the pattern of the search areas 810, 820 in which objects are detected depending on the area in which the agricultural machine 100 is located.

[0156] This allows a search suitable for the area in which the agricultural machine 100 is located to be performed.

[0157] In one embodiment, the processing device 161 may cause the patterns of the search areas 810, 820 to differ when the agricultural machine 100 is located in a first area within the field 70 and when the agricultural machine 100 is located in a second area within the field 70 that is closer to the outer edge of the field 70 than the first area.

[0158] This makes it possible to perform a search that is appropriate for when the agricultural machine 100 is located in an area close to the outer periphery of the field 70 and when it is located in an area far from the outer periphery of the field 70.

[0159] In one embodiment, the second area may be the edge of the field 712. This allows a range suitable for driving on the edge of the field 712 to be searched.

[0160] In one embodiment, the search areas 810, 820 include lateral search areas 810L, 810R, 820L, 820R that include areas on the sides of the agricultural machine 100, and the processing device 161 may increase the length of the lateral search areas in the front-to-rear direction when the agricultural machine 100 is located in the second area compared to when the agricultural machine 100 is located in the first area.

[0161] This allows the condition of the side of the agricultural machine 100 to be detected early.

[0162] In one embodiment, the processing device 161 may cause the patterns of the search areas 810 and 820 to be different when the agricultural machine 100 is located on the bridge 730 and when the agricultural machine 100 is located on a predetermined road 76 different from the bridge 730.

[0163] This allows a search that is appropriate for when the agricultural machine 100 is located on the bridge 730 and when it is located on a predetermined road 76 that is different from the bridge 730 to be performed.

[0164] In one embodiment, the agricultural machine 100 includes a work vehicle 100 provided with a front wheel 104F, and the processing device 161 sets the search area 820 to include an area closer to the front outer end of the front wheel 104F when the agricultural machine 100 is located on a bridge 730 than when the agricultural machine 100 is located on a specified road 76.

[0165] This allows a search for an area suitable for driving on the bridge 730.

[0166] In one embodiment, the processing device 161 may cause the patterns of the search areas 810, 820 to differ when the agricultural machine 100 is located on a road 76 adjacent to a waterway 720 and when the agricultural machine 100 is located on a road 76 that is not adjacent to a waterway 720.

[0167] This allows a search that is appropriate for when the agricultural machine 100 is located on a road 76 adjacent to a waterway 720 and when the agricultural machine 100 is located on a road 76 that is not adjacent to a waterway 720.

[0168] In one embodiment, the search areas 810, 820 include lateral search areas 810L, 810R, 820L, 820R that include areas to the sides of the agricultural machine 100, and the processing device 161 may increase the fore-and-aft length of the lateral search areas when the agricultural machine 100 is located on a road 76 adjacent to the waterway 720 compared to when the agricultural machine 100 is located on a road 76 that is not adjacent to the waterway 720.

[0169] This allows a search to be made for an area of ​​the road 76 adjacent to the waterway 720 that is suitable for travel.

[0170] In one embodiment, the processing device 161 may cause the patterns of the search areas 810, 820 to differ when the agricultural machine 100 is located at the entrance / exit 741 of the barn 740 and when the agricultural machine 100 is located in a third area different from the entrance / exit 741 of the barn 740.

[0171] This makes it possible to perform a search that is appropriate for both the time when the agricultural machine 100 is located at the entrance 741 of the barn 740 and the time when the agricultural machine 100 is not located at the entrance 741 of the barn 740 .

[0172] In one embodiment, the search areas 810, 820 include lateral search areas 810L, 810R, 820L, 820R that include areas on the sides of the agricultural machine 100, and the processing device 161 may increase the length of the lateral search areas in the front-to-rear direction when the agricultural machine 100 is located at the entrance / exit 741 of the barn 740 compared to when the agricultural machine 100 is located in the third area.

[0173] This allows searching for an area suitable for driving around the entrance 741 of the barn 740.

[0174] In one embodiment, the processing device 161 may cause the patterns of the search areas 810, 820 to be different when the agricultural machine 100 is located in a fourth area within the field 70 and when the agricultural machine 100 is located in a fifth area within the field 70 that is closer to the ridges 750 or the crop planting area 760 of the field 70 than the fourth area.

[0175] This allows for a search that is appropriate for when the agricultural machine 100 is located in an area close to the ridge 750 or the crop planting area 760 of the field 70 and when it is not located there.

[0176] In one embodiment, the processing device 161 may cause the patterns of the search areas 810 and 820 to differ depending on whether the agricultural machine 100 is in a position to be loaded onto the transporter 770 or not.

[0177] This makes it possible to perform a search that is appropriate for both when the agricultural machine 100 is in a position to be loaded onto the transporter 770 and when it is not.

[0178] In one embodiment, the agricultural machine 100 includes a work vehicle 100 and an implement 300 connected to the work vehicle 100, and the processing device 161 may change the pattern of the search areas 810, 820 depending on the implement 300 connected to the work vehicle 100.

[0179] This allows a search suitable for the implement 300 connected to the work vehicle 100 to be performed.

[0180] In one embodiment, multiple types of implements 300 of different sizes can be connected to the work vehicle 100, and the processing device 161 may change the pattern of the search areas 810, 820 depending on the size of the implement 300 connected to the work vehicle 100.

[0181] This allows a search to be performed that is appropriate for the size of the implement 300 connected to the work vehicle 100.

[0182] In one embodiment, the positional relationship between the work vehicle 100 and the implement 300 connected to the work vehicle 100 is changeable, and the processing device 161 may change the pattern of the search areas 810, 820 in accordance with the change in the positional relationship between the work vehicle 100 and the implement 300.

[0183] This allows a search to be performed according to changes in the positional relationship between the work vehicle 100 and the implement 300.

[0184] In one embodiment, the sensing system 10 further includes a positioning device 110 that detects the position of the agricultural machine 100 and outputs the position data, and a storage device 170 that stores map data of the area in which the agricultural machine 100 moves, and the processing device 161 may determine the area in which the agricultural machine 100 is located based on the position data and the map data.

[0185] Using the positioning device 110, the area in which the agricultural machine 100 is located can be determined.

[0186] In one embodiment, the agricultural machine 100 may be equipped with the above-described sensing system 10. This allows a search suitable for the area in which the agricultural machine 100 is located to be performed.

[0187] In an embodiment, the agricultural machine 100 may further include a travel device 240 that causes the agricultural machine 100 to travel, and a control device 160 that controls the operation of the travel device 240 and automatically drives the agricultural machine 100. This makes it possible to perform a search that is appropriate for the area in which the automatically traveling agricultural machine 100 is located.

[0188] A sensing method according to an embodiment of the present disclosure is a sensing method for a mobile agricultural machine 100, and includes sensing the environment around the agricultural machine 100 using one or more sensors 140 and outputting sensing data, detecting objects located in search areas 810, 820 around the agricultural machine 100 based on the sensing data, and changing the pattern of the search areas 810, 820 for detecting objects depending on the area in which the agricultural machine 100 is located.

[0189] This allows a search suitable for the area in which the agricultural machine 100 is located to be performed.

[0190] The technology of the present disclosure is particularly useful in the field of agricultural machinery such as tractors, harvesters, rice transplanters, riding tillers, vegetable transplanters, mowers, seed sowing machines, fertilizer applicators, or agricultural robots.

[0191] 1: Agricultural management system, 10: Sensing system, 50: GNSS satellite, 60: Reference station, 70: Field, 72: Work area, 74: Headland, 76: Road, 78: Work site, 80: Network, 100: Work vehicle, 101: Vehicle body, 102: Prime mover (engine), 103: Transmission, 104: Wheels, 105: Cabin, 106: Steering device, 107: Driver's seat, 108: Coupling device, 110: Positioning device, 111: GNSS receiver, 112: RTK receiver, 115: Inertial measurement unit (IMU), 116: Processing circuit, 120: Camera, 130: Obstacle sensor, 140: LiDAR sensor, 150: Sensor group, 152: Steering wheel sensor, 154: Turning angle sensor, 156: Rotation sensor, 160: Control system, 161: Processing device, 170: Storage device, 180: Control device, 181-185: ECU, 190: Communication device, 200: Operation terminal, 210: Operation switch group, 220: Buzzer, 240: Drive device, 300: Work machine, 340: Drive device, 380: Control device, 390: Communication device, 400: Terminal device, 420: Input device, 430: Display device, 450: Storage device, 460: Processor, 470: ROM, 480: RAM, 490: Communication device, 600: Management device, 660: Processor, 650: Storage device, 670: ROM, 680: RAM, 690: Communication device, 710: ridge, 712: ridge edge area, 720: waterway, 722: area adjacent to waterway, 730: bridge, 732: bridge area, 734: river, 740: barn, 741: entrance / exit, 742: entrance / exit area, 750: ridge, 752: ridge area, 760: crop row, 762: crop row area, 770: transport vehicle, 771: ladder rail, 772: loading work area, 810: first search area, 820: second search area, 830: sensing area

Claims

DEPCT6726 / 08 / 25671. Detection system for mobile agricultural machinery comprising: one or more sensors installed in the agricultural machinery that detect the surrounding environment, extract detection data, and a processing unit that detects an object in the search area surrounding the agricultural machinery based on the detection data. The processing unit then modifies the search area pattern for object detection in accordance with the location of the agricultural machinery.

2. Detection system according to claim 1, where the processing unit differentiates the search area pattern between the case where the agricultural machinery is located in the first area of ​​the field and the case where it is located in the second area of ​​the field, where the second area is closer to the outer edge of the field than the first area.

3. Detection system according to claim 2, where the second area is the edge of the embankment.4.

5. Any one of the detection systems under claim 1 through 4 where the processing unit uses a different search area pattern between cases where the agricultural machinery is located on a bridge and cases where the agricultural machinery is located on a pre-defined road other than a bridge.

6. Any detection system under claim 5 where the agricultural machinery includes a vehicle with one front wheel and the processing unit sets the search area to be closer to the outer edge of the front wheel in the case where the agricultural machinery is located on a bridge than in the case where the agricultural machinery is located on a pre-defined road.

7. The detection system under Claim 1 through 6, where the processor makes the search area pattern different when the agricultural machinery is located on a road adjacent to a waterway compared to when the agricultural machinery is located on a road not adjacent to a waterway.

8. The detection system under Claim 7, where the search area includes a lateral search area that includes the sides of the agricultural machinery, and the processor makes the lateral search area longer in the longitudinal direction when the agricultural machinery is located on a road adjacent to a waterway than when the agricultural machinery is located on a road not adjacent to a waterway.

9. The detection system under Claim 1 through 8, where the processor uses a different search area pattern when the agricultural machinery is located at the barn entrance compared to when the agricultural machinery is located in a third area other than the barn entrance. 10.

11. Any one of the detection systems under Claim 1 through 10, where the processor is used, causes the search area to differ in pattern between cases where the agricultural machinery is located at the barn entrance and cases where the agricultural machinery is located at the third area.

12. Any one of the detection systems under Claim 1 through 11, where the processor is used, causes the search area pattern to differ between cases where the agricultural machinery is located at the fourth area of ​​the field and cases where the agricultural machinery is located at the fifth area of ​​the field, with the fifth area closer to the ridge or the cultivated area of ​​the field than the fourth area. 13.

14. A detection system under claim 13 in which groups of different sized tool types can be connected to the work vehicle and the processing unit changes the search area layout in accordance with the size of the tool connected to the work vehicle.

15. A detection system under claim 13 or 14 in which the positional relationship between the work vehicle and the tool connected to the work vehicle can be changed and the processing unit changes the search area layout in accordance with changes in the positional relationship between the work vehicle and the tool. 16.

17. Agricultural machinery incorporating a detection system as defined in any of the claims 1 through 15, which also includes a positioning device used to detect the position of the agricultural machinery to obtain position information, and a data acquisition device used to store map data related to the area where the agricultural machinery is operating, which the processing unit uses to determine the area where the agricultural machinery is located based on position and map data.

17. Agricultural machinery incorporating a detection system as defined in any of the claims 1 through 16.

18. Agricultural machinery incorporating a system as defined in claim 17, which also includes a propulsion device that enables the agricultural machinery to move, and a controller used to control the propulsion device in a way that allows the agricultural machinery to move autonomously. 19.Detection methods for mobile agricultural machinery include: detecting the environment surrounding the agricultural machinery using one or more sensors to obtain detection data; detecting objects in the search area surrounding the agricultural machinery based on detection data; and detecting changes in the search area pattern to identify objects consistent with the location of the agricultural machinery.

Images