Agricultural machine, and operation support system for agricultural machine

Abstract

To estimate the position of an agricultural machine in an environment in which multiple trees grow thick.SOLUTION: An operation support system is for an agricultural machine that travels in a farm field where agricultural products are planted. The operation support system comprises: an underground radar device that is configured to radiate an electromagnetic wave to the ground and receive a reflected wave from the underground, and acquires sensing data including information related to underground parts of the agricultural products in the ground; and a processor that refers to the relative displacement of the agricultural machine with respect to the underground parts during the travel in the farm field, and estimates the position of the agricultural machine.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present disclosure relates to an agricultural machine and a driving assistance system for the agricultural machine. [Background technology]

[0002] As the next generation of agriculture, research and development is being conducted on smart agriculture that utilizes ICT (Information and Communication Technology) and IoT (Internet of Things). Research and development is also being conducted on the automation and unmanned operation of work vehicles such as tractors used in farm fields. For example, work vehicles that run with automatic steering using positioning systems such as GNSS (Global Navigation Satellite System) that enable precise positioning have been put into practical use. Patent Documents 1 to 3 disclose examples of work vehicles that perform automatic steering based on the results of positioning performed using GNSS.

[0003] Meanwhile, autonomous moving objects that use distance sensors such as LiDAR (Light Detection and Ranging) are also being developed. For example, Patent Document 4 discloses an example of a work vehicle that uses LiDAR to automatically navigate between rows of crops in a farm field. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2017 / 208306 [Patent Document 2] JP 2020-104617 A [Patent Document 3] JP 2020-12680 A [Patent Document 4] JP 2019-154379 A Summary of the Invention [Problem to be solved by the invention]

[0005] In environments where trees are densely distributed, such as orchards such as vineyards or forests, the leaves growing at the top of the trees form a canopy that acts as an obstacle or multiple reflector for radio waves from satellites. In such environments, accurate positioning using GNSS is difficult. In environments where GNSS cannot be used, it is possible to use SLAM (Simultaneous Localization and Mapping), which performs position estimation and map creation simultaneously. However, there are various challenges to be overcome in practical use of a mobile object that moves autonomously or with automatic steering in an environment with many trees using SLAM. For example, there is a challenge that maps created in the past cannot be used continuously because the distribution of tree leaves changes significantly with the change of seasons. [Means for solving the problem]

[0006] A driving assistance system according to one aspect of the present disclosure is a system for an agricultural machine traveling in a field where crops are planted, and includes an underground radar device configured to emit electromagnetic waves to the ground and receive reflected waves from underground, and to obtain sensing data including information about the underground portion of the crop underground, and a processing device that estimates the position of the agricultural machine by referring to the relative displacement of the agricultural machine with respect to the underground portion while traveling in the field.

[0007] A driving assistance system according to another aspect of the present disclosure is a system for an agricultural machine traveling in a field planted with crops, and includes an underground radar device configured to emit electromagnetic waves to the ground and receive reflected waves from underground, and to acquire sensing data including information about the underground portions of the crops in the ground, and a processing device that creates an underground map including the underground portions based on the sensing data, and when the underground radar device acquires multiple pieces of sensing data, each of which includes information about the underground portions of crops in the same field, the processing device creates the underground map based on the latest sensing data acquired latest among the multiple sensing data.

[0008] An agricultural machine according to one aspect of the present disclosure includes the driving assistance system described above, a drive device for driving the agricultural machine, and Equipped with.

[0009] The general or specific aspects of the present disclosure may be realized by an apparatus, a system, a method, an integrated circuit, a computer program, or a computer-readable non-transitory storage medium, or any combination thereof. The computer-readable storage medium may include a volatile storage medium or a non-volatile storage medium. The apparatus may be composed of multiple devices. When the apparatus is composed of two or more devices, the two or more devices may be arranged in one device or may be arranged separately in two or more separate devices. Effect of the Invention

[0010] According to an embodiment of the present disclosure, even in an environment where positioning using GNSS is difficult, it is possible to realize automatic steering or autonomous movement of agricultural machinery by utilizing sensing data including information about the underground parts or root systems of crops underground, which is acquired by a Ground Penetrating Radar (GPR) device. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 is a diagram for explaining an overview of an agricultural management system according to an exemplary embodiment of the present disclosure. [Figure 2A] FIG. 1 is a side view showing a schematic example of a work vehicle and an implement coupled to the work vehicle. [Figure 2B] FIG. 11 is a side view showing a schematic diagram of another configuration example of the work vehicle. [Diagram 3] FIG. 2 is a block diagram showing an example of the configuration of a work vehicle and an implement. [Figure 4] FIG. 1 is a conceptual diagram showing an example of a work vehicle performing positioning using RTK-GNSS. [Diagram 5] 4 is a diagram showing an example of an operation terminal and an operation switch group provided inside the cabin. FIG. [Figure 6] 2 is a block diagram illustrating an example of a hardware configuration of a management device and a terminal device. [Figure 7A] FIG. 1 is a diagram illustrating a state in which a ground penetrating radar device searches underground. [Figure 7B] FIG. 1 is a block diagram showing a configuration example of an underground radar device. [Figure 7C] FIG. 2 is a diagram illustrating an example of the arrangement of antenna units of an underground radar device. [Figure 8] FIG. 1 is a schematic diagram showing the distribution of underground parts of crops planted in a field. [Figure 9] FIG. 2 is a diagram illustrating a synthetic radar image. [Figure 10] 1 is a diagram showing a schematic diagram of an example of a work vehicle moving on a plane between time t1 and time t2. FIG. [Figure 11A] 10 is a diagram for explaining the displacement of a work vehicle relative to an underground portion when the work vehicle only moves in translation; FIG. [Figure 11B] 10 is a diagram for explaining the displacement of a work vehicle relative to an underground portion when the work vehicle only moves in translation; FIG. [Figure 11C] 10 is a diagram for explaining the displacement of a work vehicle relative to an underground portion when the work vehicle only moves in translation; FIG. [Figure 12A]10A and 10B are diagrams for explaining the amount of displacement and the angle of rotation of a work vehicle relative to an underground portion when the movement of the work vehicle involves translation and rotation. [Figure 12B] 10A and 10B are diagrams for explaining the amount of displacement and the angle of rotation of a work vehicle relative to an underground portion when the movement of the work vehicle involves translation and rotation. [Figure 13] 11 is a flowchart illustrating a procedure for self-location estimation using a feature point matching technique. [Figure 14] 10 is a diagram showing a schematic diagram of matching of feature points of the shape of a part of an underground portion between adjacent frames; FIG. [Figure 15] 10 is a diagram showing a schematic view of a rotation angle θ of a shape of a part of an underground portion. FIG. [Figure 16] 10 is a diagram for explaining how estimated values ​​of the amount of movement Δd and amount of change Δφ in orientation of a work vehicle are periodically updated. FIG. [Figure 17] 1 is a diagram illustrating an example of a work vehicle that automatically travels along a target route in a farm field; [Figure 18] 4 is a flowchart showing an example of a steering control operation during automatic driving. [Figure 19A] FIG. 2 is a diagram showing an example of a work vehicle traveling along a target route. [Figure 19B] FIG. 13 is a diagram showing an example of a work vehicle in a position shifted to the right from the target route. [Figure 19C] FIG. 13 is a diagram showing an example of a work vehicle in a position shifted to the left from a target route. [Figure 19D] FIG. 13 is a diagram showing an example of a work vehicle facing in an inclined direction with respect to a target route. [Figure 20] FIG. 1 is a diagram illustrating an example of a situation in which a plurality of work vehicles are automatically traveling on roads inside and outside a farm field. [Figure 21] FIG. 2 is a diagram showing a schematic diagram of the shape of the underground parts formed by crops in a farm field. [Figure 22] FIG. 13 is a diagram showing an example of a target route set based on a rootstock, which is part of the underground portion of a tree in a vineyard. [Figure 23]FIG. 1 is a diagram illustrating an example of a vineyard environment in which a work vehicle travels. [Figure 24] 1A to 1C are diagrams illustrating an example of a driving route of a work vehicle during underground map generation and during automatic driving. [Diagram 25] FIG. 1 is a diagram illustrating an example in which a work vehicle travels across rows of furrows on which crops are planted. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] (Definition of terms) In this disclosure, "agricultural machinery" refers to machinery used for agricultural purposes. Examples of agricultural machinery include tractors, harvesters, rice transplanters, riding management machines, vegetable transplanters, mowers, sowing machines, fertilizer applicators, and agricultural mobile robots. Not only may a work vehicle such as a tractor function as an "agricultural machinery" by itself, but also may a work vehicle and an implement attached to or towed by the work vehicle function as a single "agricultural machinery". Agricultural machinery performs agricultural work such as plowing, sowing, pest control, fertilization, planting crops, or harvesting on the ground in a field. These agricultural works are sometimes referred to as "ground work" or simply "work". Traveling of a vehicle-type agricultural machine while performing agricultural work is sometimes referred to as "work travel".

[0013] "Autonomous driving" means that the movement of the agricultural machine is controlled by the action of a control device, not by manual operation by the driver. An agricultural machine that performs automatic driving may be called an "automatic driving agricultural machine" or a "robot agricultural machine". During automatic driving, not only the movement of the agricultural machine but also the operation of agricultural work (for example, the operation of the working machine) may be automatically controlled. When the agricultural machine is a vehicle-type machine, the traveling of the agricultural machine by automatic driving is called "automatic driving". The control device may control at least one of steering, adjustment of the moving speed, and starting and stopping of the movement required for the movement of the agricultural machine. When controlling a work vehicle equipped with a working machine, the control device may control operations such as raising and lowering the working machine, starting and stopping the operation of the working machine. The movement by automatic driving may include not only the movement of the agricultural machine toward the destination along a predetermined route, but also the movement of the agricultural machine following a tracking target. An agricultural machine that performs automatic driving may move partially based on the instruction of a user. In addition, an agricultural machine that performs automatic driving may operate in a manual driving mode in which the agricultural machine moves by manual operation of the driver, in addition to the automatic driving mode. Steering an agricultural machine by the action of a control device, rather than manually, is called "automatic steering." A part or all of the control device may be external to the agricultural machine. Communication of control signals, commands, data, etc. may take place between the agricultural machine and a control device external to the agricultural machine. An agricultural machine that performs automatic driving may move autonomously while sensing the surrounding environment, without a human being being involved in controlling the movement of the agricultural machine. An agricultural machine capable of autonomous movement can travel unmanned in a field or outside the field (e.g., a road). During autonomous movement, the agricultural machine may detect obstacles and perform obstacle avoidance operations.

[0014] A "work plan" is data that schedules one or more agricultural works to be performed by an agricultural machine. The work plan may include, for example, information indicating the sequence of agricultural works to be performed by the agricultural machine and the field on which each agricultural work is to be performed. The work plan may include information on the scheduled date and time for each agricultural work to be performed. The work plan may be created by a processing device that communicates with the agricultural machine to manage the agricultural work, or a processing device mounted on the agricultural machine. The processing device may create a work plan based on information input by a user (such as an agricultural manager or farm worker) operating a terminal device. In this specification, a processing device that communicates with the agricultural machine to manage the agricultural work is referred to as a "management device." The management device may manage the agricultural work of multiple agricultural machines. In that case, the management device may create a work plan that includes information on each agricultural work performed by each of the multiple agricultural machines. The work plan may be downloaded by each agricultural machine and stored in a storage device. Each agricultural machine can automatically head to the field to perform the scheduled agricultural work according to the work plan and perform the agricultural work.

[0015] An "environmental map" is data that expresses the positions or areas of objects that exist in the environment in which the agricultural machine moves using a specified coordinate system. The coordinate system that defines the environmental map may be, for example, a world coordinate system such as a geographic coordinate system fixed relative to the Earth. The environmental map may include information other than the positions of objects that exist in the environment (for example, attribute information and other information). The environmental map includes maps of various formats, such as point cloud maps or grid maps. Data of local maps or partial maps that are generated or processed in the process of constructing the environmental map are also referred to as "environmental map" or "environmental map data".

[0016] An "underground map" is data that expresses the position or distribution of objects existing underground in a field where an agricultural machine travels, using a specified coordinate system. An example of an object existing underground is the underground part of a crop. The underground part includes rhizomes and roots, and is also called a root system. The coordinate system that defines the underground map may also be a world coordinate system fixed relative to the earth, similar to an environmental map. The underground map may include information other than the position (e.g., attribute information and other information) about objects existing underground. The underground map includes maps of various formats, such as a point cloud map or a grid map. Data of a local map or a partial map that is generated or processed in the process of constructing the underground map is also called an "underground map" or "underground map data".

[0017] "Farm road" means a road that is used mainly for agricultural purposes. Farm roads are not limited to roads paved with asphalt, but also include unpaved roads covered with dirt or gravel. Farm roads include roads (including private roads) that are exclusively passable by vehicle-type agricultural machinery (e.g., work vehicles such as tractors) and roads that are also passable by general vehicles (passenger cars, trucks, buses, etc.). Work vehicles may automatically travel on general roads in addition to farm roads. General roads are roads that have been developed for the traffic of general vehicles.

[0018] A "pose" is the "position and orientation" of an object. A pose in two-dimensional space is defined by three coordinate values, for example (x, y, θ). Here, (x, y) are coordinate values ​​in the XY coordinate system, which is a world coordinate system fixed to the Earth, and θ is the angle with respect to the reference direction. θ corresponds to the yaw angle among the roll angle, pitch angle, and yaw angle in three-dimensional space. The yaw angle represents the amount of rotation around the vertical axis of the agricultural machinery.

[0019] (Embodiment) Hereinafter, an embodiment of the present disclosure will be described. However, more detailed explanation than necessary may be omitted. For example, detailed explanation of already well-known matters and overlapping explanation of substantially the same configuration may be omitted. This is to avoid the following explanation becoming unnecessarily redundant and to facilitate understanding by those skilled in the art. Note that the inventor provides the attached drawings and the following explanation so that those skilled in the art can fully understand the present disclosure, and does not intend to limit the subject matter described in the claims by them. In the following description, components having the same or similar functions are given the same reference numerals.

[0020] The following embodiments are illustrative, and the technology of the present disclosure is not limited to the following embodiments. For example, the numerical values, shapes, materials, steps, order of steps, layout of the display screen, and the like shown in the following embodiments are merely examples, and various modifications are possible as long as no technical contradiction occurs. In addition, one aspect can be combined with another aspect as long as no technical contradiction occurs.

[0021] Hereinafter, an embodiment in which the technology of the present disclosure is applied to a work vehicle such as a tractor, which is an example of an agricultural machine, will be mainly described. The technology of the present disclosure is not limited to work vehicles such as tractors, but can also be applied to other types of agricultural machines.

[0022] FIG. 1 is a diagram for explaining an overview of an agricultural management system according to an exemplary embodiment of the present disclosure. The agricultural management system shown in FIG. 1 includes a work vehicle 100, a terminal device 400, and a management device 600. The agricultural management system may further include an edge computer. The terminal device 400 is a computer used by a user who remotely monitors the work vehicle 100. The management device 600 is a computer managed by a business operator who operates the agricultural management system. The work vehicle 100, the terminal device 400, and the management device 600 can communicate with each other via a network 80. Although one work vehicle 100 is illustrated in FIG. 1, the agricultural management system may include multiple work vehicles or other agricultural machines.

[0023] The underground radar device in this embodiment and a processing device that processes sensing data output from the underground radar device work together to function as a driving assistance system. Alternatively, the underground radar device, the processing device, and a control device provided in the work vehicle 100 can work together to function as a driving assistance system. The processing device is, for example, a processing device provided in the work vehicle 100, a processor in the management device 600, or an edge computer.

[0024] Driving assistance systems can also be retrofitted to work vehicles that do not have these functions. Such systems can be manufactured and sold independently of the work vehicles. Computer programs used in such systems can also be manufactured and sold independently of the work vehicles. The computer programs can be provided, for example, stored in a computer-readable non-transitory storage medium. The computer programs can also be provided by downloading via a telecommunications line (for example, the Internet).

[0025] The work vehicle 100 in this embodiment is a tractor. The work vehicle 100 can be equipped with a work implement at either the rear or the front, or both. The work vehicle 100 can travel within a farm field while performing agricultural work according to the type of work implement. The work vehicle 100 may also travel within or outside a farm field without a work implement attached.

[0026] The work vehicle 100 has an automatic driving function. That is, the work vehicle 100 can travel by the action of a control device, not manually. The control device in this embodiment is provided inside the work vehicle 100, and can control both the speed and steering of the work vehicle 100. The work vehicle 100 can travel automatically not only in a field, but also outside the field (for example, on a road).

[0027] The work vehicle 100 is equipped with devices such as a GNSS receiver used for positioning. A control device of the work vehicle 100 automatically drives the work vehicle 100 based on the position of the work vehicle 100 and information on the target route. The control device controls the operation of the work machine in addition to controlling the driving of the work vehicle 100. This allows the work vehicle 100 to perform agricultural work using the work machine while automatically driving in a field. Furthermore, the work vehicle 100 can automatically drive along roads outside the field (e.g., farm roads or general roads) along the target route. The work vehicle 100 automatically drives along roads outside the field while utilizing data output from sensing devices such as a camera or LiDAR sensor.

[0028] The management device 600 is a computer that manages agricultural work by the work vehicle 100. The management device 600 may be, for example, a server computer that centrally manages information about a farm field on the cloud and supports agriculture by utilizing data on the cloud. The management device 600, for example, creates a work plan for the work vehicle 100 and causes the work vehicle 100 to perform agricultural work according to the work plan. The management device 600 generates a target route in the farm field based on information input by a user using the terminal device 400 or other devices. The management device 600 may further generate and edit an environmental map based on data collected by the work vehicle 100 or other moving bodies using a sensing device such as a LiDAR sensor. The management device 600 transmits the generated work plan, target route, and environmental map data to the work vehicle 100. The work vehicle 100 automatically moves and performs agricultural work based on the data.

[0029] The terminal device 400 is a computer used by a user located away from the work vehicle 100. The terminal device 400 shown in FIG. 1 is a laptop computer, but is not limited thereto. The terminal device 400 may be a stationary computer such as a desktop PC (personal computer), or may be a mobile terminal such as a smartphone or a tablet computer. The terminal device 400 may be used to remotely monitor the work vehicle 100 or remotely operate the work vehicle 100. For example, the terminal device 400 can display on a display an image captured by one or more cameras (imaging devices) equipped on the work vehicle 100. The terminal device 400 can also display on a display a setting screen for a user to input information required to create a work plan for the work vehicle 100 (for example, a schedule for each farm work). When a user inputs the required information on the setting screen and performs a transmission operation, the terminal device 400 transmits the input information to the management device 600. The management device 600 creates a work plan based on the information. The terminal device 400 may further have a function of displaying, on the display, a setting screen for allowing the user to input information required for setting a target route.

[0030] An example of the system configuration according to this embodiment will be described in more detail below.

[0031] [1. Configuration example] 2A is a side view that shows a schematic example of a work vehicle 100 and a work implement 300 coupled to the work vehicle 100. The work vehicle 100 in this embodiment can operate in both a manual driving mode and an automatic driving mode. In the automatic driving mode, the work vehicle 100 can travel unmanned. The work vehicle 100 can be automatically driven both inside and outside a field.

[0032] As shown in Fig. 2A, the work vehicle 100 includes a vehicle body 101, a prime mover (engine) 102, and a speed changer (transmission) 103. The vehicle body 101 is provided with wheels 104 with tires, and a cabin 105. The wheels 104 include a pair of front wheels 104F and a pair of rear wheels 104R. A driver's seat 107, a steering device 106, an operation terminal 200, and a group of switches for operation are provided inside the cabin 105. When the work vehicle 100 travels for work in a field, crawlers instead of tires may be attached to one or both of the front wheels 104F and the rear wheels 104R.

[0033] The work vehicle 100 may be equipped with at least one sensing device that senses the environment surrounding the work vehicle 100 and the underground, and a processing device that processes sensing data output from the at least one sensing device. In the example shown in Fig. 2A, the work vehicle 100 is equipped with multiple sensing devices. The sensing devices include multiple cameras 120, multiple obstacle sensors 130, and an underground radar device 140.

[0034] The cameras 120 may be installed, for example, on the front, rear, left and right sides of the work vehicle 100. The cameras 120 capture images of the environment around the work vehicle 100 and generate image data. The images captured by the cameras 120 may be output to a processing device mounted on the work vehicle 100 and transmitted to a terminal device 400 for remote monitoring. The images may be used to monitor the work vehicle 100 during unmanned driving. The cameras 120 may also be used to generate images for recognizing surrounding objects or obstacles, white lines, signs, or indications when the work vehicle 100 travels on roads outside the field (farm roads or general roads).

[0035] The underground radar device 140 in the example of FIG. 2A is disposed under the front of the vehicle body 101. The underground radar device 140 may be disposed in other positions, such as the central lower part of the vehicle body 101 or the rear lower part of the vehicle body 101. The number of underground radar devices used is not limited to one, and may be two or more. FIG. 2B is a side view showing another configuration example of the work vehicle 100. The work vehicle 100 may be equipped with, for example, two underground radar devices 140. In this case, one may be disposed under the front of the vehicle body 101, and the other may be disposed under the rear of the vehicle body 101. Alternatively, in cases where it is difficult to attach the underground radar device to the rear of the vehicle body due to interference with the work machine, the underground radar device may be attached to the work machine that performs ground work on the ground, or the underground radar device may be attached to the bottom surface of the vehicle body 101.

[0036] The underground radar device 140 is configured to radiate electromagnetic waves to the ground and receive reflected waves from the ground. The underground radar device 140 is an exploration device that explores and detects the presence of underground objects by utilizing the phenomenon in which electromagnetic waves are reflected at physical boundaries in the ground due to changes in the relative dielectric constant. The underground radar device 140 repeatedly acquires sensing data including information on the spatial distribution of structures existing in the ground. The spatial distribution may be a distribution in a two-dimensional or three-dimensional space. Hereinafter, the spatial distribution will be simply referred to as "distribution". The underground radar device 140 in this embodiment repeatedly acquires sensing data including information on the distribution of the underground parts of crops in the ground in the field while the work vehicle 100 is traveling in the field.

[0037] Fig. 7A is a diagram showing a schematic diagram of an underground exploration by an underground radar device 140. Fig. 7B is a block diagram showing a configuration example of the underground radar device 140. Fig. 7C is a diagram showing a schematic diagram of an arrangement example of an antenna unit 141 of the underground radar device 140.

[0038] The underground radar device 140 in this embodiment includes a plurality of antenna units (multi-channel) 141 each including a transmitting antenna T and a receiving antenna R, a receiver 142 connected to each of the plurality of receiving antennas R, a pulse generator 143 connected to each of the plurality of transmitting antennas T, a control device 144 that controls the operation of the receiver 142 and the pulse generator 143, and a storage device 145. The underground radar device 140 illustrated in FIG. 7B and FIG. 7C is a three-dimensional underground radar device including a plurality of antenna units 141. The length of the underground radar device 140 along the width direction of the vehicle body 101 (Y direction shown in FIG. 7C) is, for example, 100 cm to 200 cm. The plurality of transmitting antennas T are arranged at equal intervals (for example, several tens of mm) along the width direction of the vehicle body 101, and the plurality of receiving antennas R can also be arranged at equal intervals (for example, several tens of mm) along the width direction of the vehicle body 101. According to the 3D underground radar device, it is possible to simultaneously obtain underground exploration data of multiple underground planes parallel to the XZ plane shown in FIG. 7A at intervals between the receiving antennas R. Based on the underground exploration data, the distribution of the underground part of the ground can be obtained as high-resolution 3D information and visualized. This 3D information can be visualized, and thus 3D image data can be obtained. In addition, the 2D or 3D shape of the underground part can be estimated based on the 3D information showing the distribution of the underground part of the ground.

[0039] The underground radar device 140 can transmit and receive electromagnetic waves in a frequency band from 200 MHz to 3000 MHz, for example, according to a step frequency method. However, the radar method or modulation method is not limited to this and may be any method. For example, an impulse method may be used as the radar method. The underground radar device 140 can generate a time domain waveform corresponding to the reflected wave from the ground by emitting electromagnetic waves to the ground while stepwise increasing the sine wave frequency between the above-mentioned frequency bands, receiving the reflected wave from the ground, and performing an inverse Fourier transform on the frequency spectrum data obtained. Since the reflected waveform depends on the position or shape of the reflector 77, it becomes possible to estimate or identify the position (depth in the Z direction shown in FIG. 7C) or shape of the reflector 77 based on the received signal.

[0040] The sensing data output from the underground radar device 140 may be processed by the control system of the work vehicle 100, the management device 600, or an edge computer. In this embodiment, the control system of the work vehicle 100 processes the sensing data output from the underground radar device 140 and performs self-position estimation based on the relative displacement of the work vehicle 100 with respect to the underground portion.

[0041] The work vehicle 100 may be equipped with one or more LiDAR sensors. The LiDAR sensor repeatedly outputs sensor data indicating the distance and direction to each measurement point of an object present in the surrounding environment, or the two-dimensional or three-dimensional coordinate values ​​of each measurement point, while the work vehicle 100 is mainly traveling outside the field. The sensor data output from the LiDAR sensor may be processed by a control system of the work vehicle 100. The control device can detect objects such as obstacles present in the vicinity of the work vehicle 100 based on the sensor data.

[0042] Please refer to FIG. 2A again. The obstacle sensors 130 shown in FIG. 2A are provided at the front and rear of the cabin 105. The obstacle sensors 130 may be arranged at other locations. For example, one or more obstacle sensors 130 may be provided at any positions on the sides, front, and rear of the vehicle body 101. The obstacle sensor 130 may include, for example, a laser scanner or an ultrasonic sonar. The obstacle sensor 130 is used to detect surrounding obstacles during autonomous driving and to stop or detour the work vehicle 100.

[0043] The work vehicle 100 in this embodiment further includes a GNSS unit 110. However, the GNSS unit is not essential. The GNSS unit 110 includes a GNSS receiver. The GNSS receiver may include an antenna for receiving signals from GNSS satellites and a processor for calculating the position of the work vehicle 100 based on the signals received by the antenna. The GNSS unit 110 receives satellite signals transmitted from a plurality of GNSS satellites and performs positioning based on the satellite signals. GNSS is a general term for satellite positioning systems such as GPS (Global Positioning System), QZSS (Quasi-Zenith Satellite System, e.g., Michibiki), GLONASS, Galileo, and BeiDou. The GNSS unit 110 in this embodiment is provided on the top of the cabin 105, but may be provided in another position.

[0044] The GNSS unit 110 may include an inertial measurement unit (IMU). Signals from the IMU may be used to supplement the position data. The IMU may measure the tilt and minute movements of the work vehicle 100. By using data acquired by the IMU to supplement the position data based on satellite signals, the performance of positioning may be improved.

[0045] The processing device of the work vehicle 100 in this embodiment may estimate its own position using the positioning results from the GNSS unit 110 and / or the sensing data acquired by the camera 120, in addition to the sensing data acquired by the underground radar device 140. This allows the position of the work vehicle 100 to be identified with higher accuracy.

[0046] The prime mover 102 may be, for example, a diesel engine. An electric motor may be used instead of a diesel engine. The transmission 103 can change the propulsive force and travel speed of the work vehicle 100 by changing gears. The transmission 103 can also switch the work vehicle 100 between forward and reverse.

[0047] The steering device 106 includes a steering wheel, a steering shaft connected to the steering wheel, and a power steering device that assists steering by the steering wheel. The front wheels 104F are steered wheels, and the traveling direction of the work vehicle 100 can be changed by changing the turning angle (also referred to as the "steering angle"). The steering angle of the front wheels 104F can be changed by operating the steering wheel. The power steering device includes a hydraulic device or an electric motor that supplies an auxiliary force for changing the steering angle of the front wheels 104F. When automatic steering is performed, the steering angle is automatically adjusted by the force of the hydraulic device or the electric motor under the control of a control device arranged in the work vehicle 100.

[0048] A coupling device 108 is provided at the rear of the vehicle body 101. The coupling device 108 includes, for example, a three-point support device (also referred to as a "three-point link" or "three-point hitch"), a PTO (Power Take Off) shaft, a universal joint, and a communication cable. The coupling device 108 can attach and detach the working equipment 300 to and from the work vehicle 100. The coupling device 108 can change the position or attitude of the working equipment 300 by raising and lowering the three-point link using, for example, a hydraulic device. In addition, power can be sent from the work vehicle 100 to the working equipment 300 via the universal joint. The work vehicle 100 can make the working equipment 300 perform a predetermined task while pulling the working equipment 300. The coupling device may be provided at the front of the vehicle body 101. In that case, the working equipment can be connected to the front of the work vehicle 100.

[0049] 2A is a sprayer that sprays a chemical onto crops, but the work machine 300 is not limited to a sprayer. For example, any work machine, such as a seeder, a spreader, a transplanter, a mower, a rake, a baler, a harvester, a rotary tiller, or a harrow, can be connected to the work vehicle 100 and used.

[0050] 2A is capable of being driven by a driver, but may be capable of being driven only in an unmanned manner. In that case, components necessary only for a driver, such as the cabin 105, the steering device 106, and the driver's seat 107, may not be provided in the work vehicle 100. The unmanned work vehicle 100 can travel autonomously or by remote control by a user.

[0051] 3 is a block diagram showing an example configuration of the work vehicle 100 and the work machine 300. The work vehicle 100 and the work machine 300 can communicate with each other via a communication cable included in the coupling device 108. The work vehicle 100 can communicate with the terminal device 400 and the management device 600 via the network 80.

[0052] The work vehicle 100 in the example of FIG. 3 includes a GNSS unit 110, a camera 120, an obstacle sensor 130, an underground radar device 140, and an operation terminal 200, as well as a sensor group 150 for detecting the operating state of the work vehicle 100, a control system 160, a communication device 190, an operation switch group 210, a buzzer 220, and a drive device 240. These components are connected to each other so as to be able to communicate with each other via a bus. The GNSS unit 110 includes a GNSS receiver 111, an RTK receiver 112, an inertial measurement unit (IMU) 115, and a processing circuit 116. The sensor group 150 includes a steering wheel sensor 152, a turning angle sensor 154, and an axle sensor 156. The control system 160 includes a processing device 161, a storage device 170, and a control device 180. The control device 180 includes a plurality of electronic control units (ECUs) 181 to 185. The work machine 300 includes a drive unit 340, a control unit 380, and a communication unit 390. Note that Fig. 3 shows components that are relatively highly related to the operation of the autonomous driving by the work vehicle 100, and does not show other components.

[0053] The GNSS receiver 111 in the GNSS unit 110 receives satellite signals transmitted from a plurality of GNSS satellites and generates GNSS data based on the satellite signals. The GNSS data is generated in a predetermined format, such as the NMEA-0183 format. The GNSS data may include, for example, values ​​indicating the identification number, elevation angle, azimuth angle, and reception strength of each satellite from which the satellite signal is received.

[0054] The GNSS unit 110 shown in FIG. 3 performs positioning of the work vehicle 100 using RTK (Real Time Kinematic)-GNSS. FIG. 4 is a conceptual diagram showing an example of a work vehicle 100 performing positioning by RTK-GNSS. In positioning by RTK-GNSS, in addition to satellite signals transmitted from multiple GNSS satellites 50, correction signals transmitted from a reference station 60 are used. The reference station 60 may be installed near the field where the work vehicle 100 performs work travel (for example, within 10 km from the work vehicle 100). The reference station 60 generates a correction signal, for example, in an RTCM format, based on the satellite signals received from the multiple GNSS satellites 50, and transmits it to the GNSS unit 110. The RTK receiver 112 includes an antenna and a modem, and receives the correction signal transmitted from the reference station 60. The processing circuit 116 of the GNSS unit 110 corrects the positioning result by the GNSS receiver 111 based on the correction signal. By using RTK-GNSS, it is possible to perform positioning with an accuracy of, for example, a few centimeters. Position information including latitude, longitude, and altitude information is acquired by highly accurate positioning using RTK-GNSS. The GNSS unit 110 calculates the position of the work vehicle 100 at a frequency of, for example, about 1 to 10 times per second.

[0055] The positioning method is not limited to RTK-GNSS, and any positioning method (such as interferometric positioning or relative positioning) that can obtain position information with the required accuracy can be used. For example, positioning may be performed using a Virtual Reference Station (VRS) or a Differential Global Positioning System (DGPS). If position information with the required accuracy can be obtained without using a correction signal transmitted from the reference station 60, the position information may be generated without using a correction signal. In that case, the GNSS unit 110 does not need to be equipped with the RTK receiver 112.

[0056] Even when RTK-GNSS is used, in places where a correction signal from the reference station 60 cannot be obtained (for example, on a road far from a field), the position of the work vehicle 100 is estimated by other methods without relying on the signal from the RTK receiver 112. For example, the position of the work vehicle 100 can be estimated by matching data output from the camera 120 and / or LiDAR sensor with a highly accurate environmental map. Also, when the work vehicle 100 travels through a field, the position of the work vehicle 100 can be estimated by matching sensing data output from the underground radar device 140 with a highly accurate underground map.

[0057] The GNSS unit 110 in this embodiment further includes an IMU 115. The IMU 115 may include a three-axis acceleration sensor and a three-axis gyroscope. The IMU 115 may include an orientation sensor such as a three-axis geomagnetic sensor. The IMU 115 functions as a motion sensor and can output signals indicating various quantities such as the acceleration, speed, displacement, and attitude of the work vehicle 100. The processing circuit 116 can estimate the position and orientation of the work vehicle 100 with higher accuracy based on the signal output from the IMU 115 in addition to the satellite signal and the correction signal. The signal output from the IMU 115 can be used to correct or complement the position calculated based on the satellite signal and the correction signal. The IMU 115 outputs signals at a higher frequency than the GNSS receiver 111. Using the high-frequency signal, the processing circuit 116 can measure the position and orientation of the work vehicle 100 at a higher frequency (for example, 10 Hz or more). Instead of the IMU 115, a three-axis acceleration sensor and a three-axis gyroscope may be provided separately. The IMU 115 may be provided as a device separate from the GNSS unit 110.

[0058] The camera 120 is an imaging device that captures the environment around the work vehicle 100. The camera 120 includes an image sensor, such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS). The camera 120 may also include an optical system including one or more lenses, and a signal processing circuit. The camera 120 captures the environment around the work vehicle 100 while the work vehicle 100 is traveling, and generates image (e.g., video) data. The camera 120 can capture video at a frame rate of, for example, 3 frames per second (fps) or more. The images generated by the camera 120 can be used, for example, when a remote observer checks the environment around the work vehicle 100 using the terminal device 400. The images generated by the camera 120 may be used for positioning or obstacle detection. As shown in FIG. 2A, multiple cameras 120 may be provided at different positions on the work vehicle 100, or a single camera may be provided. A visible camera for generating a visible light image and an infrared camera for generating an infrared image may be provided separately. Both a visible camera and an infrared camera may be provided as cameras for generating images for monitoring. The infrared camera may also be used to detect obstacles at night.

[0059] The obstacle sensor 130 detects objects present around the work vehicle 100. The obstacle sensor 130 may include, for example, a laser scanner or an ultrasonic sonar. When an object is present closer than a predetermined distance from the obstacle sensor 130, the obstacle sensor 130 outputs a signal indicating the presence of an obstacle. A plurality of obstacle sensors 130 may be provided at different positions of the work vehicle 100. For example, a plurality of laser scanners and a plurality of ultrasonic sonars may be disposed at different positions of the work vehicle 100. By providing such a large number of obstacle sensors 130, blind spots in monitoring obstacles around the work vehicle 100 can be reduced.

[0060] The steering wheel sensor 152 measures the rotation angle of the steering wheel of the work vehicle 100. The turning angle sensor 154 measures the turning angle of the front wheels 104F which are steered wheels. The measurement values ​​by the steering wheel sensor 152 and the turning angle sensor 154 are used for steering control by the control device 180.

[0061] The axle sensor 156 measures the rotational speed of the axle connected to the wheels 104, i.e., the number of rotations per unit time. The axle sensor 156 may be a sensor that uses, for example, a magnetoresistive element (MR), a Hall element, or an electromagnetic pickup. The axle sensor 156 outputs, for example, a numerical value indicating the number of rotations per minute (unit: rpm) of the axle. The axle sensor 156 is used to measure the speed of the work vehicle 100.

[0062] The drive device 240 includes various devices necessary for the travel of the work vehicle 100 and the driving of the work implement 300, such as the prime mover 102, the transmission 103, the steering device 106, and the coupling device 108 described above. The prime mover 102 may include an internal combustion engine, such as a diesel engine. The drive device 240 may include an electric motor for traction instead of or in addition to the internal combustion engine.

[0063] The buzzer 220 is an audio output device that emits a warning sound to notify of an abnormality. For example, the buzzer 220 emits a warning sound when an obstacle is detected during automatic driving. The buzzer 220 is controlled by the control device 180.

[0064] An example of the processing device 161 is a microprocessor or a microcontroller. The processing device 161 mainly processes sensing data output from the underground radar device 140. For example, the processing device 161 detects an underground part in the field based on sensing data repeatedly output from the underground radar device 140 while the work vehicle 100 is traveling in the field, and performs self-position estimation while referring to the relative displacement of the work vehicle 100 with respect to the underground part. The processing device 161 also repeats the operation of generating local underground map data showing the distribution of the detected underground part and recording it in the storage device 170. The processing device 161 can generate underground map data by connecting the local underground map data generated while the work vehicle 100 travels in the entire field or a section of the field. The underground map data may be generated or edited by a processor or an edge computer in the management device 600. It is also possible to cause the control device 180 described later to execute a series of processes performed by the processing device 161.

[0065] The storage device 170 includes one or more storage media such as a flash memory or a magnetic disk. The storage device 170 stores various data generated by the GNSS unit 110, the camera 120, the obstacle sensor 130, the underground radar device 140, the sensor group 150, the processing device 161, and the control device 180. The data stored in the storage device 170 may include an underground map of the field in which the work vehicle 100 travels, an environmental map of the environment in which the work vehicle 100 travels, and data of a target route for automatic driving. The environmental map includes, for example, information on a plurality of fields in which the work vehicle 100 performs farm work and roads in the surrounding area. The environmental map and the target route may be generated by a processor in the management device 600. Note that the control device 180 may have a function of generating or editing the environmental map and the target route. The control device 180 can edit the environmental map and the target route acquired from the management device 600 according to the traveling environment of the work vehicle 100. The storage device 170 also stores the data of the work plan received by the communication device 190 from the management device 600 .

[0066] The storage device 170 also stores computer programs that cause the processing device 161 and each ECU in the control device 180 to execute various operations described below. Such computer programs may be provided to the work vehicle 100 via a storage medium (e.g., a semiconductor memory or an optical disk) or an electric communication line (e.g., the Internet). Such computer programs may be sold as commercial software.

[0067] The control device 180 includes a plurality of ECUs, for example, an ECU 181 for speed control, an ECU 182 for steering control, an ECU 183 for implement control, an ECU 184 for automatic driving control, and an ECU 185 for route generation.

[0068] The ECU 181 controls the speed of the work vehicle 100 by controlling the prime mover 102 , the transmission 103 , and the brakes, all of which are included in the drive device 240 .

[0069] The ECU 182 controls the steering of the work vehicle 100 by controlling the hydraulic device or electric motor included in the steering device 106 based on the measurement values ​​of the steering wheel sensor 152 .

[0070] The ECU 183 controls the operation of the three-point link, the PTO shaft, and the like included in the coupling device 108 in order to cause the work machine 300 to perform a desired operation. The ECU 183 also generates a signal for controlling the operation of the work machine 300, and transmits the signal from the communication device 190 to the work machine 300.

[0071] The ECU 184 performs calculations and control to realize autonomous driving based on data output from the GNSS unit 110, the camera 120, the obstacle sensor 130, the processing device 161, and the sensor group 150. For example, the ECU 184 identifies the position of the work vehicle 100 based on data output from at least one of the GNSS unit 110, the camera 120, and the processing device 161. The ECU 184 may identify the position of the work vehicle 100 further based on data output from the LiDAR sensor. When the work vehicle 100 travels particularly in a farm field, the processing device 161 may perform self-location estimation based on sensing data output from the underground radar device 140, either alone or in cooperation with the ECU 184. The ECU 184 may correct the position of the work vehicle 100 estimated by the processing device 161 based on data acquired by the camera 120 or the LiDAR sensor. Outside the field, the ECU 184 estimates the position of the work vehicle 100 using data output from the camera 120 or the LiDAR sensor. For example, the ECU 184 may estimate the position of the work vehicle 100 by matching the data output from the camera 120 or the LiDAR sensor with an environmental map. During autonomous driving, the ECU 184 performs calculations necessary for the work vehicle 100 to travel along a target route based on the estimated position of the work vehicle 100. The ECU 184 sends a speed change command to the ECU 181 and a steering angle change command to the ECU 182. In response to the speed change command, the ECU 181 changes the speed of the work vehicle 100 by controlling the prime mover 102, the transmission 103, or the brakes. In response to the steering angle change command, the ECU 182 changes the steering angle by controlling the steering device 106.

[0072] While work vehicle 100 is traveling, ECU 185 recognizes obstacles present around work vehicle 100 based on data output from camera 120 and obstacle sensor 130. ECU 185 also determines the destination of work vehicle 100 based on the work plan stored in storage device 170, and can determine a target route from the start point of work vehicle 100's movement to the destination point.

[0073] Through the operation of these ECUs, the control device 180 cooperates with the processing device 161 to realize autonomous driving. During autonomous driving, the control device 180 controls the drive device 240 based on the measured or estimated position of the work vehicle 100 and the target route. In this way, the control device 180 can cause the work vehicle 100 to travel along the target route.

[0074] The multiple ECUs included in the control device 180 can communicate with each other according to a vehicle bus standard such as CAN (Controller Area Network). A faster communication method such as in-vehicle Ethernet (registered trademark) may be used instead of CAN. In FIG. 3, the processing device 161 and the ECUs 181 to 185 are shown as individual blocks, but their respective functions may be realized by multiple ECUs. An in-vehicle computer that integrates at least some of the functions of the processing device 161 and the ECUs 181 to 185 may be provided. The control device 180 may include ECUs other than the ECUs 181 to 185, and any number of ECUs may be provided depending on the functions. For example, the processing device 161 may be implemented in the control device 180 as an ECU. Each ECU includes a processing circuit including one or more processors.

[0075] The communication device 190 is a device including a circuit for communicating with the work machine 300, the terminal device 400, and the management device 600. The communication device 190 includes a circuit for transmitting and receiving signals conforming to the ISOBUS standard, such as ISOBUS-TIM, between the communication device 390 of the work machine 300. This allows the work machine 300 to perform a desired operation and to acquire information from the work machine 300. The communication device 190 may further include an antenna and a communication circuit for transmitting and receiving signals between the communication devices of the terminal device 400 and the management device 600 via the network 80. The network 80 may include, for example, a cellular mobile communication network such as 3G, 4G, or 5G, and the Internet. The communication device 190 may have a function for communicating with a mobile terminal used by a supervisor located near the work vehicle 100. Communication may be performed between such a mobile terminal in accordance with any wireless communication standard, such as cellular mobile communication such as Wi-Fi (registered trademark), 3G, 4G, or 5G, or Bluetooth (registered trademark).

[0076] The operation terminal 200 is a terminal through which a user performs operations related to the traveling of the work vehicle 100 and the operation of the work implement 300, and is also referred to as a virtual terminal (VT). The operation terminal 200 may include a display device such as a touch screen, and / or one or more buttons. The display device may be, for example, a display such as a liquid crystal or an organic light-emitting diode (OLED). By operating the operation terminal 200, a user can perform various operations such as switching the automatic driving mode on / off, recording or editing an environmental map, setting a target route, and switching the work implement 300 on / off. At least some of these operations can also be realized by operating the operation switch group 210. The operation terminal 200 may be configured to be removable from the work vehicle 100. A user at a location away from the work vehicle 100 may operate the detached operation terminal 200 to control the operation of the work vehicle 100. The user may control the operation of the work vehicle 100 by operating a computer on which necessary application software is installed, such as a terminal device 400, instead of the operation terminal 200.

[0077] 5 is a diagram showing an example of the operation terminal 200 and the operation switch group 210 provided inside the cabin 105. Inside the cabin 105, the operation switch group 210 including a plurality of switches that can be operated by a user is arranged. The operation switch group 210 may include, for example, a switch for selecting a gear stage of the main transmission or the sub-transmission, a switch for switching between an automatic driving mode and a manual driving mode, a switch for switching between forward and reverse, and a switch for raising and lowering the work machine 300. Note that in the case where the work vehicle 100 only performs unmanned driving and does not have a function of manned driving, the work vehicle 100 does not need to be provided with the operation switch group 210.

[0078] The drive device 340 in the work machine 300 shown in Fig. 3 performs operations necessary for the work machine 300 to perform a predetermined task. The drive device 340 includes devices according to the application of the work machine 300, such as a hydraulic device, an electric motor, or a pump. The control device 380 controls the operation of the drive device 340. The control device 380 causes the drive device 340 to perform various operations in response to signals transmitted from the work vehicle 100 via the communication device 390. In addition, a signal according to the state of the work machine 300 can also be transmitted from the communication device 390 to the work vehicle 100.

[0079] Next, the configurations of the management device 600 and the terminal device 400 will be described with reference to Fig. 6. Fig. 6 is a block diagram illustrating a schematic hardware configuration of the management device 600 and the terminal device 400.

[0080] The management device 600 includes a storage device 650, a processor 660, a ROM (Read Only Memory) 670, a RAM (Random Access Memory) 680, and a communication device 690. These components are connected to each other via a bus so that they can communicate with each other. The management device 600 can function as a cloud server that manages the schedule of farm work in a field performed by the work vehicle 100 and supports agriculture by utilizing the data it manages. A user can input information required for creating a work plan using the terminal device 400 and upload the information to the management device 600 via the network 80. The management device 600 can create a schedule of farm work, that is, a work plan, based on the information. The management device 600 can further generate or edit an underground map and an environmental map. Each of the underground map and the environmental map may be distributed from a computer external to the management device 600.

[0081] The communication device 690 is a communication module for communicating with the work vehicle 100 and the terminal device 400 via the network 80. The communication device 690 can perform wired communication conforming to a communication standard such as IEEE1394 (registered trademark) or Ethernet (registered trademark). The communication device 690 may perform wireless communication conforming to the Bluetooth (registered trademark) standard or the Wi-Fi standard, or cellular mobile communication such as 3G, 4G, or 5G.

[0082] The processor 660 may be, for example, a semiconductor integrated circuit including a central processing unit (CPU). The processor 660 may be realized by a microprocessor or a microcontroller. Alternatively, the processor 660 may be realized by a field programmable gate array (FPGA) equipped with a CPU, a graphics processing unit (GPU), an application specific integrated circuit (ASIC), an application specific standard product (ASSP), or a combination of two or more circuits selected from these circuits. The processor 660 sequentially executes a computer program stored in the ROM 670, which describes a group of instructions for executing at least one process, to realize a desired process.

[0083] The ROM 670 is, for example, a writable memory (e.g., PROM), a rewritable memory (e.g., flash memory), or a read-only memory. The ROM 670 stores a program that controls the operation of the processor 660. The ROM 670 does not need to be a single storage medium, and may be a collection of multiple storage media. A part of the collection of multiple storage media may be a removable memory.

[0084] The RAM 680 provides a working area for temporarily loading, at boot time, the control program stored in the ROM 670. The RAM 680 does not have to be a single storage medium, and may be a collection of multiple storage media.

[0085] The storage device 650 mainly functions as a database storage. The storage device 650 may be, for example, a magnetic storage device or a semiconductor storage device. An example of a magnetic storage device is a hard disk drive (HDD). An example of a semiconductor storage device is a solid state drive (SSD). The storage device 650 may be a device independent of the management device 600. For example, the storage device 650 may be a storage device connected to the management device 600 via the network 80, such as a cloud storage.

[0086] The terminal device 400 includes an input device 420, a display device 430, a storage device 450, a processor 460, a ROM 470, a RAM 480, and a communication device 490. These components are connected to each other via a bus so as to be able to communicate with each other. The input device 420 is a device for converting instructions from a user into data and inputting the data to a computer. The input device 420 may be, for example, a keyboard, a mouse, or a touch panel. The display device 430 may be, for example, a liquid crystal display or an organic EL display. The processor 460, the ROM 470, the RAM 480, the storage device 450, and the communication device 490 are described in the hardware configuration example of the management device 600, and the description thereof will be omitted.

[0087] [2. Underground parts of crops] Crops consist of parts that exist above ground and parts that exist underground. The former are called the above-ground parts, and the latter are called the underground parts or root systems. The underground parts include roots and rhizomes. The roots of crops grow irregularly underground as they grow. In addition, the way roots grow can vary from year to year due to various factors that affect root growth, such as pruning techniques, irrigation, and the year. Below, we will explain the general structure of the underground parts using grapes as an example of a crop.

[0088] The majority of the underground part of the grapevine is located within a depth of 100 cm below the ground, and individual roots can grow to a depth of 9 m. Most of the fine roots are located between 10 and 60 cm below the ground, with the greatest density of roots occurring at a depth of about 20 cm below the ground. The taproot is usually found between 18 and 80 cm below the ground. Root distribution may depend on soil characteristics, impermeable layers, and the variety of root stock. Grapevine roots are known to be highly branched and can grow to various depths and directions. Several studies have shown that grapevine roots grow primarily between flowering and véraison (i.e., as the grapes mature and begin to color), typically not near harvest time, and very little growth after harvest or in early spring. Roots can also grow during the dormant period if the soil temperature is warm enough.

[0089] In this way, the distribution of the underground part of a field where crops are planted is specific to each individual crop and forms a unique pattern of the underground part. Each of the multiple crops planted in the field forms a unique pattern of the underground part. Therefore, by combining the underground parts of the multiple crops, a unique pattern of the entire underground part of the field is formed as a whole field. The distribution of the entire underground part of the field may change with the growth of the roots due to the various factors mentioned above. In this respect, the dynamic underground part is different from static structures that may exist in the ground, such as pipes or rocks.

[0090] FIG. 8 is a diagram showing a schematic distribution of underground parts of crops planted in a field. FIG. 8 shows an example focusing on underground parts 700 of nine crops among the crops. The underground parts of each of the crops are spread throughout the ground of the field to form the entire underground part. The distribution or shape of the underground parts differs for each individual crop. Each root extends irregularly in a three-dimensional space. For example, the standard root size (diameter) of grapes is about 0.5 cm to 2 cm, and thick roots may have a diameter of 2 cm or more, and thin roots may have a diameter of 0.5 cm or less. Grape roots may reach a point, for example, 2 m to 3 m away from the trunk in the horizontal direction (direction perpendicular to the depth direction of the ground).

[0091] While the work vehicle 100 is traveling in the field, the underground radar device 140 acquires sensing data including information about the underground part 700 of the crop in the ground. When the work vehicle 100 repeatedly travels back and forth along the X direction or Y direction shown in FIG. 8, for example, the underground radar device 140 acquires sensing data including three-dimensional information showing the distribution of the underground part 700 in the ground. Each piece of sensing data is observation data obtained based on the reflected wave from the ground received by the receiving antenna of the underground radar device 140. For example, by using microwaves of 1500 MHz to 2000 MHz, it is possible to detect roots that exist at a depth of about 30 cm from the ground and have a diameter of 5 mm or more. By using microwaves of a relatively low frequency of about 400 MHz, it is possible to detect roots that exist at a depth of about 2 m from the ground if the roots have a diameter of 2 cm to 4 cm.

[0092] Again, refer to FIG. 7A and FIG. 7C. When the work vehicle 100 travels, for example, in the X direction, each of the multiple receiving antennas R arranged at a predetermined interval in the width direction (Y direction) of the work vehicle 100 receives a reflected wave from the ground, and thus the vertical section profiles of the underground part in the vertical section parallel to the XZ plane shown in FIG. 8 are obtained as many as the number of receiving antennas R. The processing device 161 can generate a three-dimensional data group showing the distribution of the underground part in the entire field by connecting the multiple vertical section profiles in the Y direction. The three-dimensional data group can be visualized by modeling. The processing device 161 can estimate the shape of the underground part based on the data group obtained by observation. The image generated based on the data group obtained by observation is called a radar image. The radar image includes, for example, a part or all of the shape of the underground part.

[0093] When the work vehicle 100 is equipped with a pair of underground radar devices at the front and rear of the vehicle body 101, one of them may be shifted in the width direction of the work vehicle (Y direction shown in FIG. 7C) by, for example, half the distance between the antenna units 141 relative to the other. This makes it possible to obtain longitudinal section profile data with twice the resolution in the width direction of the work vehicle compared to the case where one underground radar device is used. Obtaining a large amount of data at high resolution is useful for estimating the position or shape of underground parts underground or for constructing a highly accurate underground map showing the distribution of underground parts, as will be described later.

[0094] When the work vehicle 100 has completed traveling through the entire field or one section, a three-dimensional data group containing information on the distribution of the underground parts of the entire field or one section is obtained. The three-dimensional data group can be sliced ​​in the depth direction underground parallel to the XY plane. By slicing the three-dimensional data group in the depth direction, distribution information of the underground parts in a two-dimensional plane (horizontal cross section) at a desired depth is obtained, and a horizontal cross section profile S of the underground parts in the horizontal cross section is obtained based on the distribution information. The processing device 161 can generate a composite radar image in the depth direction by combining a plurality of radar images corresponding to a plurality of horizontal cross section profiles S0 to Sn at different depths. Each radar image contains root distribution information reflecting the shape of the roots at each depth.

[0095] The processing device 161 may analyze the distribution pattern of data points distributed in a two-dimensional or three-dimensional space using, for example, a spatial analysis method for the observed data group. The distribution of the data points may be expressed as raster data, or may be visualized using, for example, a kernel method. Alternatively, the processing device 161 may calculate the density of the point distribution and generate an image that visualizes them. The image generated in this way is also an example of the radar image described above.

[0096] FIG. 9 is a diagram showing a schematic diagram of a composite radar image. FIG. 9 illustrates a composite radar image obtained by synthesizing a plurality of radar images from the ground to a certain depth. The processing device 161 may generate a composite radar image in a depth range where roots are distributed at a high density, for example, by synthesizing a plurality of radar images corresponding to a plurality of horizontal cross-sectional profiles in that depth range. The processing device 161 may generate a moving image of, for example, 10 fps or more using data of continuous radar images or synthetic radar images in a time series. Hereinafter, the moving image generated in this way is called a "radar moving image." Each frame constituting the radar moving image includes distribution information of the underground part at the point of the work vehicle 100 in the field, and the distribution of the underground part shown by each frame differs between the frame groups.

[0097] Before generating the radar image or synthetic radar image data, the processing device 161 may apply correction parameters determined from the soil conditions in the field, such as the amount of moisture, nutrients, and soil properties contained in the soil, to the sensing data acquired by the underground radar device 140 to perform preprocessing to correct the sensing data. By using the preprocessed three-dimensional data, the accuracy of estimating the position or shape of the underground part in the ground can be improved.

[0098] [3. Self-position estimation] Next, an example of a method for estimating the position of the work vehicle 100 relative to the underground portion of the earth will be described with reference to Figures 10 to 16. In reality, the work vehicle 100 travels on the ground of a farm field that is not flat, so the attitude of the work vehicle 100 is represented by a roll angle, a pitch angle, and a yaw angle. However, in the following description, for simplicity, it is assumed that the work vehicle 100 moves in a two-dimensional space (i.e., a plane), and the attitude is represented mainly by the yaw angle.

[0099] The work vehicle 100 travels in a field where crops are planted. In one aspect of this embodiment, the processing device 161 of the work vehicle 100 refers to the relative displacement of the work vehicle 100 with respect to the underground portion when the work vehicle 100 travels in the field, and estimates the position of the work vehicle 100. Specifically, the work vehicle 100 travels in the field along a target route. The processing device 161 processes the sensing data output from the underground radar device 140 to determine the direction and magnitude of the relative displacement of the work vehicle 100 with respect to the underground portion, and estimates the relative position of the work vehicle 100 with respect to a reference point associated with the target route based on the direction and magnitude of the displacement. The self-location estimation of the present disclosure is not limited to automatic driving along a target route, and can also be applied to automatic driving when a target route is not set, or manual driving.

[0100] As described above, each group of crops planted in a field forms a unique pattern of the underground part. Therefore, the shape or distribution of the roots included in the radar image generated based on the sensing data acquired by the underground radar device 140 while the work vehicle 100 is traveling in the field changes depending on the position of the work vehicle 100. The processing device 161 determines the relative displacement of the work vehicle 100 with respect to the underground part by tracking the change in the shape or distribution of the roots between adjacent frames of the radar moving image. In this embodiment, the processing device 161 extracts a partial shape as a specific pattern from the shape of the roots included in the radar image of one frame in the time-series frame group, and determines the relative displacement of the work vehicle 100 with respect to the underground part by tracking in which direction and how much the specific pattern moves between adjacent frames. The processing device 161 performs self-location estimation based on the determined displacement.

[0101] FIG. 10 is a diagram showing a schematic example of a work vehicle 100 moving on a plane between time t1 and time t2. FIG. 10 shows a schematic diagram of the position and orientation of the work vehicle 100 moving on the plane. The position coordinates of the work vehicle 100 are indicated by an XY coordinate system. The XY coordinate system is a coordinate system that is fixed to the earth and is a coordinate system that is a coordinate system of the world coordinate system Σ wFIG. 10 shows a moving body coordinate system Σ V The moving body coordinate system Σ V is a uv coordinate system in which the direction in which the front of the work vehicle 100 faces is the u-axis direction, and the direction obtained by rotating the u-axis 90 degrees counterclockwise is the v-axis direction. V The position of the origin of the moving body coordinate system Σ changes as the work vehicle 100 moves. V The orientation of changes with the change in orientation of the work vehicle 100, that is, with rotation.

[0102] The pose (i.e., position and orientation) of the work vehicle 100 at time t1 is defined as r1, and the pose of the work vehicle 100 at time t2 is defined as r2. Pose r1 is defined, for example, by the position indicated by coordinates (x1, y1) and the orientation indicated by angle θ1. Here, the orientation of the work vehicle 100 is defined as the direction in which the front of the work vehicle 100 faces. In addition, the positive direction of the X-axis is defined as the reference direction of the angle, and the counterclockwise direction is defined as the positive direction of the angle. Pose r2 is defined by the position indicated by coordinates (x2, y2) and the orientation indicated by angle θ2.

[0103] 10, between time t1 and time t2, the position of the work vehicle 100 moves (translates) by Δd1 in the direction of angle θ1, and the orientation of the work vehicle 100 rotates counterclockwise by Δφ1 (=θ2-θ1). In this way, the movement of the work vehicle 100 is a combination of "translation" and "rotation."

[0104] In this example, when the distance Δd1 is sufficiently short, the traveling direction of the work vehicle 100 is the moving body coordinate system Σ V Therefore, the following equation 1 holds:

number

[0105] In this embodiment, Δd1 and Δφ1 are estimated by applying a feature point matching method to a radar image. If the work vehicle 100 is equipped with an internal sensor such as an inertial measurement unit (IMU), Δφ1 can also be estimated using the output from the internal sensor. If the time difference from time t1 to time t2 is short, for example, 100 milliseconds or less, the distance Δd1 is sufficiently short and Equation 1 holds. As time progresses from t1 to t2, t3, . . ., it is possible to estimate changes in the position and orientation (i.e., pose) of the work vehicle 100 by periodically updating the estimated values ​​of Δd1 and Δφ1. In other words, if an initial pose, for example (x1, y1, θ1), is known, the estimated value of the pose of the work vehicle 100 thereafter can be periodically updated. However, the accuracy of estimating the position of the work vehicle 100 can be improved by applying machine learning to the estimation of the pose based on feature point matching. Alternatively, a satellite positioning system or SLAM technology may be used in combination to obtain a highly accurate estimate of the work vehicle's 100 position.

[0106] 11A to 11C are diagrams for explaining the displacement of the work vehicle 100 with respect to the underground portion when the work vehicle 100 moves only in translation. w , and the moving body coordinate system Σ V For simplicity, the world coordinate system Σ w In the figure, a group of alphabetical characters is placed underground, and an underground radar device senses the group of characters while a work vehicle moves over the group of characters. Radar images B1, B2, and B3 acquired at time tn-1, time tn, and time tn+1, respectively, are shown by rectangular dotted lines. Here, between adjacent frames, radar image B2, which is the current frame, and radar image B1, which is the preceding frame preceding the current frame, may be referred to as the second frame and the first frame, respectively. In this case, radar images B3 and B2 may be referred to as the second frame and the first frame, respectively.

[0107] The position and orientation of the characters in the radar image are determined by the position and orientation of the underground radar device, and the position coordinates of the characters are defined in the image coordinate system. Strictly speaking, the image coordinate system is the moving body coordinate system Σ V However, in the following explanation, the image coordinate system and the moving object coordinate system Σ V The image coordinate system and the moving object coordinate system Σ V Since the relationship between these two coordinate systems is known, one coordinate system can be made to coincide with the other coordinate system by rotating it by a known angle and translating it by a known distance.

[0108] In the example shown in FIG. 11B, for example, when focusing on the character E present in both adjacent radar images B1 and B2, the position of the character E moves in the image coordinate system due to the translational motion of the work vehicle 100. This movement is a parallel movement due to the translational motion of the work vehicle 100, and as a result, a movement vector M having a magnitude (amount of movement) and a direction indicating the displacement of the character E between time tn-1 and time tn is obtained. However, the direction in this example is invariant. To determine the movement vector M, for example, a block matching method can be used with the pattern of the character E extracted from the radar image B1 as a template. In the illustrated example, when focusing on the character A, the movement vector M is similarly obtained.

[0109] In the example shown in FIG. 11C, since the letter E does not exist in radar image B3, attention is now focused on, for example, the letter F that exists in both adjacent radar images B2 and B3. Due to the translational motion of the work vehicle 100, the position of the letter F moves in the image coordinate system, and a motion vector M indicating the displacement of the letter F between time tn and time tn+1 is obtained. In this way, when the motion of the work vehicle 100 is only translational, the orientation is unchanged, and the motion vector M can be determined relatively easily. Image coordinate system and moving body coordinate system Σ V and the mobile coordinate system Σ V and the world coordinate system Σ w Since the relationship between these two is known, the world coordinate system Σ wThe amount of displacement of the work vehicle 100 at that point, that is, the distance Δd1 shown in FIG. 10, can be determined.

[0110] When the work vehicle 100 travels in a farm field, it may perform a rotational motion. Figures 12A and 12B are diagrams for explaining the displacement amount and rotation angle of the work vehicle 100 with respect to the underground part when the motion of the work vehicle 100 involves translation and rotation. Figures 12A and 12B show the world coordinate system Σ w , and the moving body coordinate system Σ V In the example shown in FIG. 12B, when attention is focused on, for example, the letter E present in both adjacent radar images B1 and B2, the letter E rotates in the image coordinate system due to the rotational motion (e.g., yaw rotation) of the work vehicle 100. Furthermore, the position of the letter E moves in the image coordinate system due to the translational motion of the work vehicle 100. This movement is a parallel movement due to the translational motion of the work vehicle 100. The magnitude and rotation angle θ (direction) of the movement vector M in this case can be determined, for example, using a feature mapping method described later. When attention is focused on the letter A, the magnitude of displacement and rotation angle θ are determined in a similar manner.

[0111] An example of a procedure for self-location estimation using a feature point matching method will be described with reference to Fig. 13. However, any known method or algorithm may be adopted for the self-location estimation of the present disclosure, other than the feature point matching method.

[0112] Fig. 13 is a flowchart illustrating a procedure for self-location estimation using a feature point matching method. Fig. 14 is a diagram showing a schematic diagram of a state where feature points of a shape of a part of an underground part are matched between adjacent frames. Fig. 15 is a diagram showing a schematic diagram of a rotation angle θ of the shape of a part of an underground part.

[0113] In this embodiment, the processing device 161 estimates the relative position of the work vehicle 100 with respect to the underground portion by executing the processes of steps S221 to S226 shown in Fig. 13. However, this series of processes may be performed, for example, by the management device 600 or an edge computer, or may be performed by the processing device 161 and the edge computer. For example, a library for feature point matching provided by OpenCV (Open Source Computer Vision Library) can be used.

[0114] (Step S221) The processing device 161 processes the sensing data output from the underground radar device 140 at a predetermined period (for example, a time determined by a scan rate) and generates a radar video image of, for example, 10 fps or more. The processing device 161 extracts the shape of a part of the underground part from the first frame, which is the preceding frame of the adjacent first and second frames. FIG. 14 illustrates an example of the shape F1 of a part of the underground part extracted from the first frame. The circumscribing rectangle of the shape F1 is indicated by a solid-line rectangle. For example, the processing device 161 binarizes the image of the first frame and extracts the contour of the shape F1 from the binarized image.

[0115] (Step S222) The processing device 161 detects feature points of the extracted shape F1 and calculates feature amounts. The processing device 161 detects the coordinates of unique points in the image coordinate system, such as corners, line intersections, or edges of the shape F1. The processing device 161 calculates, for example, values ​​that express the uniqueness of the detected feature points as vectors or binary codes as feature amounts. An example of an algorithm used to detect feature points and calculate feature amounts is AKAZE (Accelerated KAZE). The feature amounts may be calculated using deep learning. This can improve the accuracy of matching, which will be described later.

[0116] (Step S223) The processing device 161 extracts the shape of a part of the underground part from the second frame of the adjacent first and second frames, and generates a mask. The processing device 161 detects feature points of the mask and calculates feature amounts. FIG. 14 illustrates an example of a shape F2 of a part of the underground part extracted from the second frame (hereinafter, referred to as "mask F2"). The circumscribing rectangle of the shape F2 is shown by a solid-line rectangle. For example, the processing device 161 binarizes the image of the second frame, generates a mask F2 from the binarized image, and extracts the contour of the shape included in the mask F2. The processing device 161 detects feature points of the shape included in the mask F2 and calculates feature amounts, similar to step S222.

[0117] (Step S224) The processing device 161 compares the feature amount of the shape F1 calculated in step S222 with the feature amount of the mask F2 calculated in step S223. The processing device 161 may perform matching using, for example, a KNN (K Nearest Neighbor) method.

[0118] (Step S225) The processing device 161 determines whether the shape of the mask F2 extracted from the second frame matches the shape F1 extracted from the first frame. For example, the processing device 161 can determine that the shape of the mask F2 matches the shape F1 when the number of matched feature points is equal to or greater than a threshold value. When the shape of the mask F2 matches the shape F1, the process proceeds to the next step S226. On the other hand, when the number of matched feature points is less than the threshold value, the process returns to step S223. In step S223, the processing device 161 extracts the shape of a part different from the part of the underground part previously extracted from the second frame, and regenerates and updates the mask F2. The processing device 161 repeatedly executes the processes of steps S223 to S225 until the shape of the mask F2 matches the shape F1.

[0119] (Step S226) The processing device 161 estimates a transformation matrix based on the correspondence between feature points of the shape F1 and the shape included in the mask F2, and calculates the amount of movement (the magnitude of the movement vector M shown in FIG. 15) and the direction (the rotation angle θ shown in FIG. 15). An example of the transformation matrix is ​​an affine transformation matrix. The processing device 161 determines the distance Δd1 shown in FIG. 10 by converting the amount of movement in the image coordinate system into the amount of movement in the world coordinate system, and determines the rotation angle θ as the amount of change Δφ1 in the direction of the work vehicle 100. However, the amount of change Δφ1 in the direction of the displacement may be determined based on a signal output from the IMU. By using the rotation angle θ determined based on the output signal from the IMU as a known variable in estimating the transformation matrix, the accuracy of estimating the amount of movement of the displacement can be improved.

[0120] FIG. 16 is a diagram for explaining the periodic updating of the estimated values ​​of the movement amount Δd and the change amount Δφ of the orientation of the work vehicle 100 as time progresses from t0 to t1, t2, t3, and so on. When the processing device 161 estimates the movement amount Δd and the change amount Δφ in step S226, it substitutes the movement amount Δd and the change amount Δφ into the above-mentioned formula 1 each time and updates the pose rn(xn,yn,θn) of the work vehicle 100 in the world coordinate system. As a result, when the initial position at time=t0 is set as the reference point, the relative position of the work vehicle 100 with respect to the reference point is estimated at each time. However, the reference point may be set to any position coordinate in the world coordinate system. When the position coordinate of the reference point in the world coordinate system is known, the absolute position and orientation of the work vehicle 100 in the world coordinate system are determined based on the relative position of the work vehicle 100 with respect to the reference point.

[0121] According to the method of self-localization of the present embodiment, the direction and magnitude of the displacement of the work vehicle relative to the underground portion can be determined by tracking the change in the distribution of the roots between successive frames. This method does not require an underground map, which will be described later, and is therefore less affected by the growth of the roots. Since devices that were previously required for positioning, such as GNSS, LiDAR sensors, or IMUs, are not particularly required, it is possible to reduce product costs.

[0122] The processing device 161 in the above-mentioned example determines the magnitude and direction of the displacement using a radar image including the shape of the underground portion estimated from the distribution of the underground portion. However, instead of the image including the shape of the underground portion, the processing device 161 may estimate the magnitude and direction of the displacement by, for example, calculating the density of the point distribution and utilizing an image in which the density is visualized. Alternatively, the processing device 161 may use a group of three-dimensional data showing the distribution of the underground portion to analyze the similarity of spatial objects by spatial analysis, and estimate the magnitude and direction of the displacement based on the analysis results.

[0123] [4. Creating an underground map] When underground map data exists, the processing device 161 may perform self-location estimation based on the sensing data output from the underground radar device 140 and the underground map data. However, as described above, there are periods when the roots of crops grow, periods when the roots do not grow, and periods when the roots may grow depending on the soil conditions. Therefore, after a certain period of time has passed since the underground map data was generated, the underground map data may not accurately reflect the current shape of the roots and may not be suitable as a map to be used for self-location estimation. Therefore, while it is possible to reuse the underground map during the period when the roots do not grow, it is desirable to regenerate or update the underground map data during the period when the roots grow.

[0124] The processing device 161 of the work vehicle 100 in another aspect of this embodiment creates an underground map including the underground part based on the sensing data output from the underground radar device 140. It is assumed that the underground radar device 140 has acquired multiple sensing data while the work vehicle 100 has previously traveled through the same field multiple times. In this case, the sensing data acquired by the underground radar device 140 each time the work vehicle 100 travels is transmitted to, for example, the management device 600 and recorded in the storage device 650. For example, each piece of sensing data acquired on different days may include information about the underground part of the crop in the same field. In that case, the processing device 161 generates underground map data based on the latest sensing data acquired latest among the multiple sensing data. As a result, the processing device 161 can create an underground map reflecting the latest information on the distribution of the underground part, and can perform self-location estimation using the latest underground map even if the roots have grown.

[0125] The sensing data required for generating the underground map data may be collected while a work vehicle equipped with an underground radar device is traveling. Alternatively, the sensing data may be collected by a worker moving through a field while pushing a conventional hand-held underground radar device equipped with a pair of transmitting and receiving antennas (single channel). In addition, the collection of sensing data by the underground radar device and the positioning by the GNSS unit 110 may be performed simultaneously. This allows the processing device 161 to add location information (geographic information) including information on latitude, longitude, and altitude to the sensing data. The sensing data to which the geographic information has been added may be recorded in, for example, the storage device 170. The underground map data includes, for example, information on the distribution of the underground part, the shape estimated from the distribution of the underground part, or the density of the roots.

[0126] When the underground map data is generated, the work vehicle 100 can estimate its own position using the underground map data. The processing device 161 estimates its own position by matching the underground shape or distribution detected based on the sensing data repeatedly output from the underground radar device 140 while the work vehicle 100 is traveling in the field with the underground map data.

[0127] When the underground map data includes information on the distribution of underground areas, the self-location estimation can be achieved, for example, by matching the position coordinates in a three-dimensional space of the observation point obtained from the sensing data with the position coordinates of each data point of the data group showing the distribution of underground areas included in the underground map data. For example, an algorithm such as an ICP matching method or an NDT matching method can be used for the matching in this example. Alternatively, when the underground map data includes information on the shape of the underground area, for example, the above-mentioned radar image can be used as a template, and the template can be matched with the shape of the underground area included in the underground map data by a block matching method.

[0128] When the work vehicle 100 travels in the field within a predetermined period from the time when the underground map was created based on the latest sensing data, the processing device 161 may estimate the relative position of the work vehicle 100 on the underground map based on the sensing data acquired by the underground radar device 140. If the underground map has been corrected or updated since it was created, the time when the correction or update occurred corresponds to the time when the map was created. For example, the management device 600 may manage or determine whether or not a predetermined period has passed. This management or determination may be performed on an hourly or daily basis. Here, the predetermined period means, for example, the seasons of spring, summer, autumn, and winter, or a certain period in the growth process of a crop (for example, a period when roots do not grow).

[0129] When the work vehicle 100 travels through a field after a predetermined period has elapsed since the underground map was created based on the latest sensing data, the processing device 161 may update the underground map based on the sensing data acquired by the underground radar device 140. Alternatively, when the work vehicle 100 travels through a field after a predetermined period has elapsed since the underground map was created based on the latest sensing data, the processing device 161 may perform self-location estimation according to, for example, the processing procedure shown in Fig. 13 described above, without referring to the underground map data. In this way, whether or not it is possible to refer to existing underground map data, or whether or not the underground map data should be updated, may be managed by a computer on an hourly or daily basis, for example, taking into account the growth period of crop roots or the season.

[0130] According to this embodiment, for example, during periods when crop roots are not growing, the processing device 161 reuses the underground map, and during periods when crops are growing, the processing device 161 can update the underground map or, without using the underground map, estimate its own position by referring to the relative displacement of the work vehicle 100 with respect to the underground part while traveling in the field.

[0131] [5. Automatic driving operation] First, an example of the operation of the automatic driving by the work vehicle 100 will be described. The work vehicle 100 in this embodiment can automatically travel both in and outside the field. In the field, the work vehicle 100 drives the work implement 300 while traveling along a preset target route to perform a predetermined agricultural work. When the obstacle sensor 130 detects an obstacle while traveling in the field, the work vehicle 100 stops traveling, emits a warning sound from the buzzer 220, and transmits a warning signal to the terminal device 400. In the field, the position of the work vehicle 100 is estimated based on the sensing data output from the underground radar device 140 as described above. Alternatively, the positioning of the work vehicle 100 may be partially performed based on the data output from the GNSS unit 110. On the other hand, outside the field, the work vehicle 100 automatically travels along a target route set on a farm road or a general road outside the field. When traveling outside the field, the work vehicle 100 travels using data acquired by the camera 120 or the LiDAR sensor. When the work vehicle 100 detects an obstacle outside the field, the work vehicle 100 avoids the obstacle or stops there. Outside the field, the position of the work vehicle 100 is estimated based on the positioning data output from the GNSS unit 110 as well as the data output from the camera 120 or the LiDAR sensor.

[0132] An example of the operation of the work vehicle 100 when it travels autonomously within a farm field will now be described.

[0133] FIG. 17 is a diagram showing an example of a work vehicle 100 that automatically travels along a target route in a field. In this example, the field includes a work area 72 in which the work vehicle 100 performs work using the work implement 300, and a headland 74 located near the outer periphery of the field. Which area of ​​the field on the map corresponds to the work area 72 or the headland 74 can be set in advance by the user. The target route in this example includes multiple parallel main routes P1 and multiple turning routes P2 that connect the multiple main routes P1. The main route P1 is located within the work area 72, and the turning route P2 is located within the headland 74. Each main route P1 shown in FIG. 17 is a straight route, but each main route P1 may include a curved portion. The dashed line in FIG. 17 represents the working width of the work implement 300. The working width is set in advance and recorded in the storage device 170. The working width may be set and recorded by the user operating the operation terminal 200 or the terminal device 400. Alternatively, the working width may be automatically recognized and recorded when the work machine 300 is connected to the work vehicle 100. The interval between the multiple main paths P1 may be set according to the working width. The target path may be created based on the user's operation before the automatic operation is started. The target path may be created so as to cover, for example, the entire work area 72 in the field. The work vehicle 100 automatically travels back and forth from the start point of the work to the end point of the work along the target path as shown in FIG. 17. The reference point described above is, for example, associated with or determined to the start point of the work on the target path, which is the initial position. Note that the target path shown in FIG. 17 is merely an example, and the method of determining the target path is arbitrary.

[0134] Next, an example of control by the control device 180 during automatic operation in a farm field will be described.

[0135] First, an overview of steering control during automatic driving will be described. The processing device 161 in this embodiment calculates a position deviation between the target route and the relative position of the work vehicle 100. The control device 180 controls the traveling of the work vehicle 100 using the position deviation calculated by the processing device 161. The work vehicle 100 may further include an orientation measurement device that measures the orientation of the work vehicle 100 with respect to a reference line along the target route. An example of the orientation measurement device is the IMU mentioned above. The processing device 161 may calculate an orientation deviation indicating the angle difference between the orientation of the work vehicle 100 and the direction of the reference line based on the output from the orientation measurement device. The control device 180 may control the traveling of the work vehicle 100 using the position deviation and the orientation deviation.

[0136] An example of the operation of steering control during automatic driving will be described in more detail below.

[0137] FIG. 18 is a flowchart showing an example of the operation of steering control during automatic driving executed by the control device 180. The control device 180 performs automatic steering by executing the operations of steps S121 to S125 shown in FIG. 18 while the work vehicle 100 is traveling. The speed is maintained at a preset speed, for example. The control device 180 acquires data indicating the position of the work vehicle 100 estimated by the processing device 161 while the work vehicle 100 is traveling (step S121). Next, the control device 180 calculates the position deviation between the position of the work vehicle 100 and the target route (step S122). The position deviation represents the distance between the position of the work vehicle 100 at that time and the target route. The control device 180 determines whether the calculated position deviation exceeds a preset threshold value (step S123). If the position deviation exceeds the threshold value, the control device 180 changes the steering angle by changing the control parameters of the steering device included in the drive device 240 so that the position deviation becomes smaller. If the position deviation does not exceed the threshold in step S123, the operation of step S124 is omitted. In the following step S125, the control device 180 determines whether or not a command to end the operation has been received. The command to end the operation may be issued, for example, when a user remotely instructs the automatic driving to be stopped or when the work vehicle 100 reaches the destination. If the command to end the operation has not been issued, the process returns to step S121, and the same operation is executed based on the newly estimated position of the work vehicle 100. The control device 180 repeats the operations of steps S121 to S125 until a command to end the operation is issued. The above operations are executed by the ECUs 182, 184 in the control device 180.

[0138] 18, the control device 180 controls the driving device 240 based only on the position deviation between the position of the work vehicle 100 estimated by the processing device 161 and the target route, but the control may also take into consideration the azimuth deviation. For example, when the azimuth deviation, which is the angular difference between the orientation of the work vehicle 100 estimated by the processing device 161 or the orientation of the work vehicle 100 identified by the GNSS unit 110 having an IMU and the direction of the target route, exceeds a preset threshold, the control device 180 may change a control parameter (e.g., steering angle) of the steering device of the driving device 240 in accordance with the deviation.

[0139] Hereinafter, an example of steering control by the control device 180 will be described more specifically with reference to FIGS. 19A to 19D.

[0140] FIG. 19A is a diagram showing an example of the work vehicle 100 traveling along the target route P. FIG. 9B is a diagram showing an example of the work vehicle 100 in a position shifted to the right from the target route P. FIG. 9C is a diagram showing an example of the work vehicle 100 in a position shifted to the left from the target route P. FIG. 9D is a diagram showing an example of the work vehicle 100 facing in an inclined direction with respect to the target route P. In these figures, a reference line is set along the target route, and a reference point is set on the target route. For example, a pose indicating the position and orientation of the work vehicle 100 estimated by the processing device 161 is expressed as r(x, y, θ) as described with reference to FIG. 10. (x, y) are coordinates representing the position of the reference point of the work vehicle 100 in the XY coordinate system, which is the world coordinate system. In the examples shown in FIG. 19A to FIG. 19D, the reference point of the work vehicle 100 is at the position where the GNSS antenna on the cabin is installed, but the position of this reference point is arbitrary. θ is an angle representing the estimated or measured orientation of the work vehicle 100. In the illustrated example, the target path P is parallel to the Y axis, but in general, the target path P is not necessarily parallel to the Y axis.

[0141] As shown in FIG. 19A, if the position and orientation of the work vehicle 100 do not deviate from the target route P, the control device 180 maintains the steering angle and speed of the work vehicle 100 without changing them.

[0142] 19B, when the position of the work vehicle 100 has shifted to the right from the target route P, the control device 180 changes the steering angle so that the travel direction of the work vehicle 100 tilts to the left and approaches the route P. At this time, the speed may be changed in addition to the steering angle. The magnitude of the steering angle may be adjusted according to the magnitude of the position deviation Δx, for example.

[0143] 19C, when the position of the work vehicle 100 has shifted to the left from the target route P, the control device 180 changes the steering angle so that the travel direction of the work vehicle 100 tilts to the right and approaches the route P. In this case, the speed may also be changed in addition to the steering angle. The amount of change in the steering angle may be adjusted according to the magnitude of the position deviation Δx, for example.

[0144] As shown in FIG. 19D, when the position of the work vehicle 100 is not far from the target route P but the orientation is different from the direction of the target route P, the control device 180 changes the steering angle so that the azimuth deviation Δθ becomes smaller. In this case, the speed may also be changed in addition to the steering angle. The magnitude of the steering angle may be adjusted, for example, according to the magnitudes of the position deviation Δx and the azimuth deviation Δθ. For example, the smaller the absolute value of the position deviation Δx, the larger the change amount of the steering angle according to the azimuth deviation Δθ may be. When the absolute value of the position deviation Δx is large, the steering angle is changed significantly to return to the route P, so that the absolute value of the azimuth deviation Δθ inevitably becomes large. Conversely, when the absolute value of the position deviation Δx is small, it is necessary to bring the azimuth deviation Δθ closer to zero. For this reason, it is appropriate to relatively increase the weight of the azimuth deviation Δθ for determining the steering angle (i.e., the control gain).

[0145] Control techniques such as PID control or MPC control (model predictive control) can be applied to the steering control and speed control of the work vehicle 100. By applying these control techniques, the control that brings the work vehicle 100 closer to the target path P can be made smooth.

[0146] If an obstacle is detected by one or more obstacle sensors 130 while traveling, the control device 180 stops the work vehicle 100. At this time, the control device 180 may cause the buzzer 220 to emit an alarm or send an alarm signal to the terminal device 400. If it is possible to avoid the obstacle, the control device 180 may control the drive device 240 to avoid the obstacle.

[0147] The work vehicle 100 in this embodiment is capable of autonomous driving not only in a field but also outside the field. Outside the field, the control device 180 can detect objects (e.g., other vehicles or pedestrians) that exist at a relatively distant position from the work vehicle 100 based on data output from the camera 120 or the LiDAR sensor. The control device 180 can realize autonomous driving on roads outside the field by performing speed control and steering control so as to avoid the detected objects.

[0148] In this way, the work vehicle 100 in this embodiment can automatically travel inside and outside the field without a human driver. FIG. 20 is a diagram showing an example of a situation in which a plurality of work vehicles 100 are automatically traveling inside the field 70 and on the road 76 outside the field 70. At least one of an environmental map of an area including a plurality of fields and roads around the field, an underground map of each of the plurality of fields, and a target route is recorded in the storage device 170. The environmental map and the target route may be generated by the management device 600 or the ECU 185. When the work vehicle 100 travels on a road, the work vehicle 100 travels along the target route while sensing the surroundings using a sensing device such as the camera 120 with the work implement 300 raised.

[0149] FIG. 21 is a diagram showing a schematic diagram of the shape of the underground part 700 formed by the crop group in the field. FIG. 22 is a diagram showing an example of the target route P set based on the rootstock (part connected to the trunk) 701, which is a part of the underground part of the tree in the vineyard. The processing device 161 may generate distribution data of the rootstock based on three-dimensional data regarding the distribution of the underground part 700. The processing device 161 may determine the position coordinates of the rootstock 701 of each tree on the underground map from the distribution data of the rootstock. The processing device 161 may set the target route P based on the distribution data of the rootstock. In FIG. 22, the position of the rootstock 701 of each tree on the underground map is indicated by a circle mark, and the target route P for the work vehicle 100 to travel between the adjacent rootstock rows 17 is illustrated. The processing device 161 in this example determines a curve connecting the parts of the rootstock 701 included in each rootstock row 17, and determines a curve or a broken line passing through the center of the two adjacent rootstock rows 17 as the target route P.

[0150] [6. Application Examples] Next, some application examples of the self-location estimation method according to this embodiment will be described.

[0151] The self-location estimation method according to the present embodiment is preferably used for a work vehicle performing agricultural work in a vineyard, for example. A vineyard is generally an environment where multiple trees grow thickly, making it difficult to use GNSS. In such an environment, the self-location estimation method according to the present embodiment can be highly effective.

[0152] FIG. 23 is a diagram showing a schematic example of an environment of a vineyard in which a work vehicle 100 travels. In this example, the work vehicle 100 uses a work machine to perform a predetermined task (e.g., mowing, pest control, etc.) while traveling among a plurality of tree rows 20 in a vineyard. In a vineyard, the sky is blocked by branches and leaves, making automatic travel using GNSS difficult. Also, in a vineyard, the outer shape of the leaves of the trees or the hedges changes significantly depending on the season. Therefore, it is difficult to use a point cloud map created by the conventional SLAM technology throughout the year. In contrast, according to this embodiment, it is possible to perform self-location estimation based on the relative displacement of the work vehicle 100 with respect to the underground part. Furthermore, it is possible to create an underground map suitable for an orchard such as a vineyard, and it is possible to perform automatic travel of the work vehicle 100 using the underground map data.

[0153] FIG. 24 is a diagram showing an example of a travel route of the work vehicle 100 during underground map generation and automatic travel. The work vehicle 100 travels, for example, between a plurality of tree rows 20 in a vineyard along a target route P shown by an arrow in FIG. 24. In the figure, the line segments included in the target route P are drawn as straight lines, but the route on which the work vehicle 100 actually travels may include a meandering portion. In the example of FIG. 24, the work vehicle 100 travels between adjacent tree rows 20 while repeating straight and turning movements. This type of travel is performed automatically by the work vehicle 100 while estimating its own position using, for example, an underground map. Note that positioning may be performed based on the GNSS signal at a timing when the GNSS unit 110 can receive a GNSS signal. For example, at the timing of turning on the target route P shown in FIG. 24, there are no leaves that block the GNSS signal, so positioning based on the GNSS signal is possible.

[0154] FIG. 25 is a diagram showing a schematic example of a work vehicle traveling across rows of ridges on which crops are planted. FIG. 25 shows a farm field in which crops are planted in ridges 12 formed in rows. For example, the target route P may be set along a crop row 13 which is a collection of crops planted in the ridges 12. Multiple crop rows 13 may be formed on one ridge 12. The work vehicle 100 may travel along the target route P on a work passage 14 between adjacent ridges 12 while estimating its own position based on the distribution of the underground parts of the crops in the ground of the ridges 12. The work vehicle 100 may travel across two or more ridges 12.

[0155] The sensing data acquired by the underground radar device 140 can be used to estimate the self-position and create an underground map, as well as to acquire information about the growth of crops. For example, the thickness or density of roots depends greatly on the amount of water, nutrients, soil texture, etc. contained in the soil. Therefore, by measuring the diameter or density of roots from the sensing data, it becomes possible to grasp the growth status of crops. For example, the measured diameter or density of roots can be linked to the amount of water, nutrients, soil texture, etc., and managed centrally on the cloud.

[0156] The method for estimating self-location and the method for creating an underground map according to the present embodiment can be implemented in various types of agricultural machines traveling in fields where the underground parts of crops are formed underground, or in driving assistance systems for agricultural machines. Since the underground parts including the roots still remain in the ground of a field after the crops are harvested by a harvester or the like, it is possible to estimate self-location and create an underground map using information on the underground parts even after the crops have been harvested.

[0157] A driving assistance system according to an embodiment of the present disclosure is a system for a work vehicle (or agricultural machine) 100 traveling in a field where crops are planted. The driving assistance system may include an underground radar device 140 configured to radiate electromagnetic waves to the ground and receive reflected waves from the ground, and to acquire sensing data including information about the underground part of the crop underground, and a processing device 161 that estimates the position of the work vehicle 100 by referring to the relative displacement of the work vehicle 100 with respect to the underground part while traveling in the field. According to the driving assistance system, for example, even in an environment where positioning by GNSS is difficult, it is possible to realize automatic steering or autonomous movement of the work vehicle 100 by utilizing the sensing data including information about the underground part or root system of the crop underground acquired by the underground radar device 140.

[0158] The work vehicle 100 may travel in the field along a target route P. The processing device 161 processes the sensing data to determine the direction and magnitude of the relative displacement of the work vehicle 100 with respect to the underground portion, and estimates the relative position of the work vehicle 100 with respect to a reference point associated with the target route P based on the direction and magnitude of the displacement. With this configuration, it is possible to realize automatic steering or autonomous movement of the work vehicle 100 based on the estimated relative position of the work vehicle 100 with respect to the reference point associated with the target route P.

[0159] The driving assistance system may include an underground radar device 140 configured to radiate electromagnetic waves to the ground and receive reflected waves from the ground, and to acquire sensing data including information on the underground parts of crops in the ground, and a processing device 161 that creates an underground map including the underground parts based on the sensing data. Assume that the underground radar device 140 acquires a plurality of sensing data each including information on the underground parts of crops in the same field. In this case, the processing device 161 may create an underground map based on the latest sensing data acquired latest among the plurality of sensing data. According to the driving assistance system, the processing device 161 can create an underground map that reflects the latest information on the distribution of the underground parts. As a result, even if the roots have grown, self-location estimation can be realized using the latest underground map.

[0160] The work vehicle 100 can travel along a target route in a farm field. The processing device 161 can estimate the relative position of the work vehicle 100 on the underground map based on the sensing data acquired by the underground radar device 140. This makes it possible to estimate the relative position of the work vehicle 100 on the underground map.

[0161] The driving assistance system may include a control device 180 that controls the traveling of the work vehicle 100 based on information on the relative position of the work vehicle 100 and the target route P. This provides a driving assistance system that further includes the control device 180 that controls the traveling of the work vehicle 100 in addition to the processing device 161.

[0162] The processing device 161 can calculate the position deviation between the target route P and the relative position of the work vehicle 100. In this case, the control device 180 can use the position deviation to control the traveling of the work vehicle 100. This realizes the traveling control of the work vehicle 100 based on the position deviation.

[0163] The driving assistance system may further include an orientation measurement device that measures the orientation of the work vehicle 100 with respect to a reference line along the target route P. An example of the orientation measurement device is an IMU. The processing device 161 may calculate an orientation deviation indicating the angular difference between the orientation of the work vehicle 100 and the direction of the reference line based on the output from the orientation measurement device. In this case, the control device 180 may control the traveling of the work vehicle 100 using the position deviation and the orientation deviation. This realizes traveling control of the work vehicle 100 that takes into account the orientation deviation in addition to the position deviation.

[0164] The processing device 161 may apply correction parameters determined from the soil conditions in the field to the sensing data to correct the sensing data. This makes it possible to accurately estimate the position or shape of the underground part in the ground by using the preprocessed sensing data.

[0165] The processing device 161 can obtain information on the growth of the crop based on the sensing data. For example, by measuring the diameter or density of the roots from the sensing data, it becomes possible to grasp the growth status of the crop.

[0166] The underground radar device 140 may be provided at the front of the work vehicle 100. In this case, the driving assistance system may further include another underground radar device provided at the rear of the work vehicle 100. With this configuration, it is possible to obtain more sensing data from the two underground radar devices, thereby obtaining more information about the underground portion in the ground. As a result, it is possible to accurately estimate the position or shape of the underground portion in the ground.

[0167] When the work vehicle 100 travels through the field within a specified period from the time when the underground map is created based on the latest sensing data, the processing device 161 can estimate the relative position of the work vehicle 100 on the underground map based on the sensing data acquired by the underground radar device 140.

[0168] When the work vehicle 100 travels through a field after a predetermined period of time has elapsed since the underground map was created based on the latest sensing data, the processing device 161 can update the underground map based on the sensing data acquired by the underground radar device 140.

[0169] Alternatively, when the work vehicle 100 travels through a field after a predetermined period of time has elapsed since the underground map was created based on the latest sensing data, the processing device 161 can estimate the position of the work vehicle 100 by referring to the relative displacement of the work vehicle 100 with respect to the underground portion while traveling through the field.

[0170] According to such a configuration, for example, during periods when crop roots are not growing, the processing device 161 reuses the underground map, and during periods when crops are growing, the processing device 161 can update the underground map or, without using the underground map, estimate its own position by referring to the relative displacement of the work vehicle 100 with respect to the underground part while traveling in the field.

[0171] The work vehicle 100 may be equipped with the driving assistance system described above and a drive unit that drives the work vehicle 100. The work vehicle 100 may further be equipped with a work implement 300 that performs ground work on the ground. In this case, the underground radar device 140 may be attached to the work implement 300. For example, such a configuration is advantageous when it is difficult to attach the underground radar device to the rear of the vehicle body due to interference with the work implement 300.

[0172] This specification discloses the solutions described in the following items.

[0173] [Item 1] A driving assistance system for an agricultural machine traveling in a field planted with crops, comprising: a subsurface radar device configured to radiate electromagnetic waves to the ground and receive reflected waves from the ground, and to obtain sensing data including information about an underground portion of the crop in the ground; a processing device that estimates a position of the agricultural machine by referring to a relative displacement of the agricultural machine with respect to the underground portion while the agricultural machine is traveling in the field; A driving assistance system equipped with

[0174] [Item 2] The agricultural machine travels through the field along a target route; The processing device includes: Processing the sensing data to determine a direction and a magnitude of a relative displacement of the agricultural machine with respect to the underground portion; 2. A driving assistance system as described in item 1, which estimates a relative position of the agricultural machine with respect to a reference point associated with the target route based on a direction and magnitude of the displacement.

[0175] [Item 3] A driving assistance system for an agricultural machine traveling in a field planted with crops, comprising: a subsurface radar device configured to radiate electromagnetic waves to the ground and receive reflected waves from the ground, and to obtain sensing data including information about an underground portion of the crop in the ground; A processing device that creates an underground map including the underground area based on the sensing data; Equipped with A driving assistance system in which, when the underground radar device acquires multiple pieces of sensing data, each of which includes information about the underground parts of crops in the same field, the processing device creates the underground map based on the latest sensing data acquired latest among the multiple pieces of sensing data.

[0176] [Item 4] The agricultural machine travels through the field along a target route; 4. The driving assistance system according to item 3, wherein the processing device estimates a relative position of the agricultural machine on the underground map based on the sensing data acquired by the underground radar device.

[0177] [Item 5] 5. A driving assistance system according to item 2 or 4, comprising a control device that controls traveling of the agricultural machine based on information on the relative position of the agricultural machine and the target route.

[0178] [Item 6] The processing device calculates a position deviation between the target path and a relative position of the agricultural machine, 6. The driving assistance system according to item 5, wherein the control device controls the traveling of the agricultural machine using the position deviation.

[0179] [Item 7] Further comprising a direction measuring device that measures a direction of the agricultural machine with respect to a reference line along the target route, the processing device calculates an orientation deviation indicating an angular difference between a direction of the agricultural machine and a direction of the reference line based on an output from the orientation measurement device; 7. The driving assistance system according to item 6, wherein the control device controls the traveling of the agricultural machine using the position deviation and the orientation deviation.

[0180] [Item 8] 8. The driving assistance system according to any one of items 1 to 7, wherein the processing device applies a correction parameter determined from a soil condition in the field to the sensing data to correct the sensing data.

[0181] [Item 9] 9. A driving assistance system according to any one of items 1 to 8, wherein the processing device acquires information regarding the growth of the crop based on the sensing data.

[0182] [Item 10] the ground penetrating radar device is provided at a front portion of the agricultural machine, 10. The driving assistance system according to any one of items 1 to 9, further comprising another ground penetrating radar device provided at a rear of the agricultural machine.

[0183] [Item 11] 4. The driving assistance system according to claim 3, wherein when the agricultural machine travels through the field within a predetermined period from the time when the underground map is created based on the latest sensing data, the processing device estimates a relative position of the agricultural machine on the underground map based on the sensing data acquired by the underground radar device.

[0184] [Item 12] Item 4. The driving assistance system according to item 3, wherein when the agricultural machine travels through the field after a predetermined period of time has elapsed since the underground map was created based on the latest sensing data, the processing device updates the underground map based on the sensing data acquired by the underground radar device.

[0185] [Item 13] Item 4. The driving assistance system according to item 3, wherein when the agricultural machine travels through the field after a predetermined period of time has elapsed since the underground map was created based on the latest sensing data, the processing device refers to the relative displacement of the agricultural machine with respect to the underground portion while traveling through the field and estimates the position of the agricultural machine.

[0186] [Item 14] A driving assistance system according to any one of items 1 to 13, A drive unit for driving the agricultural machine; Agricultural machinery equipped with

[0187] [Item 15] A work machine for performing ground work on the ground is provided, Item 15. The agricultural machine according to item 14, wherein the ground penetrating radar device is attached to the work machine. [Industrial Applicability]

[0188] The technology of the present disclosure can be applied to the operation of agricultural machinery such as a tractor, a harvester, a rice transplanter, a riding tiller, a vegetable transplanter, a grass cutter, a seeding machine, a fertilizer applicator, or an agricultural robot. [Explanation of symbols]

[0189] 50 GNSS satellite, 60 Reference station, 70 Field, 72 Working area, 74 Headland, 76 Road, 80 Network, 100 Work vehicle, 101 Vehicle body, 102 Prime mover (engine), 103 Transmission, 104 Wheels, 105 Cabin, 106 Steering device, 107 Driving Seat, 108... coupling device, 110... positioning device, 111... GNSS receiver, 112... RTK receiver, 115... inertial measurement unit (IMU), 116... processing circuit, 120... camera, 130... obstacle sensor, 140... underground radar device, 141... antenna unit, 150... sensor group, 152... steering wheel sensor, 154 156...turning angle sensor, 160...rotation sensor, 161...processing device, 170...storage device, 180...control device, 181-185...ECU, 190...communication device, 200...operation terminal, 210...operation switch group, 220...buzzer, 240...driving device, 250...receiver, 300...working machine, 340...drive operating device, 380...control device, 390...communication device, 400...terminal device, 420...input device, 430...display device, 450...storage device, 460...processor, 470...ROM, 480...RAM, 490...communication device, 600...management device, 660...processor, 670...storage device, 670...ROM, 680...RAM

Claims

1. A driving assistance system for an agricultural machine traveling in a field planted with crops, comprising: a subsurface radar device configured to radiate electromagnetic waves to the ground and receive reflected waves from the ground, and to obtain sensing data including information about an underground portion of the crop in the ground; a processing device that estimates a position of the agricultural machine by referring to a relative displacement of the agricultural machine with respect to the underground portion while the agricultural machine is traveling in the field; A driving assistance system equipped with

2. The agricultural machine travels through the field along a target route; The processing device includes: Processing the sensing data to determine a direction and a magnitude of a relative displacement of the agricultural machine with respect to the underground portion; The driving assistance system according to claim 1 , further comprising: estimating a relative position of the agricultural machine with respect to a reference point associated with the target route based on a direction and a magnitude of the displacement.

3. A driving assistance system for an agricultural machine traveling in a field planted with crops, comprising: a subsurface radar device configured to radiate electromagnetic waves to the ground and receive reflected waves from the ground, and to obtain sensing data including information about an underground portion of the crop in the ground; A processing device that creates an underground map including the underground area based on the sensing data; Equipped with A driving assistance system in which, when the underground radar device acquires multiple pieces of sensing data, each of which includes information about the underground parts of crops in the same field, the processing device creates the underground map based on the latest sensing data acquired latest among the multiple pieces of sensing data.

4. The agricultural machine travels through the field along a target route; The driving assistance system according to claim 3 , wherein the processing device estimates a relative position of the agricultural machine on the underground map based on the sensing data acquired by the underground radar device.

5. 5. The driving assistance system according to claim 2, further comprising a control device that controls traveling of the agricultural machine based on information on the relative position of the agricultural machine and the target route.

6. The processing device calculates a position deviation between the target path and a relative position of the agricultural machine, The driving assistance system according to claim 5 , wherein the control device controls traveling of the agricultural machine using the position deviation.

7. Further comprising a direction measuring device that measures a direction of the agricultural machine with respect to a reference line along the target route, the processing device calculates an orientation deviation indicating an angular difference between a direction of the agricultural machine and a direction of the reference line based on an output from the orientation measurement device; The driving assistance system according to claim 6 , wherein the control device controls traveling of the agricultural machine by using the position deviation and the orientation deviation.

8. The driving assistance system according to claim 1 , wherein the processing device applies a correction parameter determined from a soil condition in the field to the sensing data to correct the sensing data.

9. The driving assistance system according to claim 1 , wherein the processing device acquires information about the growth of the crop based on the sensing data.

10. the ground penetrating radar device is provided at a front portion of the agricultural machine, The driving assistance system according to claim 1 , further comprising another ground penetrating radar device provided at a rear portion of the agricultural machine.

11. 4. The driving assistance system according to claim 3, wherein when the agricultural machine travels in the field within a predetermined period from a time point when the underground map is created based on the latest sensing data, the processing device estimates a relative position of the agricultural machine on the underground map based on the sensing data acquired by the underground radar device.

12. 4. The driving assistance system according to claim 3, wherein when the agricultural machine travels through the field after a predetermined period of time has elapsed since the underground map was created based on the latest sensing data, the processing device updates the underground map based on the sensing data acquired by the underground radar device.

13. 4. The driving assistance system according to claim 3, wherein when the agricultural machine travels through the field after a predetermined period of time has elapsed since the underground map was created based on the latest sensing data, the processing device refers to a relative displacement of the agricultural machine with respect to the underground portion while traveling through the field, and estimates a position of the agricultural machine.

14. A driving assistance system according to any one of claims 1 to 4, A drive unit for driving the agricultural machine; Agricultural machinery equipped with

15. A work machine for performing ground work on the ground is provided, The agricultural machine according to claim 14, wherein the ground penetrating radar device is attached to the work machine.

Images