Vegetation detection system, vegetation detection method, and program

Abstract

To provide a vegetation detection system that can accurately determine vegetation even in environments and situations where accurate vegetation detection is difficult. [Solution] The vegetation detection system 1 comprises at least one radar device 10 and a vegetation determination unit 23 that determines whether a target is vegetation based on the dispersion of the point cloud related to the target detected by the radar device. The vegetation determination unit 23 determines whether a target 40 is vegetation based on the point cloud acquired by the radar device 10. The vegetation determination unit 23 comprises a coordinate transformation unit 231, a point cloud storage unit 232, a filter unit 233, a clustering unit 234, a depth direction dispersion determination unit 235, and a vegetation detailed information calculation unit 236.

Description

Technical Field

[0001] The present invention relates to a vegetation detection system, a vegetation detection method, and a program.

Background Art

[0002] There is a technology for detecting vegetation using electromagnetic waves such as LiDAR. For example, Patent Document 1 discloses a technology that performs principal component analysis on a measurement point group included in each voxel, and determines the presence or absence of vegetation in the voxel based on the analysis result.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, with conventional determination technologies, it is difficult to detect vegetation behind fog or dust, or vegetation behind leaves. In addition, with conventional determination technologies, it is difficult to distinguish between support posts and vegetation.

[0005] Accordingly, the present invention aims to provide a vegetation detection system that can accurately determine vegetation even in environments or situations where accurate vegetation detection is difficult.

Means for Solving the Problems

[0006] The vegetation detection system of the present invention includes at least one radar device, and a vegetation determination unit that determines that a target is vegetation based on the variance of a point group related to the target detected by the radar device.

Effects of the Invention

[0007] According to the present invention, a vegetation detection system can be provided that can accurately determine vegetation even in environments and situations where accurate vegetation detection is difficult. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows a schematic configuration of a detection device according to one embodiment of the present invention. [Figure 2] This is a block diagram showing the hardware configuration of a computer in a radar system. [Figure 3] This is a flowchart illustrating the general processing of the detection device in this embodiment. [Figure 4] This flowchart specifically illustrates the operation of the radar device detection process. [Figure 5] This figure shows the overall processing flow in the vegetation detection system of the present invention. [Figure 6] This figure shows the processing flow for dataset generation in the vegetation detection system of the present invention. [Figure 7] This figure shows the processing flow for vegetation determination in the vegetation detection system of the present invention. [Figure 8] This figure shows the conditions under which vegetation is identified in the vegetation detection system of the present invention. [Figure 9] This figure shows a situation in which the vegetation detection system of the present invention does not identify the area as vegetation. [Figure 10] This figure shows an image of how the radar device in the vegetation detection system of the present invention is attached to the side. [Figure 11] This figure shows an image of how the radar device in the vegetation detection system of the present invention is attached to the rear. [Figure 12] This figure shows an image of how the radar device in the vegetation detection system of the present invention is mounted on top. [Figure 13] This figure shows an image illustrating the multiple mounting configurations of radar devices in the vegetation detection system of the present invention. [Modes for carrying out the invention]

[0009] Hereinafter, a vegetation detection system according to an embodiment of the present invention will be described. It should be noted that the present invention is not limited by the following embodiments. Further, each drawing referred to in the following description only schematically shows shapes, sizes, and positional relationships to an extent that allows the content of the present invention to be understood. That is, the present invention is not limited only to the shapes, sizes, and positional relationships illustrated in each drawing.

[0010] Figure 1 is a diagram showing a schematic configuration of a vegetation detection system 1 according to the present embodiment. The vegetation detection system 1 includes a radar device 10 and a vegetation determination unit 23.

[0011] Although only one radar device 10 is illustrated in Figure 1, the vegetation detection system 1 may be provided with a plurality of radar devices. Further, the connection between each device may be configured by wire, or may be connected via a wireless network. In addition, the vegetation detection system 1 includes, as a control device, for example, one or more processors and the like, and includes, as a storage device, one or more memories, a hard disk drive and the like. Further, the radar device 10 and the vegetation determination unit 23 may be configured as separate bodies respectively.

[0012] In the present embodiment, as will be described later, the vegetation detection system 1 uses the radar device 10 to transmit transmission waves toward the surrounding environment of the moving object 2, receives reflected waves from targets existing in the surrounding environment of the moving object 2, processes the reflected waves to periodically acquire point cloud information around the moving object 2, and detects vegetation such as trees in the surroundings based on the point cloud information acquired by the radar device 10.

[0013] The radar device 10 includes a transmission antenna 11, a reception antenna 12, a signal generation unit 13, a transmission control unit 14, an oscillation unit 15, a switch 16, an ADC (Analog to Digital Converter) 17, a distance calculation unit 18, a speed calculation unit 19, a threshold setting unit 20, a detection unit 21, and an angle calculation unit 22. The radar device 10 adopts an FCM (Fast Chirp Modulation) scheme, but is not limited to the FCM scheme, and may be a pulse scheme.

[0014] The transmitting antenna 11 is composed of one or a plurality of antenna elements, and radiates the transmission wave S1 oscillated by the oscillation unit 15 toward the target 40. The receiving antenna 12 is composed of one or a plurality of antenna elements, and receives the electromagnetic wave R1 (reflected signal) reflected by the target 40.

[0015] The signal generation unit 13 generates a signal pattern of a periodic transmission signal. The transmission control unit 14 is connected to the signal generation unit 13 and controls the operation of the signal generation unit 13. The oscillation unit 15 is connected to the signal generation unit 13, and oscillates the transmission signal based on the signal pattern from the signal generation unit 13.

[0016] The ADC 17 is connected to the receiving antenna 12, and converts the signal received by the receiving antenna 12 into a digital signal. The oscillation unit 15 and the ADC 17 can be directly connected via the switch 16. The distance calculation unit 18 calculates the relative distance of the target 40 with respect to the radar apparatus 10 based on the converted digital signal.

[0017] The speed calculation unit 19 calculates the speed of the moving body 2 based on the converted digital signal. The threshold setting unit 20 sets a detection threshold to be used by the detection unit 21. The detection unit 21 detects a peak in the distance direction. The angle calculation unit 22 obtains the arrival angle (horizontal angle and elevation / depression angle) at the distance detected by the detection unit 21.

[0018] Note that the radar apparatus 10 according to the present embodiment adopts the FCM (Fast Chirp Modulation) scheme as an example as described above, but is not limited to the FCM scheme, and may be a pulse scheme. In addition, the radar apparatus 10 may be disposed at a lower portion of the moving body 2, may be disposed at an upper portion of the moving body 2, or may be disposed at both the upper portion and the lower portion of the moving body 2. Further, the moving body 2 on which the radar apparatus 10 is mounted may be an agricultural machine, a construction work machine, or a general automobile.

[0019] The vegetation determination unit 23 determines whether the target 40 is vegetation based on the point cloud acquired by the radar device 10. The vegetation determination unit 23 includes a coordinate transformation unit 231, a point cloud storage unit 232, a filter unit 233, a clustering unit 234, a depth-direction dispersion determination unit 235, and a vegetation detailed information calculation unit 236.

[0020] The coordinate transformation unit 231 transforms the point cloud information obtained by the radar device 10 from the coordinate system of the radar device 10 to the coordinate system of the center of the moving object 2. The point cloud storage unit 232 stores the point cloud information transformed by the coordinate transformation unit 231.

[0021] The filter unit 233 filters the point cloud information accumulated by the point cloud storage unit 232. Specifically, the filter unit 233 filters the point cloud information against at least one of the point cloud intensity information, point cloud velocity information, and point cloud three-dimensional position information acquired by the radar device 10, thereby removing point cloud information that would be noise for the vegetation detection process. The coordinate transformation unit 231, point cloud storage unit 232, and filter unit 233 perform data preprocessing in the vegetation determination unit 23. The clustering unit 234, depth direction dispersion determination unit 235, and vegetation detailed information calculation unit 236 perform vegetation determination.

[0022] The clustering unit 234 performs clustering on the point cloud information that has been filtered by the filtering unit 233.

[0023] The depth-direction dispersion determination unit 235 selects one cluster from the one or more clusters classified by the clustering unit 234, and uses information on the depth-direction dispersion of the point cloud in the selected cluster to determine that the cluster is vegetation.

[0024] The depth-direction dispersion determination unit 235 determines the depth-direction dispersion of each cluster as seen from the radar device 10 based on the clustering results, and determines whether or not the cluster is vegetation. The depth-direction dispersion determination unit 235 compares the determined dispersion with a predetermined threshold, and if the detected dispersion is greater than or equal to the threshold, it can be determined to be vegetation. The threshold can be adjusted according to the environment, for example, by lowering the threshold in environments with a lot of vegetation where the branch spread is small. The information regarding the intensity of the point cloud described above is the RCS (Radar Cross Section), which indicates the reflectivity of the target to the incident transmitted wave.

[0025] The storage device 30 is composed of one or more memory devices, hard disk drives, etc., and stores various information in the vegetation detection system 1. For example, the storage device 30 stores point clouds acquired by the radar device 10.

[0026] The vegetation determination unit 23 performs the functions of each part by describing the functions of each part in a program and executing it using a computer equipped with a processor and storage device. However, each part may be configured with individual hardware and software. The program is stored in the computer's storage device. This computer also includes a communication unit that performs data communication with the vegetation determination unit, etc., and an input / output unit that constitutes a data input / output interface to the vegetation detection system 1.

[0027] Figure 2 is a block diagram showing the hardware configuration of the computer in the radar device 10. As shown in Figure 2, the radar device 10 comprises a radar device control unit 100, an input / output unit 106, a communication unit 107, and a storage unit 108. The radar device control unit 100 has a processor 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a bus 104, and an input / output interface 105. The radar device control unit 100 of the radar device 10 may be a general-purpose personal computer capable of performing various functions by installing various programs, or it may be a computer embedded in dedicated hardware.

[0028] The processor 101 performs various calculations and processes. The processor 101 is, for example, a CPU (central processing unit), MPU (micro processing unit), SoC (system on a chip), DSP (digital signal processor), GPU (graphics processing unit), ASIC (application specific integrated circuit), PLD (programmable logic device), or FPGA (field-programmable gate array). Alternatively, the processor 101 is a combination of several of these. Furthermore, the processor 101 may be a combination of these with hardware accelerators, etc.

[0029] The processor 101, ROM 102, and RAM 103 are interconnected via a bus 104. The processor 101 performs various processes according to the program recorded in ROM 102 or the program loaded into RAM 103. Part or all of the program may be incorporated into the circuitry of the processor 101.

[0030] Bus 104 is also connected to input / output interface 105. The input / output unit 106, the communication unit 107, and the storage unit 108 are connected to input / output interface 105.

[0031] The input / output unit 106 is electrically connected to the input / output interface 105 by wire or wireless connection. The input / output unit 106 consists of, for example, an input unit such as a keyboard and mouse, and an output unit such as a display for displaying captured images and a speaker for amplifying sound. The input / output unit 106 may also have an integrated configuration of display and input functions, such as a touch panel.

[0032] The communication unit 107 is a device for the processor 101 to communicate with other devices. The communication unit 107 is, for example, a short-range communication means such as Wi-Fi. Other devices include, for example, a vegetation detection system 1 and a vegetation determination unit (not shown). These devices are controlled by the radar device control unit 100 via the communication unit 107. However, these devices may also be controlled by the radar device control unit 100 via a wired connection without going through the communication unit 107. The storage unit 108 is, for example, a storage device such as a hard disk drive (HDD) or a solid-state drive (SSD).

[0033] The hardware configuration shown in Figure 2 is merely an example and is not limited to this configuration. In addition to being composed of various processing units such as single processors, multi-processors, and multi-core processors, a combination of these various processing units and processing circuits such as ASICs (Application Specific Integrated Circuits) and FPGAs (Field-Programmable Gate Arrays) may be adopted to realize a functional processor configuration. The radar device 10 may not have a storage unit 108, but rather a configuration in which a separate storage unit 108 is provided may be adopted.

[0034] Figure 3 is a flowchart illustrating the overview of the processing of the vegetation detection system 1 in this embodiment. Through the processing illustrated in Figure 3, the vegetation detection system 1 determines that each cluster detected by the radar device 10 is vegetation. The processing by the vegetation detection system 1 is started, for example, by activating the mobile unit 2 and supplying power to the vegetation detection system 1.

[0035] In step S1, the vegetation determination unit 23 acquires radar device mounting information, such as the position, height, and angle of the radar device 10 relative to the ground, from the mobile body 2's storage device (not shown). This radar device mounting information is used in the coordinate transformation described later.

[0036] In step S2, the radar device 10 periodically detects detection points on the target 40 around the moving object 2 and provides detection information to the vegetation determination unit 23 (radar device detection processing (described later with respect to Figure 4)). The detection information includes the relative velocity between the target 40 and the radar device 10, three-dimensional position information including the height of the target 40 at each detection point, and the received intensity of the reflected wave. The detection information is stored chronologically in the storage device (not shown) of the vegetation detection system 1 for each frame corresponding to each period.

[0037] In step S3, the coordinate transformation unit 231 acquires point cloud information detected by the radar device 10. The point cloud information includes three-dimensional position information (position information in the x, y, and z directions) with the position of the radar device 10 as the origin, as well as angle, amplitude, and velocity. Next, the coordinate transformation unit 231 transforms the center coordinates of the point cloud information from the coordinate system of the radar device 10 to the coordinate system of the mobile object 2. Specifically, the coordinate transformation unit 231 uses the radar device mounting information acquired in step S1 to transform the position information of the point cloud from the coordinate system of the radar device 10 to the coordinate system of the center of the mobile object 2.

[0038] Specifically, if one radar device 10 is mounted on the left side and one on the right side of the moving object 2, in order to identify that the targets 40 in the overlapping detection areas detected by each of the two radar devices 10 are the same, it is necessary to convert the positional information of the point clouds detected by each device into a common coordinate system.

[0039] For example, the positional information of the point clouds detected by each of the two radar devices 10 is represented in a polar coordinate system consisting of distance and azimuth, and a common coordinate system is represented in a Cartesian coordinate system (xy coordinate system) with the midpoint between the two radar devices 10 as the origin. Note that the coordinate systems used by each radar device 10 and the common coordinate system are not limited to these and can be arbitrarily selected.

[0040] The coordinate transformation unit 231 transforms the point cloud position information detected by each of the two radar devices 10 into a coordinate system common to both radar devices 10. In the example above, the point cloud position information in the polar coordinate system set for each of the two radar devices 10 is transformed into point cloud position information in a common Cartesian coordinate system. As a result, the point cloud position information detected by each of the two radar devices 10 is transformed into point cloud position information (x,y) in a Cartesian coordinate system.

[0041] The coordinate transformation unit 231 corrects the position information (x,y) of the transformed Cartesian coordinate system point cloud to the coordinate system (Cartesian coordinate system) of the center of the moving object 2 (the centroid of the moving object 2).

[0042] Furthermore, the coordinate transformation unit 231 adjusts the point cloud position information (x,y) using the radar device mounting information acquired in step S1 described above. As a result, the point cloud position information (x,y) in the Cartesian coordinate system takes into account the position, height, and angle with respect to the ground where the radar device 10 is mounted on the mobile body 2. The above description explains the coordinate transformation when one radar device 10 is mounted on the left and one on the right side of the mobile body 2, but it is not limited to this and can also be applied when only one radar device 10 is mounted on the front side of the mobile body 2. In addition, the coordinate transformation unit 231 may correct the point cloud position information by considering the attitude of the mobile body 2 using the position information of the mobile body 2 and IMU (Inertial Measurement Unit) information.

[0043] The point cloud storage unit 232 acquires and stores three-dimensional position information, intensity information, and velocity information for the point cloud information transformed by the coordinate transformation unit 231. The acquired three-dimensional position information, intensity information, and velocity information may be for just one frame of the point cloud information, or it may be for multiple frames. The point cloud storage unit 232 may also determine the number of frames of the point cloud information in advance and delete old frames each time a new frame is accumulated so that the number of frames is always predetermined. Furthermore, the point cloud storage unit 232 may acquire point cloud information while the moving object 2 is moving and store the acquired three-dimensional position information, intensity information, and velocity information for the point cloud information.

[0044] In step S4, the filter unit 233 performs filtering on the point cloud information using position information, amplitude information, height information, velocity information, RCS, etc. By filtering using height information and amplitude information, the filter unit 233 removes noise from the point cloud and can detect the target 40 with high accuracy. The amplitude filter may be set to a predetermined value, or a relative threshold may be created using the amplitude values ​​of the point clouds of surrounding points.

[0045] The filter unit 233 may use height information from the point cloud data to filter out point cloud data corresponding to relatively low positions in order to remove the influence of detected weeds, ground, etc. Alternatively, the filter unit 233 may use RCS to filter the point cloud data in order to remove factors that hinder accurate vegetation detection.

[0046] In step S5, the clustering unit 234 performs clustering on the point cloud information filtered by the filter unit 233 using an algorithm of a known clustering method such as DBSCAN (Density-based spatial clustering of applications with noise), and classifies it into one or more clusters.

[0047] The clustering unit 234 performs clustering on the point cloud information using either three-dimensional position information or two-dimensional position information. Furthermore, the clustering unit 234 may perform clustering on the point cloud information by adding intensity information and velocity information. In addition, the clustering unit 234 may determine the parameters of the clustering method from the extent of branch spread or, if the trees are arranged in rows, the spacing between rows of trees. By performing the clustering process as described above, the clustering unit 234 can separate the point cloud information into one cluster and classify it into one or more clusters.

[0048] In step S6, the depth-direction dispersion determination unit 235 determines that the target 40 is vegetation based on the clustered point cloud information.

[0049] In step S7, the depth-direction dispersion determination unit 235 determines that the target 40 is vegetation in the vegetation determination. If it is determined to be vegetation (step S7: YES), the process moves to step S8; if it is not determined to be vegetation (step S7: NO), the process returns to step S2.

[0050] In step S8, the vegetation detailed information calculation unit 236 calculates vegetation detailed information based on the point cloud information acquired by the radar device 10, and then terminates the process.

[0051] Next, the radar device detection process will be explained. Figure 4 is a flowchart that specifically shows the operation of the radar device detection process (step S2) in the processing flow of the vegetation detection system 1 illustrated in Figure 3. In step S11, the signal generation unit 13, the transmission control unit 14, and the oscillation unit 15 perform a transmission process in which they generate a transmission signal of a predetermined frequency and radiate a transmission wave from the transmitting antenna 11 toward the front of the moving body 2.

[0052] In step S12, the receiving antenna 12 receives the reflected wave, which is the transmitted wave reflected by one or more targets 40, as a received signal, and the ADC 17 performs reception processing to convert the received signal into a digital signal.

[0053] In step S13, the distance calculation unit 18 performs a distance calculation to determine the relative distance between the radar device 10 and the target 40 based on the digital signal.

[0054] In step S14, the velocity calculation unit 19 performs velocity calculations based on the digital signal. Specifically, the velocity calculation unit 19 calculates relative velocity information between one or more targets 40 and the radar device 10 based on the received signal. In this way, the velocity calculation unit 19 can detect the velocity relative to the target 40 through velocity calculations.

[0055] In step S15, the threshold setting unit 20 sets a detection threshold for detecting the peak to be used by the detection unit 21. Specifically, the threshold setting unit 20 adjusts the detection threshold based on the mounting position, height, angle, etc., of the receiving antenna 12.

[0056] In step S16, the detection unit 21 detects a peak in the distance direction using a predetermined method such as CFAR (Constant False Alarm Rate).

[0057] In step S17, the angle calculation unit 22 performs angle calculation processing to determine the arrival angle (horizontal angle and elevation angle) of the reflected wave at the distance detected by the detection unit 21.

[0058] In step S18, the radar device 10 corrects for any shift in the detection position caused by the radar device 10 tilting due to the effects of road irregularities, etc., by using IMU (Inertial Measurement Unit) information. The radar device 10 supplies the corrected detection information (i.e., point cloud information) to the vegetation determination unit 23 and terminates the process.

[0059] (Installation location of radar device 10) The installation location of the radar device 10 will be explained with reference to Figures 10 and 12. In the following explanation, a vehicle will be used as an example of a mobile object. As mentioned above, vehicles include, for example, agricultural machinery and general automobiles. The mobile object may also be something other than a vehicle, such as a drone. When the mobile object is a vehicle, for example, it travels among vegetation. The vegetation detection system 1 is particularly useful in places where vegetation is planted regularly.

[0060] Figure 10 is a plan view of the moving object 2 and the target 40. A plan view means viewing the object from a direction perpendicular to the ground. Arrow D10 in Figure 10 indicates a direction parallel to the ground. The direction of arrow D10 is defined as the horizontal direction D10. Arrow D1 in Figure 10 indicates the direction of movement of the moving object 2. The direction of arrow D1 is defined as the direction of movement D1.

[0061] In the example shown in Figure 10, the mobile body 2 moves along the alignment direction of the targets 40. The end of the mobile body 2 that is perpendicular to the direction of movement D1 and parallel to the horizontal direction D10 is defined as the mobile body side 2S. In the example shown in Figure 10, the radar device 10 is attached to the mobile body side 2S. The radar device 10 is attached to the side of the mobile body side 2S that faces the targets 40.

[0062] The illumination range B of the radar device 10 is shown as a sector-shaped area B. The azimuth angle R2 is defined as the spread angle of the illumination range B in the horizontal direction D10. The central axis D2 of the main beam of the radar device 10 is shown as a dashed line D2. The central axis D2 of the main beam is oriented perpendicular to the direction of movement D1 in a plan view.

[0063] Figure 12 shows the moving object 2 and the target 40 as viewed from a direction parallel to the ground G. Arrow D11 in Figure 12 indicates a direction perpendicular to the ground G. The direction of arrow D11 is defined as the up and down direction D11. As a reference for up and down, the side away from the ground G is considered up.

[0064] The upper end of the mobile body 2 is defined as the upper part 2T of the mobile body. In the example shown in Figure 12, the radar device 10 is attached to the upper part 2T of the mobile body. More specifically, the radar device 10 is attached to the upper part 2T of the mobile body on the side 2S of the mobile body. In other words, the radar device 10 is attached to the part where the side 2S of the mobile body and the upper part 2T of the mobile body intersect. Thus, the installation position of the radar device 10 can be on the side of the mobile body 2. More specifically, the installation position of the radar device 10 can be on the side and above the mobile body 2.

[0065] The radar device 10 is mounted so as to have a wide field of view in the elevation and depression direction so that it can detect the height direction of vegetation. This will be explained with reference to Figures 10 and 12. As shown in Figure 12, the elevation and depression angle R1 is defined as the spread angle D11 in the vertical direction of the illumination range B.

[0066] The radar device 10 is mounted such that its elevation angle R1 is greater than its azimuth angle R2. The elevation angle R1 can be, for example, 120 degrees. The azimuth angle R2 can be, for example, 30 degrees. The radar device 10 is usually mounted such that its azimuth angle R2 is greater than its elevation angle R1. In this embodiment, the radar device 10 is mounted such that its elevation angle R1 is greater than its azimuth angle R2. As a result, height information of the target 40 can be obtained.

[0067] Line D3 in Figure 12 is a line extending horizontally D10 from the radar device 10. Line D3 is defined as the radar device horizontal line D3. The range of the elevation angle R1 above the radar device horizontal line D3 is defined as the elevation angle R1T. The range of the elevation angle R1 below the radar device horizontal line D3 is defined as the depression angle R1D.

[0068] The radar device 10 is installed so that the elevation angle R1T is greater than the depression angle R1D. For example, if the elevation / depression angle R1 is 120 degrees, the elevation angle R1T can be set to 90 degrees and the depression angle R1D to 30 degrees. This allows for obtaining a wide range of height information about the target 40.

[0069] The installation location of the radar device 10 shown in Figure 10 is just one example. The radar device 10 can be mounted at various locations on the mobile body 2. Refer to Figures 11 and 13 to describe other mounting examples of the radar device 10. Figures 11 and 13 show the mobile body 2 and the target 40 in a plan view, similar to Figure 10.

[0070] Another mounting example of the radar device 10 shown in Figure 11 will be described. As shown in Figure 11, the end of the mobile body 2 opposite to the direction of movement D1 is defined as the rear part 2B of the mobile body. The radar device 10 may be mounted on the rear part 2B of the mobile body. More specifically, the radar device 10 may be mounted at the point where the rear part 2B of the mobile body and the side part 2S of the mobile body intersect. When the radar device 10 is mounted on the rear part 2B of the mobile body, the radar device 10 may be mounted so that the central axis D2 of the main beam is tilted in the direction of movement D1. Mounting the radar device 10 on the rear part 2B of the mobile body increases the degree of freedom in the placement of devices that can be provided on the mobile body 2.

[0071] Another mounting example of the radar device 10 shown in Figure 13 will be described. In the example shown in Figure 13, unlike the example shown in Figure 10, the radar device 10 is attached to both sides 2S of the mobile body 2. In Figure 13, the radar device 10 attached to one side 2S of the mobile body is shown as the first radar device 10A, and the radar device 10 attached to the other side 2S of the mobile body is shown as the second radar device 10B. In this other mounting example of the radar device 10 shown in Figure 13, vegetation can be efficiently detected, for example, when there are targets 40 on both sides of the mobile body 2's direction of movement D1.

[0072] (Point cloud detection) Point cloud detection by the radar device 10 will now be explained. The radar device 10 detects a point cloud related to the detection target within the illumination range B. Figure 8 is a diagram showing an example of detection points 300 by the radar device 10. Figure 8 shows the moving object 2 and the target 40 in a plan view, similar to Figure 10. The detection points 300 are points detected by the radar device 10 from the detection target within the illumination range B. A collection of multiple detection points 300 detected by the radar device 10 is defined as a point cloud. As shown in Figure 8, the radar device 10 acquires a point cloud within the illumination range B.

[0073] The radar device 10 can perform periodic processing in point cloud detection. Periodic processing means that the radar device 10 detects point clouds at regular intervals and processes those point clouds. The radar device 10 can detect point clouds on a moving object 2 while it is in motion. Therefore, it detects point clouds at regular intervals and combines multiple point clouds to perform vegetation determination, etc. This makes it possible to perform more accurate vegetation determination.

[0074] (Vegetation detection processing) The flow of the vegetation detection process will be explained with reference to Figures 5 to 7. Figure 5 is a diagram showing the overall processing flow in the vegetation detection system 1 of the present invention. Figure 5 is a processing flow that has been reconstructed from the flowchart illustrating the overview of the processing of the detection device shown in Figure 3, from the perspective of vegetation detection processing. Parameter setting S21 shown in Figure 5 corresponds to radar mounting information acquisition S1 in Figure 3, similarly, point cloud data reception S22 of the radar device in Figure 5 corresponds to radar detection processing S2 and three-dimensional point cloud information acquisition and storage S3 in Figure 3, and dataset generation S23 in Figure 5 is a process that includes filtering processing S4 in Figure 3.

[0075] S21 is a step in which the radar device 10 sets parameters for acquiring point clouds. Examples of parameters include the period for point cloud detection during periodic processing. The installation position, installation direction, and number of radar devices 10 may also be set as parameters.

[0076] S22 is the step in which the vegetation determination unit receives the point cloud data (point cloud information) detected by the radar device 10.

[0077] (Dataset generation) The dataset generation in S23 will be explained with reference to Figure 6. Figure 6 is a diagram showing the processing flow of dataset generation in the vegetation detection system of the present invention. The dataset generation in S23 includes steps S31 to S36. These will be explained in order below.

[0078] S31 is a step in which a coordinate transformation is performed. In S31, the radar device coordinates are transformed into reference coordinates. The radar device coordinates are coordinates in a coordinate system with the radar device 10 as the origin. These radar device coordinates are transformed into reference coordinates used for vegetation determination.

[0079] S4 is the step in which filtering is performed, as mentioned above.

[0080] Step S33 is the step of adding the data from the latest frame. In S33, when the radar device 10 detects point clouds at regular intervals, it acquires newly detected point cloud data (data from the latest frame) and adds it to the existing data. This allows for the accumulation of a predetermined number of frames in the dataset. Vegetation determination is then performed based on this dataset. There is no particular limit to the predetermined number of frames. The predetermined number of frames can be set appropriately according to the movement speed of the moving object and the point cloud detection period. For example, the predetermined number of frames can be between 3 and 5.

[0081] S34 is a step to determine whether the number of frames is within a predetermined number (predetermined number of frames). If the number of frames is within the predetermined number (S34, Yes), the step proceeds to S36. If the number of frames is not within the predetermined number (S34, No), the step proceeds to S35.

[0082] Step S35 is the step of deleting the oldest frame if the number of frames is not within a predetermined number. This deletes the data from the old frames, allowing subsequent vegetation determination to be performed using only the latest predetermined number of frames.

[0083] Step S36 is the step of generating a radar point cloud by merging a predetermined number of frames. In S36, a predetermined number of frames are merged to generate a dataset. In S36, the process of S3 shown in Figure 5 is completed.

[0084] (Vegetation assessment) The vegetation determination in S6 in Figure 5 will be explained with reference to Figure 7. Figure 7 is a flowchart showing the flow of vegetation determination in the vegetation detection system of the present invention. The vegetation determination in S6 includes steps S41 to S50. These will be explained in order below.

[0085] S41 is the step of performing clustering on the dataset (radar point cloud) generated in S36. Clustering can be performed using common methods such as KD-Tree.

[0086] S42 is the number of clusters generated n c This is the step to obtain the number of clusters obtained as a result of the clustering in S41.

[0087] Step S43 is the step where i=0. In S43, the index is initialized.

[0088] S44 is i <n c This is the step to determine if it is true. S44 is the step to determine the loop continuation condition. <n c If so, the step proceeds to S45 and the loop continues. <n c Otherwise, the step proceeds to S51 and the loop ends.

[0089] S45 is the depth-direction variance value of the point cloud within cluster i, dep v This is the step to calculate [i]. In S45, the variance value in the depth direction of the point cloud is calculated using the reference coordinates transformed in S31.

[0090] S46 is the depth-direction variance value dep of the point cloud within cluster i calculated in S45. v This step determines whether [i] is greater than or equal to a predetermined value. If the variance is greater than or equal to the predetermined value (S46, Yes), the step proceeds to S47. In S47, cluster i is determined to be vegetation. If the variance is not greater than or equal to the predetermined value (S46, No), the step proceeds to S48. In S48, cluster i is not determined to be vegetation. Note that the predetermined value of the variance in S46 can be set in advance, for example, based on the verification results of actual examples.

[0091] Thus, in the vegetation determination of this embodiment, vegetation is determined based on the dispersion of point clouds in the radar device 10. Specifically, based on the clustering results, the dispersion in the depth direction from the radar device's viewpoint of the point clouds included in each cluster is calculated, and vegetation is determined. Specifically, if the dispersion in the depth direction is greater than or equal to a predetermined threshold, vegetation is determined.

[0092] Step S49 is the step to calculate detailed information about the vegetation. If cluster i is vegetation, the steps proceed from S47 to S49. In S49, detailed information about the vegetation is calculated for the cluster determined to be vegetation. Detailed information about the vegetation includes, for example, the height of the vegetation, the spread of the branches, the amount of leaves, the depth of the trees, and the growth status.

[0093] Detailed vegetation information can also be used as indicators for various tasks. Detailed vegetation information can be calculated based on cluster depth, height, volume, point cloud spread, number of detected point clouds, point cloud density, and point cloud intensity information. Here, "cluster volume" refers to the volume obtained by the following steps (1) and (2): (1) Identify the endpoints of the cluster based on the clustering results. (2) The volume of the area enclosed by the identified endpoints is taken as the cluster volume. Note that the cluster volume is determined as needed, for example, when calculating detailed vegetation information based on volume. A map reflecting the calculated detailed vegetation information may also be created. This map can be used as a work map. Note that for point cloud intensity information, information on RCS (Radar Cross Section), which indicates the reflectivity of the target to the incident transmitted wave, may be used.

[0094] Step S50 is the step where i++ (incrementing index i) is performed. After S48 or S49, the step proceeds to S50. In S50, i++ is performed, increasing the loop processing counter by one. After S50, the step returns to S44 again. This process is repeated, and when the result of the determination in S44 is No, the vegetation determination S4 ends. As shown in Figure 5, when the vegetation determination S4 ends, the vegetation detection process also ends.

[0095] As described above, in the vegetation detection process of this embodiment, a dataset of a predetermined number of frames is accumulated and used for vegetation determination. The vegetation determination unit 23 determines whether each cluster is vegetation. By accumulating frames, the detection results of the radar device 10, which are superimposed in time, can be utilized.

[0096] Furthermore, vegetation determination may be performed considering a pre-created map. This map can be created in advance by considering the movement speed information of the moving object 2, or by using SLAM (Simultaneous Localization and Mapping).

[0097] The determination of whether or not the cluster in S46 is vegetation will be explained with reference to Figures 8 and 9. Figure 9 is a diagram showing the moving object 2 and the target 40 in a plan view, similar to Figure 8. As shown in S46, the vegetation detection system of this embodiment utilizes the transparency of the radar device and determines that the target detected by the radar device is vegetation based on the dispersion of the detection points of the radar device in the depth direction. Here, the depth direction refers to the direction away from the radar device 10 along the central axis D2 of the main beam.

[0098] The boxed area C1 in Figure 8 represents the first cluster C1, which was the target of evaluation in Figure 8. The boxed area C2 in Figure 9 represents the second cluster C2, which was the target of evaluation in Figure 9. It is assumed that the first cluster C1 includes vegetation, while the second cluster C2 does not include vegetation.

[0099] The detection point 300 included in the first cluster C1 is shown as the first detection point 301. The detection point 300 included in the second cluster C2 is shown as the second detection point 302. The distribution range of detection points 300 is defined as the point cloud range 32. The point cloud range 32 in the first cluster C1 is shown as the first point cloud range 321. The point cloud range 32 in the second cluster C2 is shown as the second point cloud range 322. Let L be the length of the point cloud range 32 in the direction parallel to the central axis D2 of the main beam. Let L1 be the length of the first point cloud range 321 in the direction parallel to the central axis D2 of the main beam. Let L2 be the length of the second point cloud range 322 in the direction parallel to the central axis D2 of the main beam.

[0100] A long L means that the distribution of detection points 300 in the depth direction of the radar device 10 is wide. Conversely, a short L means that the distribution of detection points 300 in the depth direction of the radar device is narrow. When the radar device 10 detects vegetation, L becomes large because vegetation has branches that include leaves. On the other hand, when the radar device 10 detects a support column, L becomes small because the column is thin and does not spread out in the depth direction.

[0101] L1 shown in Figure 8 is longer than L2 shown in Figure 9. This is because the first cluster C1 includes vegetation, while the second cluster C2 does not include vegetation but includes support structures.

[0102] In this way, based on the extent of the point cloud's distribution in the depth direction, the radar device 10 can determine that the detected target is vegetation. Therefore, in the vegetation detection system 1 of this embodiment, the variance value in the depth direction of the point cloud within a cluster is calculated as an indicator of the extent of the point cloud's distribution in the depth direction. If the variance value is greater than or equal to a predetermined value, the target is determined to be vegetation. Conversely, if the variance value is less than the predetermined value, the target is determined not to be vegetation.

[0103] Each function of the vegetation detection system 1 may be implemented through the collaboration of hardware and software. Alternatively, each function may be implemented using hardware alone. Here, hardware refers to, for example, electronic circuits.

[0104] This section describes how each function of the vegetation detection system 1 can be implemented using software. The functions of the vegetation detection system 1 can be implemented by a program that makes a computer function as the vegetation detection system 1. This program is designed to make each part of the vegetation detection system 1 function on the computer.

[0105] In this case, the vegetation detection system 1 includes a computer having at least one control device and at least one storage device as hardware for executing a program. An example of a control device is a processor. An example of a storage device is memory. By executing a program using the control device and storage device, each of the functions described in the embodiment is realized.

[0106] The program may not be temporary, but may be recorded on one or more computer-readable recording media. The recording media may be provided in the vegetation detection system 1. Alternatively, the recording media may be provided in a device other than the vegetation detection system 1. In the latter case, the program may be supplied to the vegetation detection system 1 via any wired or wireless transmission medium.

[0107] At least some of the functions of each component may be implemented by logic circuits. For example, an integrated circuit in which logic circuits functioning as components are formed is included within the scope of this disclosure.

[0108] As described above, the vegetation detection system 1 can be configured primarily with a computer and a computer program, as explained in Figure 2, for example, regarding the radar device control unit 100. The computer has a CPU, RAM, ROM, etc. The computer program works in conjunction with the computer to enable the computer to function as the vegetation detection system 1. By using a computer, the vegetation detection system 1 of this embodiment can be realized.

[0109] As described above, the vegetation detection system of this embodiment can correctly determine whether an object is vegetation or not based on the point cloud detected by the radar device. Examples of vegetation include trees growing in an environment where trees are arranged regularly. On the other hand, examples of non-vegetation include empty areas and support posts.

[0110] In this embodiment of the vegetation detection system, a radar device is used to detect objects deep within fog and leaves. This allows for more accurate detection. Specifically, a radar device that can be used robustly even in fog and environments where other sensors are weak (fog, dust, direct sunlight, nighttime) is used to determine that the detected object is vegetation. Furthermore, the transparency of the radar device is used to distinguish between vegetation and other obstacles.

[0111] Furthermore, in this embodiment of the vegetation detection system, the accuracy of the detection can be improved by installing the radar device so that it can view a wide range of elevation angles. In addition, if vegetation is detected, the growth status of that vegetation can be estimated.

[0112] Furthermore, the radar equipment used does not need to be specially designed. Therefore, the vegetation detection system of this embodiment can be applied to existing mobile devices. In addition, since radar equipment has a wider field of view than ultrasonic sensors, the number of radar equipment units that need to be installed can be reduced compared to the case of ultrasonic sensors.

[0113] The embodiments of the present invention have been described above. The present invention is not limited to the embodiments described above, and various modifications, variations, and combinations are possible.

[0114] One embodiment described above is a vegetation detection system 1 comprising at least one radar device 10 and a vegetation determination unit 23 that determines that a detected target is vegetation based on the dispersion of the point cloud detected by the radar device 10. Such a vegetation detection system can provide a system that can accurately determine vegetation even in environments where other sensors are weak, including fog, dust, direct sunlight, and nighttime.

[0115] In the vegetation detection system 1, the radar device 10 may be installed to enable detection over a wide range in the elevation and depression directions. Such a vegetation detection system allows for more accurate determination of vegetation. When installed to detect a wide range in the elevation and depression directions, the detection range in the elevation and depression directions may be wider than the detection range in the azimuth direction.

[0116] In the vegetation detection system 1, the radar device 10 may be mounted on the mobile body 2 in a direction intersecting the direction of movement of the mobile body 2. Such a vegetation detection system allows a mobile object to efficiently detect vegetation while it is in motion.

[0117] In the vegetation detection system 1, the vegetation determination unit 23 may determine that the detected target is vegetation if the dispersion of the point cloud in the depth direction as viewed from the radar device 10 is above a certain threshold. Such a vegetation detection system can more accurately determine whether or not something is vegetation.

[0118] In the vegetation detection system 1, the vegetation determination unit 23 may perform clustering on the dataset, and the vegetation determination unit 23 may perform the determination for each cluster obtained from the clustering process. This type of vegetation detection system can reliably determine whether or not a detected target is vegetation.

[0119] In the vegetation detection system 1, the vegetation determination unit 23 may estimate detailed vegetation information for clusters determined to be vegetation, including at least one of the following: size of the vegetation, growth status of the vegetation, density of the vegetation leaves, height of the vegetation branches, and spread of the vegetation branches. With such a vegetation detection system, it is possible to proceed with vegetation-related work based on the estimated detailed information.

[0120] In the vegetation detection system 1, the vegetation determination unit 23 may estimate the detailed information of the vegetation using at least one of the cluster shape, position, volume, point cloud density, number of detected point clouds, intensity information, and velocity information. Such vegetation detection systems allow for the accurate estimation of detailed information.

[0121] In the vegetation detection system 1, the vegetation determination unit 23 may create a dataset from a predetermined number of frames of data acquired from the radar device 10 and perform the determination using the dataset. Such a vegetation detection system can more accurately determine whether or not something is vegetation.

[0122] A vegetation detection method for detecting vegetation using a vegetation detection system comprising at least one radar device for detecting vegetation and a vegetation determination unit for determining that a target detected by the radar device is vegetation, the method comprising the steps of detecting the target with the radar device and determining that the target detected by the radar device is vegetation based on the dispersion of the point cloud detected by the radar device. This vegetation detection method provides a method for accurately determining vegetation even in environments and situations where accurate vegetation detection is difficult.

[0123] A vegetation detection system comprising at least one radar device for detecting vegetation and a vegetation determination unit for determining that a target detected by the radar device is vegetation, wherein a program causes a computer to perform the steps of detecting the target with the radar device and determining that the target detected by the radar device is vegetation based on the dispersion of the point cloud detected by the radar device. Such programs provide the ability to accurately determine vegetation even in environments and situations where accurate vegetation detection is difficult.

[0124] The vegetation detection system of the present invention comprises at least one radar device, The system includes a vegetation determination unit that determines that a target is vegetation based on the dispersion of the point cloud related to the target detected by the radar device. Such a vegetation detection system can accurately determine vegetation even in environments and situations where accurate vegetation detection is difficult.

[0125] In the vegetation detection system described above, the radar device may be installed so as to have a wide field of view in the elevation / depression direction so as to be able to detect the height direction of the vegetation. Such a vegetation detection system makes it easier to detect vegetation and more accurately determine whether or not something is vegetation.

[0126] In the vegetation detection system described above, the radar device may be mounted on the moving body in a direction intersecting the direction of movement of the moving body. Such a vegetation detection system makes it easier to determine whether a continuous area of ​​objects is vegetation.

[0127] In the vegetation detection system described above, the vegetation determination unit may determine that the target is vegetation if the dispersion of the point cloud related to the target in the depth direction as seen from the radar device is above a certain threshold. Such a vegetation detection system can accurately determine whether something is vegetation.

[0128] In the vegetation detection system described above, the vegetation determination unit may create a dataset from a predetermined number of point cloud data frames acquired from the radar device, and use the dataset to determine that the target is vegetation. Such a vegetation detection system allows for a more accurate determination of whether something is vegetation.

[0129] In the vegetation detection system described above, the vegetation determination unit may perform clustering on the dataset and determine that each of the clusters obtained from the clustering process is vegetation. Such a vegetation detection system can accurately determine whether something is vegetation.

[0130] In the vegetation detection system described above, the vegetation determination unit may estimate detailed information about the vegetation for the object determined to be vegetation, including at least one of the following: the size of the vegetation, the growth status of the vegetation, the density of the vegetation leaves, the height of the vegetation branches, and the spread of the vegetation branches. With such a vegetation detection system, if the target is vegetation, detailed information about that target can be obtained.

[0131] In the vegetation detection system described above, the vegetation determination unit may estimate the detailed information of the vegetation using at least one of the following: the shape, position, volume, point cloud density, number of detected point clouds, intensity information, and velocity information of the cluster. With such a vegetation detection system, if the target is vegetation, detailed information about that target can be obtained.

[0132] The vegetation detection method of the present invention comprises at least one radar device, A vegetation detection method for detecting vegetation using a vegetation detection system comprising a vegetation determination unit that determines that a target detected by the radar device is vegetation, The steps include detecting the target using the radar device, The method includes the step of determining that the target is vegetation based on the dispersion of the point cloud related to the target detected by the radar device. This vegetation detection method allows for accurate determination of vegetation even in environments and situations where accurate vegetation detection is difficult.

[0133] The present invention relates to a vegetation detection system comprising at least one radar device and a vegetation determination unit that determines that a target detected by the radar device is vegetation. On the computer, The steps include detecting the target using the radar device, The system is configured to perform the following steps: determine that the target is vegetation based on the dispersion of the point cloud related to the target detected by the radar device. Such programs allow for accurate vegetation detection even in environments and situations where accurate vegetation detection is difficult. [Explanation of Symbols]

[0134] 1. Vegetation detection system 2 Mobile Units 10 Radar equipment 21 Detection unit 23 Vegetation determination section 40 Target

Claims

1. At least one radar device, A vegetation detection system comprising: a vegetation determination unit that determines that a target is vegetation based on the dispersion of a point cloud related to a target detected by the radar device.

2. The vegetation detection system according to claim 1, wherein the radar device is installed so as to have a wide field of view in the elevation angle direction so as to be able to detect the height direction of the vegetation.

3. The vegetation detection system according to claim 1 or 2, wherein the radar device is mounted on a moving body in a direction intersecting the direction of movement of the moving body.

4. The vegetation detection system according to claim 1 or 2, wherein the vegetation determination unit determines that the target is vegetation if the dispersion of the point cloud relating to the target in the depth direction as seen from the radar device is greater than or equal to a certain threshold.

5. The vegetation detection system according to claim 1 or 2, wherein the vegetation determination unit creates a dataset from a predetermined number of point cloud data obtained from the radar device, and determines that the target is vegetation using the dataset.

6. The vegetation detection system according to claim 5, wherein the vegetation determination unit performs clustering on the dataset and determines that each of the clusters obtained by the clustering process is vegetation.

7. The vegetation detection system according to claim 6, wherein the vegetation determination unit estimates detailed information about the vegetation, including at least one of the following for the object that has been determined to be vegetation: the size of the vegetation, the growth status of the vegetation, the density of the leaves of the vegetation, the height of the branches of the vegetation, and the spread of the branches of the vegetation.

8. The vegetation detection system according to claim 7, wherein the vegetation determination unit estimates the detailed information of the vegetation using at least one of the shape, position, volume, point cloud density, number of detected point clouds, intensity information, and velocity information of the cluster.

9. At least one radar device, A vegetation detection method for detecting vegetation using a vegetation detection system comprising a vegetation determination unit that determines that a target detected by the radar device is vegetation, The steps include detecting the target using the radar device, A vegetation detection method comprising the step of determining that the target is vegetation based on the dispersion of the point cloud related to the target detected by the radar device.

10. A vegetation detection system comprising at least one radar device and a vegetation determination unit that determines that a target detected by the radar device is vegetation, On the computer, The steps include detecting the target using the radar device, A program that performs the steps of determining that the target is vegetation based on the dispersion of the point cloud related to the target detected by the radar device.

Images