Multi-sensor fusion detection fruit tree accurate targeting spraying method for ROS system

Abstract

The invention discloses a multi-sensor fusion detection fruit tree accurate targeting spraying method for an ROS system. The method comprises the following steps: data acquisition: acquiring laser radar data through a single-line laser radar, acquiring rotary encoder data through a rotary encoder, and acquiring Imu sensor data through an Imu sensor; data fusion of a three-dimensional point cloud set: fusing laser radar data, rotary encoder data and Imu sensor data to obtain the three-dimensional point cloud set of the fruit tree; segmenting the three-dimensional point cloud set according to an oblique triangular prism segmentation method; constructing a beveled triangular prism fruit tree canopy model; calculating the fruit tree canopy volume of the canopy area in the fruit tree according to the bevel triangular prism fruit tree canopy model; the fruit tree canopy volume is converted into the pesticide applying duty ratio of the canopy area in the fruit tree, and a spraying machine is guided to apply pesticide and spray. The method can accurately obtain the information and position of the fruit tree, is not liable to be interfered by a rain and fog environment, is high in target detection precision, can achieve precise spraying, and improves the pesticide utilization rate.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent control of agricultural orchard machinery and equipment, and in particular to a method for precise target spraying of fruit trees using multi-sensor fusion detection in a ROS system. Background Art

[0002] ROS (Robot Operating System) is a computer operating system architecture designed for robot software development. It is an open source meta-level operating system (post-operating system) that provides services similar to operating systems, including hardware abstract description, underlying driver management, execution of shared functions, inter-program message passing, and program distribution package management. It also provides some tools and libraries for acquiring, building, writing, and executing multi-machine fusion programs. Robots can improve work efficiency and are widely used. For example, they are used in agricultural orchards to perform tasks such as picking fruits and spraying pesticides.

[0003] Pesticide spraying is an important means of disease prevention and control in the production of orchard crops, and it is also the most effective and commonly used chemical control method in the field of agricultural plant protection. However, the existing robot pesticide application method usually only uses a single application amount for continuous single spraying operations in the operation area. In the absence of comprehensive information on the growth status of fruit trees and planting conditions, it often causes problems such as low pesticide utilization, excessive pesticide residues in agricultural products, and chemical environmental pollution. The application of sensors in precision pesticide application technology can solve the above problems well.

[0004] Precision pesticide application technology is inseparable from the precise target detection of sensors, especially when the distribution of orchard plants is uneven. The use of precise target detection technology can achieve more efficient pesticide application. At present, the existing target detection technology is mainly aimed at specific sensor technologies, such as acoustic sensors, infrared sensors and visual sensors; however, the orchard environment is complex and changeable. The use of visual sensor target detection technology has the disadvantages of long response time and great influence of rain and fog environment in the orchard operating environment due to the complexity of the algorithm. After being affected by rain and fog, the target detection accuracy decreases. The use of infrared sensors and ultrasonic sensors lacks the ability to obtain fruit tree information and is easily disturbed by rain and fog. In addition, the detection accuracy is low and the fruit tree information and location cannot be accurately obtained. Summary of the invention

[0005] The purpose of the present invention is to overcome the above-mentioned problems and provide a method for precise target spraying of fruit trees using multi-sensor fusion detection for a ROS system. The method can accurately obtain fruit tree information and location, is not easily disturbed by rain and fog environments, has high target detection accuracy, can achieve precise spraying, and improve pesticide utilization.

[0006] The purpose of the present invention is achieved through the following technical solutions:

[0007] A multi-sensor fusion detection method for fruit trees with precise target spraying for ROS system comprises the following steps:

[0008] (1) Data acquisition: The laser radar data is acquired through the single-line laser radar, the rotary encoder data is acquired through the rotary encoder, and the Imu sensor data is acquired through the Imu sensor;

[0009] (2) Data fusion 3D point cloud set: The lidar data, rotary encoder data and Imu sensor data are fused to obtain the 3D point cloud set of the fruit tree;

[0010] (3) Segment the 3D point cloud set according to the oblique truncated triangular prism segmentation method;

[0011] (4) Construct a canopy model of truncated triangular prism fruit trees;

[0012] (5) Calculate the canopy volume of the fruit tree in the middle canopy area of ​​the fruit tree based on the truncated triangular prism fruit tree canopy model;

[0013] (6) The canopy volume of the fruit tree is converted into the pesticide application duty ratio of the canopy area of ​​the fruit tree to guide the sprayer to spray the pesticide.

[0014] The working principle of the above-mentioned multi-sensor fusion detection method for precise target spraying of fruit trees is:

[0015] The single-line laser radar, rotary encoder and Imu sensor constitute a multi-sensor. Data is collected through the multi-sensor, and each data is fused into a three-dimensional point cloud set. Based on the three-dimensional point cloud set of the single-line laser radar, the oblique triangular prism fruit tree canopy model is constructed based on the oblique triangular prism segmentation method, so as to obtain the canopy volume of the fruit tree. By fusing the laser radar data, the rotary encoder data and the Imu sensor data, the three-dimensional point cloud set accumulated on both sides of the sprayer can be obtained during the driving process of the sprayer, that is, the three-dimensional point cloud data. By dividing the depth information of the point cloud data, it is distinguished whether the point cloud data is on the right or left side of the sprayer. After data fusion, according to the accumulated point cloud data, the oblique triangular prism fruit tree canopy model is constructed based on the oblique triangular prism segmentation method. The canopy volume of each fruit tree in the canopy area is calculated by the oblique triangular prism fruit tree canopy model. The canopy volume of multiple fruit trees in the canopy can be summed to obtain the total canopy volume of each fruit tree. The total volume of the truncated triangular prisms in the corresponding canopy area is divided by the total volume of the cuboids in the corresponding canopy area of ​​the fruit tree to obtain the PWM duty cycle of the canopy area of ​​each fruit tree, that is, the duty cycle of the pesticide application in the canopy area.

[0016] A preferred embodiment of the present invention, wherein, in step (1), the laser radar data is point cloud data; by dividing the depth information of the point cloud data, it is distinguished whether the point cloud data is on the right side or the left side of the sprayer.

[0017] Preferably, in step (1), the set of point cloud data obtained by scanning a certain data block by the single-line laser radar is P i =(P i,0 ,P i,1 ,...,P i,j ), where i is the data block number and j is the number of point clouds in the data block;

[0018] In the point cloud data set of each data block, there are j+1 scanning points in total. The point cloud data contains the scanning point distance information, scanning point intensity information and point cloud angle information. The depth information and height information of each scanning point can be obtained through the trigonometric function relationship between the scanning point distance information and the point cloud angle information.

[0019] In a single scanning cycle of a single-line laser radar, according to the angular range of the single-line laser radar scanning and the number of all scanning points in the scanning area, the angular spacing of each scanning point can be obtained, so that the angle set of the point cloud in a data block is α i =(α i,0 ,α i,1 ,…,α i,j );

[0020] The distance information of each scanning point of a single-line laser radar in a single cycle is defined as:

[0021] ρ i =(ρ i,0 ,ρ i,1 ,…,ρ i,j )

[0022] The depth information of each scanning point in a single cycle is:

[0023] d i =(d i,0 ,d i,1 ,…,d i,j )

[0024] Based on the target detection scanning system, the depth information of each scanning point in a single cycle is expressed as:

[0025] d i,j =ρ i,j *cosα i,j

[0026] The scanning field of view of the single-line laser radar corresponds to the front view of the sprayer body. i,j When it is greater than 0, it means that the point cloud is distributed on the left side of the sprayer. i,j When it is less than 0, it means that the point cloud is distributed on the right side of the sprayer.

[0027] Preferably, in step (2), during the spraying process on the ground, the sprayer applies pesticides to a single canopy of the fruit tree on one side. The point cloud data scanned by the single-line laser radar is on a two-dimensional plane, the rotary encoder data is the forward distance of the sprayer, and the Imu sensor data is the forward direction of the sprayer. The rotary encoder determines the accumulated point cloud data in the z-axis direction by recording the forward distance of the sprayer to form a three-dimensional point cloud set of the fruit tree. By measuring the distance between the sprayer and the trunks of the fruit trees on both sides and the forward direction detected by the Imu sensor, the point cloud data of each scanning cycle in the single-line laser radar is filtered, and the point cloud that does not belong to the forward direction point cloud detected by the current Imu and the point cloud outside the distance of the fruit tree trunk are filtered out, so that the formed three-dimensional point cloud set is valid data belonging to the current forward direction of the sprayer.

[0028] Preferably, in step (3) and step (4), the set of filtered point cloud data is defined as p′=(p 0 ,p 1 ,...,p j ); the set of point cloud data after trunk filtering of the nth scanning cycle of the single-line laser radar is recorded as:

[0029] p n ′=(p 0,n ,p 1,n ,…,p j,n )

[0030] A single point cloud data in the point cloud data set can be expressed as:

[0031] p j,n =9ρ j,n ,θ j,n ,z j,n )

[0032] Among them, ρ j,n is the distance of the jth scanning point in the nth scanning cycle, θ j,n is the angle information of the jth scanning point in the nth scanning cycle, z j,n The position of the sprayer when obtaining the jth scanning point in the nth scanning cycle;

[0033] According to the oblique truncated triangular prism segmentation method, the scanning point p j,n 、p j+1,n 、p j,n+1 The constructed triangle is the oblique section of a right triangular prism, and the distance between the fruit tree trunks is used as depth information to construct a truncated triangular prism differential model; the truncated triangular prism differential model is a truncated triangular prism fruit tree canopy model, and the volume of the truncated triangular prism is calculated through the truncated triangular prism fruit tree canopy model.

[0034] Preferably, in step (5) and step (6), in the vertical direction, the canopy of the fruit tree is divided into a plurality of regions from top to bottom, each region corresponding to a nozzle group at the end of the sprayer; for each region, a scanning point with the smallest depth information of the scanning point is found, and the depth information of the scanning point is recorded as d min , taking the scanning point as the critical point of each area, the canopy volume of fruit trees in the canopy area of ​​each area is calculated.

[0035] Preferably, one area corresponds to one rectangular volume block; after calculating the volume of the rectangular volume block corresponding to each canopy, the scanning points in each cycle of the single-line laser radar are classified according to the angle information according to the upper and lower limit ranges of the nozzle angles in the nozzle group set on the touch screen of the sprayer, and the truncated triangular prism volume of each canopy area in adjacent cycles is calculated by the truncated triangular prism fruit tree canopy model; then the sum of the truncated triangular prism volume of each canopy area is calculated, and the sum of the truncated triangular prism volume of each canopy area is the canopy volume of the fruit tree in each canopy area; the sum of the truncated triangular prism volume of each canopy area is divided by the volume of the corresponding rectangular volume block to obtain the pesticide application duty ratio of each canopy area; the calculation formula is as follows:

[0036]

[0037] Among them, the application duty cycle is the PWM duty cycle of the solenoid valve in the nozzle group, P represents the application duty cycle, V s It is expressed as the sum of the volumes of the obliquely truncated triangular prisms in each canopy area, V i The volume expressed as a cuboid volume.

[0038] Preferably, when the sprayer moves forward a distance of the sprayer body length, the solenoid valve nozzle group executes the spraying duty cycle of the corresponding detection position in the previous spraying cycle.

[0039] Preferably, when the single-line laser radar completes one spraying cycle detection, the spraying duty cycle of the solenoid valve nozzle group obtained from the previous round of corresponding position detection is executed to achieve spraying of the target at the periodic position of the sprayer.

[0040] Preferably, the single-line laser radar, rotary encoder and Imu sensor are all connected to an industrial computer; the industrial computer is connected to the solenoid valve nozzle group. The single-line laser radar, rotary encoder and Imu sensor collect data, and after processing by the industrial computer, according to the duty cycle of the canopy area, the solenoid valve nozzle group is controlled to achieve precise spraying.

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] 1. The multi-sensor fusion detection method for precise target spraying of fruit trees in the present invention can accurately obtain the information and position of fruit trees by fusion of laser radar data, rotary encoder data and Imu sensor data through the coordinated work of single-line laser radar, rotary encoder data and Imu sensor data. The single-line laser radar will not be disturbed by rain and fog environment, which can improve the target detection accuracy.

[0043] 2. The multi-sensor fusion detection method for precise targeted spraying of fruit trees in the present invention segments the three-dimensional point cloud set by the truncated triangular prism segmentation method. The canopy volume of each fruit tree canopy area can be accurately calculated based on the truncated triangular prism fruit tree canopy model. The canopy volume of the fruit tree is converted into the duty cycle of pesticide application that needs to be adjusted for each canopy area, which can achieve precise spraying and improve the utilization rate of pesticides. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 The present invention is a flow chart of a method for precise targeted spraying of fruit trees using multi-sensor fusion detection of ROS systems.

[0045] Figure 2 It is a schematic diagram of the structure of the target spray system in the present invention.

[0046] Figure 3 This is a scanning field diagram of the single-line laser radar in the present invention.

[0047] Figure 4 It is a schematic diagram of the target detection scanning system in the present invention.

[0048] Figure 5 It is a schematic diagram of the truncated triangular prism differential model (truncated triangular prism fruit tree canopy model) in the present invention.

[0049] Figure 6 It is a schematic diagram of the fruit tree canopy area division in the present invention.

[0050] Figure 7 This is a schematic diagram of the sprayer position for target application in the present invention. DETAILED DESCRIPTION

[0051] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention is further described below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0052] See also Figure 1-Figure 2 This embodiment discloses a method for multi-sensor fusion detection of fruit trees with precise target spraying for ROS system, comprising the following steps:

[0053] (1) Data acquisition: The laser radar data is acquired through the single-line laser radar, the rotary encoder data is acquired through the rotary encoder, and the Imu sensor data is acquired through the Imu sensor;

[0054] (2) Data fusion 3D point cloud set: The lidar data, rotary encoder data and Imu sensor data are fused to obtain the 3D point cloud set of the fruit tree;

[0055] (3) Segment the 3D point cloud set according to the oblique truncated triangular prism segmentation method;

[0056] (4) Construct a canopy model of truncated triangular prism fruit trees;

[0057] (5) Calculate the canopy volume of the fruit tree in the middle canopy area of ​​the fruit tree based on the truncated triangular prism fruit tree canopy model;

[0058] (6) The canopy volume of the fruit tree is converted into the pesticide application duty ratio of the canopy area of ​​the fruit tree to guide the sprayer to spray the pesticide.

[0059] See also Figure 1-Figure 2 The working principle of the above multi-sensor fusion detection method for precise target spraying of fruit trees is:

[0060] The single-line laser radar, rotary encoder and Imu sensor constitute a multi-sensor. Data is collected through the multi-sensor, and each data is fused into a three-dimensional point cloud set. Based on the three-dimensional point cloud set of the single-line laser radar, the oblique triangular prism fruit tree canopy model is constructed based on the oblique triangular prism segmentation method, so as to obtain the canopy volume of the fruit tree. By fusing the laser radar data, the rotary encoder data and the Imu sensor data, the three-dimensional point cloud set accumulated on both sides of the sprayer can be obtained during the driving process of the sprayer, that is, the three-dimensional point cloud data. By dividing the depth information of the point cloud data, it is distinguished whether the point cloud data is on the right or left side of the sprayer. After data fusion, according to the accumulated point cloud data, the oblique triangular prism fruit tree canopy model is constructed based on the oblique triangular prism segmentation method. The canopy volume of each fruit tree in the canopy area is calculated by the oblique triangular prism fruit tree canopy model. The canopy volume of multiple fruit trees in the canopy can be summed to obtain the total canopy volume of each fruit tree. The total volume of the truncated triangular prisms in the corresponding canopy area is divided by the total volume of the cuboids in the canopy area of ​​the corresponding fruit tree to obtain the PWM duty cycle of the canopy area of ​​each fruit tree.

[0061] See also Figure 1-Figure 3 In step (1), the laser radar data is point cloud data; by dividing the depth information of the point cloud data, it is distinguished whether the point cloud data is on the right side or the left side of the sprayer.

[0062] In this embodiment, the specific calculation process of the fruit tree canopy volume is:

[0063] See also Figure 3In step (1), the single-line laser radar is a 3i-T1 single-line laser radar. Assume that the set of point cloud data obtained by scanning a certain data block by the single-line laser radar is P i =(P i,0 ,P i,1 ,...,i i,j ), where i is the data block number and j is the number of point clouds in the data block; in this embodiment, the number of data blocks is 8, and the 8 data blocks are arranged counterclockwise in the scanning field of view of the single-line laser radar. The value range of i is 0-7. In this embodiment, the single-line laser radar has 144 scanning points per data block at a scanning frequency of 20 Hz, so the value range of j is 0-143.

[0064] See also Figure 4 In the point cloud data set of each data block, there are j+1 scanning points in total. For the point cloud data of the single-line laser radar, the scanning point distance information and scanning point intensity information are provided in the data message; in the scanning point distance information, since the point cloud data is evenly distributed in the 8 scanning areas, the angle information of the point cloud is implied in the scanning point distance sequence. The depth information and height information of each scanning point can be obtained through the trigonometric function relationship between the scanning point distance information and the angle information of the point cloud. The scanning system is established with the single-line laser radar at the installation position of the sprayer as the origin. The scanning coordinate system of the target detection is as follows: Figure 4 As shown in the figure, H is the installation height of the single-line laser radar set by the touch screen, ρ is the distance information of the scanning point, α is the angle information of the point cloud, d is the depth information of the scanning point, and h is the height information of the scanning point.

[0065] See also Figure 3-Figure 4 In a single scanning cycle of a single-line laser radar, according to the angle range of the single-line laser radar scanning and the number of all scanning points in the scanning area, the angle spacing of each scanning point can be obtained, so that the angle set of the point cloud in a certain data block is α i =(α i,0 ,α i,1 ,…,α i,j ); The angular range of the single-line laser radar scan is 270°, and 1152 scanning points are evenly distributed in the scanning area in a counterclockwise order, so the angular spacing of each scanning point is 270° / 1152=0.234375°; The angular relationship of each scanning point of the single-line laser radar in a single cycle can be expressed as:

[0066] α i,j =0.234375*(144i+j)-45

[0067] The distance information of each scanning point of a single-line laser radar in a single cycle is defined as:

[0068] ρi =(ρ i,0 ,ρ i,1 ,…,ρ i,j )

[0069] The depth information of each scanning point in a single cycle is:

[0070] d i =(d i,0 ,d i,1 ,…,d i,j )

[0071] The height information of the scanning point is:

[0072] h i =(h i,0 ,h i,1 ,…,h i,j )

[0073] Based on the target detection scanning system, the depth information and height information of each scanning point in a single cycle are expressed as:

[0074] d i,j =ρ i,j *cosα i,j

[0075] h i,j =H+ρ i,j *sinα i,j

[0076] In the above formula, when d i,j When it is greater than 0, it means that the scanning point is in the 0 to 3 data blocks of the single-line laser radar scanning area; when d i,j When it is less than 0, it means that the scanning point is in the 4th to 7th data block of the single-line laser radar scanning area. i,j The positive and negative values ​​of correspond to the point cloud information on the left and right sides of the single-line laser radar. The scanning field of view of the single-line laser radar corresponds to the front view of the sprayer body. i,j When it is greater than 0, it means that the point cloud is distributed on the left side of the sprayer. i,j When it is less than 0, it means that the point cloud is distributed on the right side of the sprayer.

[0077] In this embodiment, a truncated triangular prism fruit tree canopy model is constructed based on the truncated triangular prism segmentation method. The truncated triangular prism segmentation method is described below based on the target detection scanning system of this embodiment.

[0078] See also Figure 5In step (2), when the sprayer is applying pesticides on the ground, only the targeted spraying of the canopy on one side of the fruit tree is considered. First, the point cloud data (point cloud information) scanned by the single-line laser radar is on the two-dimensional plane of x, y. The rotary encoder data is the forward distance of the sprayer, and the Imu sensor data is the forward direction of the sprayer. The rotary encoder determines the accumulated point cloud data (point cloud information) in the z-axis direction by recording the forward distance of the sprayer to form a three-dimensional point cloud set of the fruit tree. By measuring the distance D between the sprayer and the trunks of the fruit trees on both sides, trunk The point cloud data of each scanning cycle in the single-line laser radar is filtered based on the forward direction detected by the Imu sensor, and the point cloud that does not belong to the forward direction point cloud detected by the current Imu and the point cloud outside the distance of the fruit tree trunk is filtered out to ensure that the three-dimensional point cloud data (three-dimensional point cloud set) of the fruit tree formed belongs to the forward direction of the current sprayer and is valid. Similarly, taking the point cloud of the fruit tree on the left side of the sprayer as an example, the point cloud data set p on the left side of the sprayer forward direction is defined as {p 1 ,p 2 ,p 3 ,p 4}, rewrite the set p as p = {p 0 ,p 1 ,…,p j}, where j is the point cloud number mark of the single-line laser radar single-side scanning area, and the value range is 0-575. If the depth information d of each scanning point in the set P is determined j >D trunk , filter out this type of point cloud, and define the set of filtered point cloud data as:

[0079] p′=(p 0 ,p 1 ,...,p j )

[0080] The point cloud data set after tree trunk filtering in the nth scanning cycle of the single-line laser radar is recorded as:

[0081] p n ′=(p 0,n ,p 1,n ,…,p j,n )

[0082] A single point cloud data in the point cloud data set can be expressed as:

[0083] p j,n =(ρ j,n ,θ j,n ,z j,n )

[0084] Among them, ρ j,n is the distance of the jth scanning point in the nth scanning cycle, θ j,nis the angle information of the jth scanning point in the nth scanning cycle, z j,n is the position of the sprayer when the jth scanning point is obtained in the nth scanning cycle. j,n , its angle information θ j,n In the target detection scanning system, it can be expressed as:

[0085] θ j,n =0.234375*j-45

[0086] For the position of the sprayer when the jth scanning point is obtained in the nth scanning cycle, if the sprayer operates at a speed of 0.5m / s, since the scanning frequency of the single-line laser radar is 20Hz, it can be concluded that the horizontal resolution of the adjacent scanning cycles of the single-line laser radar in the forward direction of the sprayer is 0.5 / 20=0.025m=2.5cm, that is, when the sprayer moves forward 2.5cm, the single-line laser radar completes a scanning cycle. It can be further concluded that the horizontal resolution of adjacent scanning points in a single scanning cycle is

[0087] 2.5 / 1152≈0.00217cm / point, that is, the sprayer acquires a scanning point every time it moves forward 0.00217cm. In this system, the horizontal distance between adjacent scanning points is ignored. Therefore, for scanning point p j,n =(ρ j,n ,θ j,n ,z j,n ), z j,n It can be simplified to z n , where the sprayer position z is taken at the first scanning point in each scanning cycle 1,n As z n The scan point p j,n =(ρ j,n ,θ j,n ,z j,n ) is converted to Cartesian coordinates, then p j,n The scan points can be expressed as:

[0088] p j,n =(ρ j,n *cosθ j,n ,H+ρ j,n *sinθ j,n ,z n )

[0089] See also Figure 5 In the oblique truncated triangular prism segmentation method, the scanning point p j,n ,p j+1,n ,p j,n+1The triangle is the oblique section of a right triangular prism. The distance between the trunks of the fruit trees is used as the depth information to construct a truncated triangular prism differential model. The truncated triangular prism differential model is a truncated triangular prism fruit tree canopy model. The volume of the truncated triangular prism is calculated by the truncated triangular prism fruit tree canopy model. Figure 5 shown.

[0090] See also Figure 5 In the calculation method of the volume of the oblique truncated triangular prism, the idea of ​​segmentation is used to calculate the volume. The oblique truncated triangular prism is divided into a right triangular prism and a pyramid. The volumes of the prism and pyramid are calculated respectively to obtain the volume of the oblique truncated triangular prism. In the process of solving the pyramid volume, the cross-section scanning point p j,n ,p j+1,n ,p j,n+1 Get the depth information d respectively ρ =ρ*cosθ, find the point with the largest depth information among the three scanning points and record it as ρ max , the following is a classification discussion on the method of calculating the volume of a pyramid:

[0091] (1) When there is only one point with the largest depth information, the segmented pyramid is a quadrangular pyramid. Figure 4 .7, when scanning point p j+1,n is the maximum point of depth information, and A′p is j,n p j,n+1 The volume of a tetrahedron with C' as a right trapezoid, where the height of the tetrahedron is the height H of the right triangle A'B'C' with A'C' as the base. The height H of the tetrahedron is calculated by A'B'*B'C'=A'C'*H, where the formulas for the lengths of the sides are as follows:

[0092] A′B′=ρ j+1,n *sinθ j+1,n -ρ j,n *sinθ j,n

[0093] B′C′=z n+1 -z n

[0094]

[0095] For the A′b′ side length formula, since the angular resolution of the laser radar is 0.234375°, sinθ j+1,n The sine value formula can be rewritten as:

[0096] sinθ j+1,n = sin(θ j,n +0.234375°)=sinθ j,n *cos0.234375°+cosθ j,n*sin0.234375°=sinθ j,n *0.999+cosθ j,n *0.004≈sinθ j,n Therefore, the formula for the length of the side A′B′ can be simplified to

[0097] A′B′=(ρ j+1,n -ρ j,n )sinθ j,n

[0098] Combining the B′C′ side length formula, A′C′ side length formula, and A′B′ side length simplified formula, we get the height of the tetrahedron as:

[0099]

[0100] For the right trapezoid A′p j,n p j,n+1 The area of ​​C', and the upper base A'p of the trapezoid are calculated respectively. j,n and the lower base p j,n+1 C′:

[0101] A′p j,n =ρ j+1,n cosθ j+1,n -ρ j,n cosθ j,n

[0102] p j,n+1 C′=ρ j+1,n cosθ j+1,n -ρ j,n+1 cosθ j,n+1

[0103] On the trapezoid, base A′p j,n and the lower base p j,n+1 In C′, since the scanning points of the same order in different scanning cycles have the same angle, cosθ j,n = cosθ j,n+1 , so according to the area formula of the right trapezoid S = (A′p j,n +p j,n+1 C′)*A′C′ / 2 and the volume formula of the pyramid The volume of the tetrahedron is:

[0104]

[0105] At this time, the volume of the right triangular prism ABC-A'B'C' is: (D 1 is the distance between the sprayer and one side of the tree trunk. 1 is the distance of the left trunk)

[0106]

[0107] Similarly, when scanning point p j,n+1 and scanning point p j,n When they are the maximum depth information points, the volume of the tetrahedron and the corresponding right-angled triangular prism are:

[0108]

[0109] (2) When there are two scanning points with the largest depth information, the segmented pyramid is a triangular pyramid. At this time, find the point with the smallest depth information among the three scanning points. When the scanning point p j+1,n When it is the minimum point of depth information, the volume of the segmented triangular pyramid and the volume of the right triangular prism are:

[0110]

[0111] Similarly, when scanning point p j,n+1 and scanning point p j,n When they are the minimum points of depth information, the volume of the triangular pyramid and the corresponding right-angled triangular prism are:

[0112]

[0113] (3) When there are three points with the largest depth information, that is, the differential model is a right-angled triangular prism, the volume of the triangular prism can be directly calculated as:

[0114]

[0115] Since the distance resolution of the point cloud in the single-line laser radar data packet is 2mm, it is difficult for the depth information of the three scanning points to be equal. Therefore, the first volume classification case is used as the model for calculation. According to the above formula, the volume differential model of the truncated triangular prism is obtained by adding the volumes of the cone and the triangular prism in the three cases respectively, and then merged into:

[0116]

[0117] See also Figure 1 and Figure 6 In step (5) and step (6), the calculation process of the application duty cycle in this embodiment is:

[0118] In the vertical direction, the canopy of the fruit tree is divided into multiple areas from top to bottom. In this embodiment, it is divided into 4 areas, and each area corresponds to the nozzle group at the end of the sprayer. For each canopy area, find the scanning point ρ with the smallest depth information of the scanning point, and record the depth information of the scanning point as d min, taking the scan point as the critical point of each canopy area, calculate the canopy volume of the fruit tree in each area. In the process of dividing the canopy of the fruit tree, one area corresponds to a rectangular volume block, that is, it is divided into four rectangular volume blocks. Taking the fruit tree on the left side of the sprayer as an example, combined with the above known quantities, the canopy is divided into four rectangular volume blocks, among which the volume of the rectangular volume block of the i-th area is:

[0119] V i =d min (tanθ 2 -tanθ 1 )(D 1 -d min )(z n+1 -z n )

[0120] Among them, θ 1 ,θ 2 are the lower and upper limits of the nozzle angles set in each nozzle group. Figure 6 shown.

[0121] See also Figure 1 and Figure 6 , after calculating the volume of the rectangular volume block corresponding to each canopy, the scanning points in each cycle of the single-line laser radar are classified according to the angle information according to the upper and lower limits of the nozzle angle in the nozzle group set on the touch screen of the sprayer. The truncated triangular prism volume of each canopy area in adjacent cycles is calculated through the truncated triangular prism fruit tree canopy model; then the sum of the truncated triangular prism volume of each canopy area is calculated, and the sum of the truncated triangular prism volume of each canopy area is the canopy volume of the fruit tree in each canopy area; the sum of the truncated triangular prism volume of each canopy area is divided by the volume of the corresponding rectangular volume block to obtain the pesticide application duty ratio of each canopy area; the calculation formula is as follows:

[0122]

[0123] Among them, the application duty cycle is the PWM duty cycle of the solenoid valve in the nozzle group, P represents the application duty cycle, V s It is expressed as the sum of the volumes of the obliquely truncated triangular prisms in each canopy area, V i The volume expressed as a cuboid volume.

[0124] See also Figure 1 and Figure 6After the sprayer starts the target spraying operation, the single-line laser radar detects the canopy information of the fruit trees in front of the sprayer. At this time, the target system calculates the sum of the truncated triangular prism volumes of the canopy areas in each area in the adjacent cycle according to the truncated triangular prism differential model, and calculates the spraying duty cycle of each canopy area. At this time, the nozzle group at the end of the sprayer waits for the spraying control to be executed. When the sprayer moves forward a distance of the sprayer body length, the solenoid valve nozzle group executes the spraying duty cycle of the corresponding detection position in the previous spraying cycle. According to the pre-set sprayer body length, select the appropriate sprayer operating speed and design the sprayer spraying control plan. In this embodiment, the sprayer operating speed is selected as v=0.5m / s. At this time, the sprayer obtains a scanning point every time it moves forward 0.00217cm. At this speed, the forward resolution of the scanning point of the single-line laser radar is high, which meets the calculation requirements of the truncated triangular prism differential model.

[0125] See also Figure 7 , when spraying the target at the sprayer position, by setting the sprayer body length L, the time it takes for the sprayer to move forward one sprayer body length is calculated as seconds, at a scanning frequency of 20Hz, the single-line laser radar collects a single-line point cloud every 0.05 seconds. Therefore, before each variable target application action is executed, the number of point cloud scanning lines collected by the single-line laser radar is 20t=40L. Therefore, in the engineering code, the point cloud in a sprayer application cycle is stored in the point cloud two-dimensional array targetFrame[40L]

[1152] , and the array row label is used as the sprayer target position record. When the single-line laser radar completes an application cycle detection, the application duty cycle of the solenoid valve nozzle group obtained from the previous round of corresponding position detection is executed to realize the sprayer's periodic position target application. The schematic diagram of the sprayer position target is shown in the figure. Figure 7 shown.

[0126] In the actual operation of the sprayer, in order to make the position alignment more accurate, the line mark of the point cloud array is corrected in real time, the data of the rotary encoder and Imu sensor are collected at a frequency of 20Hz, the pulse increment Δp and speed increment Δv of the rotary encoder between adjacent scanning cycles are calculated, and the coefficient k is corrected for the sprayer's target position line mark:

[0127]

[0128] See also Figure 2The multi-sensor fusion detection method for precise target spraying of fruit trees is applied to the target spraying system, which includes a single-line laser radar, a rotary encoder, an Imu sensor, an industrial computer and a solenoid valve nozzle group; the solenoid valve nozzle group includes a nozzle and a solenoid valve, and the solenoid valve is a PWM controlled solenoid valve; the single-line laser radar, the rotary encoder, the Imu sensor, and the PWM controlled solenoid valve are all connected to the industrial computer. The single-line laser radar, the rotary encoder and the Imu sensor collect data, and after processing by the industrial computer, according to the duty cycle of the canopy area, the solenoid valve nozzle group is controlled to achieve precise spraying.

[0129] The Imu sensor is the N100WP nine-axis Imu sensor; the main control and data acquisition are carried out using a Linux industrial computer.

[0130] The above is a preferred embodiment of the present invention, but the embodiment of the present invention is not limited to the above content. Any other changes, modifications, substitutions, combinations, and simplifications made without departing from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A multi-sensor fusion detection method for fruit trees with precise target spraying for ROS system, characterized in that: The following steps are involved: (1) Data acquisition: The laser radar data is acquired through the single-line laser radar, the rotary encoder data is acquired through the rotary encoder, and the Imu sensor data is acquired through the Imu sensor; (2) Data fusion 3D point cloud set: The lidar data, rotary encoder data and Imu sensor data are fused to obtain the 3D point cloud set of the fruit tree; (3) Segment the 3D point cloud set according to the oblique truncated triangular prism segmentation method; (4) Construct a canopy model of truncated triangular prism fruit trees; (5) Calculate the canopy volume of the fruit tree in the middle canopy area of ​​the fruit tree based on the truncated triangular prism fruit tree canopy model; (6) The canopy volume of the fruit tree is converted into the pesticide application duty ratio of the canopy area of ​​the fruit tree to guide the sprayer to spray the pesticide.

2. The method for accurate target spraying of fruit trees by multi-sensor fusion detection according to claim 1 is characterized in that: In step (1), the laser radar data is point cloud data; by dividing the depth information of the point cloud data, it is distinguished whether the point cloud data is on the right side or the left side of the sprayer.

3. The method for accurate target spraying of fruit trees by multi-sensor fusion detection according to claim 2 is characterized in that: In step (1), the single-line laser radar scans a certain data block to obtain a set of point cloud data P i =(P i,0 ,P i,1 ,...,P i,j ), where i is the data block number and j is the number of point clouds in the data block; In the point cloud data set of each data block, there are j+1 scanning points in total. The point cloud data contains the scanning point distance information, scanning point intensity information and point cloud angle information. The depth information and height information of each scanning point can be obtained through the trigonometric function relationship between the scanning point distance information and the point cloud angle information. In a single scanning cycle of a single-line laser radar, according to the angular range of the single-line laser radar scanning and the number of all scanning points in the scanning area, the angular spacing of each scanning point can be obtained, so that the angle set of the point cloud in a data block is α i =(α i,0 ,α i,1 ,…,α i,j ); The distance information of each scanning point of a single-line laser radar in a single cycle is defined as: r i =(ρ i,0 ,r i,1 ,…,r i,j ) The depth information of each scanning point in a single cycle is: d i =(d i,0 ,d i,1 ,…,d i,j ) Based on the target detection scanning system, the depth information of each scanning point in a single cycle is expressed as: d i,j =ρ i,j *things i,j The scanning field of view of the single-line laser radar corresponds to the front view of the sprayer body. i,j When it is greater than 0, it means that the point cloud is distributed on the left side of the sprayer. i,j When it is less than 0, it means that the point cloud is distributed on the right side of the sprayer.

4. The method for accurate target spraying of fruit trees by multi-sensor fusion detection according to claim 3 is characterized in that: In step (2), the sprayer applies pesticides on one side of the canopy of the fruit tree while applying pesticides on the ground. The point cloud data scanned by the single-line laser radar is on a two-dimensional plane. The rotary encoder data is the forward distance of the sprayer, and the Imu sensor data is the forward direction of the sprayer. The rotary encoder determines the accumulated point cloud data in the z-axis direction by recording the forward distance of the sprayer to form a three-dimensional point cloud set of the fruit tree. By measuring the distance between the sprayer and the trunks of the fruit trees on both sides and the forward direction detected by the Imu sensor, the point cloud data of each scanning cycle in the single-line laser radar is filtered, and the point cloud that does not belong to the forward direction point cloud detected by the current Imu and the point cloud outside the distance of the fruit tree trunk are filtered out, so that the formed three-dimensional point cloud set is valid data belonging to the current forward direction of the sprayer.

5. The method for accurate target spraying of fruit trees by multi-sensor fusion detection according to claim 4 is characterized in that: In steps (3) and (4), the set of filtered point cloud data is defined as p ′ =(p0,p1,...,p j ); the set of point cloud data after trunk filtering of the nth scanning cycle of the single-line laser radar is recorded as: p n ′=(p 0,n ,p 1,n ,…,p j,n ) A single point cloud data in the point cloud data set can be expressed as: p j,n =(ρ j,n ,i j,n ,z j,n ) Among them, ρ j,n is the distance of the jth scanning point in the nth scanning cycle, θ j,n is the angle information of the jth scanning point in the nth scanning cycle, z j,n The position of the sprayer when obtaining the jth scanning point in the nth scanning cycle; According to the oblique truncated triangular prism segmentation method, the scanning point p j,n 、p j+1,n 、p j,n+1 The constructed triangle is the oblique section of a right triangular prism, and the distance between the fruit tree trunks is used as depth information to construct a truncated triangular prism differential model; the truncated triangular prism differential model is a truncated triangular prism fruit tree canopy model, and the volume of the truncated triangular prism is calculated through the truncated triangular prism fruit tree canopy model.

6. The method for accurate target spraying of fruit trees by multi-sensor fusion detection according to claim 5 is characterized in that: In step (5) and step (6), in the vertical direction, the canopy of the fruit tree is divided into multiple regions from top to bottom, and each region corresponds to the nozzle group at the end of the sprayer; for each region, the scanning point with the smallest depth information of the scanning point is found, and the depth information of the scanning point is recorded as d min , taking the scanning point as the critical point of each area, the canopy volume of fruit trees in the canopy area of ​​each area is calculated.

7. The method for accurate target spraying of fruit trees by multi-sensor fusion detection according to claim 6 is characterized in that: One area corresponds to one rectangular volume block; after calculating the volume of the rectangular volume block corresponding to each canopy, the scanning points in each cycle of the single-line laser radar are classified according to the angle information according to the upper and lower limit ranges of the nozzle angles in the nozzle group set on the touch screen of the sprayer, and the truncated triangular prism volume of each canopy area in adjacent cycles is calculated through the truncated triangular prism fruit tree canopy model; then the sum of the truncated triangular prism volume of each canopy area is calculated, and the sum of the truncated triangular prism volume of each canopy area is the canopy volume of the fruit tree in each canopy area; the sum of the truncated triangular prism volume of each canopy area is divided by the volume of the corresponding rectangular volume block to obtain the pesticide application duty ratio of each canopy area; The calculation formula is as follows: Among them, the application duty cycle is the PWM duty cycle of the solenoid valve in the nozzle group, P represents the application duty cycle, V s It is expressed as the sum of the volumes of the obliquely truncated triangular prisms in each canopy area, V i The volume expressed as a cuboid volume.

8. The method for accurate target spraying of fruit trees by multi-sensor fusion detection according to claim 1 is characterized in that: When the sprayer moves forward a distance equal to the length of the sprayer body, the solenoid valve nozzle group executes the spraying duty cycle of the corresponding detection position in the previous spraying cycle.

9. The method for accurate target spraying of fruit trees by multi-sensor fusion detection according to claim 1 is characterized in that: When the single-line laser radar completes a pesticide application cycle detection, the pesticide application duty cycle of the solenoid valve nozzle group obtained from the previous round of corresponding position detection is executed to realize the spraying of the sprayer to the target at a periodic position.

10. The method for accurate target spraying of fruit trees by multi-sensor fusion detection according to claim 1 is characterized in that: The single-line laser radar, rotary encoder and Imu sensor are all connected to an industrial computer; and the industrial computer is connected to a solenoid valve nozzle group.

Images