Ground-air cooperative orchard parameter detection method and system

Abstract

The invention provides a ground-air cooperative orchard parameter detection method and system, the method is applied to a central control device in the system, and the system further comprises a ground acquisition device and a low-altitude acquisition device which are in communication connection with the central control device. The method comprises the following steps: controlling a low-altitude acquisition device to obtain low-altitude point cloud data and a global orchard map of a target orchard, performing path planning according to the global orchard map to obtain an orchard map route, controlling a ground acquisition device to acquire ground point cloud data based on the orchard map route, performing registration fusion on the low-altitude point cloud data and the ground point cloud data, and obtaining an orchard map route; the porosity is calculated according to the complete orchard point cloud data, a low-altitude supplementary collection target area is determined from the target orchard according to the porosity, a low-altitude collection device is controlled to collect supplementary collection point cloud data of the target area, and fruit tree parameters of the target area are calculated according to the supplementary collection point cloud data.

Description

Technical Field

[0001] The present disclosure relates to the fields of intelligent agricultural machinery and equipment and point cloud technology processing, and in particular to a ground-to-air coordinated orchard parameter detection method and system. Background Art

[0002] In modern agriculture, the rapid and accurate collection of fruit tree parameters can improve agricultural production efficiency and increase the degree of automated management of fruit trees. At the same time, accurate fruit tree parameters can not only more accurately locate the areas where diseases or pests occur and implement targeted prevention and control measures, but also understand the specific growth conditions and needs of each fruit tree and improve the efficiency of fruit tree irrigation, pesticide application and fertilization.

[0003] The traditional collection of orchard point cloud data mainly relies on a single collection method. However, when facing a complex fruit tree planting environment, the single collection method is often difficult to achieve comprehensive collection due to the collection angle and other reasons. The incompleteness of the orchard point cloud data not only affects the accuracy of the 3D reconstruction of the fruit trees, but also seriously affects the accurate calculation of the fruit tree parameters.

[0004] The content of the background technology section is only the information known to the inventor personally, and does not mean that the above information has entered the public domain before the application date of this disclosure, nor does it mean that it can become the prior art of the present disclosure. Summary of the invention

[0005] The present disclosure provides a ground-to-air coordinated orchard parameter detection method and system to avoid at least one of the above-mentioned technical problems.

[0006] In a first aspect, the present disclosure provides a ground-to-air coordinated orchard parameter detection method, the method being applied to a central control device in a ground-to-air coordinated orchard parameter detection system, the system further comprising a ground collection device and a low-altitude collection device respectively connected to the central control device for communication; the method comprising:

[0007] Controlling the low-altitude acquisition device to acquire low-altitude fruit tree data of a target orchard, and determining low-altitude point cloud data and a global orchard map of the target orchard based on the low-altitude fruit tree data, wherein the acquisition height of the low-altitude acquisition device is higher than the height of the fruit trees in the target orchard, and the height between the low-altitude acquisition device and the ground is less than a preset height threshold;

[0008] Performing path planning according to the global orchard map to obtain an orchard map route, and controlling the ground acquisition device to acquire ground point cloud data of the target orchard based on the orchard map route;

[0009] The low-altitude point cloud data and the ground point cloud data are registered and fused to obtain complete orchard point cloud data, and the porosity corresponding to each fruit tree in the target orchard is calculated according to the complete orchard point cloud data;

[0010] According to the porosity corresponding to each of the fruit trees, a low-altitude supplementary collection target area having a porosity outside a preset porosity range is determined from the target orchard;

[0011] The low-altitude acquisition device is controlled to acquire the supplementary acquisition point cloud data of the target area, and the fruit tree parameters of the target area are calculated based on the supplementary acquisition point cloud data.

[0012] In some embodiments, the ground acquisition device includes: a crawler chassis, and a first aluminum frame mounted on the crawler chassis, a real-time dynamic positioning RTK module, a dual-degree-of-freedom gimbal, a radar, a ground inertial measurement unit IMU, and a host, wherein the dual degrees of freedom include lifting and pitching;

[0013] Wherein, the crawler vehicle chassis provides power to control the ground collection device to move in the target orchard;

[0014] The first aluminum frame includes a vertical slide rail, which is used to install the dual-degree-of-freedom pan-tilt head to provide a lifting track for the dual-degree-of-freedom pan-tilt head. The first aluminum frame is also used to install the display component of the host.

[0015] The dual-degree-of-freedom gimbal includes a motor, the radar, and the IMU, wherein the motor is used to control the pitch angle and lifting of the dual-degree-of-freedom gimbal to provide the radar with a collection height and collection angle;

[0016] The radar, the IMU, and the RTK module are used to acquire the ground point cloud data;

[0017] The host is used to display the ground point cloud data and transmit the ground point cloud data to the central control device.

[0018] In some embodiments, the dual-degree-of-freedom pan-tilt head includes two lifting motors, and the two lifting motors are respectively installed on two vertical slide rails of the first aluminum frame, wherein the vertical slide rail has a preset track length, and a groove is provided at each preset interval length, and a support buckle is installed inside each of the two sides of the vertical slide rail connected to the dual-degree-of-freedom pan-tilt head, and the support buckle fits into the groove, and the support buckle supports the dual-degree-of-freedom pan-tilt head to move upward when extended, and the support buckle includes a counter, and when the dual-degree-of-freedom pan-tilt head moves upward, the support buckle is retracted, triggering the counter to count;

[0019] When the ground acquisition device is in an initial state, the dual-degree-of-freedom pan-tilt head is located at the bottom of the vertical slide rail, and the vertical slide rail provides an upward supporting force for the dual-degree-of-freedom pan-tilt head;

[0020] During the process of collecting the ground point cloud data, if it is detected that the radar collection height is less than a preset collection height threshold, the support buckle is controlled to pop up, and the target collection height of the radar is calculated, and each preset height interval is taken as an ascending value, and the ascending value closest to the target collection height is selected to control the movement of the lifting motor, and the counter is controlled to count at each preset height. When the value in the counter is consistent with the value corresponding to the collection height, the lifting motor is controlled to stop working, so as to control the dual-degree-of-freedom pan / tilt head to be fixed at the corresponding position of the vertical slide rail by the support buckle;

[0021] When the collection of the ground point cloud data is completed, the support buckle is controlled to be recovered, so that the lifting motor controls the dual-degree-of-freedom pan-tilt head to descend to the lowest point of the vertical slide rail.

[0022] In some embodiments, the radar carrying part carried by the dual-degree-of-freedom gimbal and the lifting motor are connected by a rotating motor; the method further includes:

[0023] If the radar collects the pitch angle when collecting the ground point cloud data, and it is determined that the sparsity of the ground point cloud data is greater than a preset threshold due to the collected pitch angle, the pitch angle of the radar is adjusted by controlling the dual-degree-of-freedom gimbal through the rotating motor to supplement the collection of the ground point cloud data and obtain ground point cloud data that reaches the preset threshold.

[0024] In some embodiments, the controlling the low-altitude collection device to collect low-altitude fruit tree data of the target orchard includes: controlling the low-altitude collection device to collect low-altitude fruit tree data of the target orchard based on a bow-shaped flight path;

[0025] And, performing path planning according to the global orchard map to obtain the orchard map route includes: performing global bow-shaped full coverage path planning according to a heuristic search algorithm to obtain the orchard map route.

[0026] In some embodiments, the registering and fusing the low-altitude point cloud data and the ground point cloud data to obtain complete orchard point cloud data includes:

[0027] Constructing a reference point cloud set based on the ground point cloud data;

[0028] Constructing a point cloud set to be registered according to the low-altitude point cloud data;

[0029] Calculate the geometric distance between any two points in the reference point cloud set and the point cloud set to be registered;

[0030] Taking minimizing the variance of the geometric distance as an iterative goal, the reference point cloud set and the to-be-registered point cloud set are registered and fused to obtain the complete orchard point cloud data.

[0031] In some embodiments, the step of calculating the porosity corresponding to each fruit tree in the target orchard according to the complete orchard point cloud data includes:

[0032] voxelize the canopy of fruit trees in the target orchard according to the complete orchard point cloud data to obtain a plurality of voxels;

[0033] Calculate the total projection area of ​​the multiple volume elements and the two-dimensional projection area of ​​the fruit tree canopy;

[0034] The porosity corresponding to each of the fruit trees is calculated based on the total projection area and the two-dimensional projection area.

[0035] In some embodiments, the step of determining a low-altitude supplementary collection target area having a porosity outside a preset porosity range from the target orchard according to the porosity corresponding to each of the fruit trees includes:

[0036] From the porosities corresponding to the fruit trees, determine at least one target fruit tree whose porosity is less than the preset porosity range;

[0037] Assigning identifiers to the at least one target fruit tree in a sequence starting from zero, and superimposing the identifiers and geographic information into a preset virtual orchard map, wherein the complete orchard point cloud data includes the geographic information;

[0038] The coordinate information corresponding to the identifier of the at least one target fruit tree is extracted from the preset virtual orchard map to obtain the low-altitude supplementary collection target area.

[0039] In some embodiments, the low-altitude acquisition device includes a camera; and controlling the low-altitude acquisition device to acquire the supplementary acquisition point cloud data of the target area includes:

[0040] For any fruit tree among the at least one target fruit tree, determining a crown center of the any fruit tree;

[0041] When the camera points to the center of the tree crown, the low-altitude collection device is controlled to take the center of the tree crown as the center of the circle, and through the tracking control algorithm, it is possible to realize circumnavigation of the arbitrary fruit tree, obtain the target image data collected by the camera, and obtain the supplementary collection point cloud data.

[0042] In a second aspect, the present disclosure further provides a ground-air coordinated orchard parameter detection system, the system comprising: a central control device, a ground collection device, and a low-altitude collection device; wherein:

[0043] The central control device is respectively connected to the ground collection device and the low-altitude collection device for communication;

[0044] The central control device is used to execute the method as described in any one of the first aspects.

[0045] The present embodiment provides a ground-to-air coordinated orchard parameter detection method and system, wherein the method is applied to a central control device in the system, and the system further comprises a ground acquisition device and a low-altitude acquisition device respectively connected to the central control device in communication; the method comprises: controlling the low-altitude acquisition device to acquire low-altitude fruit tree data of a target orchard, and determining low-altitude point cloud data and a global orchard map of the target orchard based on the low-altitude fruit tree data, wherein the acquisition height of the low-altitude acquisition device is higher than the height of the fruit trees in the target orchard, and the height between the low-altitude acquisition device and the ground is less than a preset height threshold; performing path planning based on the global orchard map to obtain an orchard map. The method comprises the following steps: determining a route of the target orchard map, and controlling the ground acquisition device to acquire the ground point cloud data of the target orchard based on the route of the orchard map; registering and fusing the low-altitude point cloud data and the ground point cloud data to obtain the complete orchard point cloud data, and calculating the porosity corresponding to each fruit tree in the target orchard according to the complete orchard point cloud data; determining the low-altitude supplementary acquisition target area whose porosity is outside the preset porosity range from the target orchard according to the porosity corresponding to each fruit tree; controlling the low-altitude acquisition device to acquire the supplementary acquisition point cloud data of the target area, and calculating the fruit tree parameters of the target area according to the supplementary acquisition point cloud data. The method can achieve the technical effect of determining the fruit tree parameters from multiple acquisition methods, thereby improving the accuracy and reliability of determining the fruit tree parameters. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0047] Figure 1 A schematic diagram of a working scenario of a ground-air coordinated orchard parameter detection system according to an embodiment of the present disclosure;

[0048] Figure 2 It is a schematic diagram of the ground-air coordinated orchard parameter detection method according to an embodiment of the present disclosure;

[0049] Figure 3 It is a structural schematic diagram of a low-altitude collection device according to an embodiment of the present disclosure;

[0050] Figure 4 It is a schematic diagram of the structure of the ground collection device according to an embodiment of the present disclosure.

[0051] The above drawings show clear embodiments of the present disclosure, which will be described in more detail below. These drawings and text descriptions are not intended to limit the scope of the present disclosure in any way, but to illustrate the concepts of the present disclosure to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0052] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0053] It should be understood that the terms "including" and "having" and any variations thereof in the embodiments of the present disclosure are intended to cover but not exclude inclusion. For example, a product or device comprising a series of components is not necessarily limited to those components explicitly listed, but may include other components not explicitly listed or inherent to these products or devices.

[0054] In the embodiments of the present disclosure, the term "and / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent three situations: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0055] The term "plurality" in the embodiments of the present disclosure refers to two or more than two, and other quantifiers are similar thereto.

[0056] The terms "first", "second", "third", etc. in the present disclosure are used to distinguish similar or similar objects or entities, and are not necessarily meant to limit a specific order or sequence, unless otherwise indicated. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, for example, they can be implemented in an order other than those given in the diagrams or descriptions of the embodiments of the present disclosure.

[0057] The term "unit / module" as used in this disclosure refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or a combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0058] The present invention provides a ground-to-air coordinated orchard parameter detection method, which is applied to a central control device in a ground-to-air coordinated orchard parameter detection system. The system also includes a ground collection device and a low-altitude collection device that are respectively communicatively connected to the central control device.

[0059] See also Figure 1 , Figure 1 Schematic diagram of the working scene of the ground-air coordinated orchard parameter detection system of the embodiment of the present disclosure. Figure 1 As shown, the working scene includes a ground-air coordinated orchard parameter detection system and a target orchard. The ground-air coordinated orchard parameter detection system includes a central control device (not shown in the figure), a ground collection device, and a low-altitude collection device. The target orchard includes multiple fruit trees. Figure 1 Only some of the fruit trees are shown, and the actual number of fruit trees collected can be more.

[0060] Before executing the ground-to-air coordinated orchard parameter detection method disclosed in the present invention, the central control device, the ground collection device, and the low-altitude collection device are powered on and all perform initialization operations, with the central control device running and the ground collection device and the low-altitude collection device on standby.

[0061] Among them, the ground collection device, low-altitude collection device and central control device are all equipped with an operating system, so it is necessary to initialize the ground collection device, low-altitude collection device and central control device. The initialization operation can release the process pressure of the ground collection device, low-altitude collection device and central control device, and reduce the probability of failure of the ground collection device, low-altitude collection device and central control device due to system resource problems.

[0062] Among them, after the central control device is started, it can establish communication with the ground collection system and the low-altitude collection system through the wireless network.

[0063] Exemplarily, the central control device starts the operating system, initializes the wireless network, and starts the roscore node and the central control node corresponding to the central control device. Roscore is the master node of ROS (Robot Operating System), which is responsible for initializing the system and running a name service and a parameter service. Before any ROS node (i.e., ROS program) starts running, roscore is started first, and the central control node is responsible for sending status control instructions to the ground collection device and the low-altitude collection device, allowing the ground collection device and the low-altitude collection device to enter different states such as collection and waiting. At the same time, the central control node receives and processes the data collected by the ground collection device and the low-altitude collection device. The ground collection device and the low-altitude collection device also start the Ubuntu operating system, establish a wireless network connection with the central control device, form a distributed ROS operating system with the central control device, start the corresponding node to receive the control instructions sent by the central control node, and check the communication quality to ensure that it meets the data transmission requirements.

[0064] Among them, the ROS operating system is a meta-operating system that provides a software framework for robots. It provides a series of services, libraries and tools to help software developers create robot applications. Its core functions include messaging, services, package management, build systems, tools and libraries. Through this operating system, developers can more easily write reusable robot software, thereby achieving rapid development, testing and deployment.

[0065] See also Figure 2 , Figure 2 FIG. 1 is a schematic diagram of a ground-air coordinated orchard parameter detection method according to an embodiment of the present disclosure. Figure 2 As shown, the method includes:

[0066] S201: Control the low-altitude collection device to collect low-altitude fruit tree data of the target orchard, and determine the low-altitude point cloud data and the global orchard map of the target orchard based on the low-altitude fruit tree data, wherein the collection height of the low-altitude collection device is higher than the height of the fruit trees in the target orchard, and the height between the low-altitude collection device and the ground is less than a preset height threshold.

[0067] Figure 3 Schematic diagram of the structure of the low-altitude collection device of the embodiment of the present disclosure. Figure 3 As shown, the low-altitude acquisition device includes: an aluminum alloy frame 1, blades 2, a take-off and landing frame 3, an aluminum frame platform 4, a power assembly 5, a single-chip microcomputer and a real-time operating system 6, an accelerometer 7, a gyroscope 8, a magnetic compass module 9, a camera 10, and a real-time kinematic (RTK) module 11. Among them, the camera 10 can be a visible light camera.

[0068] The low-altitude collection device is about 10 meters from the ground and about 2 meters higher than the fruit trees, so as to ensure that the camera 10 can normally collect the fruit trees in the target orchard.

[0069] The propeller blades 2, the landing gear 3, the aluminum frame platform 4, the power assembly 5, the single chip microcomputer and the real-time operating system 6, the accelerometer 7, the gyroscope 8, the magnetic compass module 9, and the camera 10 are mounted on the aluminum alloy frame 1 in a direct or indirect manner.

[0070] The lower end surface of the aluminum alloy frame 1 is provided with a take-off and landing frame 3 and a camera 10. The take-off and landing frame 3 is used to assist the take-off and landing of the low-altitude acquisition device to ensure that the low-altitude acquisition device can be stable and safe during take-off and landing.

[0071] The aluminum alloy frame 1 is internally loaded with a power assembly 5 for providing power for the low-altitude collection device to ascend, descend, and translate in the air. The power assembly 5 is directly connected to the four blades 2 of the low-altitude collection device, and the blades 2 are used to assist the power assembly 5 in providing power for the low-altitude collection device.

[0072] The upper end of the aluminum alloy frame 1 is directly connected to the aluminum frame platform 4. The aluminum frame platform 4 is divided into two layers, both of which are square structures, and the two layers are connected by four aluminum columns at the four corners. The first layer of the aluminum frame platform 4 is equipped with a single-chip microcomputer and a real-time operating system 6, an accelerometer 7, a gyroscope 8, and a magnetic compass module 9; the second layer of the aluminum frame platform 4 is equipped with an RTK module 11.

[0073] The low-altitude acquisition device receives the acquisition instruction from the central control device, drives the accelerometer 7, gyroscope 8, magnetic compass 9, RTK module 11 and camera 10, obtains the data corresponding to the accelerometer 7, gyroscope 8, magnetic compass 9, RTK module 11 and camera 10, unlocks the power system 5, and executes the take-off command.

[0074] Among them, after receiving the collection instruction from the central control device, the accelerometer 7, the gyroscope 8, the magnetic compass module 9, the camera 10, and the RTK module 11 collect the low-altitude fruit tree data of the target orchard according to the collection instruction. For example, the camera 10 collects visible light images, and the RTK module 11 collects real-time dynamic positioning information (such as posture, etc.).

[0075] In some embodiments, in order to save communication resources and other effects, the low-altitude collection device transmits the first visible light image in each collection angle to the central control device in real time during the collection process, so that the central control device can determine whether there is any abnormality in the current collection environment.

[0076] After the acquisition instruction is completed, the camera 10 stops working, and the accelerometer 7, gyroscope 8, and magnetic compass module 9 continue to work. When the low-altitude acquisition device receives the return instruction sent by the central control device, the low-altitude acquisition performs the return operation based on the return instruction. When the low-altitude acquisition device completes the return landing, the accelerometer 7, gyroscope 8, and magnetic compass module 9 stop working.

[0077] It is understandable that before the low-altitude collection device enters the collection phase to collect low-altitude fruit tree data, it is necessary to start the low-altitude collection device, specifically, to start the power component driver, accelerometer driver, gyroscope driver, magnetic compass driver, RTK driver, and camera driver.

[0078] Exemplarily, the low-altitude acquisition device is started, and the central control device establishes communication with the low-altitude acquisition device through a wireless communication (Wireless Fidelity, WIFI) module. After the communication is established, the low-altitude acquisition device feeds back to the central control processing device. After the central control processing device obtains the feedback, it sends a start instruction to the low-altitude acquisition device through the WIFI module. After the low-altitude acquisition device receives the instruction, it starts the gyroscope driver, accelerometer driver, magnetic compass driver, RTK driver, camera driver, and power component driver.

[0079] The power drive unit driver initializes the motor and ESC to ensure that the motor can operate normally. It uses pressure sensing to check whether the propellers are installed properly. At the same time, the power drive unit also detects the battery power and issues a warning when the power is less than 25%.

[0080] The accelerometer driver and the gyroscope driver drive the above peripheral hardware to initialize and calibrate, and at the same time communicate with the single-chip microcomputer and the real-time operating system 6, and let them use 6 bytes to send the detected data to the single-chip microcomputer and the real-time operating system 6. The data can provide the attitude information of the low-altitude acquisition device, among which the accelerometer driver sends the acceleration of the XYZ three axes, and the gyroscope driver sends the angular velocity of the XYZ three axes.

[0081] The magnetic compass driver is used to initialize and calibrate the magnetic compass hardware. The detected information is the angle value of the low-altitude acquisition device relative to the magnetic north direction. At the same time, it communicates with the microcontroller and the real-time operating system 6, and uses 2 bytes of data to send the collected data to the microcontroller and the real-time operating system 6.

[0082] RTK driver, responsible for initializing the RTK module. The RTK module is a high-precision positioning technology module that uses a fixed base station and a mobile receiver to obtain real-time geographic location information. It can perform operations such as data reception and processing, coordinate conversion, etc. The RTK module is connected to the single-chip microcomputer and the real-time operating system 6, so that the single-chip microcomputer and the real-time operating system 6 can read and process the data collected by the RTK module. After processing, the data can be used to calculate the real-time position of the low-altitude acquisition device, thereby achieving high-precision navigation and positioning.

[0083] The camera driver is mainly responsible for the initialization, parameter configuration, and image acquisition of the camera. Here, the resolution is set to 1600*1300 and the image is acquired at a frame rate of 30fps. At the same time, the camera establishes communication with the microcontroller and the real-time operating system 6. When performing the flight mission to acquire images, the microcontroller and the real-time operating system 6 control its shooting angle.

[0084] It is worth noting that before the low-altitude collection device collects low-altitude orchard data, the low-altitude collection device can be controlled to run in trial operation to determine whether the low-altitude collection device is working normally, so that when the low-altitude collection device is working normally, the low-altitude collection device can be controlled to collect low-altitude orchard data.

[0085] In combination with the above examples, during the trial operation, the low-altitude acquisition device can collect the current sensor group data (such as the above examples) images, acceleration, angular velocity, geographic location information, etc. The low-altitude acquisition device transmits the current sensor group data to the central control device through the single-chip microcomputer and the real-time operating system 6.

[0086] Correspondingly, the central control device receives the current sensor group data transmitted by the low-altitude acquisition device, and performs fusion processing on the current sensor group data to obtain the current posture of the low-altitude acquisition device, and determines whether the low-altitude acquisition device is working normally based on the posture.

[0087] Exemplarily, the central control device sends instructions to the low-altitude acquisition device through the WIFI module. The low-altitude acquisition device acquires the current sensor group data. The current sensor group data is transmitted back to the central control device based on the WIFI module.

[0088] Furthermore, the central control device calculates and fuses the 6-bit byte data from the accelerometer 7 and gyroscope 8 peripherals, and the 2-bit byte data from the magnetic compass module 9 peripheral. The roll angle (Roll) and pitch angle (Pitch) of the low-altitude acquisition device can be obtained by analyzing the accelerometer 7 data, the rotation angle of the low-altitude acquisition device can be obtained by integrating the angular velocity obtained by the gyroscope 8, and the yaw angle (Yaw) of the low-altitude acquisition device can be obtained from the magnetic compass module 9 data. At the same time, the central control device can judge the posture of the low-altitude acquisition device in the air according to the three-dimensional posture of the low-altitude acquisition device, namely Roll, Pitch and Yaw, and further control the action of the low-altitude acquisition device by sending instructions.

[0089] For example, if the central control device determines that the low-altitude collection device can work normally by judging the posture of the low-altitude collection device in the air, it will control the low-altitude collection device to enter the collection stage (or called the collection process, etc.); if the central control device determines that the low-altitude collection device cannot work normally by judging the posture of the low-altitude collection device in the air, it will control the low-altitude collection device to perform a return operation.

[0090] In some embodiments, controlling the low-altitude collection device to collect low-altitude fruit tree data of the target orchard in S201 includes: controlling the low-altitude collection device to collect low-altitude fruit tree data of the target orchard based on a bow-shaped flight path.

[0091] Exemplarily, when the low-altitude acquisition device enters the acquisition process, the target orchard can be collected using a bow-shaped flight path, the collected low-altitude orchard data is stored in the memory of the low-altitude acquisition device, and at least part of the collected image is transmitted back to the central control device. Accordingly, the central control device receives at least part of the image transmitted by the low-altitude acquisition device, and can obtain the low-altitude orchard data from the memory of the low-altitude acquisition device, so that the central control device processes the low-altitude orchard data to obtain low-altitude point cloud data and a global orchard map.

[0092] For example, the central control device sends a work instruction to the low-altitude collection device through the WIFI module, and the low-altitude collection device enters the collection process and uses the bow-shaped route collection method to collect orchard data.

[0093] On the collection route, the low-altitude collection device sets the shooting angle of camera 10 to 90°, that is, directly facing the ground; the relative flight altitude of the low-altitude collection device is set to 25 meters, and the flight speed is set to 5m / s; during the collection process, part of the collected images will be returned to the central control device in real time. In order to reduce the large-scale transmission of data, the low-altitude collection device will only select the first captured image at the beginning and middle shooting positions of each straight route of the bow-shaped route and return it. After the selection is completed, the collected images to be returned will be down-sampled and returned to the central control device.

[0094] The principle of downsampling is to downsample an image I with a size of p*q by s times, that is, to obtain a resolution image with a size of (p / s)*(q / s). In I, s is the common divisor of p and q. Here, the average pooling method is applied to downsample the image I, and each image to be transmitted back to the central control device is divided into several rectangular areas, and the average value of all elements in each sub-area is output.

[0095] As shown in Formula 1, the average pixel value of each sub-region is obtained, and finally the matrix synthesized by all the average values ​​is the down-sampled output image, Formula 1:

[0096]

[0097] Among them, y kij Represents the rectangular area R related to the kth feature map ij The average pooling output value, x kpq Represents the element at (p,q) in the rectangular region, R ij Represents the rectangular area R ij The number of elements in .

[0098] When the low-altitude collection device completes collection according to the route, it will return to the starting point.

[0099] In some embodiments, S201 determines the global orchard map based on the low-altitude fruit tree data, including: creating a new three-dimensional model task and filling in the task name; importing the low-altitude fruit tree data (such as an image folder) collected by the low-altitude collection device in the task bar; setting parameters, selecting the reconstruction type as a three-dimensional point cloud model, the mapping scene as a normal scene, the reconstruction clarity as high, and the calculation mode as unit price calculation; after the settings are completed, reconstruction is performed, and after waiting for the reconstruction to be completed, a global orchard map can be obtained.

[0100] S202: Perform path planning according to the global orchard map to obtain an orchard map route, and control the ground acquisition device to acquire ground point cloud data of the target orchard based on the orchard map route.

[0101] Figure 4 FIG. 1 is a schematic diagram of the structure of the ground collection device according to an embodiment of the present disclosure. Figure 4 As shown, the ground acquisition device includes: a crawler chassis 11, a first aluminum frame 12, an RTK module 13, a dual-degree-of-freedom (lift, pitch) gimbal 14, a radar 15, a ground inertial measurement unit (IMU) 16, and a host 17 (the display part of the host 17 is shown in the figure - display component). The radar 15 can be a 3D laser radar.

[0102] The first aluminum frame 12, RTK module 13, dual-degree-of-freedom (lift and pitch) gimbal 14, radar 15, IMU 16, and display part of host 17 are all mounted on the crawler chassis 1, and the crawler chassis 1 provides power to control the ground acquisition device to move in the target orchard.

[0103] The first aluminum frame 12 is assembled from 12 aluminum materials of different lengths after cutting and grinding. A vertical slide rail is installed on the top 30 cm of the vertical aluminum material for docking and mounting a dual-degree-of-freedom (lifting, pitching) pan-tilt head 14, providing a lifting track for the dual-degree-of-freedom (lifting, pitching) pan-tilt head 14.

[0104] The dual-degree-of-freedom (lift, pitch) platform 14 is printed by a 3D printer and has four motors built in the shell. Two of them are responsible for controlling the pitch angle of the dual-degree-of-freedom (lift, pitch) platform 14, and the other two are responsible for controlling the lifting and lowering of the dual-degree-of-freedom (lift, pitch) platform 14, which are mainly used to provide a better acquisition height and acquisition angle for the radar 15.

[0105] The crawler vehicle chassis 1 has a crawler vehicle battery, a motor driver, and a stepper motor installed in its housing. The crawler vehicle battery provides energy for the crawler vehicle chassis, and the motor driver is connected to the battery, the host 17, and the stepper motor. The stepper motor is connected to the crawler drive wheel. The host 17 controls the stepper motor to the motor driver, and the stepper motor drives the crawler vehicle to move.

[0106] The dual-degree-of-freedom (lifting, pitching) pan / tilt platform 14 is installed on the first aluminum frame 12 as follows: Figure 4As shown. The two lifting motors in the dual-degree-of-freedom (lifting, pitching) pan-tilt head 14 are respectively installed on the two vertical rails of the first aluminum frame 12. The total length of this vertical rail is 30 cm, and there is a groove every 2 cm on the vertical rail. A support buckle is installed inside the dual-degree-of-freedom (lifting, pitching) pan-tilt head 14 connecting both sides of the vertical rail. This support buckle is adapted to the internal groove of the rail, the upper surface is a slope, and the lower surface is a plane. When the support buckle is extended, it only supports the dual-degree-of-freedom (lifting, pitching) pan-tilt head 14 to move upward, and a counter is installed inside the support buckle. When the dual-degree-of-freedom (lifting, pitching) pan-tilt head 14 moves upward, the support buckle is retracted, triggering the counter to count.

[0107] In the initial state, the dual-degree-of-freedom (elevation, pitch) pan-tilt platform 14 of the ground acquisition device is located at the bottom of the vertical track. The vertical track will provide an upward supporting force to the dual-degree-of-freedom (elevation, pitch) pan-tilt platform 14 to reduce the damage to the supporting buckle.

[0108] During the collection process of the ground collection device, when the central control device detects that the collection height of the radar 15 is low (preset collection height threshold), the control support buckle pops up, calculates the optimal rise height (target collection height), takes every 2cm (preset height interval) as a rise value, selects the rise value closest to the optimal rise height for movement, counts 1 every 2cm, and when the value in the counter is consistent with the value corresponding to the collection height, the lifting motor stops working and the dual-degree-of-freedom (lift, pitch) pan-tilt 14 is buckled and fixed in the middle of the vertical track (corresponding position). When the ground collection device completes data collection, the control support buckle is recovered so that the dual-degree-of-freedom (lift, pitch) pan-tilt 14 drops to the lowest point of the vertical track.

[0109] The preset acquisition height threshold is the height when α1<α2. The target acquisition height is the height when α1=α2. α1 is the angle between the midpoint of the radar 15 and the top of the tree canopy scanned by the radar 15; α2 is the angle between the midpoint of the radar 15 and the bottom of the tree canopy scanned by the radar 15.

[0110] The dual-degree-of-freedom (elevation, pitch) platform 14 carrying the radar 15 and the IMU 16 is connected to its elevation part by two rotating motors. The two rotating motors are controlled by the host 17. When a problem is detected in the pitch angle of radar 15 data collection, the dual-degree-of-freedom (elevation, pitch) platform 14 is controlled to adjust the pitch angle of radar 15.

[0111] Among them, when the radar 15, IMU 16 and RTK module 13 are working, the radar 15 continuously collects point cloud information of the surrounding environment, and the IMU 16 and RTK module 13 provide real-time posture and position information to obtain ground point cloud data.

[0112] Similarly, before the ground acquisition device enters the acquisition phase to collect ground point cloud data, it is necessary to start the ground acquisition device. To start the ground acquisition device, specifically, the IMU driver, RTK module driver, and radar driver can be started, and the central control device can control the trial operation of the ground acquisition device to detect whether the ground acquisition device is working normally.

[0113] Exemplarily, the ground acquisition device is started, and the central control device establishes communication with the ground acquisition device through the WIFI module. After the communication is established, the ground acquisition device feeds back to the central control device. After the central control device obtains the feedback, it sends a start instruction to the ground acquisition device through the WIFI module. After the ground acquisition device receives the instruction, it starts the mapping program of the host 17. The mapping program is a launch file written based on the Ubuntu system integrated with the ROS operating system. The file links other node programs, including IMU driver, RTK module driver, and radar driver.

[0114] Among them, the launch file is a startup file in the ROS operating system that does not use a predefined tag language (Extensible Markup Language, XML) format. It can link multiple node programs, which enables it to start multiple nodes at the same time. It is suitable for complex ROS operating systems and multi-node collaborative working scenarios.

[0115] The IMU driver is mainly responsible for running and receiving data from the IMU16. The IMU16 can provide data such as acceleration, angular velocity, Euler angles, and four elements for estimating the position and posture of the ground acquisition device. In the ROS operating system, the driver can calibrate, convert, and publish the IMU data in the mapping program so that the ground processing device can obtain the attitude estimation in the reference coordinate system.

[0116] The RTK module driver provides real-time geographic location information by using fixed base stations and mobile receivers. It can perform operations such as data reception and processing, communication interface, coordinate conversion, etc. The RTK module 13 is connected to the ground processing device host 17 so that the ground processing device can read and process the data collected by the RTK module 13. After processing, these data will be published in the mapping program, and these data can be used to calculate the real-time position of the ground processing device, thereby achieving high-precision navigation and positioning.

[0117] The radar driver is connected to the host 17 of the ground processing device through Ethernet. The radar driver can initialize the radar 15, configure the basic parameters of the laser radar, calibrate and calibrate the data, etc. After the radar driver is started, the data scanned by the radar 15 is parsed (the digital signal of the radar is converted into a distance value, and these distance values ​​are organized into point cloud data), filtered (such as radius filtering, voxel filtering, etc.), and then the data is released in the mapping program.

[0118] When the mapping program is started, the IMU driver, RTK module driver, and radar driver are started at the same time. After starting the above programs, each sensor starts to collect data and publishes the data in the mapping program. The mapping program starts to build a map based on the data provided by each sensor, and transmits the mapping data to the central control device through the wireless data transmission module. The central control device detects whether the ground acquisition device is working normally through the received data.

[0119] If the ground collection device works normally, the ground collection device is controlled to enter the collection process. Exemplarily, the central control device performs global bow-shaped full coverage path planning according to the heuristic search algorithm to obtain the orchard map route. The central control device transmits the orchard map path to the ground collection device to control the ground collection device to collect the ground point cloud data of the target orchard based on the orchard map route.

[0120] Specifically, the central control device makes a global bow-shaped full coverage path planning for the path according to the A* algorithm. The A* algorithm is a heuristic search algorithm that can determine the global optimal path in a static network according to the cost function and the heuristic function: f(n) = g(n) + h(n), where n represents the current node, f(n) represents the weight calculated for node n, g(n) is the cost function, which represents the actual cost from the current starting point to the current node n, and h(n) is the heuristic function, which represents the actual cost from the current node n to the target point T.

[0121] The ground collection device reads the orchard map route, and the central control device sends a start command to the ground collection device. The ground collection device collects data in the target orchard based on the bow-shaped full coverage path as the path target. Since the road conditions in the target orchard are relatively complex, unknown obstacles may appear on the original path. Therefore, the key points in the global path planning obtained by the A* algorithm are used as the intermediate target points of the dynamic window method (Dynamic Window Approaches, DWA) algorithm to implement local path planning to avoid position obstacles. The main idea of ​​the DWA algorithm is to determine a sampling speed space in the speed space that meets the hardware constraints of the ground collection device according to the position and speed of the current device (the ground collection device in this embodiment), calculate the trajectory within a certain time of moving at this speed, use the evaluation function to evaluate the trajectory, and finally use the best evaluated trajectory as the movement speed of the ground collection device. For the ground collection device, the sampling space usually has a speed limit, as shown in Formula 2, Formula 2:

[0122] V m ={(v,ω)|v∈[v min ,v max ],ω∈[ω min ,ω max ]}

[0123] Among them, v min and v max is the maximum linear velocity of the ground acquisition device, ω min and ω max is the maximum angular velocity of the ground acquisition device.

[0124] The acceleration limit can be expressed by equation 3:

[0125] V d ={(v,ω)|(v∈[v c -a vmax ·Δt,v c +a vmax ·Δt],ω∈[ω c -a ωmax ·Δt,ω c +a ωmax ·Δt]}

[0126] Among them, v c and ω c is the linear velocity and angular velocity at the current moment, a vmin and a vmax are the maximum linear acceleration and angular acceleration of the ground acquisition device, respectively.

[0127] The environmental obstacle limitation can be expressed by Equation 4:

[0128]

[0129] Among them, dist(v, ω) represents the shortest distance between the corresponding simulated trajectory and the obstacle at the current speed, where the speed and acceleration space limits can be obtained according to the hardware parameters of the ground acquisition device, and the environmental obstacle limits can be obtained by radar 15. The final speed sampling space is the intersection of the three. After determining the above sampling space, the DWA algorithm uniformly samples in the space at a certain sampling interval, and calculates each group of different speeds (v, ω) based on the motion model of the ground acquisition device to predict the posture of the ground acquisition device at the next moment, and obtains the optimal speed (v, ω) through the evaluation function. The evaluation function can be expressed by Formula 5, Formula 5:

[0130] G(v,ω)=α·heading(v,ω)+β·dist(v,ω)+γ·velocity(v,ω)

[0131] Among them, heading(v,ω) is the azimuth evaluation function, dist(v,ω) is the distance evaluation function, velocity(v,ω) is the speed evaluation function, and α, β, and γ are the evaluation function coefficients.

[0132] The ground collection device starts from the starting point (preset) and uses the above-mentioned path collection method to collect data. It uses a radar 15 with a degree of freedom of 2, an IMU 16 and an RTK module 13 to collect data on the fruit trees in the target orchard and publish the data to the mapping program. During the acquisition process, the data obtained by each sensor is published to the mapping program in real time. When the program receives the point cloud data of the radar 15, it first pre-processes the point cloud data of the radar 15. The pre-processing includes using the IMU16 data to estimate the movement of the radar 15 during the scanning process, performing corresponding motion compensation on the point cloud data, converting the point cloud data into a raster map format, and then optimizing the current frame pose through scan-to-map (matching the scanned data with the existing map to determine the actual position coordinates (x, y, theta) of the ground acquisition device) feature matching (matching the point cloud data of the radar 15 through corner point and surface point optimization); and performing pose estimation based on the data provided by the IMU16 and RTK module 13 to further improve the positioning accuracy; relying on the optimized pose estimation and point cloud data, the point cloud data is projected into the raster map, and the map information is updated to obtain ground point cloud data. At the same time, according to the results of loop closure detection, such as the results of iterative closest point (ICP) between the current frame and the sub-image around the candidate frame, the current frame is matched with the sub-image around the candidate frame to obtain more comprehensive and accurate ground point cloud data.

[0133] In some embodiments, if the radar collects the pitch angle when collecting the ground point cloud data, and it is determined that the sparsity of the ground point cloud data is greater than a preset degree threshold due to the collected pitch angle, the pitch angle of the laser radar is adjusted by controlling the dual-degree-of-freedom gimbal through the rotating motor to supplement the collection of the ground point cloud data and obtain ground point cloud data that reaches the preset degree threshold.

[0134] For example, if the fruit tree collection effect is not ideal due to factors such as the pitch angle of the collection, obvious obstructions next to the fruit tree, etc., the central control device will feedback to the ground collection device through the WIFI module. The ground collection device enters the supplementary collection stage. The ground collection device adjusts the height and pitch angle of the radar 15 through the dual-degree-of-freedom (lift, pitch) pan-tilt 14, and collects fruit trees with unsatisfactory collection effects after adjusting to a suitable posture. At the same time, the ground collection device pre-processes the collected data and returns it to the central control device in real time through the WIFI module. When the ground collection device completes the collection, it automatically returns to the starting point.

[0135] S203: registering and fusing the low-altitude point cloud data and the ground point cloud data to obtain complete orchard point cloud data, and calculating the porosity corresponding to each fruit tree in the target orchard based on the complete orchard point cloud data.

[0136] In some embodiments, the step of registering and fusing the low-altitude point cloud data and the ground point cloud data to obtain complete orchard point cloud data in S203 includes the following steps:

[0137] Step 1: Construct a reference point cloud set based on the ground point cloud data.

[0138] Step 2: Construct a point cloud set to be registered based on the low-altitude point cloud data.

[0139] Step 3: Calculate the geometric distance between any two points in the reference point cloud set and the point cloud set to be registered.

[0140] Step 4: Taking minimizing the variance of the geometric distance as the iteration goal, the reference point cloud set and the to-be-registered point cloud set are registered and fused to obtain the complete orchard point cloud data.

[0141] Exemplarily, the central control device registers and fuses the ground point cloud data and the low-altitude point cloud data through an ICP algorithm.

[0142] The ground acquisition device and the low-altitude acquisition device respectively use their respective acquisition starting points as the origin, and the initial motion direction as the x-axis and the vertical direction as the z-axis to establish a right-handed coordinate system; the ground point cloud data obtained by the ground acquisition device is obtained based on their respective coordinate systems to construct a reference point cloud set P, and the reference point cloud set P has N data points, denoted as P = {x1, x2, ..., xN}; Construct the point cloud set Q to be registered using the low-altitude point cloud data obtained by the low-altitude acquisition device. There are N data points in the point cloud set Q to be registered, denoted as Q = {y1, y2, …, y N}.

[0143] Calculate the geometric distance between two randomly selected points in the reference point cloud set P and the point cloud set Q to be registered through Equation 6. When the variance of the geometric distances between all points in the reference point cloud set P and the point cloud set Q to be registered and their corresponding points is minimized, the pairwise corresponding points can be determined as the same point in the real situation collected by their respective acquisition methods. Equation 6:

[0144]

[0145] where, R represents the rotation matrix, t represents the translation matrix, y i is a point in the point cloud set Q to be registered, and x i is a point in the reference point cloud set P; when the Gaussian distance variance between the points in the point cloud set Q to be registered after being moved by the rotation matrix and the translation matrix and the points in the reference point cloud set P is minimized, the rotation matrix R and the translation matrix t calculated at this time are the rotation value and translation value required for the point cloud set Q to be registered relative to the reference point cloud set P.

[0146] For the convenience of calculation, the central control device can calculate the centroids of the point cloud set Q to be registered and the P of the reference point cloud set. Among them, the centroid of the reference point cloud set P is represented by Equation 7, and the centroid of the point cloud set Q to be registered is represented by Equation 8. Equation 7:

[0147]

[0148] Equation 8:

[0149]

[0150] where, t can be simplified to Equation 9. Equation 9:

[0151] t = h P - Rh Q

[0152] Correspondingly, substitute Equation 9 back into Equation 6, and set the minimum threshold d. When the minimum threshold d < E(R, t), recalculate the rotation matrix R and the translation matrix t; when the minimum threshold d > E(R, t), the condition is satisfied and the iteration terminates, outputting the rotation matrix R and the translation matrix t.

[0153] Furthermore, use the obtained rotation matrix R and translation matrix t to perform coordinate transformation on the point cloud set Q to be registered to achieve registration and fusion of the low-altitude point cloud data and the ground point cloud data.

[0154] In some embodiments, calculating the porosity corresponding to each fruit tree in the target orchard according to the complete orchard point cloud data in S203 includes the following steps:

[0155] Step 1: voxelize the fruit tree canopy of the target orchard according to the complete orchard point cloud data to obtain a plurality of voxels.

[0156] Step 2: Calculate the total projection area of ​​the multiple voxels and the two-dimensional projection area of ​​the fruit tree canopy.

[0157] Step 3: Calculate the porosity corresponding to each of the fruit trees according to the total projection area and the two-dimensional projection area.

[0158] Exemplarily, the central control device calculates the porosity using the voxel method. First, the fruit tree canopy is voxelized, and the point cloud data of the fruit tree canopy is divided into an individual element, and the size of the voxel is calculated. After obtaining the size of the voxel, the central control device can obtain the voxel coordinates (a, b, c) according to formula 10, and use the voxel coordinates to replace the coordinates of the point cloud data of the fruit tree canopy, thereby reducing the redundancy of the point cloud. Formula 10:

[0159]

[0160] Among them, a, b, c are the voxel coordinates; x min ,y min 、z min is the minimum value of the point cloud coordinates; x, y, z are the coordinates of the current point; L is the side length of the voxel; int() is the rounding function.

[0161] The central control device can calculate the total projected area of ​​the volume element, use the Alpha-shape algorithm to calculate the two-dimensional projected area, and calculate the porosity p according to formula 11. Formula 11:

[0162]

[0163] Among them, s voxel is the total projected area, s canopy is the two-dimensional projected area.

[0164] The number of voxels N can be calculated by formula 12:

[0165]

[0166] Among them, x max is the maximum value of the x-axis of the point cloud; min is the minimum value of the point cloud x-axis; L is the length of the voxel side.

[0167] The total projected area can be calculated using formula 13:

[0168] S voxel =N*L 2

[0169] The side length of the voxel can be calculated by formula 14:

[0170]

[0171] Among them, a is the preset scale factor, which is generally 1-2; x min ,y min is the minimum value of the point cloud coordinates; x max ,y max is the maximum value of the point cloud coordinates; N p is the total number of point clouds of the complete orchard point cloud data.

[0172] S204: According to the porosity corresponding to each of the fruit trees, a low-altitude supplementary collection target area having a porosity outside a preset porosity range is determined from the target orchard.

[0173] In some embodiments, S204 may include the following steps:

[0174] Step 1: From the porosities corresponding to the fruit trees, determine at least one target fruit tree that is outside the preset porosity range.

[0175] Step 2: assigning identifiers to the at least one target fruit tree in a sequence starting from zero, and superimposing the identifiers and geographic information onto a preset virtual orchard map, wherein the complete orchard point cloud data includes the geographic information.

[0176] Step 3: extracting coordinate information corresponding to the identifier of the at least one target fruit tree from the preset virtual orchard map to obtain the low-altitude supplementary collection target area.

[0177] For example, due to the different voxel sizes' different abilities to accommodate the number of point clouds, larger voxels can make up for the uneven density of point clouds caused by multi-site sampling, but too large voxels will also result in too much space that does not belong to a single tree being included in the voxel; smaller voxels can accommodate point clouds with a finer grid to fit the actual situation of the point cloud data, but too small voxels will lead to an underestimation of canopy porosity. To determine the appropriate voxel side length, the root mean square error (RMSE), overall accuracy (TA), and relative analytical error (RPD) are used for accuracy assessment.

[0178] Among them, the root mean square error can be calculated based on formula 15, formula 15:

[0179]

[0180] Among them, y i is the measured value of LAI of the ith tree; y i is the estimated LAI value of the i-th tree; i is the measured average value of LAI; n is the number of samples.

[0181] The overall accuracy can be calculated based on Equation 16:

[0182]

[0183] The relative analysis error can be calculated based on Equation 17:

[0184]

[0185] Where SD is the standard deviation, which can be calculated based on Formula 18:

[0186]

[0187] The value of RMSE is greater than 0; the closer it is to 0, the higher the accuracy. When RPD>1.4, the accuracy is good, and vice versa.

[0188] Furthermore, the central control device extracts the center point of the single tree in the segmentation step, assigns identifiers to the fruit trees that need to be supplemented (i.e., the at least one target fruit tree) in a sequence starting from zero, and superimposes the geographic information and identifiers into the virtual orchard map through a geographic information system (GIS). The geographic information is determined based on the complete orchard point cloud data. For example, the complete orchard point cloud data after registration and fusion includes geographic information.

[0189] When the central control device obtains the label (identifier) ​​of the designated fruit tree (target fruit tree), it extracts the corresponding Global Positioning System (GPS) coordinate information from the virtual orchard map, and then transmits it to the low-altitude collection device to control the low-altitude collection device to collect point cloud data based on the coordinate information, thereby obtaining the fruit tree parameters of the designated fruit tree.

[0190] S205: Control the low-altitude acquisition device to acquire the supplementary acquisition point cloud data of the target area, and calculate the fruit tree parameters of the target area according to the supplementary acquisition point cloud data.

[0191] According to the above analysis, the low-altitude acquisition device includes a camera, and controlling the low-altitude acquisition device to acquire the supplementary acquisition point cloud data of the target area in S205 includes the following steps:

[0192] Step 1: For any fruit tree among the at least one target fruit tree, determine the crown center of the any fruit tree.

[0193] Step 2: When the camera points to the center of the tree crown, the low-altitude acquisition device is controlled to take the center of the tree crown as the center of the circle, and through the tracking control algorithm, it is realized to fly around the arbitrary fruit tree, and the target image data collected by the camera is obtained to obtain the supplementary acquisition point cloud data.

[0194] Exemplarily, the central control device can mark the target fruit trees that need to be supplemented by the low-altitude collection device, and plan the path for the location of the marked target fruit trees. After the low-altitude collection device reaches the first target fruit tree according to the path planning, it determines the crown center of the fruit tree through the global orchard map, adjusts the camera angle toward the crown center, and uses the crown center as the center of the circle. Through the tracking control algorithm, the low-altitude collection device rotates around the tree in a circle within 2 meters from the crown for about 1 minute to capture RGB and depth data. According to the preset acquisition rate, 1800 frames of images are generated for each target fruit tree. The camera's read frame rate is very high, such as 30 frames per second, with sufficient overlap between frames, and the camera's maximum depth perception distance can reach 5.46 meters, which can fully collect the depth data of the target fruit tree and better reconstruct and analyze the model in 3D.

[0195] The tracking control algorithm can be expressed by equations 19 to 21. Equation 19:

[0196] x(t)=cx+r·cos(t)

[0197] Formula 20:

[0198] y(t)=cy+r·sin(t)

[0199] Formula 21:

[0200] Among them, (cx, cy) is the center of the circle, r is the radius, Equations 19 and 20 represent the position of the low-altitude collection device on the circumference, and Equation 21 represents the distance error from the low-altitude collection device to the circular path.

[0201] The central control device makes the distance error as small as possible (e.g., close to zero) to control the low-altitude acquisition device to perform supplementary acquisition in this case. After the low-altitude acquisition device completes the acquisition of the first target fruit tree, it determines the second target fruit tree according to the marking, and repeats the steps of collecting low-altitude point cloud data described in the above example to complete the acquisition of all target fruit trees. For example, after the low-altitude acquisition device completes the aerial photography, the target image data in the low-altitude acquisition device is transmitted to the central control device to obtain supplementary acquisition point cloud data.

[0202] In some embodiments, the fruit tree parameters include tree height and canopy volume parameters, etc. In this embodiment, by calculating the fruit tree parameters such as tree height and canopy volume parameters, support can be provided for subsequent variable spraying, reducing pesticides and increasing income, etc.

[0203] In order to improve the calculation speed of the supplementary collected point cloud data, the central control device can filter the supplementary collected point cloud data and use the voxel downsampling algorithm to calculate the obtained three-dimensional point cloud. Among them, the centroid or center of the voxel grid replaces the points in the grid to maintain the shape of the point cloud. At the same time, in order to ensure the accuracy of the segmentation and avoid the influence of some points close to the ground on the segmentation, the direct filter filters out part of the tree bottom point cloud, and then uses the density-based spatial clustering (Density-Based Spatial Clustering ofApplications with Noise, DBSCAN) algorithm to segment multiple target fruit trees into a single fruit tree. After obtaining the point cloud of the single fruit tree, as shown in Formula 22, the center point (x, y, z) of the point cloud is obtained to continue the calculation. Formula 22:

[0204]

[0205] Among them, x, y are the average values ​​of each point; z is the approximate half tree height; N is the total number of point clouds of a single fruit tree after segmentation; x i ,y i , z i is the coordinate corresponding to the i-th point.

[0206] Finally, a k-dimensional tree (kd-tree) is established for the supplementary collected point cloud data, and then the K nearest neighbors are obtained by searching. The average value of the K nearest neighbors is used to estimate the height z value of the tree bottom and top. Take (x, y, z-a1h) as the bottom point of the tree, calculate the average height of the K1 points closest to it, take (x, y, z+a2h) as the vertex of the tree, and calculate the average height of the K2 points closest to it. a1 and a2 are adjustable factors, where a1∈[0.6,1] and a2∈[0.8,1.5]. The tree height parameter calculation is shown in Formula 23. Formula 23:

[0207]

[0208] in, is the average height of the tree base; is the average height of the treetops, is the height of the fruit tree.

[0209] Adaptive slicing is used to calculate the canopy volume. The thickness and number of slices are intelligently adjusted in the vertical direction to ensure that the horizontal slice projections of the same layer are similar, and the horizontal projections of different layers are significantly different. The number of slices is reduced, so that the canopy volume can be estimated more accurately to adapt to the specific shape and size of the canopy, improving the credibility and feasibility of the algorithm. The specific ideas of the algorithm are as follows:

[0210] (1) The fruit tree canopy point cloud is divided into N layers along the vertical z direction, and the initial thickness of each layer is bi = b0 (i = 1, 2, ..., m).

[0211] (2) Because three points form a plane, the number of point clouds in each layer is counted during stratification, and layers with less than three points are merged with adjacent layers.

[0212] (3) Project the point cloud of each layer onto the horizontal XY plane.

[0213] (4) Use the Alpha-shapes algorithm to traverse the points of the plane, connect the outermost points to form a two-dimensional plane, calculate the projection area of ​​each layer, denoted as Si, and achieve stratification. For example, the canopy point cloud is evenly divided into 50 layers in the vertical direction and distinguished by different colors.

[0214] (5) As shown in Formula 24, the area ratios between adjacent layers are calculated, and the average of all ratios is calculated. Formula 24:

[0215]

[0216] Among them, i is the i-th layer; Ai is the area ratio of the i-th layer; Amean is the mean of the area ratio; N is the total number of segmentation layers.

[0217] (6) As shown in Formula 25, point cloud slices with a value greater than the average ratio are classified into one category, and those with a value less than the average ratio are classified into another category. Adjacent point cloud slices are compared to see if they are of the same category. If they are of the same category, they are merged, otherwise they are not merged. Formula 25:

[0218]

[0219] Among them, C i is the category number of the i-th slice.

[0220] (7) If the adjacent slice C i and C i+1 If they belong to the same category, they are merged into one category, as shown in Formula 26.

[0221] Formula 26:

[0222] C i+1 =C i

[0223] (8) The thickness of the merged segments is directly added, and the number of adaptive slices becomes 33 layers.

[0224] (9) In summary, after the adaptive slicing is completed, the canopy volume is calculated according to Formula 27. Formula 27:

[0225]

[0226] Where Vi is the volume of the i-th layer; Δh i is the height difference between the i-th layer and the i+1-th layer; Si and S i+1 is the projection area of ​​the i-th layer and the i+1-th layer.

[0227] According to an embodiment of the present disclosure, the present disclosure also provides a ground-air coordinated orchard parameter detection system, the system comprising: a central control device, a ground collection device, and a low-altitude collection device; wherein,

[0228] The central control device is respectively connected to the ground collection device and the low-altitude collection device for communication;

[0229] The central control device is used to execute the method described in the above embodiment.

[0230] Obviously, those skilled in the art can make various changes and modifications to the present disclosure without departing from the spirit and scope of the present disclosure. Thus, if these modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is also intended to include these modifications and variations.

Claims

1. A ground-air collaborative orchard parameter detection method. It is characterized in that The method is applied to a central control device in a ground-air coordinated orchard parameter detection system, wherein the system further comprises a ground collection device and a low-altitude collection device respectively connected to the central control device for communication; the method comprises: Controlling the low-altitude acquisition device to acquire low-altitude fruit tree data of a target orchard, and determining low-altitude point cloud data and a global orchard map of the target orchard based on the low-altitude fruit tree data, wherein the acquisition height of the low-altitude acquisition device is higher than the height of the fruit trees in the target orchard, and the height between the low-altitude acquisition device and the ground is less than a preset height threshold; Performing path planning according to the global orchard map to obtain an orchard map route, and controlling the ground acquisition device to acquire ground point cloud data of the target orchard based on the orchard map route; The low-altitude point cloud data and the ground point cloud data are registered and fused to obtain complete orchard point cloud data, and the porosity corresponding to each fruit tree in the target orchard is calculated according to the complete orchard point cloud data; According to the porosity corresponding to each of the fruit trees, a low-altitude supplementary collection target area having a porosity outside a preset porosity range is determined from the target orchard; The low-altitude acquisition device is controlled to acquire the supplementary acquisition point cloud data of the target area, and the fruit tree parameters of the target area are calculated based on the supplementary acquisition point cloud data.

2. The method according to claim 1, It is characterized in that The ground acquisition device includes: a crawler chassis, and a first aluminum frame mounted on the crawler chassis, a real-time dynamic positioning RTK module, a dual-degree-of-freedom gimbal, a radar, a ground inertial measurement unit IMU, and a host, wherein the dual degrees of freedom include lifting and pitching; Wherein, the crawler vehicle chassis provides power to control the ground collection device to move in the target orchard; The first aluminum frame includes a vertical slide rail, which is used to install the dual-degree-of-freedom pan-tilt head to provide a lifting track for the dual-degree-of-freedom pan-tilt head. The first aluminum frame is also used to install the display component of the host. The dual-degree-of-freedom gimbal includes a motor, the radar, and the IMU, wherein the motor is used to control the pitch angle and lifting of the dual-degree-of-freedom gimbal to provide the radar with a collection height and collection angle; The radar, the IMU, and the RTK module are used to acquire the ground point cloud data; The host is used to display the ground point cloud data and transmit the ground point cloud data to the central control device.

3. The method according to claim 2, It is characterized in that The dual-degree-of-freedom pan-tilt head includes two lifting motors, which are respectively mounted on two vertical slide rails of the first aluminum frame, wherein: The vertical slide rail has a preset track length, and a groove is provided at each preset interval length. A support buckle is installed inside each of the two sides of the vertical slide rail connected to the dual-degree-of-freedom gimbal. The support buckle fits the groove. When the support buckle is extended, it supports the dual-degree-of-freedom gimbal to move upward. The support buckle includes a counter. When the dual-degree-of-freedom gimbal moves upward, the support buckle is retracted, triggering the counter to count. When the ground acquisition device is in an initial state, the dual-degree-of-freedom pan-tilt head is located at the bottom of the vertical slide rail, and the vertical slide rail provides an upward supporting force for the dual-degree-of-freedom pan-tilt head; During the process of collecting the ground point cloud data, if it is detected that the radar collection height is less than a preset collection height threshold, the support buckle is controlled to pop up, and the target collection height of the radar is calculated, and each preset height interval is taken as an ascending value, and the ascending value closest to the target collection height is selected to control the movement of the lifting motor, and the counter is controlled to count at each preset height. When the value in the counter is consistent with the value corresponding to the collection height, the lifting motor is controlled to stop working, so as to control the dual-degree-of-freedom pan / tilt head to be fixed at the corresponding position of the vertical slide rail by the support buckle; When the collection of the ground point cloud data is completed, the support buckle is controlled to be recovered, so that the lifting motor controls the dual-degree-of-freedom pan-tilt head to descend to the lowest point of the vertical slide rail.

4. The method according to claim 3, It is characterized in that The radar carrying part carried by the dual-degree-of-freedom gimbal and the lifting motor are connected by a rotating motor; the method further includes: If the radar collects the pitch angle when collecting the ground point cloud data, and it is determined that the sparsity of the ground point cloud data is greater than a preset threshold due to the collected pitch angle, the pitch angle of the radar is adjusted by controlling the dual-degree-of-freedom gimbal through the rotating motor to supplement the collection of the ground point cloud data and obtain ground point cloud data that reaches the preset threshold.

5. The method according to claim 1, It is characterized in that The controlling the low-altitude collection device to collect low-altitude fruit tree data of the target orchard comprises: controlling the low-altitude collection device to collect low-altitude fruit tree data of the target orchard based on a bow-shaped flight path; And, performing path planning according to the global orchard map to obtain the orchard map route includes: performing global bow-shaped full coverage path planning according to a heuristic search algorithm to obtain the orchard map route.

6. The method according to claim 1, It is characterized in that The registering and fusing the low-altitude point cloud data and the ground point cloud data to obtain complete orchard point cloud data includes: Constructing a reference point cloud set based on the ground point cloud data; Constructing a point cloud set to be registered according to the low-altitude point cloud data; Calculate the geometric distance between any two points in the reference point cloud set and the point cloud set to be registered; Taking minimizing the variance of the geometric distance as an iterative goal, the reference point cloud set and the to-be-registered point cloud set are registered and fused to obtain the complete orchard point cloud data.

7. The method according to claim 6, It is characterized in that The step of calculating the porosity corresponding to each fruit tree in the target orchard according to the complete orchard point cloud data comprises: voxelize the canopy of fruit trees in the target orchard according to the complete orchard point cloud data to obtain a plurality of voxels; Calculate the total projection area of ​​the multiple volume elements and the two-dimensional projection area of ​​the fruit tree canopy; The porosity corresponding to each of the fruit trees is calculated based on the total projection area and the two-dimensional projection area.

8. The method according to claim 7, It is characterized in that The step of determining, from the target orchard, a low-altitude supplementary collection target area having a porosity outside a preset porosity range according to the porosity corresponding to each of the fruit trees, comprises: Determining at least one target fruit tree located outside the preset porosity range from the porosities corresponding to the fruit trees; Assigning identifiers to the at least one target fruit tree in a sequence starting from zero, and superimposing the identifiers and geographic information into a preset virtual orchard map, wherein the complete orchard point cloud data includes the geographic information; The coordinate information corresponding to the identifier of the at least one target fruit tree is extracted from the preset virtual orchard map to obtain the low-altitude supplementary collection target area.

9. The method according to claim 7, It is characterized in that The low-altitude acquisition device includes a camera; the controlling the low-altitude acquisition device to acquire the supplementary acquisition point cloud data of the target area includes: For any fruit tree among the at least one target fruit tree, determining a crown center of the any fruit tree; When the camera points to the center of the tree crown, the low-altitude collection device is controlled to take the center of the tree crown as the center of the circle, and through the tracking control algorithm, it is possible to realize circumnavigation of the arbitrary fruit tree, obtain the target image data collected by the camera, and obtain the supplementary collection point cloud data.

10. A ground-air coordinated orchard parameter detection system, It is characterized in that The system includes: a central control device, a ground collection device, and a low-altitude collection device; wherein, The central control device is respectively connected to the ground collection device and the low-altitude collection device for communication; The central control device is used to execute the method according to any one of claims 1 to 9.

Images