An in-situ fine panoramic three-dimensional reconstruction device and growth state detection method for field crop phenotypes based on a unmanned aerial vehicle platform

Abstract

The application discloses a kind of field crop phenotype in situ fine panoramic three-dimensional reconstruction device and growth state detection method based on unmanned aerial vehicle platform.Belongs to the field of agricultural intelligence, through the data fusion of laser radar and monocular camera carried by unmanned aerial vehicle, environmental perception is carried out to obtain crop position;According to crop position information, unmanned aerial vehicle navigates to crop position, and uses internal reflection type panoramic imaging system to finely reconstruct three-dimensional of crop;Three-dimensional vigor model of acquisition point crop is obtained and feature extraction is carried out, and abnormal crop is extracted by comparing three-dimensional vigor model, and the cause of abnormality is analyzed.To this end, disease and insect pests, weeds or lack of fertilizer detection is carried out, and further decision on crop health is made.The application solves the problem of low efficiency and difficult inspection of traditional manual inspection to obtain crop data.Provides effective help for the development of precision agriculture, and the structure is simple and compact and the operation efficiency is high.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of agricultural intelligence, and particularly relates to a field crop phenotype in-situ fine panoramic three-dimensional reconstruction device based on a UAV platform and a growth state detection method. BACKGROUND

[0002] Fine agriculture is a modern agricultural concept. The core of fine agriculture is to use modern information technology and high-tech means to manage the agricultural production process finely to realize efficient, environmentally friendly and sustainable development of agricultural production. In order to realize more fine agricultural management, the existing technology mainly relies on manual inspection to collect crop data. However, this method is not only inefficient, but also prone to large errors. Especially in complex terrain conditions such as mountains, the work of inspecting crops becomes more difficult. In order to better promote the practice of fine agriculture, it is necessary to find a more efficient and accurate monitoring and data collection method.

[0003] With the continuous development of science and technology, the UAV technology has been widely used in various fields. For example, using a UAV equipped with a mechanical arm for picking can greatly improve the picking efficiency, and using a UAV for crop disease and pest inspection can greatly reduce the labor intensity, promoting the upgrading of agricultural production methods. SUMMARY

[0004] The application provides a field crop phenotype in-situ fine panoramic three-dimensional reconstruction device based on a UAV platform and a growth state detection method, which solves the problem that traditional manual inspection for obtaining crop data is not only inefficient, but also has large errors, and for crops in complex environments, there is a situation of inspection difficulty, which seriously restricts the development of fine agriculture. After setting the target point, autonomous mapping can be carried out without manual intervention, which is convenient to use, saves a lot of time and labor cost, provides convenience for obtaining crop environment in complex environment, and provides effective help for the development of fine agriculture.

[0005] In order to achieve the above object, the present application is realized by the following technical scheme: a field crop phenotype in-situ fine panoramic three-dimensional reconstruction device based on a UAV platform, comprising a ground end and a UAV end, the ground end and the UAV end are connected through wireless communication, the UAV end comprises a rack, the rack is provided with an on-board computer, a flight controller, a model airplane battery, a laser radar, a monocular camera, four motor drive assemblies and four electronic speed controllers, the on-board computer is connected with the flight controller, the laser radar, the monocular camera and the wireless communication module through a data interface, and the flight controller is connected with the electronic speed controllers through PWM signal lines; the four motor drive assemblies are connected with the four electronic speed controllers through motor lines; the model airplane battery is connected with the on-board computer, the flight controller, the laser radar and the monocular camera through a bus; an industrial camera and a ring light source are arranged below the rack, and a UAV arm is fixed to an inner reflective cylinder;

[0006] The rack comprises an upper bottom plate, a lower bottom plate and four arms fixed between the upper bottom plate and the lower bottom plate;

[0007] The on-board computer, the laser radar and the monocular camera are all fixed to the bottom of the lower bottom plate, and the laser radar is arranged above the monocular camera;

[0008] The flight controller is fixed to the top of the upper bottom plate, and the model airplane battery is fixed between the upper bottom plate and the lower bottom plate;

[0009] The motor drive assembly comprises a brushless motor and a paddle, and the output end of the brushless motor is fixed to one end of the rotating shaft of the paddle;

[0010] The brushless motor is fixed to the top of the arm, and the four electronic speed controllers are respectively fixed to the top of one arm for controlling the rotating speed of the brushless motor;

[0011] The inner reflective cylinder is made of opaque acrylic, the inside is a reflective film, and the industrial camera and the ring light source are located at the top of the light shielding cylinder.

[0012] The flight controller is built-in and integrated with a plurality of sensors, including but not limited to a barometer, an accelerometer, a gyroscope, an electronic compass, a GPS and a wireless communication module.

[0013] The wireless communication module is used for wireless communication with the ground end, receiving the control signal of the ground end, and transmitting the control signal to the on-board computer;

[0014] The laser radar and the monocular camera are both used for acquiring regional image data;

[0015] The on-board computer is used for collecting regional image data and UAV flight path planning, and performing three-dimensional reconstruction and real-time positioning.

[0016] The application further provides that the on-board computer comprises an MCU module, an environment sensing module, a path planning module, a three-dimensional reconstruction module and a crop growth detection module, and the MCU module is connected with the environment sensing module, the path planning module, the three-dimensional reconstruction module and the crop growth detection module respectively.

[0017] The application further provides that the industrial camera is fixed between the bottom plate and the upper plate, and the annular light source is fixed at the lower part of the bottom plate.

[0018] The application further provides that the environment sensing is used for collecting regional image data, and the regional scene environment sensing and the three-dimensional coordinate positioning of crops are carried out in combination with the multi-sensor data fusion of the laser radar and the monocular camera.

[0019] The application further provides that the path planning module is used for transmitting real-time flight control instructions to the flight controller after the position of the crops is determined.

[0020] The application further provides that the three-dimensional reconstruction module is used for providing light for the industrial camera by the annular light source after the unmanned aerial vehicle reaches a plant position, and the industrial camera is used for capturing images of the plant by the industrial camera in combination with the internal reflection type reflector bucket to carry out panoramic three-dimensional reconstruction.

[0021] The application further provides a growth state detection method of a field crop phenotype in-situ fine panoramic three-dimensional reconstruction device based on an unmanned aerial vehicle platform, comprising the following steps:

[0022] Step (1) environment sensing, the point cloud data collected by the laser radar is preprocessed, the ground information is filtered out by straight-through filtering, the points with large color difference from the plant itself are removed by color filtering, including soil and leaf edge miscellaneous redundant points, and the outliers are removed by statistical filtering;

[0023] Step (2) path planning, the unmanned aerial vehicle is navigated to the plant position by using the A* algorithm;

[0024] Step (3) three-dimensional reconstruction, based on the images collected by the panoramic imaging system, the camera can capture the image formed by the real view point on the target object, and also can capture the image formed by the virtual view point on the circular track after the point is reflected by the internal reflection mirror;

[0025] Step (4) crop growth detection, the crop growth detection module processes and analyzes the images and point clouds obtained by the panoramic reconstruction based on the built neural network model.

[0026] The application is further provided with: the crop growth detection module processes and analyzes the image and point cloud obtained through panoramic reconstruction based on the built neural network model.

[0027] The effective effects of the application are:

[0028] (1) The unmanned aerial vehicle end of the application can greatly improve the efficiency of surveying and mapping through autonomous flight and surveying and mapping, and can automatically plan a flight path, obtain regional image and three-dimensional information, and achieve real-time updating of the map under the setting of the on-board computer. After setting the target area, autonomous crop growth state detection can be performed without manual intervention, which is convenient to use and saves a lot of time and labor cost, provides convenience for crop environment acquisition in complex environments, and provides effective help for the development of precision agriculture.

[0029] (2) The application can obtain regional image data at a high frame rate and accurately locate the crop plants through multi-sensor information fusion of the monocular camera and the laser radar, thereby providing protection for high-quality plant panoramic three-dimensional generation.

[0030] (2) The application can accurately reconstruct the three-dimensional plant through the internal reflection panoramic reconstruction system, and the system only includes an industrial camera and an internal reflection cylinder, so that the system structure is simple and reliable, and the cost is greatly reduced.

[0031] (3) The application can process the obtained image and point cloud through the crop growth state detection module, and realize disease and pest detection, weed detection, and fertilizer deficiency detection through the setting of the neural network model, thereby achieving the purpose of real-time and comprehensive monitoring of the health condition of the plant, and providing protection for improving the growth quality and yield of the crop. BRIEF DESCRIPTION OF DRAWINGS

[0032] The above and / or additional aspects and advantages of the application will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings, in which:

[0033] Figure 1 The external structure diagram of the front side of the unmanned aerial vehicle end of the application.

[0034] Figure 2 The external structure diagram of the bottom view of the unmanned aerial vehicle end of the application.

[0035] Figure 3 is the front view of the unmanned aerial vehicle of the present application.

[0036] Figure 4 is the system principle diagram of the on-board computer of the present application.

[0037] Figure 5 is the panoramic imaging schematic diagram of the present application.

[0038] Figure 6 is the horizontal stack processing of the three-dimensional reconstruction process of the present application.

[0039] Figure 7 is the depth information calculation of the three-dimensional reconstruction process of the present application,

[0040] (a) is the corresponding point of the polar image, (b) is the inner light path diagram of the cylinder. DETAILED DESCRIPTION

[0041] Embodiments of the present application are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference signs represent the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the drawings are exemplary and are intended to explain the present application, and cannot be understood as a limitation of the present application.

[0042] In the present application, unless otherwise explicitly specified and limited, the terms "mounting", "connection", "connecting", "fixing" and the like should be understood in a broad sense, for example, can be fixedly connected, or can be detachably connected, or integrally connected; can be mechanically connected, or can be electrically connected; can be directly connected, or can be indirectly connected through an intermediate medium, or can be the internal communication of two elements. For those skilled in the art, the specific meanings of the above terms in the present application can be understood according to the specific circumstances.

[0043] The unmanned aerial vehicle platform-based field crop phenotype in-situ fine panoramic three-dimensional reconstruction device and growth state detection method provided by the present application comprises 1, an on-board computer; 2, a rack; 3, a flight controller; 4, a motor drive assembly; 5, a laser radar; 6, a monocular camera; 7, an industrial camera; 8, a ring light source; 9, an inner reflection type cylinder; 10, an MCU module; 11, an environment perception module; 12, a path planning module; 13, a three-dimensional reconstruction module; 14, a crop growth detection module.

[0044] The flight controller 3 is built-in and integrated with several sensors, including but not limited to barometers, accelerometers, gyroscopes, electronic compasses, GPS and wireless communication modules. Ground end workers can obtain real-time data and information of the unmanned aerial vehicle end during flight through the wireless signal transceiver, to ensure the smooth progress of the task.

[0045] As detailed in the description, the motor drive assembly includes a brushless motor and a blade. The output end of the brushless motor is fixedly connected to one end of the blade shaft. The brushless motor is fixedly connected to the top of the arm, and four electronic speed controllers are respectively fixedly connected to the top of the arm to control the rotation speed of the brushless motor.

[0046] The wireless transceiver is used to communicate wirelessly with the ground station, receive control signals from the ground station, and transmit the control signals to the onboard computer.

[0047] Both lidar and monocular cameras are used to acquire regional image data;

[0048] The onboard computer is used to collect regional image data and perform 3D reconstruction, real-time positioning, and map building. Specifically, the onboard computer includes an MCU module 10 that interfaces with the environmental perception module 11, path planning module 12, 3D reconstruction module 13, and crop growth detection module 14.

[0049] The environmental perception module 11 is used to collect regional image data and, in combination with the internal and external parameters of the lidar and monocular camera 7, to perform three-dimensional coordinate positioning of the plants in the region.

[0050] The path planning module 12 is used to transmit real-time flight control commands to the flight controller 3 after determining the target point of the plant, in conjunction with the output of the A* algorithm.

[0051] The 3D reconstruction module 13 is used to perform accurate 3D reconstruction of the plant through the polar imaging panoramic reconstruction system.

[0052] The crop growth detection module 14, based on a constructed neural network model, processes and analyzes the images and point clouds acquired through panoramic reconstruction. Specifically, it compares the 3D model of the crop at the collection point location with the predicted 3D model of crop growth in the database to obtain the growth status corresponding to the collection point location. Simultaneously, based on the comparison results, crop information from abnormal collection points is input into a pre-trained model to analyze the causes of the anomalies. The causes of the anomalies include at least an anomaly type or an anomaly type and its categories, with the anomaly type including at least one or more of pests and diseases, weeds, and fertilizer deficiency.

[0053] The working principle of this invention is as follows:

[0054] Step (1) Environmental perception involves preprocessing the point cloud data collected by the lidar. Specifically, this includes steps such as: pass-through filtering to remove ground information; color filtering to remove points with colors significantly different from the plant itself, such as redundant points of mixed colors on the soil and leaf edges; and statistical filtering to remove outliers.

[0055] Based on 2D image detection, target detection of 2D crops is performed on images acquired by a monocular camera.

[0056] Specifically, in some embodiments, the identified crop can be a point or a region on a two-dimensional image. Specifically, target detection of the two-dimensional crop is achieved by a deep learning framework such as YOLO or a machine learning model. The detection method is only illustrative, and in actual tests, a person skilled in the art can select according to actual needs as long as the two-dimensional coordinate positioning can be achieved. Details are not repeated here.

[0057] Further, the monocular camera and the laser radar are fused for multi-dimensional information, and the two-dimensional image information and the three-dimensional space point cloud information are synchronously collected to realize accurate spatial positioning of the plant individuals.

[0058] Specifically, first, the three-dimensional point cloud data captured by the laser radar is converted according to the external parameter matrix and mapped to the two-dimensional imaging plane of the camera. Combined with the camera intrinsic matrix parameters, accurate projection of the point cloud data on the camera imaging plane is realized. The formula is as follows:

[0059]

[0060] In the formula, (X, Y, Z) is the coordinate of a point cloud in the laser radar coordinate system, is the external parameter matrix from the laser radar coordinate system to the camera coordinate system, (u, v) is the projection coordinate of the point cloud in the image pixel coordinate system, f x and f y are the focal lengths of the camera in the X and Y directions, u0 and v0 are the coordinates of the principal point (the intersection of the optical axis and the image plane) of the image plane, Z c represents the depth of the point in the camera coordinate system.

[0061] In order to align the space between the camera and the three-dimensional laser radar, joint calibration is needed to obtain the projection matrix from the laser radar frame to the camera plane. Then the external parameter matrix between the camera and the laser radar coordinate system is calculated to realize the consistency of the coordinates of the two in space. The formula is as follows:

[0062] a m,i :θ m,i X+d m,i = 0

[0063] a n,i :θ n,i X+d n,i = 0

[0064] In the formula, a m,i and a n,i respectively represent the plane model of the camera and the laser radar; θ is the plane normal vector; d is the distance from the plane to the far point.

[0065] By minimizing the objective function, R and T are obtained when the following objective function formula takes the minimum value. For each point in each sample point cloud, the projection position of the point on the camera image is calculated, and the error between the predicted position and the true position of the point is calculated. However, the weighted sum of the errors of all points is calculated, where the weight is the number of each point in the point cloud. Minimizing this error can obtain the optimal external parameter matrix, thereby realizing the joint calibration of the camera and the lidar.

[0066]

[0067] where R and T are the rotation and translation parts of the external parameter matrix, respectively, which describe the conversion relationship from the lidar coordinate system to the camera coordinate system; n is the number of samples, and l(i) is the number of points in the i-th sample point cloud, is the dimension coordinate of the i-th sample. θ m,i and d m,i is the projection angle and distance of the sample in the camera image, and m is a parameter in the camera intrinsic matrix.

[0068] Further, a two-dimensional vision technology is used to finely extract a region of interest (ROI) from a three-dimensional image, and a two-dimensional ROI is mapped to a truncated cone region proposal containing point cloud data through a sensor joint calibration technology, to obtain a three-dimensional spatial position of each plant.

[0069] Step (2) path planning: an A* algorithm is used to navigate the unmanned aerial vehicle to the plant position. A* is a heuristic search algorithm used to find the shortest path between two points in a graph or grid. It combines the advantages of breadth-first search and depth-first search, and balances exploration and exploitation. The working principle of the A* algorithm is as follows:

[0070] f(n)=g(n)+h(n)

[0071] where f(n) is a cost function, g(n) represents the distance from the current calculated grid point to the starting point (using Manhattan distance), and h(n) is the distance from the current calculated grid point to the end point. According to this rule, the grid with the smallest f(n) around is found as the next point for further calculation. The unvisited grid is put into the open table, and the visited grid is put into the close table. Repeat the above steps until the target node is found or the open list is empty.

[0072] Specifically, in some embodiments, the navigation path planning of the unmanned aerial vehicle can be performed by Dijkstra algorithm, depth-first search, or machine learning and deep learning. The navigation method is only illustrative here, and those skilled in the art can select according to actual needs in actual testing. As long as the unmanned aerial vehicle is navigated to the position above the target crop, further description is not needed here.

[0073] Step (3) 3D reconstruction, based on the images captured by the panoramic imaging system, the camera can capture the point on the target object through the real view point image, and also can capture the virtual view point image on the circular track after the reflection of the inner mirror.

[0074] Specifically, the light entering the cylindrical inner mirror can be reflected multiple times, and through appropriate mirror length and camera position focal length settings, an image that has only been reflected once can be obtained. Therefore, after one reflection, as shown in the system, the radial section in each direction has two virtual view points symmetrical to the optical axis. At the same time, since the complete imaging system is composed of continuous radial sections, the virtual view points have the circular track shown in the figure, with the center located on the optical axis of the camera. For a point on a radial section, the point will be imaged three times along the corresponding image radial line (polar line). Figure 5

[0075] Further, the captured images are subjected to simple image processing, as shown in Figure 6 , i.e. converting the polar line into a horizontal line form, so as to obtain the horizontal stacked images about different view points. Using a stereo matching method based on image area blocks, the cross-correlation degree between the image area blocks is taken as the similarity measurement model, so as to obtain relatively dense matching point pairs, and the three images obtained are subjected to stereo matching respectively, and the cross-correlation degree of the image area blocks is calculated as shown in the formula:

[0076]

[0077] In the above formula, f is the image to be matched, wherein (x, y) represents the pixel coordinates on the image, and the value of f(x, y) is the gray value of the pixel point. (u, v) represents the matching area of the template on the image. is the average gray value of the template image, is the average gray value of the template area in the image to be matched f(x, y), and the point with coordinates (x, y) in the image f corresponds to the point with position (x-u, y-v) in the template. If there is no corresponding point at the edge of the image f, the default gray value is 0. The summation in the formula is only for the area corresponding to the template on the image f. The template image is slid on the original image, and for each position (u, v), the normalized cross-correlation coefficient is calculated, and the position (u, v) that makes γ reach the maximum value is found. This position is the best matching position of the template image in the original image;

[0078] ​Depth point cloud reconstruction is performed: h0, h1, and h2 are distances of points P0, P1, and P2 to the center of the epipolar image, W0 is the diameter of the inner circle of the epipolar image, a is the included angle of the current radial line P0, P1, P2 and the horizontal diameter of the epipolar image, D is the diameter length of the epipolar image; point O is the camera optical center, f is the camera imaging focal length, theta is the half vertex angle of the conical internal reflector, r is the left port radius of the internal reflector, m1 and m2 are radial sections of the internal reflector, point P is a point in the scene, P1 and P2 are images of the point P with respect to two mirror surfaces, h is the distance of the point to the optical axis, and d is the horizontal distance of the point P to the camera optical center;

[0079] The coordinates (x0, y0) and (x1, y1) of points P0 and P1 in the horizontal stack image and the distances h0 and h1 of the two points to the center of the epipolar image have the following relationship:

[0080]

[0081] Taking point O as the coordinate origin, by listing the equations of the mirror section m1 and the straight line p1p1', and combining the geometric method of triangle similarity, the values of h and d can be expressed as follows:

[0082]

[0083] In the above formula, the value of d is the Z value of point P in the camera coordinate system, and the X, Y, and Z values can be expressed as follows:

[0084]

[0085] Further, by the above calculation, a three-dimensional coordinate point of the target object can be obtained, and by performing the above calculation on all matching point pairs, a three-dimensional point cloud of the target object can be obtained, and the color information of the points on the center circle of the epipolar image corresponding to the three-dimensional points can be extracted, thereby obtaining a three-dimensional point cloud with color information.

[0086] Step (4) crop growth detection: the crop growth detection module processes and analyzes the images and point clouds obtained by panoramic reconstruction based on the built neural network model. Specifically, the crop three-dimensional model of the collection point position is compared with the predicted crop growth three-dimensional model in the database to obtain the growth state corresponding to the collection point position. At the same time, based on the comparison result, the crop information of the abnormal collection point is put into the pre-trained model to analyze the abnormal reason, wherein the abnormal reason at least includes an abnormal type or an abnormal type and an abnormal category, and the abnormal type at least includes one or more of disease and pest, weed, fertilizer deficiency, and soil temperature and humidity anomaly.

[0087] Specifically, in some embodiments, the acquired characteristics of the crop at each collection point include, but are not limited to, crop height, crop width, crop leaf density, etc. Specifically, the acquisition of various characteristics of the crop is achieved by a deep learning framework such as pointnet or a machine learning model. The selection of various characteristics and the method of acquiring various characteristics are only exemplary descriptions, and in actual tests, a person skilled in the art can select according to actual needs. As long as the three-dimensional point cloud features of the collected crop are extracted, the rest is omitted here.

[0088] Specifically, in some embodiments, the database is a three-dimensional model of the crop at different growth times and under different climate environment data, and various characteristics of the crop. The selection of characteristics is only exemplary description, and in actual tests, a person skilled in the art can select according to actual needs, and the rest is omitted here.

[0089] Specifically, in some embodiments, the comparison between the crop growth three-dimensional model of the collection point position and the expected crop growth three-dimensional model is as follows: first, place them in a unified coordinate system to ensure that their central axes and bottom surfaces are completely aligned. Then, if the crop growth three-dimensional model of the collection point position fails to reach a preset coverage threshold (for example, the threshold can be 70% or 80%, and the specific value is selected by a person skilled in the art according to actual application requirements), the collection point will be considered as an abnormal collection point.

[0090] In addition, in a specific embodiment, the method further includes constructing a mature crop three-dimensional model database. When the coverage of the crop growth three-dimensional model of the collection point position to the expected crop mature three-dimensional model reaches a preset mature threshold (for example, the threshold can be 90% or 100%, and the specific value is selected by a person skilled in the art according to actual needs), the system will determine that the crop has reached a mature state.

[0091] In another embodiment, for the identified abnormal collection point, its corresponding three-dimensional model and extracted features are input into a deep learning model to analyze and determine the specific reason for the abnormality. These abnormal reasons can include pest damage, weeds, and fertilizer deficiency, etc. Pest damage data covers distribution location, species and quantity; weed data includes distribution location, species and quantity; and fertilizer deficiency data involves missing fertilizer type and degree.

[0092] Further, in some embodiments, by combining two-dimensional image recognition algorithms and three-dimensional point cloud clustering segmentation algorithms, accurate identification and extraction of pests and weeds are achieved. At the same time, by using deep reinforcement learning technology, the extracted crop characteristics are analyzed in depth to evaluate the nutrient deficiency status of the crop. The selection of these identification and extraction methods aims to provide flexibility and adaptability to meet the actual needs in different application scenarios.

[0093] It should be understood that although the present specification is described in terms of various embodiments, each of which describes only one implementation, the specification is intended to cover all possible implementations that are within the scope of the application, which is defined by the claims. One skilled in the art will readily recognize from the disclosure herein, that alternative embodiments of the present application can be constructed from a number of approaches already known in the art, which do not depart from the spirit and scope of the present application. The individual features of the various embodiments of this application each will be recognized by one of ordinary skill in the art to be an innovative application that alone would entitle the applicant to a patent, but the present application is intended to cover each and every combination of the individual features disclosed herein and any other innovative feature that would be recognized by those of ordinary skill in the art to be an innovative application that alone would entitle the applicant to a patent.

[0094] The foregoing detailed description has set forth various embodiments of the devices and / or processes via the use of specific terminology. However, embodiments thereof can be practiced without the specific details (e.g., dimensions, times, direction, order of sequence, etc.) that are described herein. It is to be understood that the foregoing detailed description of the devices and / or processes is susceptible to various modifications, adaptations, and implementations by one of ordinary skill in the art. It is to be understood that such modifications, adaptations, and implementations are intended to fall within the scope of the present application.

Claims

1. An unmanned aerial vehicle platform based in-situ fine panoramic three-dimensional reconstruction device for field crop phenotyping, characterized in that: The application relates to a ground terminal and a UAV terminal, the ground terminal and the UAV terminal are communicated through wireless communication, the UAV terminal comprises a rack, an on-board computer, a flight controller, a model battery, a laser radar, a monocular camera, four motor driving assemblies and four electronic speed controllers are arranged on the rack, the on-board computer is connected with the flight controller, the laser radar, the monocular camera and a wireless communication module through a data interface, the flight controller is connected with the electronic speed controllers through PWM signal lines, the four motor driving assemblies are connected with the four electronic speed controllers through motor lines, the model battery is connected with the on-board computer, the flight controller, the laser radar and the monocular camera through a bus, an industrial camera and a ring light source are arranged below the rack, and a UAV arm is fixed with an inner reflection type cylinder. The rack comprises an upper bottom plate, a lower bottom plate and four arms fixed between the upper bottom plate and the lower bottom plate. The on-board computer, the laser radar and the monocular camera are fixed at the bottom of the lower bottom plate, and the laser radar is arranged above the monocular camera. The on-board computer comprises a three-dimensional reconstruction module, the ring light source is used for providing light for the industrial camera, and the three-dimensional reconstruction module is used for reconstructing a panorama of a plant through images captured by the industrial camera and the inner reflection type light bucket. The flight controller is fixed at the top of the upper bottom plate, and the model battery is fixed between the upper bottom plate and the lower bottom plate. The motor driving assembly comprises a brushless motor and a paddle, and the output end of the brushless motor is fixed with one end of a paddle rotating shaft. The brushless motor is fixed at the top of the arm, and the four electronic speed controllers are respectively fixed at the top of one arm and used for controlling the rotating speed of the brushless motor. The inner reflection type cylinder is made of opaque acrylic, the inside of the cylinder is provided with a reflective film, and the industrial camera and the ring light source are arranged at the top of the light shielding cylinder. The light entering the cylindrical inner reflection mirror is reflected for many times, and only one reflected image is obtained through the length of the reflection mirror and the focal length of the camera position; after one reflection, each radial section of the system in each direction has two virtual view points which are symmetrical relative to the optical axis, and since the complete imaging system is composed of continuous radial sections, the virtual view points have a circular track with the center located on the optical axis of the camera, and for a point on a radial section, the point is imaged three times along the corresponding image radial line; Simple image processing is conducted on the captured image, that is, the polar line is converted into the form of a horizontal line, so that a horizontal stacked image about different view points is obtained; a stereo matching method based on image area blocks is used to take the cross-correlation degree of the image area blocks as a similarity measurement model, so that relatively dense matching point pairs are obtained, and the three images obtained are respectively subjected to stereo matching, and the cross-correlation degree of the image area blocks is calculated as shown in the formula: ； In the above formula The image to be matched, where Represents pixel coordinates on the image. The value is the grayscale value of that pixel. This indicates the matching region of the template on the image. It is the average grayscale value of the template image. The image to be matched is in the template region. The average gray level of the image, and the image The median coordinate is The position of the point in the template is The points correspond to each other, if in If there are no points at the edge that correspond to the template, their grayscale value is assumed to be 0. The summation in the formula only applies to the image. For the region corresponding to the template, slide the template image onto the original image, for each position... Calculate the normalized cross-correlation coefficient and find the coefficient that makes the cross-correlation coefficient equal to the normalized cross-correlation coefficient. Reaching the maximum value The position is the best matching position of the template image in the original image; Depth point cloud reconstruction is performed: , , and are the distances from the point , and to the center of the epipolar image, is the diameter of the inner circle of the epipolar image, and α is the angle between the current radial line , , and the horizontal diameter of the epipolar image, is the diameter length of the epipolar image; point is the camera optical center, is the camera imaging focal length, is the half vertex angle of the conical inner reflector, is the radius of the left port of the inner reflector, and are the radial sections of the inner reflector, point is a point in the scene, and are the images of the point with respect to the two reflector surfaces, is the distance from the point to the optical axis, is the horizontal distance from the point to the camera optical center; points and coordinates in the horizontal stack plot , have the following relationship between the distances of these two points to the center of the polar image circle , ​ ； With point O as the coordinate origin, by listing the equations of the mirror section and the straight line , and combining the geometric method of triangle similarity, the values of h and d can be expressed as follows: ； The value of the point is the point in the camera coordinate system with the value , , can be expressed as follows: ； Through the above calculation, a three-dimensional coordinate point of the target object is obtained, all the matching point pairs are subjected to the above calculation, so that a three-dimensional point cloud of the target object is obtained, and the color information of the points on the center circle of the three-dimensional point pair is extracted, so that a three-dimensional point cloud with color information is obtained.

2. The apparatus of claim 1, wherein: The flight controller is built-in and integrated with several sensors, including a barometer, an accelerometer, a gyroscope, an electronic compass, a GPS, and a wireless communication module.

3. The apparatus of claim 2, wherein: The wireless communication module is used for wireless communication with the ground terminal, receiving control signals from the ground terminal, and transmitting the control signals to the onboard computer.

4. The apparatus of claim 1, wherein: The onboard computer also includes an MCU module, an environment perception module, a path planning module, a crop growth detection module, and a database. The environment perception module is used to collect regional image data and perform environment perception and three-dimensional coordinate positioning of crops by combining multi-sensor data fusion of laser radar and monocular cameras. The path planning module is used to output navigation targets to the controller and transmit real-time flight control instructions to the flight controller after determining the crop location. The crop growth detection module processes and analyzes the images and point clouds obtained by panoramic reconstruction based on a built neural network model. The database contains three-dimensional models of crops at different growth times and climate environment data, as well as various characteristics of the crops.

5. A growth state detection method based on an unmanned aerial vehicle platform field crop phenotype in-situ fine panoramic three-dimensional reconstruction device, characterized by: The pest data includes pest distribution location, pest species, and pest quantity. The weed data includes weed distribution location, weed species, and weed quantity. The fertilizer deficiency data includes deficiency fertilizer type and deficiency degree. The steps include: Step (1) Environment perception: pre-process the point cloud data collected by the laser radar, and filter out the ground information by straight-through filtering. Color filtering removes points with large color differences from the plant itself, including soil and leaf edge miscellaneous redundant points. Statistical filtering removes outliers. Step (2) Path planning: use A* algorithm to navigate the UAV to the plant location. Step (3) Three-dimensional reconstruction: based on the images collected by the panoramic imaging system, the camera can capture the point on the target object through the real viewpoint image, and also capture the virtual viewpoint image on the circular track after the reflection of the internal mirror. Step (4) Crop growth detection: the crop growth detection module processes and analyzes the images and point clouds obtained by panoramic reconstruction based on a built neural network model. The specific process of step (3) is: The light entering the cylindrical inner reflector is reflected for many times, and the image is obtained by setting the length of the reflector and the focal length of the camera position to be reflected only once; after one reflection, each radial section of the system in each direction has two virtual view points symmetrical to the optical axis, and since the complete imaging system is composed of continuous radial sections, the virtual view points have a circular trajectory with the center on the optical axis of the camera, and for a point on a radial section, the point will be imaged three times along the corresponding image radial line; The captured image is simply image-processed, i.e., the polar line is converted into a horizontal line, and a horizontal stacked image about different view points is obtained; a stereo matching method based on image area blocks is used, the cross-correlation between the image area blocks is taken as a similarity measurement model, so that relatively dense matching point pairs are obtained, and the three images obtained are respectively subjected to stereo matching, and the cross-correlation of the image area blocks is calculated as shown in the formula: ； In the above formula The image to be matched, where Represents pixel coordinates on the image. The value is the grayscale value of that pixel. This indicates the matching region of the template on the image. It is the average grayscale value of the template image. The image to be matched is in the template region. The average gray level of the image, and the image The median coordinate is The position of the point in the template is The points correspond to each other, if in If there are no points at the edge that correspond to the template, their grayscale value is assumed to be 0. The summation in the formula only applies to the image. For the region corresponding to the template, slide the template image onto the original image, for each position... Calculate the normalized cross-correlation coefficient and find the coefficient that makes the cross-correlation coefficient equal to the normalized cross-correlation coefficient. Reaching the maximum value The position is the best matching position of the template image in the original image; Depth point cloud reconstruction is performed: , , and are the distances from the point , and to the center of the epi-image, is the diameter of the inner circle of the epi-image, and α is the angle between the current radial line , , and the horizontal diameter of the epi-image, is the length of the diameter of the epi-image; point is the camera optical center, is the camera imaging focal length, is the half vertex angle of the conical inner reflector, is the radius of the left port of the inner reflector, and are the radial sections of the inner reflector, point is a point in the scene, and are the images of the point with respect to the two reflector surfaces, is the distance from the point to the optical axis, is the horizontal distance from the point to the camera optical center; points and coordinates in the horizontal stack plot , have the following relationship between the distances of these two points to the center of the polar image circle , ​ ； With point O as the coordinate origin, by listing the equations of the mirror section and the straight line , and combining the geometric method of triangle similarity, the values of h and d can be expressed as follows: ； The value of the point is the point in the camera coordinate system with the value , , The value of the point ； Through the above calculation, a three-dimensional coordinate point of the target object is obtained, all the matching point pairs are subjected to the above calculation, and a three-dimensional point cloud of the target object is obtained, and the color information of the points on the center circle of the polar image corresponding to the three-dimensional point pairs is extracted, so that a three-dimensional point cloud with color information is obtained.

Images