Farmland boundary automatic identification method of agricultural unmanned aerial vehicle

Abstract

The invention discloses a farmland boundary automatic identification method for an agricultural unmanned aerial vehicle, and the method comprises the steps: comprehensively collecting multi-mode remote sensing data of a visible light image, a near-infrared image and a laser point cloud, combining airborne real-time positioning and attitude determination information, and carrying out the deep fusion through spatial spectrum features and elevation features, thereby obtaining a multi-modal remote sensing image; accurate recognition and vectorization output of farmland boundaries are achieved, and the problems that in the prior art, recognition precision is insufficient, the anti-interference capacity is weak, and universality of complex land parcels is poor are effectively solved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of agricultural unmanned aerial vehicles and smart agriculture, and particularly relates to a farmland boundary automatic identification method for an agricultural unmanned aerial vehicle. BACKGROUND

[0002] With the rapid development of modern precision agriculture and smart agriculture technology, accurate identification of farmland boundaries has gradually become one of the important prerequisites for efficient farmland management and precision agricultural operations. Traditional farmland boundary identification mainly adopts manual field investigation or a method based on satellite remote sensing images. However, the manual method is limited by high labor costs, low operation efficiency, and human errors. Although the satellite remote sensing image method has a wide coverage, it has limitations such as limited spatial resolution, great influence of cloud cover, and inability to effectively distinguish complex ground details.

[0003] In recent years, with the rapid maturity of unmanned aerial vehicle remote sensing technology, unmanned aerial vehicle low-altitude remote sensing has been widely applied in farmland information collection and fine management fields due to its high mobility, high spatial resolution, and flexibility, and has become a hot research direction for farmland boundary identification technology.

[0004] CN113221740A discloses a farmland boundary identification method based on an improved UNet network and a RANSAC straight line fitting algorithm. Although the identification accuracy of the farmland boundary is improved, the technical route is mainly aimed at regular rectangular or trapezoidal farmland, and has insufficient adaptability in farmland with complex boundaries, irregular plots, or large differences in vegetation coverage, which may lead to significant deviations in the identification results.

[0005] CN117788822A discloses a farmland boundary positioning method based on semantic segmentation and affine transformation. Although the identification efficiency and universality are improved to some extent, the method mainly relies on single image data and lacks comprehensive utilization of multi-dimensional information such as elevation and terrain. In the face of farmland areas with high spectral similarity of ground objects or large changes in lighting conditions, the segmentation robustness and stability are insufficient.

[0006] Therefore, the existing unmanned aerial vehicle farmland boundary identification technology still cannot simultaneously consider high precision, robustness, and universality.

[0007] In summary, the existing farmland boundary recognition technology generally has poor adaptability to complex plots, lacks multi-dimensional data fusion, and has insufficient robustness of recognition results. In view of the above problems, the present application provides a farmland boundary automatic recognition method of an agricultural unmanned aerial vehicle, which comprehensively collects multi-modal remote sensing data of visible light images, near-infrared images and laser point clouds, and combines with the real-time positioning and pose information of the airborne vehicle, uses spatial spectral features and elevation features for deep fusion, realizes accurate recognition and vector output of the farmland boundary, and effectively solves the problems of insufficient recognition accuracy, weak anti-interference ability and poor universality of complex plots in the prior art. SUMMARY

[0008] This section is intended to summarize some aspects of the embodiments of the present application and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the abstract and title of the specification to avoid obscuring the purpose of this section, the abstract and the title, and such simplifications or omissions cannot be used to limit the scope of the present application.

[0009] In view of the above existing problems, the present application is proposed.

[0010] To solve the above technical problems, the present application provides the following technical scheme: according to the preset flight route planning instruction, driving the agricultural unmanned aerial vehicle to collect continuous and overlapping multi-modal remote sensing data at low altitude above the target farmland, the multi-modal remote sensing data at least including visible light sequence images, near-infrared images and laser point clouds;

[0011] Based on the real-time positioning and pose information of the agricultural unmanned aerial vehicle, the multi-modal remote sensing data is subjected to geometric registration and radiation normalization to generate an orthographic image and a digital terrain model in a unified coordinate system;

[0012] The orthographic image and the digital terrain model are respectively input into a spatial spectral branch network and an elevation branch network for feature fusion, and an agricultural field boundary probability map is output;

[0013] The agricultural field boundary probability map is subjected to conditional random field post-processing and morphological closing operation to obtain a closed and topologically consistent agricultural field boundary vector polygon.

[0014] As a preferred scheme of the farmland boundary automatic recognition method of the agricultural unmanned aerial vehicle, the preset flight route planning instruction comprises:

[0015] The outer boundary vector file of the target farmland is imported into the ground station, and the minimum circumscribed rectangle is automatically calculated and the reference coordinate system is determined;

[0016] Based on the field of view angle of the sensor and the target ground resolution, the parallel flight path lines are generated at equal intervals according to the constraint condition that the forward overlap degree is not less than 80% and the lateral overlap degree is not less than 70%;

[0017] The continuous waypoints are divided according to the length of the track, and each waypoint is sequentially marked with three-dimensional coordinates, target speed and attitude holding time;

[0018] The track is buffered and detected by calling the obstacle database, and if interference is detected, the height of the waypoint is adjusted along the vertical direction until the obstacle avoidance is successful;

[0019] The track and waypoint information that complete the obstacle avoidance correction and meet the overlap requirement are packaged as a flight planning instruction, and the instruction is sent to the flight control system for execution.

[0020] As a preferred scheme of the agricultural unmanned aerial vehicle field boundary automatic identification method, the agricultural unmanned aerial vehicle is driven according to the preset flight planning instruction to cruise at low altitude above the target farmland to collect continuous and overlapping multi-modal remote sensing data, which comprises:

[0021] A synchronous time signal is triggered before each waypoint is reached, so that the visible light camera and the near-infrared camera are exposed at a unified timestamp;

[0022] During the uniform flight of the unmanned aerial vehicle along the track, the laser radar continuously acquires point cloud data at a scanning frequency of not less than 10HZ;

[0023] The image frames and the point cloud frames are aligned by using the airborne time synchronizer, so that the cross-modal time error is not more than 5ms;

[0024] During the idle period of the flight control loop, motion compensation is performed to correct the image smear and point cloud distortion caused by speed changes in real time;

[0025] The multi-modal remote sensing data segments after time alignment and motion compensation are cached to the onboard solid-state storage.

[0026] As a preferred scheme of the agricultural unmanned aerial vehicle field boundary automatic identification method, the orthophoto and digital terrain model in the unified coordinate system are generated, which comprises:

[0027] Based on the onboard real-time positioning and pose information of the agricultural unmanned aerial vehicle, the image and point cloud are initialized with external parameters to establish a route-level coarse registration model;

[0028] The accurate relative pose parameters between images are calculated through feature matching and bundle adjustment, and the point cloud is completed by re-projecting the point cloud according to the accurate relative pose parameters to complete fine registration;

[0029] A dense stereo reconstruction algorithm is used to generate dense point cloud from multi-view images, and the dense point cloud is fused with the original laser point cloud at the voxel level to output high-density fused point cloud;

[0030] The fused point cloud is executed with weighted interpolation to construct a regular grid digital terrain model;

[0031] The digital terrain model is taken as a projection reference to re-map and radiometrically correct the image, so as to generate an orthographic image under a unified coordinate system.

[0032] As a preferred scheme of the farmland boundary automatic identification method of the agricultural unmanned aerial vehicle, the on-board real-time positioning and pose information at least includes:

[0033] Centimeter-level three-dimensional coordinates and a heading angle output by a dual-frequency differential global navigation satellite system;

[0034] Attitude angles and angular velocities output by an inertial measurement unit composed of a three-axis gyroscope, a three-axis accelerometer and a three-axis magnetometer;

[0035] Relative height information output by an on-board barometric altimeter for height constraint in attitude solution;

[0036] The three-dimensional coordinates and the heading angle, the attitude angles and the angular velocities, and the relative height information are fused through an extended Kalman filter to generate a high-frequency attitude and heading solution stream of no less than 50 Hz.

[0037] As a preferred scheme of the farmland boundary automatic identification method of the agricultural unmanned aerial vehicle, the orthographic image and the digital terrain model are respectively input into a spatial spectrum branch network and an elevation branch network for feature fusion, and a farmland boundary probability map is output, including:

[0038] The orthographic image is cropped into image blocks that overlap and have consistent sizes, and the spatial spectrum branch network is input block by block to extract texture, chrominance and vegetation index features;

[0039] The digital terrain model is segmented into elevation blocks corresponding to the image blocks, and the elevation branch network is input to extract local elevation, slope and terrain change rate features;

[0040] The two types of features extracted are spliced through a feature fusion model, and feature weights are redistributed through a channel attention mechanism;

[0041] A pixel-level farmland boundary probability map with consistent original resolution is output through a full convolution classification layer, and an initial binary boundary mask is obtained after thresholding.

[0042] As a preferred scheme of the farmland boundary automatic identification method of the agricultural unmanned aerial vehicle, the spatial spectrum branch network is composed of four-level encoders, four-level decoders and a cross-layer jump connection channel, each level of the encoder comprises two layers of convolution batch normalization activation units, a 2x2 maximum pooling downsampling unit and a feature bypass connected with the symmetric decoder, each level of the decoder restores the resolution by 2x2 deconvolution upsampling first, then splices the corresponding encoder bypass feature, and then executes two layers of convolution batch normalization activation units in sequence to reconstruct the spatial details, and the last level of the decoder outputs a feature map with the same resolution as the input image, and the feature map is compressed by one layer of one multiplication convolution to supply subsequent fusion.

[0043] The elevation branch network is composed of three layers of voxel convolution downsampling modules, two layers of voxel convolution upsampling modules and a residual direct connection channel, each voxel convolution downsampling module comprises a 3x3x3 voxel convolution batch normalization activation unit and a voxel step-down operation with a step of two, for extracting local elevation fluctuation features, each voxel upsampling module restores the spatial size by trilinear interpolation, and then executes a 3x3x3 voxel convolution batch normalization activation unit to refine the terrain edge, and the residual direct connection channel establishes an element-by-element addition relationship between the network input and the deepest layer feature to maintain the overall terrain trend information.

[0044] After the spatial spectrum branch network and the elevation branch network are preliminarily fused by feature splicing, a channel attention mechanism is introduced to weight the spliced features, wherein the channel attention mechanism first performs global average pooling to obtain channel descriptors, then generates a weight vector through a two-level fully connected network, and adjusts the feature response in a channel-by-channel multiplication manner, so as to highlight the high-response features related to the farmland boundary and suppress redundant information.

[0045] As a preferred scheme of the farmland boundary automatic identification method of the agricultural unmanned aerial vehicle, conditional random field post-processing and morphological closing operation are performed on the farmland boundary probability map to obtain a closed and topologically consistent farmland boundary vector polygon, including:

[0046] An eight-neighbor full-connection conditional random field is constructed to optimize the pixel consistency of the farmland boundary probability map and refine the edge position;

[0047] A 3x3 structural element is used to perform morphological closing operation on the optimized image to fill the boundary gaps, and then morphological opening operation is performed to remove isolated noise;

[0048] A boundary tracking algorithm is applied to extract a closed outer ring and a hole ring, and a polyline simplification algorithm is used to compress redundant vertices to form a topologically consistent vector polygon.

[0049] As a preferred scheme of the farmland boundary automatic identification method of the agricultural unmanned aerial vehicle, the farmland boundary vector polygon at least comprises a main outer boundary ring, a hole ring, a vertex coordinate sequence, an attribute field set and a boundary outer package rectangle.

[0050] The present application has the advantages that: the present application collects multi-modal remote sensing data through low-altitude cruising of the unmanned aerial vehicle, realizes accurate data registration by combining real-time positioning and pose information, effectively improves the accuracy, robustness and applicability of farmland boundary identification by using deep feature fusion and probability optimization of spatial spectrum and elevation double-branch networks, and realizes high-quality vector expression of the farmland boundary through subsequent conditional random fields and morphological processing, so that the overall scheme can better meet the requirements of efficient operation and management of precision agriculture in complex farmland environments. BRIEF DESCRIPTION OF DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0052] Figure 1 The flowchart of the farmland boundary automatic identification method of the agricultural unmanned aerial vehicle is shown. DETAILED DESCRIPTION

[0053] In order to make the above-mentioned purposes, features and advantages of the present application more apparent and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings of the specification. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments.

[0054] Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application.

[0055] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, but the present application can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the connotation of the present application, therefore the present application is not limited to the specific embodiments disclosed below.

[0056] According to the embodiments of the present application, combined with the flowchart shown in the figure, a farmland boundary automatic identification method of an agricultural unmanned aerial vehicle specifically comprises the following steps: Figure 1

[0057] ​S1. driving the agricultural unmanned aerial vehicle to cruise above the target farmland at low altitude according to a preset flight path planning instruction to collect continuous and overlapped multi-modal remote sensing data, the multi-modal remote sensing data at least including visible light sequence images, near-infrared images and laser point clouds. It should be noted that the present step is as follows:

[0058] S1.1. importing an outer boundary vector file of the target farmland into a ground station, automatically calculating a minimum circumscribed rectangle and determining a unified reference coordinate system;

[0059] S1.2. generating parallel flight path lines at equal intervals along the long side direction of the minimum circumscribed rectangle according to the field of view angle of the airborne visible light and near-infrared dual-spectrum camera and the expected ground resolution, and calculating the recommended flight height under the constraint condition of "forward overlap degree ≥ 80%, lateral overlap degree ≥ 70%";

[0060] S1.3. dividing continuous navigation points according to the length of the flight path, and labeling three-dimensional coordinates, target speed and attitude holding time for each navigation point in turn, so as to ensure that the acceleration and deceleration buffer of the turning section is ≥ 3 seconds and the roll angle is ≤ 15°;

[0061] S1.4. searching a barrier database, performing buffer detection with a radius of twice the safety margin for each flight path, and when interference is detected, preferentially adjusting the navigation point height in the vertical direction by 5 meters step, and if the conflict still exists, offsetting the flight path in the horizontal plane by 0.5 meters step until the obstacle avoidance is successful;

[0062] S1.5. packaging the flight path and navigation point information which have been corrected by obstacle avoidance and meet the overlap requirement into flight path planning instructions, and issuing the instructions to the flight control system and locking the task number;

[0063] S1.6. 100 milliseconds before the unmanned aerial vehicle reaches each navigation point, triggering a synchronization pulse through an onboard unified timing bus to make the visible light camera and the near-infrared camera expose at the same time stamp;

[0064] S1.7. the unmanned aerial vehicle flies along the flight path at a constant speed, the visible light and near-infrared cameras collect sequence images at 2-5 frames per second, and the laser radar continuously scans at a line speed not less than 10HZ, and the point cloud frames are returned in real time;

[0065] S1.8. the onboard time synchronizer time labels the image frames and point cloud frames based on the timing pulse, so as to ensure that the cross-modal time error is ≤ 5 milliseconds;

[0066] S1.9. in the idle period (such as 2ms) of the flight control loop, using real-time navigation and attitude calculation results to perform track pixel level de-smearing processing on the images and line bundle distortion correction processing on the point clouds, so as to correct the spatial distortion caused by speed fluctuation;

[0067] S1.10, cache the time-aligned and motion-compensated visible light image, near-infrared image and laser point cloud in segment form to the on-board solid-state memory, and attach the corresponding flight attitude, GPS and ambient light metadata for subsequent geometric registration and radiometric normalization.

[0068] As an example, the visible light sequence image at least includes raw pixel matrix, exposure time, camera intrinsic parameters, real-time position and attitude metadata.

[0069] As an example, the near-infrared sequence image at least includes raw pixel matrix, exposure time, camera intrinsic parameters, real-time position and attitude metadata, and record of the center wavelength information of the photosensitive spectrum.

[0070] As an example, the laser point cloud at least includes three-dimensional coordinates, reflection intensity and time stamp, and the start and end attitude quaternions are attached to each scanning line.

[0071] In an optional embodiment, the time alignment method comprises: using hardware pulse per second (PPS) time service to synchronize all sensor local clocks, recording pulse count and microsecond-level offset for image and point cloud frames, and correcting the remaining drift by linear extrapolation.

[0072] In an optional embodiment, the motion compensation method specifically comprises:

[0073] Image end: estimate inter-frame displacement based on attitude angular rate, perform inverse re-projection combined with optical flow constraint to eliminate smearing;

[0074] Point cloud end: divide a single-frame scan into multiple beam segments along the time axis, and use linear interpolation attitude to de-distort each point.

[0075] In an optional embodiment, the buffer detection method comprises: performing buffer inflation on the flight path line, taking twice the flight safety margin as the radius, using a line-surface intersection judgment algorithm for collision detection with the obstacle polygon, and returning the nearest conflict point and recommended height increment.

[0076] Further need to be described in detail is that the calculation method of the recommended flight height specifically comprises the following steps:

[0077] (1) input the expected ground resolution (such as 3 cm per pixel) of the target farmland in the task planning interface, which is determined by the crop plant spacing, field ridge width and the minimum interpretation unit of the subsequent boundary analysis algorithm, the flight control system reads the focal length, imaging chip size and effective pixel number of the on-board visible light and near-infrared camera, and calculates the ground resolution interval that can be reached at different heights according to the intrinsic parameters, and takes the resolution not lower than the set value as the criterion to obtain a group of candidate height upper limits;

[0078] (2) Estimate the scanning width on the ground of a single track according to the camera horizontal field of view and the candidate height, and combine the width with the preset lateral overlap (no less than 70%), to calculate whether the track spacing meets the constraints; if the scanning width is too narrow to maintain sufficient overlap, automatically increase the height level until the overlap condition and the resolution condition are both met;

[0079] (3) Compare the unmanned aerial vehicle flight speed with the camera trigger frequency to evaluate whether the forward overlap can still be guaranteed to be no less than 80% at the candidate height; if the maximum camera trigger frequency is not sufficient to support this overlap, the flight control system prompts to reduce the flight speed; if the speed has reached the lower limit of safety and still does not meet the condition, the height is reduced slightly for reevaluation until the forward overlap meets the standard;

[0080] (4) Corresponding to the maximum measurement distance of the laser radar and the expected point cloud density, the flight control system checks whether the candidate height is within the effective range of the laser radar; if the height exceeds the upper limit of the range or results in a point cloud density below the threshold, the height is reduced or the laser radar scanning parameters are adjusted to ensure that the final solution can be covered by the laser radar;

[0081] (5) On the premise of meeting the imaging and point cloud requirements, retrieve the estimated maximum plant height of the target seasonal crop and the top elevation of known obstacles (such as windbreaks or shrub belts), and add a fixed safety margin (such as 5-10 meters) to the candidate height to avoid damage to the crop caused by the downwash of the propeller and to leave a margin for attitude error;

[0082] (6) Compare the final candidate height with the local airspace management regulations to confirm that it does not touch the upper limit of civil aviation or the no-fly height; if there is a conflict, the flight control system automatically adjusts to within the legal range and repeats the aforementioned overlap and resolution checks;

[0083] (7) When the resolution, overlap, point cloud density, safety margin, and flight height limit are all met, the flight control system marks the height as the recommended flight height and writes the height field into the flight path planning file for the flight control to execute; if there are multiple feasible solutions, the algorithm prioritizes the lowest height that provides higher resolution and meets energy efficiency to reduce battery consumption and shorten the task duration.

[0084] In an optional implementation, the method of performing track pixel-level de-smearing processing specifically includes:

[0085] Each image is marked with the start and end exposure time when triggered, and the flight control system retrieves the attitude record to obtain the three-dimensional attitude and position of the aircraft at the corresponding time;

[0086] Taking the start of exposure as a reference, the sequence of attitude positions from the start to the end of exposure is interpolated into a continuous trajectory to represent the aircraft motion during the entire exposure period;

[0087] For any pixel in the image, first calculate its spatial ray according to the original camera model, then project the ray back to the same ground plane according to the attitude corresponding to each micro time slice in the trajectory, and finally splice these fragments to form the corrected pixel position;

[0088] Based on the remapping result, perform rasterization resampling on the entire image to eliminate stripes, smearing and jaggies caused by body translation or roll;

[0089] Use the fast optical flow method to detect the texture difference before and after resampling, and perform interpolation compensation on the local high-frequency details to maintain image clarity.

[0090] In an optional embodiment, the method for bundle distortion correction specifically comprises:

[0091] The laser radar outputs several scanning line bundles in a single frame period, and the flight control system records the start and end time of each line bundle to form a time line bundle index table;

[0092] According to the time index, the attitude solution result is linearly interpolated to obtain the accurate attitude and position of each line bundle at the start and end time;

[0093] For each laser point in the line bundle, the body attitude at the sampling instant is calculated by doing proportional interpolation between the start and end attitudes according to the timestamp of the point, and the original coordinates of the point are then mapped back to the unified reference coordinate system to eliminate the curvature introduced by body rotation and translation;

[0094] The compensated point cloud bundle is spliced into a complete point cloud frame in time sequence to preserve continuity, and the density of overlapping points is balanced to reduce the compensation error;

[0095] The local plane residual error is calculated for the compensated point cloud, and if the residual error exceeds the threshold, the attitude weight is fine-tuned by the interpolator until the flatness of the point cloud meets the preset standard.

[0096] It should be noted that the preset flight path planning instruction is used to drive the agricultural unmanned aerial vehicle to collect continuous and overlapping multi-modal remote sensing data at a low altitude over the target farmland, achieving comprehensive, efficient and fine data collection of the target farmland. This step ensures that the image data has sufficient overlap and multi-modal characteristics, effectively compensates for the lack of single modal data in simultaneously obtaining ground object spectral information and terrain features, and ensures the completeness and diversity of the data source during subsequent farmland boundary identification, thereby improving the quality of farmland boundary identification basic data and improving the identification accuracy and robustness.

[0097] S2, based on the real-time positioning and attitude information of the agricultural unmanned aerial vehicle, perform geometric registration and radiation normalization on the multi-modal remote sensing data to generate orthophotos and digital terrain models in a unified coordinate system. It should be noted that this step:

[0098] S2.1, based on the real-time positioning and pose information of the agricultural unmanned aerial vehicle, the image and the point cloud are initialized, and a rough registration model is established; wherein:

[0099] Reading the attitude position records corresponding to the image exposure start and end time and the point cloud scanning start and end time in the high-frequency attitude and position calculation flow;

[0100] Write the calibration relationship between the camera optical center and the laser radar installation coordinate system relative to the unmanned aerial vehicle body coordinate system into the external parameter matrix;

[0101] Combined with the attitude position record and the external parameter matrix, the initial three-dimensional space projection of each image and each point cloud beam is calculated to form a rough registration model at the flight route level;

[0102] S2.2, the accurate relative pose parameters between images are calculated through feature matching and bundle adjustment, and the point cloud is reprojected according to the accurate relative pose parameters to complete the precise registration; wherein:

[0103] Extract scale-invariant features between adjacent image pairs and establish matching pairs, input the bundle adjustment solution module, and calculate the accurate relative rotation and translation;

[0104] Smoothly distribute the bundle adjustment results to the entire image sequence in time sequence to obtain an inter-frame pose curve with error compensation;

[0105] According to the inter-frame pose curve, the original laser point cloud is reprojected point by point to realize the precise registration of the point cloud and the image, and eliminate the spatial misplacement caused by the initial attitude error;

[0106] S2.3, a dense stereo reconstruction algorithm is used to generate a dense point cloud from multi-view images, which is fused with the original laser point cloud at the voxel level to output a high-density fusion point cloud; wherein:

[0107] Select three or more overlapping images to construct a multi-view stereo data block, run the dense stereo reconstruction algorithm to generate a dense image point cloud, and attach color and pixel confidence labels to each point;

[0108] Under the premise of consistent coordinate system, the dense image point cloud and the precisely registered laser point cloud are divided into voxels;

[0109] For each voxel unit, calculate the weighted average coordinates according to the point density, reflection intensity and confidence label to output a high-density fusion point cloud;

[0110] S2.4, weighted interpolation is performed on the fusion point cloud to construct a regular grid digital surface model; wherein:

[0111] Perform weighted interpolation on the fusion point cloud according to the set grid resolution, and use the neighborhood density weighted scheme to fill the sparse area;

[0112] A rule grid digital terrain model is generated, and median smoothing is performed on the grid height outliers to ensure terrain continuity.

[0113] S2.5, with the digital terrain model as the projection reference, the image is remapped and radiometrically corrected to generate an orthoimage in a unified coordinate system. Wherein:

[0114] With the digital terrain model as the stereoscopic projection surface, each image is remapped pixel by pixel to correct the tilt displacement caused by the height difference of the ground object;

[0115] Call the light atmosphere condition estimation module, based on the sun elevation angle and the unmanned aerial vehicle attitude when the image is taken, calculate the pixel incident angle and perform radiometric correction to eliminate local light differences;

[0116] Output the orthoimage consistent with the digital terrain model coordinate system, seamlessly spliced, and provide high geometric and spectral consistency input for subsequent deep network feature fusion.

[0117] Further, the airborne real-time positioning and attitude information at least includes:

[0118] The centimeter-level three-dimensional coordinates and heading angle output by the dual-frequency differential global navigation satellite system;

[0119] The attitude angle and angular velocity output by the inertial measurement unit composed of three-axis gyroscope, three-axis accelerometer and three-axis magnetometer;

[0120] The relative height information output by the airborne barometric altimeter for height constraint in attitude solution;

[0121] The three-dimensional coordinates and heading angle, attitude angle and angular velocity, and relative height information are fused by extended Kalman filter to generate a high-frequency attitude solution stream not less than 50HZ.

[0122] In an optional embodiment, the generation method of the high-frequency attitude solution stream is specifically:

[0123] The centimeter-level three-dimensional coordinates and heading angle output by the dual-frequency differential satellite positioning module in real time, the attitude angle and angular velocity output by the inertial measurement unit in real time, and the relative height information of the barometric altimeter are used as the original observation;

[0124] Run the extended Kalman filter in the airborne computing unit, take the satellite positioning parameters as the position observation, the barometric height as the height constraint, and the inertial measurement unit data as the state prediction, and update the position-attitude state vector in a loop;

[0125] The filter outputs a high-frequency attitude solution stream not less than fifty times per second, and records a unified timestamp for each moment as the only attitude reference for subsequent geometric registration of images and point clouds.

[0126] Preferably, by generating the orthographic image and the digital terrain model in the unified coordinate system, the spatial and radiation accurate alignment between the multi-modal data is realized; this step fully utilizes the advantages of the real-time positioning and pose determination technology, eliminates the spatial misplacement and radiation deviation of the data caused by the changes in the flight attitude of the unmanned aerial vehicle or the distortion of the sensor, ensures the high-precision consistency of the image and the height data, and thus improves the data registration accuracy of the farmland boundary recognition process and enhances the accuracy of the subsequent feature extraction.

[0127] S3, inputting the orthographic image and the digital terrain model into the spatial spectrum branch network and the height branch network respectively for feature fusion, and outputting a farmland boundary probability map. It should be noted that in this step:

[0128] The orthographic image is cropped into overlapping and size-consistent image blocks, which are input into the spatial spectrum branch network block by block to extract texture, chrominance and vegetation index features;

[0129] The digital terrain model is segmented into height blocks corresponding to the image blocks, which are input into the height branch network to extract local height, slope and terrain change rate features;

[0130] The two types of features extracted are spliced through the feature fusion model, and the feature weights are redistributed through the channel attention mechanism;

[0131] The full convolution classification layer outputs a pixel-level farmland boundary probability map consistent with the original resolution, and the initial binary boundary mask is obtained after thresholding.

[0132] Specifically, the spatial spectrum branch network is composed of four levels of encoders, four levels of decoders and one cross-layer jump connection channel, each level of encoder successively includes two layers of convolution batch normalization activation units, one 2x2 maximum pooling downsampling unit and one feature bypass connected with the symmetric decoder, each level of decoder first restores the resolution through 2x2 deconvolution upsampling, then splices the corresponding encoder bypass feature, and then successively executes two layers of convolution batch normalization activation units to reconstruct the spatial details, and the last level of decoder outputs a feature map consistent with the resolution of the input image, and one layer of one multiplication convolution is used to compress the channel number for subsequent fusion;

[0133] The height branch network is composed of three layers of voxel convolution downsampling modules, two layers of voxel convolution upsampling modules and one residual direct connection channel, each voxel convolution downsampling module includes one layer of 3x3x3 voxel convolution batch normalization activation unit and one voxel step two downsampling operation, which is used to extract local height fluctuation features, each voxel upsampling module restores the spatial size by three linear interpolation, and then executes one layer of 3x3x3 voxel convolution batch normalization activation unit to refine the terrain edge, and the residual direct connection channel establishes an element-by-element addition relationship between the network input and the deepest layer feature to maintain the overall terrain trend information;

[0134] After the spatial spectral branch network and the elevation branch network are initially fused through feature splicing, a channel attention mechanism is introduced to weight the spliced ​​features. The channel attention mechanism first performs global average pooling to obtain the descriptors of each channel, then generates a weight vector through a two-level fully connected network, and adjusts the feature response by channel-wise multiplication, thereby highlighting the high-response features related to farmland boundaries and suppressing redundant information.

[0135] As an example, the mathematical expression of the feature fusion model in this embodiment is as follows:

[0136]

[0137] Where A is the fused feature projected area element, α is the spatial spectral feature index weighting factor, β is the elevation feature index weighting factor, and C... s C represents the number of output channels for the spatial spectral branch. h The number of output channels for the elevation branch. Let ψ be the texture response obtained after passing the c-th spatial spectral channel through a two-dimensional Gaussian Lithra filter. d (F h Let ) represent the slope response obtained after the d-th elevation channel is filtered by a Laplace variational filter, ε be a positive value to prevent the denominator from being zero, N(·) be the zero-mean hyperbolic tangent normalization function, γ be the fusion gain coefficient, and η be the slope response obtained after the d-th elevation channel is filtered by a Laplace variational filter. k Let δ be the weight of the k-th neuron. k χ is the scaling factor for the k-th neuron. k (F cat ) for in F cat The coefficients of the k-th layer after applying the three-dimensional wavelet packet transform are σ(·), which is the Sigmoid mapping function, and P is the output pixel-level farmland boundary probability value with a range of (0,1), where a value close to 1 indicates a high-confidence boundary pixel and a value close to 0 indicates a non-boundary pixel.

[0138] Preferably, by inputting orthophotos and digital land surface models into spatial spectral branch networks and elevation branch networks respectively for feature fusion, and outputting a farmland boundary probability map, a refined and multi-dimensional analysis of farmland boundary features is achieved. This step fully leverages the comprehensive discrimination capabilities of different features such as spectrum, texture, and elevation for farmland boundaries, making up for the shortcomings of traditional single-feature dimension identification of farmland boundaries, which suffers from misjudgment and omission, and effectively improving the accuracy of boundary identification and adaptability to complex plots. Thus, it achieves the beneficial effects of enhancing the robustness of boundary identification and improving the accuracy of farmland boundary identification in complex scenarios.

[0139] S4. Perform conditional random field post-processing and morphological closing operations on the farmland boundary probability map to obtain closed and topologically consistent farmland boundary vector polygons. Note the following in this step:

[0140] S4.1, construct eight-neighbor full connection conditional random field, optimize pixel consistency of farmland boundary probability map, and refine edge position; wherein:

[0141] On the farmland boundary probability map P, an eight-neighbor full connection graph is established for each pixel, and the edge weight is composed of three parts: the prediction confidence difference item of the same pixel, which is used to retain the original network output; the spatial distance item, which increases exponentially with the Euclidean distance between pixels, is used to suppress long-distance jumps; the spectral and elevation joint similarity item is calculated using the four-channel spectral values of the orthophoto and the digital terrain model elevation values to prevent misdiffusion caused by different crops or slope surfaces;

[0142] An iterative mean field inference is used to minimize the energy function for 5-10 rounds, and after each iteration, the energy of the whole map is monitored, and when the decrease is less than the preset threshold, the convergence is stopped in advance to reduce the calculation delay;

[0143] The pixel consistency optimized probability map P' is output, which has steeper gradients and higher confidence of boundary pixels, providing a fine edge for subsequent morphological processing;

[0144] S4.2, perform morphological closing operation on the optimized image using a 3x3 structure element to fill in boundary gaps, and then perform morphological opening operation to remove isolated noise; wherein:

[0145] Thresholding P' to generate a binary image B, with a threshold of 0.5;

[0146] On B, perform one dilation immediately followed by one erosion (closing operation) using a 3x3 square structure element to fill in boundary cracks and connect broken pixel chains;

[0147] Perform one reverse sequence (opening operation) on the closing operation result, also using a 3x3 structure element, first erosion and then dilation, to remove isolated noise regions with an area less than 9 pixels;

[0148] Get a smooth, connected, and island-free refined binary image B'.

[0149] S4.3, apply the boundary tracking algorithm to extract closed outer rings and hole rings, and use the polyline simplification algorithm to compress redundant vertices to form a topologically consistent vector polygon. Wherein:

[0150] Call the improved Moore-Neighbor chain code algorithm on B', and track the pixel boundary layer by layer from outside to inside: first extract the largest closed outer ring as the main outer boundary, and then recursively extract the inner ring surrounded by it and with an area greater than the threshold as the hole ring;

[0151] The vertex coordinate sequence is recorded in the boundary tracking order, and the Douglas-Peucker polyline simplification algorithm is used for vertex compression of each polyline, and the threshold is set to 0.2 meters, so as to reduce redundant points and maintain geometric accuracy;

[0152] The self-intersection and direction of the main outer ring and each hole ring are checked: the main outer ring is clockwise, and the hole ring is counterclockwise; if self-intersection is found, it is automatically repaired by segment reordering to ensure topological consistency.

[0153] As an example, the farmland boundary vector polygon at least includes a main outer boundary ring, a hole ring, a vertex coordinate sequence, a set of attribute fields, and a boundary bounding rectangle, wherein:

[0154] The main outer boundary ring is composed of a closed multi-segment polyline arranged in a clockwise order, and is used to represent the outer limit of the target farmland. The overall shape can be approximately rectangular, approximately trapezoidal, or irregular freeform, depending on the actual farmland outer contour.

[0155] The hole ring is composed of 0 or more counterclockwise sequentially arranged closed polylines, and is used to identify non-cultivated areas inside the farmland. The corresponding geometric shape of the hole ring can be a pond with an approximate circular shape, a farmhouse with an approximate rectangular shape, or a forest belt vacancy with an arbitrary shape.

[0156] The vertex coordinate sequence records the two-dimensional coordinates of each corner point in a unified geographic coordinate system in the connection order of the boundary polyline, and is used to accurately describe the position of the boundary corner.

[0157] The set of attribute fields records the plot number, boundary length, internal and external ring closure, internal area, and generation timestamp information for the main outer boundary ring and each hole ring, respectively, which is used for subsequent agricultural operation management and historical comparison analysis.

[0158] The boundary bounding rectangle is a set of fields representing the four corner coordinates of the minimum bounding rectangle attached to the vector file, which is used for fast spatial indexing and visual scaling.

[0159] The above geometric information and attribute information are encapsulated in the GeoJSON polygon object structure, so that they are compatible with the interface specifications of the agricultural unmanned aerial vehicle ground station, the precision spraying system, and the variable rate fertilizer control terminal, and can directly drive the subsequent operation track generation and execution.

[0160] Preferably, by obtaining a closed and topologically consistent farmland boundary vector polygon, the refinement of the farmland boundary recognition result is realized, the boundary hole, boundary fracture, and irregular burr problems in the preliminary recognition result are effectively eliminated, the continuity, closure, and topological correctness of the boundary contour are ensured, and the boundary vector data directly used for agricultural production is generated; thereby improving the practicality of the farmland boundary recognition result and ensuring the efficiency of the subsequent operation of precision agriculture.

[0161] It should be noted that the above examples are only used to illustrate the technical solutions of the present application but not to limit the present application. Although the present application is described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or equivalently replaced, without departing from the spirit and scope of the technical solutions of the present application, which should be covered in the scope of the claims of the present application.

Claims

1. An agricultural field boundary automatic recognition method of an agricultural unmanned aerial vehicle, characterized by, Comprise: According to the preset flight planning instruction driving agricultural unmanned aerial vehicle in the target farmland above low altitude cruise collection continuous overlap multi-modal remote sensing data, the multi-modal remote sensing data at least contains visible light sequence image, near-infrared image and laser point cloud; Based on the real-time positioning and attitude information of the agricultural unmanned aerial vehicle, the multi-modal remote sensing data is executed geometric registration and radiation normalization, and the orthographic image and digital surface model under the unified coordinate system are generated; The orthographic image and the digital surface model are input into the spatial spectrum branch network and the height branch network respectively for feature fusion, and the farmland boundary probability graph is output; The farmland boundary probability graph is executed conditional random field post-processing and morphological closing operation, and the closed and topologically consistent farmland boundary vector polygon is obtained. 2.The method of claim 1, wherein, The preset flight planning instruction comprises: Import the outer boundary vector file of the target farmland into the ground station, automatically calculate the minimum circumscribed rectangle and determine the reference coordinate system; Based on the sensor field of view angle and the target ground resolution, the parallel flight path is generated with equal interval according to the constraint condition that the forward overlap degree is not less than 80% and the lateral overlap degree is not less than 70%; According to the flight path length, continuous navigation points are divided, and each navigation point is labeled with three-dimensional coordinates, target speed and attitude holding time; Call the obstacle database to buffer detect the flight path, if interference is detected, adjust the navigation point height along the vertical direction until the obstacle avoidance is successful; The flight path and navigation point information which complete the obstacle avoidance correction and meet the overlap degree requirement are packaged as flight planning instruction, and the instruction is executed in the flight control system. 3.The method of claim 1 or 2, wherein, The preset flight planning instruction driving agricultural unmanned aerial vehicle in the target farmland above low altitude cruise collection continuous overlap multi-modal remote sensing data, comprising: Trigger a synchronous timing signal before each navigation point arrives, so that the visible light camera and the near-infrared camera expose under the same time stamp; During the uniform flight of the unmanned aerial vehicle along the flight path, the laser radar continuously obtains point cloud data with a scanning frequency not less than 10HZ; Align the image frame and the point cloud frame by using the airborne time synchronizer, so that the cross-modal time error is not more than 5ms; During the idle period of the flight control loop, perform motion compensation to correct the image trailing and point cloud distortion caused by speed change in real time; Cache the multi-modal remote sensing data segment after time alignment and motion compensation to the onboard solid state storage. 4.The method of claim 3, wherein, The generation of orthographic image and digital surface model under the unified coordinate system comprises: Based on the real-time positioning and attitude information of the agricultural unmanned aerial vehicle, the image and the point cloud are initialized with external parameters to establish a coarse registration model at the flight path level; Calculate the accurate relative attitude parameters between images through feature matching and bundle adjustment, and complete the precise registration according to the reprojected point cloud; Generate dense point cloud by using dense stereo reconstruction algorithm on multi-view image, and fuse it with the original laser point cloud to output high-density fusion point cloud; Perform weighted interpolation on the fusion point cloud to construct a regular grid digital surface model; With the digital surface model as the projection reference, remap and radiation correct the image to generate the orthographic image under the unified coordinate system. 5.The method of claim 4, wherein, The real-time positioning and attitude information on board at least includes: The centimeter-level three-dimensional coordinates and heading angle output by the dual-frequency differential global navigation satellite system. The attitude angle and angular velocity output by an inertial measurement unit composed of a three-axis gyroscope, a three-axis accelerometer and a three-axis magnetometer; Relative height information output by an airborne barometric altimeter used for height constraint in attitude solution; The three-dimensional coordinates and heading angle, the attitude angle and angular velocity, and the relative height information are fused by extended Kalman filtering to generate a high-frequency attitude and heading solution stream of no less than 50 Hz. 6.The method of claim 1, wherein, The orthographic image and the digital terrain model are respectively input into a spatial spectrum branch network and an elevation branch network for feature fusion, and an agricultural field boundary probability map is output, including: The orthographic image is cropped into overlapping and size-consistent image blocks, which are input into the spatial spectrum branch network block by block to extract texture, chrominance and vegetation index features; The digital terrain model is segmented into elevation blocks corresponding to the image blocks, which are input into the elevation branch network to extract local elevation, slope and terrain change rate features; The two types of features extracted are spliced by a feature fusion model, and feature weights are redistributed by a channel attention mechanism; A pixel-level agricultural field boundary probability map with the same resolution as the original is output by a full convolution classification layer, and an initial binary boundary mask is obtained after thresholding. 7.The method of claim 6, wherein, The spatial spectrum branch network is composed of four levels of encoders, four levels of decoders and one cross-layer jump connection channel, each level of encoder successively includes two layers of convolution batch normalization activation units, one 2x2 maximum pooling downsampling unit and one feature bypass connected to the symmetric decoder, each level of decoder first restores the resolution by 2x2 deconvolution upsampling, then splices the corresponding encoder bypass features, and then successively executes two layers of convolution batch normalization activation units to reconstruct spatial details, and the last level of decoder outputs a feature map with the same resolution as the input image, which is compressed by one layer of convolution to reduce the number of channels for subsequent fusion; The elevation branch network is composed of three layers of voxel convolution downsampling modules, two layers of voxel convolution upsampling modules and one residual direct connection channel, each voxel convolution downsampling module includes one layer of 3x3x3 voxel convolution batch normalization activation unit and one voxel step two downsampling operation, which is used to extract local elevation fluctuation features, each voxel upsampling module restores the spatial size by trilinear interpolation, and then executes one layer of 3x3x3 voxel convolution batch normalization activation unit to refine the terrain edge, and the residual direct connection channel establishes an element-by-element addition relationship between the network input and the deepest layer feature to maintain the overall terrain trend information; After the spatial spectrum branch network and the elevation branch network are preliminarily fused by feature splicing, a channel attention mechanism is introduced to weight the spliced features, wherein the channel attention mechanism first performs global average pooling to obtain channel descriptors, then generates a weight vector through a two-level fully connected network, and adjusts the feature response by a channel-by-channel multiplication method, so as to highlight the high-response features related to the agricultural field boundary and suppress redundant information. 8.The method of claim 6, wherein the method further comprises: determining a boundary of the agricultural field based on the image data. Conditional random field post-processing and morphological closing operation are performed on the agricultural field boundary probability map to obtain a closed and topologically consistent agricultural field boundary vector polygon, including: An eight-neighbor full-connection conditional random field is constructed to perform pixel consistency optimization on the farmland boundary probability map, and to refine the edge position; A morphological closing operation is performed on the optimized image using a 3*3 structure element to fill the boundary gap, and then a morphological opening operation is performed to remove isolated noise; A boundary tracking algorithm is applied to extract the closed outer ring and hole ring, and a polyline simplification algorithm is used to compress the redundant vertices to form a vector polygon with consistent topological structure. 9.The method of claim 8, wherein, The farmland boundary vector polygon at least includes a main outer boundary ring, a hole ring, a sequence of vertex coordinates, a set of attribute fields, and a boundary bounding rectangle.

Images