A method for monitoring farmland non-agriculturalization based on spatialization of unmanned aerial vehicle video

Abstract

The application discloses a kind of based on unmanned vehicle video spatialization's cultivated land non-agricultural monitoring method.The method includes: obtaining farmland area video stream collected by unmanned vehicle, positioning and pose system data and digital elevation model data;Based on the double threshold detection algorithm of optical flow change rate and image information entropy from video stream Extraction key frame;By combining the collinearity equation optimization solution of digital elevation model, key frame is spatialized, and image coordinates are converted into geographic coordinates;Key frame image data and corresponding geographic coordinate information are fused, and the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of the input data of

Description

Technical Field

[0001] This invention belongs to the field of remote sensing monitoring and geographic information technology, and relates to a method for monitoring the non-agriculturalization of cultivated land based on UAV video spatialization. Background Technology

[0002] Farmland protection is a crucial foundation for maintaining social stability. However, with the rapid advancement of urbanization and infrastructure construction, the conversion of farmland to non-agricultural uses has become increasingly prominent, posing a serious threat to the sustainable use of farmland resources. Therefore, conducting efficient, accurate, and timely monitoring of farmland conversion to non-agricultural uses has become an urgent need for land and resources supervision.

[0003] Currently, the mainstream methods for monitoring the non-agricultural use of arable land mainly rely on satellite remote sensing image interpretation and manual on-site inspections. Although satellite remote sensing technology has a wide coverage, its image acquisition is limited by factors such as cloud cover, making high-frequency monitoring difficult and resulting in a significant lag in the detection of sudden and concealed non-agricultural activities. Furthermore, low-to-medium resolution satellite imagery has limitations in monitoring small, scattered non-agricultural plots, including insufficient spatial detail and inaccurate boundary positioning, affecting the accuracy of monitoring results. In addition, the interpretation process based on satellite imagery still relies heavily on manual interpretation, with limited automation, low efficiency, and susceptibility to subjective factors.

[0004] On the other hand, while traditional manual on-site inspections can obtain firsthand information, they are time-consuming, labor-intensive, and costly, and the inspection scope is limited, making it difficult to achieve large-scale, routine dynamic supervision. Especially in areas with inconvenient transportation or complex terrain, the implementation of manual inspections is even more difficult and prone to creating regulatory blind spots.

[0005] In recent years, UAV remote sensing technology has shown its application potential in the field of land and resources monitoring due to its advantages of flexibility, efficiency, and high resolution. However, existing technologies mostly focus on analyzing static orthophotos acquired by UAVs, failing to fully leverage the continuous and dynamic information advantages of UAV video data. Furthermore, existing interpretation methods typically separate visual recognition from spatial positioning, lacking deep coupling and utilization of geospatial information. This results in insufficient ability to distinguish between homogeneous and heterogeneous materials, and the accuracy of identifying and spatially locating the boundaries of farmland converted to non-agricultural uses still needs improvement.

[0006] Therefore, developing a monitoring method for the non-agriculturalization of cultivated land that can integrate dynamic information from UAV video with precise geospatial coordinates and achieve automated, high-precision identification and positioning is of great significance for improving the timeliness, accuracy, and intelligence of land and resources supervision. Summary of the Invention

[0007] To address the problems existing in the background technology, this invention proposes a method for monitoring the non-agriculturalization of cultivated land based on UAV video spatialization.

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows: a method for monitoring the non-agricultural use of cultivated land based on UAV video spatialization, comprising the following steps:

[0009] Acquire video streams, positioning and attitude determination system data, and digital elevation model data of farmland areas collected by drones;

[0010] Keyframes are extracted from the video stream of the farmland area, and the keyframes are selected by a dual threshold detection algorithm based on optical flow rate of change and image information entropy;

[0011] The keyframes are spatialized, and the image coordinates are converted into geographic coordinates by combining the collinearity equation optimization solution of the digital elevation model.

[0012] The image data of the keyframes are fused with their corresponding geographic coordinates;

[0013] The fused data is input into a geographic awareness deep learning model for land cover interpretation, wherein the model extracts spatial features through a built-in geographic coordinate embedding layer to identify farmland conversion targets.

[0014] The interpretation results are post-processed to generate a monitoring report on the non-agriculturalization of arable land.

[0015] Compared with existing technologies, the present invention has the following advantages: by synchronously acquiring video streams from UAVs and positioning and attitude determination systems, and combining a dual-threshold keyframe extraction algorithm based on optical flow change rate and image information entropy, it realizes the intelligent screening of keyframes with significant changes in ground features from continuous videos, effectively solving the problems of long satellite remote sensing monitoring cycles and low efficiency of manual inspections, and enabling rapid detection and identification of sudden and covert non-agricultural activities.

[0016] Video spatialization is achieved by using a collinearity equation optimization solution combined with a digital elevation model. Through iterative optimization and boundary constraint mechanisms, image coordinates are accurately converted into geographic coordinates. Then, through geographic coordinate embedding and coordinate attention mechanisms, spatial location information is deeply integrated into the deep learning model, which enhances the ability to represent the boundary between cultivated land and non-agricultural features and solves the problems of positioning deviation and boundary ambiguity in traditional methods.

[0017] By fusing keyframe image data with corresponding geographic coordinate information at the feature level, a deep learning model for geographic perception was constructed. This model possesses both the ability to recognize visual features of land features and the ability to understand spatial relationships, effectively overcoming the problem of misjudgment caused by different spectra of the same object or different objects of the same spectra, and improving the accuracy of identifying targets for the non-agriculturalization of arable land.

[0018] By performing morphological closing operations and raster-to-vector conversion in the post-processing stage, the spatial integrity of the interpretation results is optimized; by conducting dual quality checks on reprojection error and intersection-union ratio, the quality control of the entire process from spatialization accuracy to interpretation accuracy is ensured, providing reliable technical support for land and resources supervision.

[0019] This invention organically integrates advanced technologies such as UAV technology, photogrammetry, and deep learning, forming a complete technology chain from data collection, processing, analysis to result output. It reduces the intensity of manual intervention, improves monitoring efficiency, and provides an automated monitoring solution that can be scaled up for farmland protection. Attached Figure Description

[0020] Figure 1 This is a flowchart of a method for monitoring the non-agricultural use of cultivated land based on UAV video spatialization according to the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] like Figure 1 As shown, the technical solution adopted by this invention is as follows: A method for monitoring the non-agricultural use of cultivated land based on UAV video spatialization, comprising the following steps:

[0023] Acquire video streams, positioning and attitude determination system data, and digital elevation model data of farmland areas collected by drones.

[0024] Keyframes are extracted from the video stream of the farmland area, and the keyframes are selected using a dual-threshold detection algorithm based on optical flow rate of change and image information entropy.

[0025] The keyframes are spatialized by combining the collinearity equation optimization solution of the digital elevation model to convert the image coordinates into geographic coordinates.

[0026] The image data of the keyframe is fused with the corresponding geographic coordinates.

[0027] The fused data is input into a geospatial deep learning model for land cover interpretation, wherein the model extracts spatial features through a built-in geographic coordinate embedding layer to identify farmland conversion targets.

[0028] The interpretation results are post-processed to generate a monitoring report on the non-agriculturalization of arable land.

[0029] Specifically, the drone is controlled to fly according to preset flight parameters and collect video streams of the farmland area.

[0030] The flight parameters mentioned above include flight altitude, flight speed, and heading overlap.

[0031] While collecting video streams from farmland areas, the camera's pose data is simultaneously recorded through a positioning and attitude determination system.

[0032] Call the digital elevation model data at a preset resolution for subsequent spatialization processing.

[0033] By controlling the drone to fly according to preset flight parameters and collect video streams of the farmland area, based on the rigid requirements of photogrammetry for the quality of basic data, the flight parameters, such as camera focal length and sensor size, are designed by pre-matching the parameters of the airborne equipment to ensure that the collected video stream can completely and clearly carry the information of the farmland area, providing original data that meets the accuracy standards for subsequent video spatialization and non-agricultural interpretation of cultivated land.

[0034] The design of flight parameters must be based on a logical match between equipment capabilities and data requirements. The focal length and sensor size of the airborne camera are the main references. Calculations must be made to ensure the clarity of ground details in the video stream, such as crop boundaries, plot outlines, and potential traces of non-agricultural use. This is to avoid blurring or loss of ground features due to unreasonable parameter design, which would affect the accuracy of subsequent keyframe extraction, spatial coordinate calculation, and interpretation model recognition.

[0035] The flight parameters mentioned include flight altitude, flight speed, and heading overlap. The values ​​and design logic of each parameter are determined around three objectives: data accuracy, acquisition efficiency, and processing reliability, as detailed below:

[0036] The flight altitude is calculated by inversely using the ground sampling distance (GSD) formula, based on the airborne camera's focal length and sensor size. For example, if the airborne camera's focal length is 24mm and the flight altitude is set to 150 meters, the final ground sampling distance will be approximately 2.5 centimeters. This precision ensures that the microscopic features of farmland in the video stream are clearly captured, such as crop row spacing and the foundations of small non-agricultural facilities, meeting the monitoring needs for early detection and identification of non-agricultural uses in arable land.

[0037] The flight speed setting is the optimal value that balances coverage efficiency and image quality. For example, setting the flight speed to 8 m / s for a certain area can not only complete video acquisition of a large area of ​​farmland within a reasonable time, but also avoid motion blur in video frames caused by excessive flight speed. Motion blur will destroy the edge features of ground objects and affect subsequent optical flow calculation and key frame determination.

[0038] The forward overlap rate setting must ensure sufficient redundancy between adjacent video frames. For example, a forward overlap rate of 80% is recommended. This forward overlap rate provides the data foundation for subsequent ground verification control, control point position accuracy optimization, and stereo measurement during video spatialization.

[0039] Preferably, the integrity of the area coverage and the accuracy of the model can be further improved by setting the lateral overlap rate, so as to avoid the occurrence of breakpoints or sudden increase in error during spatial coordinate calculation due to insufficient inter-frame overlap.

[0040] While acquiring video streams from farmland areas, the camera's pose data is simultaneously recorded through a positioning and orientation system. This is achieved through a hardware-level time synchronization mechanism, enabling time alignment between video frame acquisition and pose data recording. This provides accurate initial values ​​for exterior orientation elements for subsequent video frame spatialization.

[0041] Furthermore, by precisely binding the camera exposure time with the GPS timestamp through a hardware trigger signal, the time difference in the generation of pose data for video frames is eliminated, avoiding incorrect pose data corresponding to a certain video frame due to time asynchrony, which in turn causes deviations in spatial coordinate calculation.

[0042] The acquisition frequency of the Positioning and Attitude (POS) system needs to cover the requirements of high-maneuverability flight scenarios for UAVs, such as turning and climbing. For example, setting the acquisition frequency to 20Hz can cover high-maneuverability flight scenarios for UAVs, such as turning and climbing, ensuring that a set of pose data is recorded every 0.05 seconds. This avoids pose data interpolation errors caused by excessively large sampling intervals and provides near real-time pose reference for each frame of video.

[0043] The recorded pose data contains complete positioning and attitude information, with an accuracy sufficient for subsequent video spatialization processing. The positioning information includes latitude, longitude, and altitude, while the attitude information includes roll, pitch, and yaw angles. This high-precision data directly serves as the parameter vector in the subsequent image-based direct positioning process using collinearity equations. The initial value is the basic input to ensure spatial positioning accuracy.

[0044] The system retrieves digital elevation model (DEM) data at a preset resolution for subsequent spatialization processing. The DEM resolution varies depending on the terrain and environmental factors such as airflow, temperature, and humidity. Here, the preset DEM resolution is 5 meters, a choice that balances accuracy and computational efficiency. This provides ground points with sufficient spatial accuracy while avoiding excessively high resolutions, such as a 1-meter DEM, which would lead to a surge in data volume and prolonged computation time. The DEM data serves as crucial for providing elevation compensation in the DEM-assisted collinearity equation optimization method.

[0045] Subsequent video frame spatialization requires calculating the planar coordinates of each pixel using the collinearity equation, which requires knowledge of the elevation values ​​of ground points. At this point, the DEM data serves as the elevation data source, and bilinear interpolation is used to represent any ground point. Calculate accurate elevation .

[0046] First, determine the four corner points of the DEM grid containing the ground point and their elevation values. Then substitute into the interpolation formula ;

[0047] in , For the corner points of this DEM grid coordinate; , For the corner points of this DEM grid Coordinates, ultimately yielding the ground point Precise elevation This elevation value provides the necessary input for the iterative solution of the collinearity equation, ensuring that video frame pixels can be accurately mapped to the geographic coordinate system, and avoiding errors in planar coordinate calculation due to missing or coarse elevation values.

[0048] Specifically, the keyframe extraction includes: calculating the optical flow vector between adjacent frames in the video stream of the farmland area, and calculating the average optical flow rate of change based on the optical flow vector.

[0049] Calculate the difference in image information entropy between adjacent frames.

[0050] When the average optical flow rate exceeds a first preset threshold and the image information entropy difference exceeds a second preset threshold, the current frame is marked as a key frame.

[0051] Furthermore, the optical flow vector between adjacent frames in the video stream of the farmland area is calculated, and the average optical flow change rate is calculated based on the optical flow vector. This step captures the subtle movements of pixels between adjacent frames through a dense optical flow algorithm, and then quantifies the overall motion intensity between frames by statistical averaging, providing a quantitative basis for the motion change dimension for subsequent key frame determination.

[0052] The Gunnar Farneback dense optical flow algorithm is used to calculate pixel-level motion vectors between adjacent frames. ,in Represents pixels Displacement in the horizontal direction, This represents the vertical displacement of the pixel. Adjacent frames refer to the current frame and the frame preceding it, i.e., two consecutive frames. To ensure the algorithm can capture multi-scale motion while controlling computational load, key parameters are set, and a three-layer image pyramid is constructed, with each layer scaled by 0.5 times, using a 15×15 pixel computation window. The pyramid structure can cover different scales of motion, from subtle jitter to significant scene changes. The 15×15 pixel window can obtain stable optical flow estimation in textured farmland areas, such as crop rows and field ridges, avoiding motion calculation deviations caused by an excessively small window.

[0053] Based on the aforementioned optical flow vector, using the formula Calculate the average rate of change of optical flow. In the formula, This represents the total number of pixels in a single frame of an image. The magnitude of the optical flow vector of a single pixel, i.e., the actual displacement of that pixel, is summed and then averaged to eliminate the interference of local abnormal pixels on motion judgment, resulting in the final value. It can accurately quantify the overall motion intensity between adjacent frames. For example, minor inter-frame motion caused by slight drone jitter in farmland areas, or significant motion caused by changes in plot boundaries, can be precisely quantified. The numerical differences are reflected.

[0054] The difference in image information entropy between adjacent frames is calculated by quantifying the texture complexity and information richness of the image through information entropy. Then, by using the difference in entropy values ​​between adjacent frames, it is determined whether there are substantial changes in the content between frames, rather than just invalid changes such as brightness fluctuations.

[0055] First, the current frame is converted into a grayscale image to eliminate redundant interference in the color channels, focusing on the texture features of the ground objects themselves. Then, the grayscale histogram of the grayscale image is calculated to obtain the grayscale level for each grayscale level. The grayscale range is 0-255, and the probability of occurrence is... This refers to the proportion of pixels at that gray level to the total number of pixels. Then, using the formula... Calculate the information entropy of a single frame image Information entropy The higher the value, the more complex the image texture and the richer the information. For example, a frame of farmland containing field ridges and variations in crop varieties. A value higher than the uniform frame of a single crop cover indicates a more singular information.

[0056] Calculate the entropy of the current frame. Information entropy of the previous frame The absolute value, that is This difference represents the difference in image information entropy between adjacent frames. For example, if adjacent frames only experience brightness fluctuations due to changes in illumination, resulting in an overall shift in grayscale values ​​but no change in texture structure, then... The value is relatively small. If adjacent frames show a significant change in texture structure due to a drone's flight covering a new area, such as moving from a wheat field to a vegetable garden, then... The values ​​are relatively large, which is used to distinguish between invalid brightness changes and valid content changes.

[0057] When the average optical flow change rate exceeds a first preset threshold and the image information entropy difference exceeds a second preset threshold, the current frame is marked as a key frame. This step uses dual thresholds and logic to accurately filter key frames, ensuring that the selected frames contain both sufficient scene motion changes and substantial differences in information content, avoiding redundant frames, such as consecutive similar frames, or invalid frames, such as frames with only brightness changes, which are then processed further.

[0058] Based on the actual scenarios captured by video in farmland areas, such as the shaking during normal drone flight and the characteristics of ground feature changes during the crop growth cycle, dual thresholds are determined.

[0059] The first preset threshold is The pixel / frame threshold effectively filters out minor drone jitters, such as small inter-frame displacements caused by airflow, resulting in invalid motion. It retains only frames with significant scene changes, such as plot boundary switching or the appearance of non-agricultural facilities.

[0060] The second preset threshold is This threshold can exclude frames with minor fluctuations in inter-frame entropy values ​​caused only by changes in lighting and shadows, and retain only frames with substantial information differences caused by changes in the texture and structure of ground objects.

[0061] The system employs an AND-based logical condition: a frame is marked as a keyframe only when both the average optical flow rate of change and the image information entropy difference simultaneously exceed a first preset threshold and a second preset threshold. This dual constraint avoids selecting jittery frames with no informational value based solely on motion changes, and also avoids selecting repetitive frames without motion changes based solely on information entropy differences. Ultimately, it intelligently extracts the most representative keyframes from a continuous video stream. These keyframes not only fully preserve the information on changes in land features in farmland areas but also reduce the computational load of subsequent video spatialization and intelligent interpretation, balancing monitoring accuracy and processing efficiency.

[0062] Specifically, the spatialization process includes: establishing collinearity equations based on keyframes, camera pose parameters, and ground point coordinates.

[0063] Elevation compensation is performed on the collinearity equation using elevation data provided by the digital elevation model.

[0064] An optimization algorithm is used to iteratively solve the collinearity equation to obtain the geographic coordinates of the pixels.

[0065] Collinearity equations are established based on keyframes. These equations include camera pose parameters and ground point coordinates. According to the collinearity condition in photogrammetry, pixels, projection centers (i.e., the camera optical center), and ground points are collinear. The collinearity equations are used to associate the pixel coordinates of the keyframes with the ground point coordinates and camera pose in the actual geographic space, providing a basic model for subsequent geographic coordinate calculation.

[0066] The mathematical form of collinear equations:

[0067] ;

[0068] ;

[0069] in, , The image plane coordinates of the pixels in the keyframe image; The camera focal length is fixed, and the equipment parameters are consistent with the camera focal length in the previous flight parameter design. , , , , , , , , The elements of the rotation matrix are determined by the camera pose parameters; , , Let be the geographic coordinates of the ground point to be determined; , , For the camera, the ground coordinates of the projection center are derived from the initial values ​​of the POS data.

[0070] The relationship between camera pose parameters and the rotation matrix is ​​that the elements of the rotation matrix in the equation are determined by the camera pose angles ( The attitude angle is calculated using a fixed formula, and specifically corresponds to the roll angle in the POS data. ), pitch angle ( ), yaw angle ( ):

[0071]

[0072]

[0073] ;

[0074]

[0075]

[0076] ;

[0077]

[0078]

[0079] ;

[0080] The correlation of these parameters ensures that the collinearity equation can accurately reflect the impact of camera pose changes on pixel coordinates, providing a basis for pose correction for the coordinate transformation between pixels and ground points.

[0081] Elevation compensation is performed on the collinearity equation using elevation data provided by a digital elevation model. The collinearity equation is then used to solve for the geographic coordinates of ground points. hour, Ground point elevation is a key unknown quantity, while the digital elevation model (DEM) can provide accurate elevation references within the region. The elevation value of any ground point can be obtained through interpolation calculation, supplementing the unknown quantity of the collinearity equation and avoiding coordinate solution errors caused by missing elevation values.

[0082] The digital elevation model has a resolution of 5 meters, which balances elevation accuracy with computational efficiency. It can cover the topographical undulations of farmland areas, such as field ridges and gentle slopes, providing elevation data that meets spatial accuracy requirements, without causing a surge in data volume or prolonged interpolation calculation time due to excessively high resolutions, such as a 1-meter DEM.

[0083] Bilinear interpolation method for elevation compensation: due to the ground points to be determined It may not fall exactly at the grid corner of the DEM, and its accurate elevation needs to be calculated using bilinear interpolation. The specific steps are as follows:

[0084] The first step is to determine the ground point. For the given digital elevation model grid, obtain the elevation values ​​of the four corner points of that grid, and record them as the elevation of the top left corner point. Elevation of the upper right corner Elevation of the lower left corner point Elevation of the lower right corner point Simultaneously record the geographic coordinates of the four corner points, the top left corner ( , (top right corner) , (bottom left corner) , (bottom right corner) , ).

[0085] Calculate the normalized coordinates of ground points within the grid: , ,in Indicates the ground point is at The orientation relative to the grid position ratio Indicates in The positional proportion of the direction.

[0086] Substitute into the bilinear interpolation formula to calculate the elevation. :

[0087] ;

[0088] This formula achieves a smooth and accurate estimation of the elevation of points within any grid by weighted averaging of the elevations of the four corner points. It avoids the abrupt elevation changes caused by simple adjacency interpolation, providing a reliable basis for the collinearity equation. Input values ​​to ensure subsequent , The accuracy of coordinate calculation.

[0089] An optimization algorithm is used to iteratively solve the collinearity equation to obtain the geographic coordinates of the pixels. The principle behind this step is the initial parameters of the collinearity equation. The data comes from POS and contains some errors. Therefore, it's necessary to optimize the algorithm's objective function by minimizing the residual between the predicted pixel coordinates and the actual observed pixel coordinates. Through iterative parameter correction, the geographic coordinates of the ground point corresponding to each pixel are ultimately calculated accurately. This enables the spatialization of keyframes.

[0090] The Levenberg-Marquardt optimization algorithm used combines the advantages of gradient descent and Gauss-Newton's method, which can converge quickly while ensuring iterative stability, avoiding the shortcomings of a single algorithm, such as slow convergence of gradient descent and easy divergence of Gauss-Newton's method.

[0091] The specific steps of iterative solution:

[0092] 1. Parameter initialization: Constructing the parameter vector The initial values ​​are taken directly from POS data, converted from latitude and longitude. , altitude Rolling corners Pitch angle Yaw angle This provides an initial starting point for iteration.

[0093] 2. Residual Vector Calculation: Residual The formula represents the difference between the model-predicted pixel coordinates and the actual observed pixel coordinates. ,in It is to put the parameters The predicted pixel coordinates (x, y) are obtained by substituting them into the collinearity equation. It is the actual observed pixel coordinates of the pixel in the keyframe, that is, the row and column number transformation value of the pixel. The smaller the residual, the closer the parameter is to the true value.

[0094] 3. Jacobian matrix calculation: Jacobian matrix Reflecting each parameter ( , To determine the impact of minute changes in parameters (e.g., parameters) on the residuals, the central difference method was used for numerical differentiation with a step size of 0.0001. This step size accurately captures the relationship between parameter changes and residuals while avoiding the derivative error caused by an excessively large step size and the computational burden caused by an excessively small step size.

[0095] 4. Parameter update: via formula Calculate parameter correction amount ,in: yes The transpose of the matrix, It is the identity matrix. It is the damping factor, with an initial value set to 0.01.

[0096] Damping factor Its function is to adjust the iteration direction: when the residual is large, Increasing the value makes the algorithm closer to gradient descent, ensuring stability. When the residual is small, By reducing the size of the algorithm, it becomes closer to the Gauss-Newton method, thus accelerating convergence.

[0097] 5. Convergence Criterion: If the norm of the parameter correction is... This indicates that the parameter change is minimal and close to the true value; or the number of iterations reaches the maximum limit of 10, then the iteration stops, and the parameter at this point... That is, the optimal parameters.

[0098] Optimal parameters Substituting the collinearity equation and combining it with the ground point elevations obtained through DEM interpolation... This allows for the accurate calculation of the geographic coordinates of the ground point corresponding to each pixel. This completes the spatialization of keyframes. At this point, the keyframes not only contain visual information but also carry precise geographic coordinate information, providing spatially attributed input data for subsequent deep learning interpretation models for geographic perception. This ensures that the interpretation results correspond to the actual geographic location and meet the spatial positioning requirements for monitoring the non-agricultural use of arable land.

[0099] Optimal parameters The process of substituting into the collinearity equation is as follows: First, from the parameter vector after optimization convergence... Extracting camera position ( ) and attitude angle ( Then, the rotation matrix elements are calculated using the attitude angles. , , , , , , , , Specifically, this is accomplished using trigonometric function relationships. For each pixel in a video keyframe, its image plane coordinates (x, y) and focal length are known. Simultaneously, based on the coarse ground position (X,Y) corresponding to this pixel, the precise elevation value is obtained from the 5-meter resolution DEM data through bilinear interpolation. Next, the parameters are substituted into the basic form of the collinearity equation, and through linearization, the ground plane coordinates (X, Y) for each pixel are directly solved. The entire process requires no further iteration because... The convergence has been optimized using the Levenberg-Marquardt optimization algorithm, ensuring the accuracy and efficiency of coordinate calculation.

[0100] Specifically, the iterative solution of the collinearity equation using an optimization algorithm includes:

[0101] The solution is obtained iteratively by constructing a solution parameter vector that includes camera position and attitude parameters;

[0102] During the iteration process, the residual is calculated based on the reprojection error of the current parameter vector, and the damping factor is dynamically adjusted according to the direction and magnitude of the residual change.

[0103] Specifically, when the residual decreases and the preset convergence condition is met, the iteration is determined to be converged and the optimal geographic coordinates are output; when the residual increases or oscillates, the iteration process is stabilized by increasing the damping factor.

[0104] After each iteration update, boundary constraints are applied to the solution parameter vector to ensure that its values ​​are within a reasonable range.

[0105] Furthermore, iterative solutions are obtained by constructing a solution parameter vector containing camera position and attitude parameters. The principle is to integrate the key unknown parameters affecting the geographic coordinate solution in the collinearity equation into a unified vector, providing a standardized iterative operation object for the Levenberg-Marquardt optimization algorithm, and ensuring the uniformity and efficiency of parameter updates and error calculation.

[0106] Construct the solution parameter vector The meaning of each component is clearly related to its physical properties. , , This refers to the ground position parameters of the camera projection center, corresponding to the latitude and longitude converted coordinates and altitude in the POS data. , , These are the camera attitude parameters, corresponding to roll, pitch, and yaw angles in the POS data. This vector completely covers all parameters that need iterative correction in the collinearity equation, avoiding computational chaos caused by parameter dispersion.

[0107] Using the initial parameters provided by the POS system ( , , , , , () as the initial value of the solution parameter vector During the iteration process, the deviation between the model's predicted values ​​and the actual observed values ​​is gradually reduced by continuously correcting each component in the vector. This transforms the synchronous optimization of multiple parameters into mathematical operations at the vector level, simplifying the calculation logic of Jacobian matrix construction and parameter update formulas, and providing a unified parameter carrier for subsequent residual calculation, damping factor adjustment, and boundary constraint application.

[0108] During the iteration process, the residual is calculated based on the reprojection error of the current parameter vector, and the damping factor is dynamically adjusted according to the direction and magnitude of the residual change. The solution accuracy of the current parameter vector is quantified by the reprojection error, and the damping factor is dynamically adjusted with the residual change as a feedback signal to achieve a dynamic balance between convergence speed and iteration stability.

[0109] Reprojection error refers to the error based on the current parameter vector. (No. The predicted pixel coordinates (X,Y) calculated by the parameter vector of the next iteration are compared with the actual observed pixel coordinates of the pixel in the keyframe. , The deviation is essentially a direct reflection of the accuracy of solving collinear equations. The residual is expressed through the residual vector. Quantification, the formula is: ,in , It is The predicted pixel coordinates calculated after substituting into the collinearity equation, and the magnitude of the residual vector The smaller the value, the closer the current parameter vector is to the optimal solution.

[0110] Damping factor This is the adjustment parameter for the Levenberg-Marquardt algorithm, initially set to 0.01. Its adjustment depends entirely on the direction and magnitude of the residual change. When the residual decreases... If this indicates that the current iteration direction is correct and the step size is reasonable, the damping factor will be scaled by 0.5 times. This makes the algorithm closer to the Gauss-Newton method, accelerating the convergence speed. As the residual increases... If the residuals show alternating increases and decreases, it indicates that the current step size is too large or there is a deviation in the iteration direction. In this case, the damping factor should be scaled by a factor of 2. This enhances the gradient descent characteristics of the algorithm and reduces the parameter update step size. The purpose of this dynamic adjustment is to prevent the algorithm from diverging when the initial parameter error is large, while simultaneously ensuring rapid convergence as it approaches the optimal solution, thus balancing iterative efficiency and stability.

[0111] Specifically, when the residual decreases and meets the preset convergence condition, the iteration is determined to be converged and the optimal geographic coordinates are output; when the residual increases or oscillates, the damping factor is increased to stabilize the iteration process, clarify the termination condition and anomaly handling mechanism of the iteration, and ensure that the iteration can not only accurately converge to the optimal solution, but also cope with residual abnormal scenarios.

[0112] The convergence condition employs a combination of two thresholds and the maximum number of iterations.

[0113] Residual norm threshold: Pixels ensure that the calculation error is controlled within the pixel level, meeting the spatial accuracy requirements of geographic coordinates.

[0114] Parameter update threshold: Solving for the update norm of the parameter vector. The position parameters are in meters, and the attitude angle parameters are in degrees. This ensures that the parameter changes are small enough to be negligible, thus avoiding excessive iteration.

[0115] Maximum number of iterations: set to 10 times to prevent infinite iteration due to local optima.

[0116] When the residual continues to decrease and meets any of the above threshold conditions, the iteration is considered to have converged. At this point, the current parameter vector is... Substituting into the collinearity equation, the resulting ground point geographic coordinates (X, Y, Z) are the optimal geographic coordinates.

[0117] When the residual increases or oscillates, the main problem is that the parameter update step size exceeds a reasonable range, causing the iteration to deviate from the optimal solution direction. Increasing the damping factor can address this. This can enhance the identity matrix in the parameter update formula. weight This forces a reduction in the parameter update step size, allowing the iteration direction to stabilize. For example, when a sudden change in the drone's flight attitude causes a large error in the initial parameters, the residual is prone to increase. Increasing the damping factor limits the parameter update magnitude, and the iteration gradually adjusts its direction to avoid divergence caused by excessively large step sizes, ensuring that the algorithm continuously approaches the optimal solution.

[0118] After each iteration update, boundary constraints are applied to the solution parameter vector to ensure that its value is within a physically reasonable range. Based on the physical characteristics of UAV flight and the spatial range of farmland area, a physical feasible region is set for the solution parameter vector to avoid parameter values ​​that do not conform to the actual scenario during the iteration process.

[0119] The constraint range is determined entirely based on the actual application scenario to ensure the physical rationality of the parameter values.

[0120] Camera position parameter constraints: , The value range is ±50 meters of the geographic coordinates of the bounding rectangle of the farmland area to avoid the camera position deviating too far from the monitoring area, which would render the geographic coordinate calculation invalid. The value range is between 140 meters and 160 meters, corresponding to a ±10 meter fluctuation range at the preset flight altitude of 150 meters, which is in line with the altitude stability of the drone flight.

[0121] Camera attitude parameter constraints: roll angle Pitch angle The value range is -5° to 5°, yaw angle The value range is 0°~360°. When the drone is used for farmland monitoring, it adopts a stable flight mode and the attitude angle will not deviate. Attitude angles outside this range are considered physically impossible scenarios.

[0122] The implementation method and function of constraints: each iteration updates the result. Then, constraints are applied using the truncation method. If a parameter component exceeds the constraint range, it is forcibly truncated to the nearest boundary value, such as... When the angle is 6°, it is truncated to 5°. This constraint eliminates physically invalid parameter values, preventing parameters from deviating from the actual scene due to iteration anomalies. For example, the camera position might exceed farmland areas, or the attitude angle might exhibit extreme values, leading to errors in geographic coordinate calculation. Simultaneously, the constraint limits the parameter search space, reducing the probability of iteration getting trapped in local optima and ensuring that the solution meets both mathematical accuracy and practical spatial logic.

[0123] Specifically, the deep learning model for geographic perception includes:

[0124] Geographic coordinate embedding processing: The geographic coordinates corresponding to each pixel in the keyframe are converted into high-dimensional feature vectors.

[0125] Furthermore, the geographic coordinates are normalized and converted to a preset numerical range.

[0126] For the normalized geographic coordinates, the sine and cosine function values ​​are calculated based on a set of preset multiple different frequencies.

[0127] All the calculated sine and cosine function values ​​are combined in dimensional order to form the high-dimensional feature vector.

[0128] The geographic coordinates are normalized and converted to a preset numerical range. The principle is to eliminate scale differences in geographic coordinates across different farmland monitoring areas, avoiding learning biases in spatial relationships caused by varying coordinate ranges. For example, the X-coordinates of different areas may differ by hundreds or even thousands of meters, ensuring the model can stably handle coordinate data from various monitoring scenarios. Its function is to uniformly map the actual geographic coordinates (X, Y) corresponding to pixels in keyframes to a preset numerical range of [0,1], making coordinates from different areas comparable. Specific implementation details and parameters are as follows:

[0129] Normalization is achieved through a linear transformation formula, calculated separately for the X and Y dimensions of the geographic coordinates. The formula is defined as follows:

[0130] Dimensional normalization formula: ;

[0131] Dimensional normalization formula: ;

[0132] in, , After spatialization, the keyframe contains the actual geographic coordinates of each pixel, in meters, consistent with the coordinate units of the DEM data.

[0133] , The minimum area of ​​farmland currently under monitoring Coordinates, maximum Coordinate values ​​are obtained by iterating through all pixels within the region. The coordinates are determined by taking the extreme values.

[0134] , The minimum area of ​​farmland currently under monitoring Coordinates, maximum Coordinate values, acquisition method and , Consistent.

[0135] [0,1] is a preset numerical range, which is selected based on the numerical sensitivity verification of the input features by the deep learning model. This avoids both excessively large coordinate values ​​that could lead to gradient explosion and excessively small coordinate values ​​that could cause feature information to be overwhelmed, thus ensuring the numerical stability of subsequent position encoding.

[0136] For normalized geographic coordinate data, sine and cosine function values ​​are calculated based on a set of preset frequencies. The principle is to utilize the periodicity and monotonicity of the sine and cosine functions to encode the relative positional relationships of the normalized coordinates into multi-dimensional numerical features. Furthermore, by using functions with different preset frequencies, spatial relationships at different scales can be captured simultaneously, such as the close proximity of adjacent pixels, the mid-distance relationship between plot boundaries and centers, and the long-distance relationship between different plots, overcoming the limitation that one-dimensional normalized coordinates can only express absolute values. Its function is to convert one-dimensional normalized coordinates (… , This is converted into multi-dimensional features, providing a foundation for the subsequent construction of high-dimensional feature vectors.

[0137] First, clarify the dimension of the normalized geographic coordinate data. First dimension In the second dimension, the function values ​​of the two sets of data need to be calculated separately. The preset different frequencies are determined by a fixed frequency factor. Define and form 64 groups of different frequencies, corresponding to dimensional indices. From 0 to 63, the frequency follows By increasing the size and decreasing the size, it can cover the spatial scale from micro-pixels to macro-plots in farmland monitoring scenarios.

[0138] For any one-dimensional normalized coordinate , for or Calculate the sine and cosine function values ​​using the following formulas:

[0139] Formula for the value of the sine function: .

[0140] Formula for cosine function value: .

[0141] in, This is the location-coded value. The dimension index of the high-dimensional feature vector. The value ranges from 0 to 127; The normalized geographic coordinates are or The value range is [0,1]. Frequency index, ranging from 0 to 63, corresponding to 64 preset frequencies. Each frequency group corresponds to two dimensions: a sine dimension and a sinusoidal dimension. Cosine dimension ; The preset frequency base is the optimal value verified by a large number of experiments, ensuring that low frequencies can capture macroscopic spatial relationships and high frequencies can capture microscopic spatial relationships. The total number of feature dimensions to be generated for single-dimensional normalized coordinates is 64 sine values ​​+ 64 cosine values. This number of dimensions matches the number of channels of the visual features of the subsequent image, laying the dimensional foundation for feature fusion.

[0142] Through this calculation, and They are converted into 128-dimensional numerical feature groups respectively, realizing a multi-dimensional and structured expression of spatial location information.

[0143] The calculated sine and cosine function values ​​are combined in dimensional order to form the high-dimensional feature vector. The scattered function values ​​are structurally integrated according to a preset dimensional order, aggregating the numerical features corresponding to two-dimensional geographic coordinates into a unified high-dimensional vector. This ensures the integrity and orderliness of the features, avoiding the inability of the model to learn consistent spatial patterns due to disordered order. Its function is to generate high-dimensional spatial feature vectors that meet the input requirements of deep learning models, providing a feature carrier with a unified format and matching dimensions for subsequent fusion with image visual features.

[0144] First, it should be clear that all sine and cosine function values ​​consist of two parts: The corresponding 128-dimensional function values ​​are 64 sine values ​​and 64 cosine values. The corresponding 128-dimensional function has 64 sine values ​​and 64 cosine values, for a total of 256 values.

[0145] Prioritize combining the first dimension of geographic coordinates Function values ​​indexed by dimension Arranged in order from 0 to 127, that is ( (sine value) → ( (cosine value) → ( (sine value) → ( (cosine value) →……→ ( (cosine value of 63), forming a 128-dimensional array. Coordinate feature sub-vectors. Then combine the second-dimensional geographic coordinates. The function value is adopted in conjunction with The exact same dimensional order is arranged as follows: → →……→ , forming 128 dimensions Coordinate feature vectors.

[0146] Finally Coordinate feature vectors and Coordinate feature vectors by in front, The subsequent sequential concatenation forms a 256-dimensional high-dimensional feature vector.

[0147] The combination order is fixed and preset, and its purpose is to ensure that the output format of each geographic coordinate embedding process is consistent, so that the model can learn stably. Dimensional spatial relationships and The correspondence rules of dimensional spatial relationships, and when the 256-dimensional vector dimension is spliced ​​with the subsequent RGB three-channel visual features in 3 dimensions, a reasonable fusion feature of 259 dimensions can be formed, avoiding the visual features being covered by spatial features due to dimensional imbalance.

[0148] By performing feature fusion processing, the high-dimensional feature vector is concatenated with the image visual features of the key frame to form a fused feature.

[0149] Furthermore, feature fusion processing presupposes clarifying the specific sources, dimensional forms, and information carried by the high-dimensional feature vector and the image visual features, ensuring that they have a spatial alignment basis, thus providing conditions for subsequent stitching. The high-dimensional feature vector is a 256-dimensional feature tensor generated through geographic coordinate embedding, with the following dimensional form: , The height of the keyframe image. The keyframe image width carries the geospatial information corresponding to each pixel in the keyframe. Specifically, this includes the relative positional relationship of pixels within the farmland monitoring area and their spatial distance relationship with surrounding features. This information is converted into numerical features that the model can compute through sine and cosine position coding, serving as the basis for distinguishing features with the same visual features from those in different locations, such as cultivated land and non-agricultural green belts.

[0150] The image visual features of a keyframe are RGB three-channel feature tensors extracted from the raw keyframe data, with a dimensional form of... These features carry information about the appearance attributes of land features, including color (e.g., the green of farmland, the gray of non-agricultural buildings) and texture (e.g., the texture of crop rows in farmland, the smooth texture of non-agricultural roads), serving as the basic visual basis for initial identification of land feature types. Two types of features... , The dimensions are strictly consistent and all match the keyframe resolution. For example, if the keyframe resolution is 1920×1080... , This ensures that the spatial position of each pixel corresponds one-to-one, meeting the spatial alignment requirements for splicing.

[0151] The principle of feature fusion processing is based on the splicing logic of spatial alignment and channel dimension expansion. By superimposing two types of features in the channel dimension, i.e., the third dimension of the feature tensor, corresponding to the 2, 5, 6, and 3 dimensions mentioned above, it achieves the organic integration of visual and spatial attributes, rather than simple feature superposition or replacement. The essence of this principle is to utilize the complementarity of the two types of features. Image visual features can provide the appearance basis of ground features, while high-dimensional feature vectors can provide the spatial basis for ground feature location. The combination of the two can solve the problem of identifying homogeneous but heterogeneous features when relying solely on visual features, such as green farmland and green non-agricultural nurseries, as well as the problem of identifying homogeneous but heterogeneous features, such as the color difference of farmland in different seasons.

[0152] Feature concatenation is a direct extension of the tensor dimension without changing the original feature values ​​and structure: Let the high-dimensional feature vector tensor be... , The number of spatial feature channels; The symbol for the set of real numbers in mathematics indicates that the range of values ​​for all elements in the characteristic tensor is real numbers.

[0153] The image visual feature tensor is , The number of visual feature channels represents the fused feature tensor formed after stitching. The feature values ​​of each pixel in the channel dimension are as follows: 256 channel values The three channel values ​​ensure that the spatial information of each pixel is bound to the visual information one by one, without any information loss or misalignment.

[0154] The specific operation of feature fusion strictly follows the rules of invariant spatial location and fixed channel order to ensure the consistency and interpretability of the fused features. First, the dimensions of the two types of feature tensors are verified to confirm... , Dimensions must be perfectly matched. If there are slight differences, they must be adjusted to be consistent through bilinear interpolation. The interpolation parameters must be adapted to the resolution during keyframe spatialization to avoid pixel-level feature mismatch due to spatial misalignment.

[0155] Secondly, the fixed channel stitching order is high-dimensional feature vectors first, followed by image visual features. That is, the first 256 channels of the fused features correspond to... The geospatial features, in dimensional order when the geographic coordinates are embedded, i.e. The 128 dimensions of the encoding The 128 dimensions of the encoding, the last 3 channels correspond to The RGB visual features are processed in the order of R, G, B channels. This order is a preset fixed rule to ensure that the feature structure is consistent in each fusion process, which facilitates the subsequent deep learning model to stably learn the feature patterns.

[0156] Finally, the fusion is completed through tensor concatenation. The operation does not modify the feature values; it only merges the channel dimensions of the two types of features, ultimately outputting a fused feature tensor. .

[0157] The function of feature fusion is to provide full-dimensional feature input for subsequent geographic perception deep learning models, supporting high-precision identification of farmland non-agriculturalization monitoring. This is specifically reflected in three aspects: First, it resolves visual ambiguity by using spatial information to help distinguish visually similar features. For example, both farmland and non-agricultural nurseries are green, but if the fused features show that a certain area is located far from contiguous farmland and close to roads, the model can determine it as non-agricultural. Second, it improves boundary recognition accuracy. The spatial information in the fused features reflects the spatial distribution patterns of features. For example, farmland is often distributed in contiguous areas, while non-agricultural buildings are often scattered, providing spatial basis for subsequent coordinate attention mechanisms, enabling the model to more accurately capture the boundaries between farmland and non-agricultural features, such as building edges and the intersection of roads and farmland. Third, it enhances the model's generalization ability. When farmland changes visually due to seasonal changes, such as turning yellowish-brown after harvest, the spatial information in the fused features, such as the spatial location attributes of the area historically being farmland, can help the model maintain correct identification and avoid misjudgments caused by fluctuations in visual features.

[0158] Specifically, the fused features are spatially enhanced using a coordinate attention mechanism to improve the ability to represent ground feature boundaries.

[0159] Furthermore, the coordinate attention mechanism performs spatial enhancement processing on the fused features by performing global pooling processing on the fused features along the height and width directions respectively, generating a feature vector in the height direction and a feature vector in the width direction.

[0160] The height-direction feature vector and the width-direction feature vector are concatenated and then subjected to convolution transformation to generate intermediate features.

[0161] The intermediate features are segmented into height-direction attention weights and width-direction attention weights.

[0162] The fused features are modulated using the height direction attention weights and width direction attention weights respectively to enhance the spatial feature representation.

[0163] The fused features are globally pooled along both the height and width directions to generate height and width feature vectors. Through directional global information aggregation, global spatial correlation information in both height and width dimensions is extracted from the fused features, avoiding interference from local pixel noise and providing a global spatial basis for the subsequent generation of attention weights. Its function is to compress the two-dimensional spatial information (height × width) of the fused features into one-dimensional directional features, achieving preliminary extraction of spatial information.

[0164] First, it is clarified that the initial form of the fusion feature is a tensor. ,in The height of the keyframe image, such as 1080 pixels. For image width, such as 1920 pixels. The number of feature channels to be fused is 259, which is 256 channels of high-dimensional spatial features + 3 channels of RGB visual features.

[0165] Global pooling along the height direction: along the width direction The dimension performs global average pooling on all pixels at each height position, calculated using the following formula:

[0166] .

[0167] This operation will be performed at each height position. ,aisle of The pixel values ​​are averaged to generate the final height-oriented feature vector. Each row corresponds to global information of a height position, which can capture the distribution pattern of land features in the vertical direction, such as the vertical boundary position of farmland and non-agricultural buildings.

[0168] Global pooling along the width direction: along the height direction The dimension performs global average pooling on all pixels at each width position, calculated using the following formula:

[0169] .

[0170] This operation will apply to each width position. ,aisle of The pixel values ​​are averaged to generate the final width-direction feature vector. Each column corresponds to global information at a width position, which can capture the distribution pattern of land features in the horizontal direction, such as the horizontal boundary position between cultivated land and non-agricultural roads.

[0171] Both types of pooling use an average calculation method to balance the contribution of different pixels, avoid the interference of extreme values ​​(such as noisy pixels) on global information, and ensure that the generated directional feature vector can truly reflect the spatial distribution trend of ground features.

[0172] The height and width feature vectors are concatenated and then convolved to generate intermediate features. By concatenating and integrating global spatial information of height and width, and then using convolution to extract the correlation features between the two types of information and reduce computational cost, a concise and correlated feature foundation is provided for the subsequent generation of attention weights. Its function is to transform separate height and width features into unified intermediate features, realizing secondary processing of spatial information.

[0173] In terms of spatial dimensions, height + width and Perform the splicing. First, put... from for , and then with ( The pieces are stitched together along the height dimension to form a stitching feature. This operation links global information in the height direction with global information in the width direction. For example, the boundary of a feature at a certain height can be linked with the boundary information at the corresponding width location, providing support for identifying cross-shaped feature boundaries, such as the intersection of roads and buildings.

[0174] The convolution transformation uses a 1×1 convolution kernel pair. Perform feature transformation and introduce Activation function, denoted as The specific formula is as follows ;

[0175] convolution kernel The size is 1×1, and the number of output channels is [number missing]. , The reduction ratio is set to 8 by default. ;

[0176] Reduction ratio That is, by changing the number of channels from Down to Reduce subsequent calculations and avoid [problems]. While avoiding computational redundancy due to large numerical values, it can also retain sufficient spatial correlation information and avoid information loss caused by excessive feature compression.

[0177] Activation function : Enhance the model's ability to fit complex spatial relationships (such as irregular feature boundaries) through nonlinear transformations, and filter out invalid linear relationship features.

[0178] Final generated intermediate features It is a simplified relational representation of global information on height and width, providing direct input for the next step of segmentation attention weights.

[0179] The intermediate features are segmented into height-direction attention weights and width-direction attention weights. After inverse segmentation along the original directional dimensions, channel recovery, and weight normalization, the intermediate features are restored to a preliminary weight structure corresponding to the height and width directions. Then, convolution and activation functions are used to adjust the weight dimensions and normalize them to the [0,1] interval, ensuring that the weights accurately match the spatial dimensions of the fused features. Its function is to generate height and width attention weights that can be directly used to modulate the fused features, achieving three-stage processing of spatial information.

[0180] Feature segmentation: based on the dimensionality ratio during splicing , Will Divided into two parts:

[0181] intermediate features in the height direction ,correspond The former OK;

[0182] Middle features in the width direction ,correspond After OK.

[0183] The segmentation strictly follows the reverse process of the splicing segmentation to ensure... Features in the original height direction Spatial Dimensions Consistent, Features in the original width direction Spatial Dimensions Consistent.

[0184] Weight generation: Perform 1×1 convolution on the two classes of intermediate features after segmentation and... Activation function (denoted as) Processing, restoring the number of channels and normalizing the weights:

[0185] Height-oriented attention weights: ,in It uses a 1×1 convolution kernel and has the following output channels: ,final The numerical range is [0,1].

[0186] Attention weights in the width direction: ,in It uses a 1×1 convolution kernel and has the following output channels: ,final The numerical range is [0,1].

[0187] 1×1 convolution kernel , Its function is to increase the number of channels from Restore to Ensure weights and fusion features The number of channels is consistent, which meets the channel-by-channel modulation requirements. Activation function The purpose is to normalize the weights to [0,1]. A weight value close to 1 indicates that the feature of height or width of the corresponding spatial location is of high importance, such as the boundary of a ground feature, while a weight value close to 0 indicates that it is of low importance, such as the background area.

[0188] The fused features are modulated using the height and width attention weights respectively to enhance spatial feature representation. By dynamically adjusting each pixel of the fused feature using height and width weights, the feature response of important spatial regions such as feature boundaries is enhanced, while feature interference from irrelevant regions such as the background is suppressed, ultimately achieving precise enhancement of spatial features. Its function is to improve the fused features' ability to represent feature boundaries, providing clearer spatial feature input for subsequent interpretation of farmland as non-agricultural land in geospatial deep learning models. Specific modulation details and formulas are as follows:

[0189] The modulation process targets fusion features Each pixel and each channel is processed independently. Calculation formula: .

[0190] The enhanced spatial features after modulation, i.e., the final spatial enhancement result of the output; The original fused features are the feature values ​​at position (h,w) and channel c, which contain both spatial and visual information; Attention weights in the height direction ,aisle The weight value is used to determine the feature importance of that height position; Attention weights in the width direction are... ,aisle The weight value is used to determine the feature importance of the width position.

[0191] The modulation logic is double-weighted. For a given pixel... The eigenvalue will be simultaneously influenced by its height. Weight and width Weight adjustment. If the pixel is located at the boundary of a feature, such as the boundary between farmland and non-agricultural buildings, then... and The weight values ​​are all close to 1, and the feature values ​​are preserved or even relatively enhanced; if the pixel is located in the background area, such as a uniformly cultivated land without boundaries, then or The weight values ​​are close to 0, and the feature values ​​are suppressed. In this way, the enhanced features... It can better highlight the spatial characteristics of land feature boundaries, improve the accuracy of subsequent models in identifying the boundaries of farmland converted to non-agricultural use, and avoid misjudgment or omission due to blurred boundary features.

[0192] Specifically, the post-processing includes performing morphological closing operations and raster-to-vector conversion on the interpretation results in sequence.

[0193] The morphological closing operation includes first performing a dilation operation on the raster interpretation results to fill the internal holes, and then performing an erosion operation to restore the outline boundaries of the ground features.

[0194] The raster-to-vector conversion process includes converting the processed raster data into vector polygon data using a contour extraction algorithm.

[0195] The post-processing addresses the raster interpretation results output by the geographic perception deep learning model. Each pixel is labeled with either cultivated land or non-agricultural land cover category, and the data format is a two-dimensional raster matrix. Optimization and format conversion are performed. Morphological closing operations are used to repair local defects in the raster interpretation results, such as internal holes and minor breaks, improving the spatial integrity of the interpretation results. Secondly, raster-to-vector conversion transforms pixel-level raster data into vector polygon data, meeting the practical application needs of precise boundary delineation, area calculation, and spatial analysis in land and resources supervision, such as boundary confirmation and area statistics for non-agricultural land parcels, thus filling the gaps in vector-level spatial operations on raster data.

[0196] Morphological closing operation processing is based on the principle of mathematical morphology's structural elements acting on the foreground region. By first dilating and then eroding in a fixed order, it repairs the internal defects of the raster interpretation results while preserving the main outline of the ground features.

[0197] The principle of dilation is to expand the foreground area in the raster interpretation result using a preset structural element. This expansion applies to pixels labeled as non-agricultural features or target feature pixels defined according to requirements, allowing the foreground area to cover surrounding small background areas, such as internal holes and small gaps, thus filling defects. A 5×5 circular structural element (B) is selected as the structural element. Compared to square or rectangular structural elements, circular structural elements avoid right-angle distortion at the corners of the foreground area during dilation, ensuring that the natural shape of feature outlines is not destroyed, such as the curved boundaries of non-agricultural buildings or the curved boundaries of irregular plots.

[0198] The mathematical expression for the expansion operation is: ,in:

[0199] The raster interpretation result is a two-dimensional pixel matrix, with foreground pixel values ​​of 1 and background pixel values ​​of 0. It is a 5×5 circular structural element, with an effective pixel value of 1 and the rest of the value of 0, and the center of the circle corresponds to the center of the structural element. The reflection of structural element B, i.e. The matrix after rotating 180° around the center ensures the symmetry of the expansion direction; For the pixel coordinates of the output raster, when the structuring element is reflected... exist Location and foreground area When there is at least one intersecting pixel, The pixel at that location is identified as a foreground pixel and has a value of 1.

[0200] The erosion operation, based on the dilation operation, uses the same 5×5 circular structural element to shrink the dilated foreground area, removing the excessive expansion of the foreground area during dilation, such as rough edges and protrusions at the boundaries, restoring the original outline of the features and avoiding shape distortion. The mathematical expression for the erosion operation is: ;

[0201] in: For the raster data after the dilation operation, the foreground pixels have filled the holes; It is a 5×5 circular structural element; structural element In pixel coordinates Translation of position, i.e. The center and The overlapping matrix.

[0202] When the structural element is translated Completely contained in the foreground area ,Right now All effective pixels correspond to When the foreground pixel, The pixel at a given position is considered a foreground pixel with a value of 1, otherwise it is considered a background pixel with a value of 0.

[0203] Erosion operations counteract the excessive expansion of the foreground area boundary caused by dilation operations, and repair the problems of boundary widening and blurring of outline that may be caused by dilation. For example, the boundary of non-agricultural roads widens due to dilation, and can be restored to a width close to the actual width after erosion. At the same time, the internal pores filled by dilation are preserved, so that the outline of the ground feature is both complete and accurate, and highly matched with the spatial form of the actual ground feature.

[0204] The combination of dilation and erosion, a closed operation, is essentially a synergistic effect that repairs internal defects and preserves the main body's shape. Dilation solves the problem of internal incompleteness, while erosion solves the problem of boundary distortion. When the two are combined, the foreground area of ​​the raster interpretation result has neither internal holes and gaps nor boundary burrs and over-expansion, providing a high-quality raster data source for subsequent raster-to-vector conversion.

[0205] The principle of raster-to-vector conversion is to use contour extraction algorithms to transform pixel-level raster data optimized by morphological closing operations into vector polygon data defined by geometric coordinates, thus achieving a leap in data format from pixel arrays to spatial geometric objects.

[0206] The Marching Squares algorithm iterates through 2×2 pixel blocks of raster data, calculates the intersection points of the contour lines of the ground feature boundary with the edge of the pixel block based on the distribution pattern of foreground and background pixels within the block, i.e., the contour line configuration, and then connects all the intersection points to form a closed polygon.

[0207] Step 1: Traverse the 2×2 pixel block to determine the contour line configuration.

[0208] After performing a closing operation on the optimized raster data, a block-by-block traversal is performed. Each traversal unit is a 2×2 pixel block containing 4 pixels, denoted as top-left D1, top-right D2, bottom-right D3, and bottom-left D4. Based on the foreground and background attributes of each pixel (1 for foreground and 0 for background), the attributes of the 4 pixels are combined into 16 contour configurations from 0 to 16. For example, D1=1, D2=1, D3=0, D4=0 corresponds to one configuration, and D1=1, D2=0, D3=0, D4=1 corresponds to another. Each configuration predefines the approximate orientation of the feature boundary within the pixel block, such as horizontal crossing, vertical crossing, or diagonal crossing. By analyzing the local attribute distribution of the pixel blocks, the location and orientation of the feature boundary are accurately located, providing a basis for subsequent intersection point calculations.

[0209] Step 2: Determine the intersection points of the contour lines and the edges of the pixel blocks using linear interpolation.

[0210] For each contour configuration, based on the attribute differences between the pixels on both sides of the pixel block edge (i.e., foreground → background or background → foreground), linear interpolation is used to calculate the coordinates of the intersection point between the boundary contour line and the edge. For example, if in the upper edge D1 and D2 of a 2×2 pixel block, D1 is foreground (1) and D2 is background (0), then the calculation logic for the intersection point coordinates is as follows: Let the coordinates of D1 be... The coordinates of D2 are Based on the linear change of pixel values, the intersection point satisfy Simplified to a midpoint, the actual alignment is fine-tuned based on pixel value gradients to ensure that the intersection coordinates accurately correspond to the actual position of the feature boundary, rather than a coarse alignment of pixel edges. This transforms the discrete boundaries of pixel edges into continuous geometric coordinate intersections, improving the positional accuracy of vector boundaries and avoiding boundary offsets caused by pixel discreteness.

[0211] Step 3: Connect the intersection points to form a closed vector polygon.

[0212] Following the traversal order of the raster data, such as from left to right or from top to bottom, the intersection points calculated from all 2×2 pixel blocks are connected sequentially in a clockwise or counterclockwise direction to form a continuous closed curve, i.e., the vector polygon corresponding to the feature. For features containing internal holes, if the closing operation does not completely fill the hole, or if the hole needs to be retained in a special scene, an outer polygon is generated simultaneously. The main boundary of the feature and the boundary of the inner polygon hole constitute a complex vector object with holes. This realizes the transformation from discrete intersection points to a complete geometric object, so that the feature boundary exists in the form of a sequence of coordinate points, possessing the basic attributes of vector data, such as boundary length and closure.

[0213] Step 4: Simplify polygon vertices to reduce data redundancy.

[0214] Vertex simplification is performed on the generated closed polygons using the Douglas-Peucker algorithm to remove redundant vertices generated by pixel block traversal, such as dense vertices on consecutive near-straight lines. While maintaining boundary accuracy, the simplification error threshold is preset to 0.5 pixels, reducing the amount of vector data and improving the efficiency of subsequent spatial analysis, such as area calculation and buffer analysis. This approach balances the accuracy of vector data with storage and computational efficiency, avoiding the problems of excessive data volume and slow analysis speed caused by redundant vertices.

[0215] Specifically, the post-processing also includes a quality control step, including: calculating the reprojection error index of spatialization accuracy, which is obtained by backprojecting the solved geographic coordinates onto the image plane and comparing them with the original pixels.

[0216] The intersection-union ratio (IUGR) is used to calculate the interpretation accuracy. The IUGR is obtained by comparing the model segmentation results with the reference annotation results.

[0217] If the reprojection error index is better than the preset spatialization accuracy threshold, and the intersection-union ratio index is better than the preset interpretation accuracy threshold, then the monitoring result is deemed qualified.

[0218] The reprojection error metric, used to calculate spatialization accuracy, utilizes the inverse operation of the collinearity equation to backproject the geographic coordinates of ground points obtained after spatialization of the video back to the image plane, yielding backprojected pixel coordinates. The deviation is then calculated between these backprojected pixel coordinates and the corresponding original pixel coordinates in the keyframes, thus quantifying the positional accuracy of the spatialization process. This metric verifies the inverse accuracy of the pixel-to-geographic coordinate transformation, avoiding deviations in geographic coordinates from their actual positions caused by iterative errors in the collinearity equation and DEM interpolation biases.

[0219] First, clarify the source of the input data. The calculated geographic coordinates... That is, the coordinates of the ground point corresponding to each pixel after spatialization processing, where , For planar coordinates, These are elevation values ​​obtained through bilinear interpolation of a 5-meter resolution DEM. Original pixel coordinates. This refers to the actual image plane coordinates of the pixels in the keyframe, obtained by combining the pixel row and column numbers with the camera sensor size and pixel size, and is consistent with the observations during the previous solution of the collinearity equation. Camera intrinsic parameters include optimal extrinsic parameters such as camera focal length. Rotation matrix elements The optimal parameters come from the algorithm iterations. Camera projection center coordinates Also derived from optimal parameters , Specifically, it refers to the set of optimal parameters obtained after iterative convergence of the Levenberg-Marquardt optimization algorithm.

[0220] The back projection process strictly follows the inverse operation of the collinearity equation, converting the geographic coordinates of ground points... Substituting into the collinearity equation, the back-projected pixel coordinates are calculated. , The formula is consistent with the collinearity equation in the previous spatialization process. Since back projection involves finding pixels from known ground points, the equation form remains unchanged:

[0221] ;

[0222] ;

[0223] The reprojection error is calculated as the Euclidean distance between the backprojected pixel and the original pixel, using the following formula:

[0224] ;

[0225] in, The reprojection error is expressed as the error per pixel (unit: pixels). In actual calculations, the average reprojection error is calculated across all verification pixels to obtain the final reprojection error index. This index is used to verify spatialization accuracy: a smaller error indicates a smaller deviation between the calculated geographic coordinates and the actual ground location, resulting in more reliable location of geographic features, such as the boundary coordinates of non-agricultural land parcels. This avoids the problem of interpreting the correct category but mislocating the correct position due to inaccurate spatialization.

[0226] The Cross-Union Ratio (CIRR) metric is used to calculate interpretation accuracy. This metric measures the accuracy of the interpretation by quantifying the spatial overlap between the segmentation results output by the geographic perception deep learning model and the actual distribution of land cover categories. It balances missed detection errors (such as mislabeling non-agricultural land cover) with false positive errors (such as mislabeling arable land cover), thus avoiding the problem that accuracy and recall alone cannot comprehensively reflect interpretation quality.

[0227] The model segmentation result, i.e., the raster data output by the deep learning model, has the same dimensions as the keyframes, and is denoted as... A pixel value of 1 indicates that the pixel is interpreted as a non-agricultural feature, and 0 indicates cultivated land. The reference annotation results, i.e., ground-value raster data obtained through manual field surveys, high-precision satellite image interpretation, etc., with dimensions consistent with the model segmentation results, are denoted as... Pixel value definition and Completely identical is an objective standard for measuring the accuracy of interpretation.

[0228] Intersection and Union (IoU) The formula for calculating the area of ​​intersection to the area of ​​union of the two result spaces is: ;

[0229] in: This refers to land features that are simultaneously segmented by the model into non-agricultural land features. And the reference marking is non-agricultural land cover. The area of ​​a pixel set is the number of pixels in the set multiplied by the actual ground area corresponding to a single pixel, calculated by GSD. For example, when GSD = 2.5cm, the area of ​​a single pixel = 2.5cm × 2.5cm = 6.25cm². , refers to the set of pixels segmented by the model into non-agricultural features S=1 or referenced as non-agricultural features G=1, and the area calculation method is the same as that for intersection.

[0230] The value range is [0,1]: the closer the value is to 1, the higher the overlap between the model's segmentation results and the actual situation, and the stronger the interpretation accuracy. A value close to 0 indicates that the interpretation results deviate greatly from the actual situation. The purpose of this indicator is to verify the accuracy of the interpretation results: to ensure that the model can correctly distinguish between cultivated land and non-agricultural land features, avoid misclassifying cultivated land as non-agricultural land, which could lead to wrongful accountability or omission of non-agricultural land features, resulting in regulatory deficiencies, and provide a reliable category basis for the subsequent statistics and disposal of non-agricultural land plots.

[0231] The qualification of monitoring results is constrained by a dual threshold, simultaneously constraining spatial accuracy and interpretation accuracy. Because monitoring of farmland conversion to non-agricultural uses requires both accurate location and correct category classification, neither can be neglected. If only the projection error is deemed acceptable, but... Failure to meet the requirements will result in the correct location but the wrong category. If only... If the standards are met but the reprojection error is not, it will result in the category being correct but the location being wrong. For example, the boundary coordinates of non-agricultural land plots may deviate from reality, which will fail to meet the actual needs of land and resources supervision.

[0232] First, clarify the setting criteria for the preset threshold, based on the actual requirements and technical feasibility of the farmland monitoring scenario. The spatial accuracy threshold is usually set as the average reprojection error ≤ 1 pixel. Since the ground sampling distance (GSD) of the key frame is about 2.5 cm, the actual ground distance corresponding to 1 pixel is 2.5 cm. This threshold ensures that the position deviation of the geographical coordinates is controlled at the centimeter level, meeting the accuracy requirements for plot boundary positioning. The interpretation accuracy threshold is usually set as , which is the commonly used passing standard for the interpretation of cultivated land non-agriculturalization in the industry. This threshold ensures that the overlap degree between the model segmentation result and the actual situation is not less than 70%, reducing the impact of missed detection and misdetection.

[0233] The judgment condition is double compliance. Only when the reprojection error index < the spatial accuracy threshold and the intersection over union index > the interpretation accuracy threshold, the cultivated land non-agriculturalization monitoring result is judged to be qualified. If any index fails to meet the standard, it is necessary to return to the previous link to check for problems. The role of this judgment link is to establish a quality threshold, ensuring that the output monitoring result has both accurate spatial position attributes and reliable category attributes, and can be directly used for actual work such as the area statistics of non-agriculturalized plots, boundary rights confirmation, and supervision and law enforcement, avoiding decision-making mistakes caused by unqualified results.

[0234] In a specific embodiment, this embodiment specifically implements the monitoring method. First, control the drone to fly according to the preset flight parameters: flight height 150 meters, flight speed 8 m / s, and heading overlap rate 80%, to collect the video stream of the farmland area. At the same time, synchronously record the pose data of the camera through the positioning and orientation system, and call the digital elevation model data with a resolution of 5 meters.

[0235] Subsequently, extract key frames from the video stream: when the average optical flow change rate between adjacent frames exceeds the first preset threshold of 0.15 pixels / frame, and the image information entropy difference exceeds the second preset threshold of 0.1, mark the current frame as a key frame.

[0236] Next, perform spatial processing on the key frames: based on the collinearity equation, use the digital elevation model with a resolution of 5 meters for elevation compensation, and use an optimization algorithm to iteratively solve the collinearity equation; set the maximum number of iterations to 10 times during the iteration process, the parameter update amount norm threshold to 0.001, and dynamically adjust the damping factor according to the reprojection error. At the same time, apply boundary constraints to the solution parameter vector.

[0237] Then, construct the input data of the geospatial-aware deep learning model: convert the geographical coordinates into a 256-dimensional high-dimensional feature vector through geographical coordinate embedding processing, splice it with the image visual features through feature fusion processing, and perform spatial enhancement processing through the coordinate attention mechanism.

[0238] The fused data is then input into a geospatial deep learning model for feature interpretation to identify farmland conversion targets.

[0239] Finally, the interpretation results are post-processed: morphological closing operations are performed using 5×5 circular structural elements, raster-to-vector conversion is performed using contour extraction algorithms, and the vertex simplification error threshold is set to 0.5 pixels; in the quality control stage, the spatialization accuracy reprojection error index is required to be better than 1 pixel, and the interpretation accuracy intersection-union ratio index is required to be better than 0.7; when both of these threshold conditions are met, the monitoring results are deemed qualified, and the final farmland non-agriculturalization monitoring report is generated.

[0240] Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for monitoring the non-agricultural use of arable land based on UAV video spatialization, characterized in that, Includes the following steps: Acquire video streams, positioning and attitude determination system data, and digital elevation model data of farmland areas collected by drones; Keyframes are extracted from the video stream of the farmland area, and the keyframes are selected by a dual threshold detection algorithm based on optical flow rate of change and image information entropy; The keyframes are spatialized, and the image coordinates are converted into geographic coordinates by combining the collinearity equation optimization solution of the digital elevation model. The image data of the keyframes are fused with their corresponding geographic coordinates; The fused data is input into a geographic awareness deep learning model for land cover interpretation, wherein the model extracts spatial features through a built-in geographic coordinate embedding layer to identify farmland conversion targets. The deep learning model for geographic awareness includes: Geographic coordinate embedding processing: Convert the geographic coordinates corresponding to the pixels of the keyframe into high-dimensional feature vectors; Through feature fusion processing, the high-dimensional feature vector is concatenated with the image visual features of the key frame to form a fused feature; The fused features are spatially enhanced using a coordinate attention mechanism to improve the ability to represent ground feature boundaries. The interpretation results are post-processed to generate a monitoring report on the non-agriculturalization of arable land.

2. The method for monitoring the non-agricultural use of cultivated land based on UAV video spatialization according to claim 1, characterized in that, The keyframe extraction includes: Calculate the optical flow vector between adjacent frames of the video stream of the farmland area, and calculate the average optical flow rate of change based on the optical flow vector; Calculate the difference in image information entropy between adjacent frames; When the average optical flow rate exceeds a first preset threshold and the image information entropy difference exceeds a second preset threshold, the current frame is marked as a key frame.

3. The method for monitoring the non-agricultural use of cultivated land based on UAV video spatialization according to claim 1, characterized in that, The spatialization process includes: Establish collinearity equations based on keyframes, camera pose parameters, and ground point coordinates; Elevation compensation is performed on the collinearity equation using elevation data provided by the digital elevation model. An optimization algorithm is used to iteratively solve the collinearity equation to obtain the geographic coordinates of the pixels in the keyframe.

4. The method for monitoring the non-agricultural use of cultivated land based on UAV video spatialization according to claim 1, characterized in that, The geographic coordinate embedding process converts geographic coordinates into high-dimensional feature vectors, including: The geographic coordinates are normalized and converted to a preset numerical range; For the normalized geographic coordinates, the sine and cosine function values ​​are calculated based on a set of preset multiple different frequencies. All the calculated sine and cosine function values ​​are combined in dimensional order to form the high-dimensional feature vector.

5. The method for monitoring the non-agricultural use of cultivated land based on UAV video spatialization according to claim 1, characterized in that, Drones collect video streams and positioning / attitude determination system data from farmland areas. Control the drone to fly according to preset flight parameters and collect video streams of the farmland area; The flight parameters mentioned above include flight altitude, flight speed, forward overlap rate, and lateral overlap rate; The camera's pose data is recorded synchronously through a positioning and attitude determination system; Call up digital elevation model data with a preset resolution.

6. The method for monitoring the non-agricultural use of cultivated land based on UAV video spatialization according to claim 3, characterized in that, The iterative solution of the collinearity equation using an optimization algorithm includes: The solution is obtained iteratively by constructing a solution parameter vector that includes camera position and attitude parameters; During the iteration process, the residual is calculated based on the reprojection error of the current parameter vector, and the damping factor is dynamically adjusted according to the direction and magnitude of the residual change. Specifically, when the residual decreases and the preset convergence condition is met, the iteration is determined to be converged and the optimal geographic coordinates are output. When the residual increases or oscillates, the damping factor is increased to stabilize the iterative process; After each iteration update, boundary constraints are applied to the solution parameter vector to ensure that its value is within a preset range.

7. The method for monitoring the non-agricultural use of cultivated land based on UAV video spatialization according to claim 1, characterized in that, The coordinate attention mechanism performs spatial enhancement processing on the fused features, including: The fused features are subjected to global pooling along the height and width directions respectively to generate feature vectors in the height and width directions; The height-direction feature vector and the width-direction feature vector are concatenated and then subjected to convolution transformation to generate intermediate features; The intermediate features are segmented into height-direction attention weights and width-direction attention weights; The fused features are modulated using the height direction attention weights and width direction attention weights respectively to enhance the spatial feature representation.

8. A method for monitoring the non-agricultural use of cultivated land based on UAV video spatialization according to claim 1, characterized in that, The post-processing includes: The interpretation results are then subjected to morphological closing operations and raster-to-vector conversion. The morphological closing operation processing includes first performing a dilation operation on the raster interpretation result to fill the internal holes, and then performing an erosion operation to restore the outline boundary of the ground feature. The raster-to-vector conversion process includes converting the processed raster data into vector polygon data using a contour extraction algorithm.

9. A method for monitoring the non-agricultural use of cultivated land based on UAV video spatialization according to claim 8, characterized in that, The post-processing also includes a quality control step, including: The reprojection error index for calculating spatialization accuracy is obtained by backprojecting the solved geographic coordinates onto the image plane and comparing them with the original pixels. The intersection-union ratio (IU) index is used to calculate the interpretation accuracy. The IU index is obtained by comparing the model segmentation results with the reference annotation results. If the reprojection error index is better than the preset spatialization accuracy threshold, and the intersection-union ratio index is better than the preset interpretation accuracy threshold, then the monitoring result is deemed qualified.

Images