A large-area surveying and mapping data real-time splicing and processing method and system based on multi-unmanned aerial vehicle cooperation

Abstract

The application discloses a kind of based on multi-unmanned aerial vehicle cooperation large-area surveying and mapping data real-time splicing and processing method and system, it is related to image splicing technical field.The application includes: a plurality of unmanned aerial vehicles respectively in the respective coverage sub-region of target area continuously collect frame picture and record spatial position information, obtain the frame picture sequence of each unmanned aerial vehicle real-time update, each frame picture sequence is filtered by quality evaluation method, and the application frame picture sequence of unmanned aerial vehicle is obtained.The application is driven by semantic segmentation rigid-flexible partition differential registration strategy, global homography constraint of rigid area and elastic deformation field decoupling optimization of flexible area, so that the geometric fidelity of building, road and other key features is not involved by non-rigid motion interference, guarantee the vectorization accuracy of surveying and mapping result, natural fitting is obtained by the frequency domain adaptive weighting and time domain continuous constraint of non-rigid deformation through elastic deformation field, weaken ghosting, blur and grid boundary fracture phenomenon.

Description

Technical Field

[0001] This invention relates to the field of image stitching technology, specifically to a method and system for real-time stitching and processing of large-area surveying data based on multi-UAV collaboration. Background Technology

[0002] Low-altitude remote sensing by drones has become an important tool for large-area topographic mapping, environmental monitoring, and disaster emergency assessment. By deploying multiple drones for collaborative operations, the data collection cycle can be significantly shortened and operational efficiency improved.

[0003] For example, patent publication number "CN122048642A", entitled "Real-time Image Stitching System and Method for UAVs Based on Navigation Assistance and Mesh Homography Modeling", includes a data synchronization acquisition module, a spatial initial value calculation module, an overlapping region determination module, a feature matching module, a local homography matrix calculation module, a directional mesh construction module, and an image reconstruction and stitching module. The data synchronization acquisition module is used to synchronously acquire airborne image sequences and real-time navigation data. This invention provides accurate spatial transformation initial values ​​for feature matching by deeply fusing real-time UAV navigation information. This enables the algorithm to transform traditional global search into targeted local precise search during the feature extraction and matching stages, thereby significantly reducing invalid computation, greatly reducing the computational complexity of feature matching, significantly improving system real-time performance, and effectively reducing the false matching rate.

[0004] The aforementioned patent divides the overlapping area into regular grids and solves an independent local homography matrix for each sub-grid. The homography matrix is ​​essentially a planar projection transformation model, which can only describe the linear mapping relationship of a planar scene under changes in camera viewpoint. However, aerial photography scenes are often affected by wind disturbances such as swaying vegetation, water ripples, and local displacements caused by moving objects (vehicles, construction machinery). These are all non-rigid motions and cannot be accurately modeled using any form of homography matrix (whether global or local), resulting in ghosting and blurring in the stitched overlapping area. To address this, a method and system for real-time stitching and processing of large-area surveying data based on multi-UAV collaboration was invented. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for real-time stitching and processing of large-area surveying and mapping data based on multi-UAV collaboration, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: The real-time stitching and processing method includes: several drones continuously acquiring frame images and recording spatial location information in their respective coverage sub-areas of the target area to obtain a real-time updated frame image sequence of each drone; filtering each frame image sequence through a quality assessment method to obtain an application frame image sequence of the drone; each drone selecting an initial key frame from its own application frame image sequence to establish a local spatial reference, and selecting key frames from the application frame image sequence to form a key frame set; Semantic segmentation is performed on each frame in the keyframe set of the UAV, dividing pixels into rigid and flexible sub-regions. The relative positional relationships and overlapping areas between keyframes are determined based on spatial location information. The cross-frame motion trajectory of feature points within each sub-region is tracked, and a spectral transform is performed on the motion trajectory to extract high-frequency reciprocating components and low-frequency trend components. The high-frequency reciprocating components are then weighted, and an inverse Fourier transform is performed on the weighted frequency domain signal to obtain the filtered displacement trajectory in the spatial domain. Using the filtered displacement trajectory as the registration basis and the semantic segmentation result as the boundary condition, a global homography transformation constraint is applied to the rigid sub-region, and a local homography transformation constraint is applied to the flexible sub-region. Smoothing constraints and temporal continuity constraints are used to construct an energy function containing data fidelity terms, stiffness constraints, and smoothing regularization terms. A pixel-level elastic deformation field is obtained by jointly solving this function within a unified optimization framework. Each UAV generates a local mosaic image based on its initial keyframe. Image coordinate transformation and resampling are performed on the overlapping areas of each UAV according to the pixel-level elastic deformation field. The fused overlapping areas are then integrated frame by frame into their respective local mosaic images. The local mosaic images of several UAVs are globally registered, and coordinate transformation is used to align each local mosaic image to a unified global coordinate system. The overlapping areas between adjacent local mosaic images are then fused to generate a global mosaic image covering the target area.

[0007] Furthermore, the first frame on the timestamp is selected from the application frame sequence as the initial keyframe, and the pose and orientation of the initial keyframe are recorded as the current coverage state. For each subsequent candidate frame, the spatial coverage difference between the candidate frame and the set of selected keyframes is calculated based on the candidate frame's own pose and orientation. The spatial coverage difference includes the viewpoint deviation and the region increment. If the viewpoint deviation exceeds a preset viewpoint threshold and the region increment exceeds a preset increment threshold, the candidate frame is determined to be an information complementary frame. The information complementary frame is added to the set of selected keyframes and the coverage state is updated. If the candidate frame falls within the existing coverage area, it is determined to be a redundant frame and is skipped. This process is repeated until the application frame sequence is processed.

[0008] Furthermore, the viewpoint deviation is determined by the azimuth difference between the field of view direction of the candidate frame and the field of view direction of the spatially nearest frame in the selected keyframe set; the region increment is determined by the proportion of the newly added region of the ground coverage area of ​​the candidate frame relative to the current overall coverage area of ​​the selected keyframe set.

[0009] Furthermore, semantic segmentation employs a lightweight convolutional neural network to perform real-time pixel-level classification of frames, labeling buildings, roads, and man-made facilities as rigid sub-regions, and vegetation, water bodies, and sandy areas as flexible sub-regions. Using the semantic segmentation results as boundary conditions, global homography transformation constraints are applied to pixels within rigid sub-regions, while local smoothing constraints are applied to pixels within flexible sub-regions to induce elastic deformation in the flexible sub-regions.

[0010] Furthermore, the global homography transformation constraint is calculated based on the matching feature points in the rigid sub-region and applied to the rigid sub-region; the local smoothing constraint is applied to the pixel displacement field in the flexible sub-region based on the non-rigid transformation model; and the temporal continuity constraint is applied based on the change amplitude of the deformation field between adjacent time points. The weight of the stiffness constraint term is adjusted based on the residual value of the feature point matching in each sub-region, and the weight of the smoothing regularization term is allocated based on the ratio of the spectral energy of the high-frequency reciprocating component to the low-frequency trend component.

[0011] Furthermore, the spectrum transformation includes performing a fast Fourier transform on the time-domain signal of the cross-frame motion trajectory of the feature points, determining the components with frequencies higher than a preset frequency threshold as high-frequency reciprocating components and assigning them zero weights, determining the components with frequencies lower than the preset frequency threshold as low-frequency trend components and assigning them standard weights, and performing an inverse Fourier transform on the frequency-domain signal after weight reduction processing to obtain the filtered displacement trajectory in the spatial domain.

[0012] Furthermore, the energy function includes a linear combination of a data fidelity term, a stiffness constraint term, and a smoothing regularization term. The data fidelity term calculates the registration residual between the pixel-level elastic deformation field and the filtered displacement trajectory. The stiffness constraint term calculates the positional deviation of pixels in the rigid sub-region after global homography transformation. The smoothing regularization term calculates the difference between the deformation vectors of adjacent pixels in the flexible sub-region and the difference between the deformation vectors at adjacent time points. The minimum value of the energy function is obtained by solving the Gauss-Newton iteration method to obtain the pixel-level elastic deformation field.

[0013] Furthermore, the quality assessment method includes calculating the frame image sharpness based on the Laplacian operator, evaluating the frame image exposure uniformity based on histogram statistics, and judging the frame image acquisition stability based on the attitude angle in the spatial location information. Frame images with sharpness lower than a preset sharpness threshold, exposure uniformity lower than a preset uniformity threshold, and attitude angle offset exceeding the allowable offset range are removed from the frame image sequence, and the frames that meet the quality requirements are retained to form the application frame image sequence.

[0014] A real-time stitching and processing system for large-area mapping data based on multi-UAV collaboration is provided. The system employs the aforementioned real-time stitching and processing method for large-area mapping data based on multi-UAV collaboration. The real-time stitching and processing system includes: a processing module: several UAVs continuously collect frame images and record spatial location information within their respective coverage sub-areas of the target area, obtaining a real-time updated frame image sequence for each UAV; filtering each frame image sequence using a quality assessment method to obtain an application frame image sequence for the UAVs; each UAV selects an initial keyframe from its respective application frame image sequence to establish a local spatial reference, and then filters keyframes from the application frame image sequence to form a keyframe set. The stitching module performs semantic segmentation on each frame in the keyframe set of the UAV, dividing pixels into rigid and flexible sub-regions. Based on spatial location information, it determines the relative positional relationships and overlapping areas between keyframes, tracks the cross-frame motion trajectories of feature points within each sub-region, performs spectral transformation on the motion trajectories, extracts high-frequency reciprocating components and low-frequency trend components, reduces the weight of the high-frequency reciprocating components, and uses the low-frequency trend components as the registration basis. Using the semantic segmentation results as boundary conditions, it applies global homography constraints to the rigid sub-regions and local smoothing and temporal continuity constraints to the flexible sub-regions, constructing a system containing... The energy functions of the data fidelity term, stiffness constraint term, and smoothing regularization term are jointly solved within a unified optimization framework to obtain the pixel-level elastic deformation field. The UAVs generate local mosaic images based on their respective initial keyframes. Based on the pixel-level elastic deformation field, the overlapping areas of each UAV are transformed and resampled. The fused overlapping areas are then integrated into their respective local mosaic images frame by frame. The local mosaic images of several UAVs are globally registered. Through coordinate transformation, each local mosaic image is aligned to a unified global coordinate system. The overlapping areas between adjacent local mosaic images are fused to generate a global mosaic image covering the target area.

[0015] Compared with the prior art, the beneficial effects of the present invention are: This method and system for real-time stitching and processing of large-area surveying data based on multi-UAV collaboration utilizes a semantic segmentation-driven rigid-flexible partitioning differential registration strategy. It decouples and optimizes the global homography constraint of rigid regions with the elastic deformation field of flexible regions, ensuring that the geometric fidelity of key features such as buildings and roads is not affected by non-rigid motion. This guarantees the vectorization accuracy of the surveying results. Non-rigid deformation is naturally fitted through frequency-domain adaptive weighting and temporal continuous constraints of the elastic deformation field, reducing ghosting, blurring, and grid boundary breakage. Even in the absence of reliable feature matching, weak-texture regions can still maintain a reasonable elastic deformation trend through temporal continuous constraints, reducing registration degradation and error accumulation.

[0016] Meanwhile, through a semantic segmentation-driven rigid-flexible partitioning registration strategy, rigid sub-regions (buildings, roads, and man-made facilities) maintain precise geometric alignment through global homography constraints, ensuring the vectorization accuracy of the surveying results; flexible sub-regions (vegetation, water bodies, and sandy areas) allow local nonlinear displacement through elastic deformation fields, effectively absorbing dynamic non-rigid movements such as wind-induced vegetation swaying and water surface ripples. Even in weakly textured areas such as large areas of water and farmland, the elastic deformation field can still maintain a reasonable deformation trend through smoothness constraints and temporal continuity constraints, solving the registration failure or degradation caused by feature scarcity in weakly textured areas.

[0017] By using partitioned differential constraints of stiffness and smoothing regularization terms in the energy function, the homography accuracy of the rigid sub-region is not affected by the elastic deformation of the flexible sub-region. At the same time, the elastic displacement of the flexible sub-region is made continuous and natural through the smoothing regularization term, which effectively avoids the splicing discontinuity and tearing problems caused by independent mesh optimization. The temporal continuity constraint enables the elastic deformation field to be smoothly transmitted between frames, and the deformation gradient of adjacent frames is constrained within a reasonable range. This suppresses the accumulation and amplification of frame-by-frame registration errors to the global splicing image, ensuring the global consistency and geometric accuracy of the splicing results in a large area under long-term and large-area operation scenarios. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the real-time splicing and processing method of the present invention; Figure 2 This is a schematic diagram of the frame processing of the present invention; Figure 3 This is a schematic diagram of the global stitching diagram of the present invention; Figure 4 This is a schematic diagram illustrating the fusion of partial stitched images from multiple UAVs according to the present invention; Figure 5 This is a schematic diagram illustrating the multi-drone corresponding multi-view splicing of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] like Figure 1 - Figure 5As shown, the present invention provides a technical solution: a real-time stitching and processing method includes: several drones continuously acquiring frame images and recording spatial position information in their respective coverage sub-areas of the target area, obtaining a real-time updated frame image sequence of each drone, filtering each frame image sequence through a quality assessment method to obtain the application frame image sequence of the drone, and each drone selecting an initial key frame from its own application frame image sequence to establish a local spatial reference, and filtering key frames from the application frame image sequence to form a key frame set; Semantic segmentation is performed on each frame in the keyframe set of the UAV, dividing pixels into rigid and flexible sub-regions. The relative positional relationships and overlapping areas between keyframes are determined based on spatial location information. The cross-frame motion trajectory of feature points within each sub-region is tracked, and a spectral transform is performed on the motion trajectory to extract high-frequency reciprocating components and low-frequency trend components. The high-frequency reciprocating components are then weighted, and an inverse Fourier transform is performed on the weighted frequency domain signal to obtain the filtered displacement trajectory in the spatial domain. Using the filtered displacement trajectory as the registration basis and the semantic segmentation result as the boundary condition, a global homography transformation constraint is applied to the rigid sub-region, and a local homography constraint is applied to the flexible sub-region. By employing sliding constraints and temporal continuity constraints, an energy function is constructed that includes data fidelity terms, stiffness constraints, and smoothing regularization terms. Within a unified optimization framework, a pixel-level elastic deformation field is jointly solved to obtain the local mosaic image. Each UAV generates a local mosaic image based on its initial keyframe. Based on the pixel-level elastic deformation field, image coordinate transformation and resampling are performed on the overlapping areas of each UAV. The fused overlapping areas are then integrated frame by frame into their respective local mosaic images. The local mosaic images of several UAVs are globally registered, and coordinate transformation is used to align each local mosaic image to a unified global coordinate system. The overlapping areas between adjacent local mosaic images are then fused to generate a global mosaic image covering the target area.

[0021] The first frame with the timestamp is selected from the application frame sequence as the initial keyframe, and the pose and orientation of the initial keyframe are recorded as the current coverage state. For each subsequent candidate frame, the spatial coverage difference between the candidate frame and the set of selected keyframes is calculated based on the candidate frame's own pose and orientation. The spatial coverage difference includes the viewpoint deviation and the region increment. If the viewpoint deviation exceeds the preset viewpoint threshold and the region increment exceeds the preset increment threshold, the candidate frame is determined to be an information complementary frame. The information complementary frame is added to the set of selected keyframes and the coverage state is updated. If the candidate frame falls within the existing coverage area, it is determined to be a redundant frame and is skipped. This process is repeated until the application frame sequence is processed.

[0022] The viewing angle deviation is determined by the azimuth difference between the field of view of the candidate frame and the field of view of the nearest frame in the selected keyframe set; the region increment is determined by the proportion of the newly added region of the ground coverage area of ​​the candidate frame relative to the current overall coverage area of ​​the selected keyframe set.

[0023] Semantic segmentation uses a lightweight convolutional neural network to perform real-time pixel-level classification of frames. Buildings, roads, and man-made facilities are labeled as rigid sub-regions, while vegetation, water bodies, and sandy areas are labeled as flexible sub-regions. Using the semantic segmentation results as boundary conditions, global homography transformation constraints are applied to pixels in rigid sub-regions, and local smoothing constraints are applied to pixels in flexible sub-regions to induce elastic deformation in the flexible sub-regions.

[0024] The global homography transformation constraint is calculated based on the matching feature points in the rigid sub-region and applied to the rigid sub-region. The local smoothing constraint is applied to the pixel displacement field in the flexible sub-region based on the non-rigid transformation model. The temporal continuity constraint is applied based on the change amplitude of the deformation field between adjacent time steps. The weight of the stiffness constraint term is adjusted based on the residual value of the feature point matching in each sub-region. The weight of the smoothing regularization term is allocated based on the ratio of the spectral energy of the high-frequency reciprocating component to the low-frequency trend component.

[0025] The spectrum transformation includes performing a fast Fourier transform on the time-domain signal of the cross-frame motion trajectory of feature points, identifying components with frequencies higher than a preset frequency threshold as high-frequency reciprocating components and assigning them zero weights, identifying components with frequencies lower than a preset frequency threshold as low-frequency trend components and assigning them standard weights, and performing an inverse Fourier transform on the frequency-domain signal after weight reduction processing to obtain the filtered displacement trajectory in the spatial domain.

[0026] The energy function is a linear combination of a data fidelity term, a stiffness constraint term, and a smoothing regularization term. The data fidelity term calculates the registration residual between the pixel-level elastic deformation field and the filtered displacement trajectory. The stiffness constraint term calculates the positional deviation of pixels in the rigid sub-region after global homography transformation. The smoothing regularization term calculates the difference between the deformation vectors of adjacent pixels in the flexible sub-region and the difference between the deformation vectors at adjacent time points. The minimum value of the energy function is obtained by solving the Gauss-Newton iteration method to obtain the pixel-level elastic deformation field.

[0027] The quality assessment method includes calculating frame sharpness based on the Laplacian operator, evaluating frame exposure uniformity based on histogram statistics, and judging frame acquisition stability based on attitude angles in spatial location information. Frames with sharpness below a preset sharpness threshold, exposure uniformity below a preset uniformity threshold, and attitude angle offset exceeding the allowable offset range are removed from the frame sequence, and frames that meet the quality requirements are retained to form the application frame sequence.

[0028] A real-time stitching and processing system for large-area surveying data based on multi-UAV collaboration is provided. The system adopts the aforementioned real-time stitching and processing method for large-area surveying data based on multi-UAV collaboration. The real-time stitching and processing system includes: a processing module: several UAVs continuously collect frame images and record spatial location information in their respective coverage sub-areas of the target area to obtain the real-time updated frame image sequence of each UAV. The frame image sequence is filtered by a quality assessment method to obtain the application frame image sequence of the UAV. Each UAV selects an initial key frame from its own application frame image sequence to establish a local spatial reference and selects key frames from the application frame image sequence to form a key frame set. The stitching module performs semantic segmentation on each frame in the keyframe set of the UAV, dividing pixels into rigid and flexible sub-regions. Based on spatial location information, it determines the relative positional relationships and overlapping areas between keyframes, tracks the cross-frame motion trajectories of feature points within each sub-region, performs spectral transformation on the motion trajectories, extracts high-frequency reciprocating components and low-frequency trend components, reduces the weight of the high-frequency reciprocating components, and uses the low-frequency trend components as the registration basis. Using the semantic segmentation results as boundary conditions, it applies global homography constraints to the rigid sub-regions and local smoothing and temporal continuity constraints to the flexible sub-regions, constructing a system containing... The energy functions of the data fidelity term, stiffness constraint term, and smoothing regularization term are jointly solved within a unified optimization framework to obtain the pixel-level elastic deformation field. The UAVs generate local mosaic images based on their respective initial keyframes. Based on the pixel-level elastic deformation field, the overlapping areas of each UAV are transformed and resampled. The fused overlapping areas are then integrated into their respective local mosaic images frame by frame. The local mosaic images of several UAVs are globally registered. Through coordinate transformation, each local mosaic image is aligned to a unified global coordinate system. The overlapping areas between adjacent local mosaic images are fused to generate a global mosaic image covering the target area.

[0029] like Figure 1 As shown, multiple drones collaboratively collect frame images of the target area along a preset flight path, simultaneously recording the spatial location information (3D coordinates, attitude angle, and timestamp) of each frame. This information is transmitted in real-time to the ground data processing center via a wireless link. The ground-based system first performs a quality assessment, eliminating blurred frames based on the Laplacian operator, frames with abnormal exposure based on histogram statistics, and frames with unstable acquisition based on attitude angle offset. This results in a sequence of application frame images. The first frame is then selected as the initial keyframe from the set. For subsequent candidate frames, the viewpoint deviation and regional increment between the frame and the keyframe set are calculated based on the spatial location information. Frames with any index exceeding a preset threshold are considered complementary and included in the set, while redundant frames are skipped. This process is repeated to continuously update the keyframe set.

[0030] Lightweight convolutional neural network semantic segmentation is performed on keyframes, labeling buildings, roads, and man-made facilities as rigid sub-regions, and vegetation, water bodies, and sandy areas as flexible sub-regions. A binary mask is used as boundary conditions to extract feature points within both rigid and flexible sub-regions and track their cross-frame motion trajectories. Discrete Fourier transform is performed on the trajectories to extract high-frequency reciprocating components (non-rigid motion noise such as vegetation swaying and water ripples) and low-frequency trend components (rigid parallax of the UAV platform). High-frequency components are weighted using an exponential decay function and then removed. An inverse Fourier transform is performed on the weighted frequency domain signal to obtain... The filtered displacement trajectory in the spatial domain is used as the registration basis. Based on this, an energy function is constructed, including data fidelity terms, stiffness constraint terms, and smoothing regularization terms. The stiffness constraint term applies global homography constraints only within rigid sub-regions to ensure geometric accuracy. The smoothing regularization term applies local smoothing constraints and temporal continuity constraints within flexible sub-regions to absorb non-rigid displacements. Pixel-level elastic deformation fields are solved through L-BFGS iterative optimization. Finally, using the initial keyframe on the UAV timestamp as the base map, coordinate transformation and resampling are performed on overlapping regions by partition. Global homography constraints are applied to rigid sub-regions. The data should be aligned, with flexible sub-regions shifted pixel-by-pixel. After weighted fusion with fade-in and fade-out, these sub-regions are integrated frame-by-frame into the overall mosaic. This process is repeated until all keyframes corresponding to a single UAV have been processed, outputting a local mosaic. These local mosaics are then fused to obtain the overall mosaic. This overall mosaic retains the high-resolution texture details of each frame and achieves pixel-level geometric registration and global spatial consistency through a unified optimization framework. It meets the requirements for data integrity, geometric accuracy, and timeliness in large-area topographic mapping, environmental monitoring, and disaster emergency assessment, and therefore can be used directly as mapping data. The semantic segmentation-driven rigid-flexible partitioning differential registration strategy decouples the global homography constraint of rigid regions with the elastic deformation field of flexible regions, ensuring that the geometric fidelity of key features such as buildings and roads is not affected by non-rigid motion, thus guaranteeing the vectorization accuracy of surveying results. Non-rigid deformation is naturally fitted through frequency domain adaptive weighting and temporal continuous constraints of the elastic deformation field, reducing ghosting, blurring, and grid boundary breakage. Even in the absence of reliable feature matching, weak texture regions can still maintain a reasonable elastic deformation trend through temporal continuous constraints, reducing registration degradation and error accumulation.

[0031] Each drone flies collaboratively within the target area according to a preset route to perform full coverage photography of the target area. To ensure complete coverage of the target area, the field of view of each drone is redundant at the boundary of the area, and the overall coverage of the frame collection is greater than the boundary of the target area.

[0032] While capturing each frame of footage, the drone records the corresponding spatial location information. This spatial location information is obtained through an onboard GNSS or INS integrated navigation system and includes: three-dimensional geographic coordinates (longitude, latitude, and altitude), three-axis attitude angles (pitch, roll, and yaw), and a capture timestamp. The spatial location information is used to determine the relative spatial orientation and approximate field of view between each frame. Due to limitations in positioning accuracy, the coordinates of the spatial location information are insufficient to support direct and precise registration between frames, but they are sufficient to serve as a basis for determining the relative positional relationship and spatial proximity between frames.

[0033] The UAV can transmit the collected frames and corresponding spatial location information to the ground data processing center in real time via a wireless data link. The ground data processing center then performs processing tasks such as quality assessment, keyframe selection, semantic segmentation, motion analysis, elastic registration, and incremental stitching. The ground data processing center is the computing node. Multiple UAVs continuously collect and generate timestamped frames. The system sequentially incorporates these frames into the frame set and incrementally completes real-time stitching based on newly added frames. At the same time, a basic screening is performed on newly added frames during stitching to exclude frames with poor quality, thereby ensuring the quality of the stitched images. Through a rigid-flexible partitioning registration strategy driven by semantic segmentation, rigid sub-regions (buildings, roads, and artificial facilities) maintain accurate geometric alignment through global homography constraints, ensuring the vectorization accuracy of the surveying results. Flexible sub-regions (vegetation, water bodies, and sandy areas) allow local nonlinear displacement through elastic deformation fields, effectively absorbing dynamic non-rigid movements such as wind-induced vegetation swaying and water surface ripples. Even in areas with weak texture, such as large water areas and farmland, the elastic deformation field can still maintain a reasonable deformation trend by relying on smoothness constraints and temporal continuity constraints, thus solving the problem of registration failure or degradation caused by the lack of features in areas with weak texture.

[0034] The two quantitative metrics for keyframe determination are: first, the viewpoint deviation, which measures the viewpoint change brought about by the candidate frame by calculating the azimuth difference between the viewpoint direction of the candidate frame and the viewpoint direction of the spatially nearest frame in the selected keyframe set; and second, the area increment, which measures the amount of new information brought by the candidate frame by calculating the proportion of the ground coverage area of ​​the candidate frame relative to the current overall coverage area of ​​the selected keyframe set. Only when a candidate frame has sufficient viewpoint difference and sufficient new coverage area is it determined to be an information-complementary frame and included in the set; otherwise, it is determined to be a redundant frame and skipped.

[0035] like Figure 2As shown, the quality assessment and keyframe selection sub-process details the independent frame quality screening and keyframe selection process performed by each UAV. Using the original frame sequence as input, a triple quality assessment is first performed sequentially: a sharpness assessment based on the Laplacian operator, an exposure uniformity assessment based on the grayscale histogram uniformity index, and an acquisition stability assessment based on attitude angle offset. Frames failing to meet preset thresholds in any dimension are discarded. Only after passing all three screenings can the frame sequence be used for application. Based on this, the first frame at the timestamp in the application frame sequence is selected as the initial keyframe, and a local spatial reference is established. Subsequently, the spatial coverage difference (including viewpoint deviation and region increment) between each subsequent candidate frame and the selected keyframe set is calculated. If the viewpoint deviation exceeds a preset viewpoint threshold and the region increment exceeds a preset increment threshold, it is determined to be inaccurate. Complementary frames are included in the keyframe set; if a candidate frame falls sufficiently within the existing coverage area, it is considered a redundant frame and skipped. This process is repeated until all candidate frames have been processed, outputting a keyframe set with the maximum information content and minimum redundancy. Three quality assessment metrics are used to filter the frame sequence: sharpness is assessed based on the Laplacian operator, which is sensitive to gradient changes in image edges and details; frames with insufficient sharpness will introduce blur during stitching; exposure uniformity is assessed based on histogram statistics, using the grayscale distribution uniformity index to determine if there is local overexposure or underexposure; and acquisition stability is assessed based on attitude angle, calculating the Euclidean distance between the current frame attitude angle and the flight path design attitude angle; frames with unstable attitude will produce geometric distortion. The combination of these three assessment methods ensures that the frames entering subsequent processing stages meet the quality requirements in terms of sharpness, exposure, and stability.

[0036] The quality assessment method includes three aspects: calculating frame sharpness based on the Laplacian operator, evaluating frame exposure uniformity based on histogram statistics, and judging frame acquisition stability based on attitude angles in spatial location information. The Laplacian operator is used to quantitatively evaluate frame sharpness. The Laplacian operator is a second-order differential operator that is highly sensitive to edge and detail information in an image. The sharper the image, the more dramatic the gradient changes at edges and details; the larger the Laplacian response value. Histogram statistics are used to evaluate frame exposure uniformity. First, the frame image is converted to grayscale, and its grayscale histogram H(k), k=0,1,...,255, is calculated. The uniformity index U of the grayscale distribution is then calculated. ; Where μH is the mean of the gray-level distribution, calculated using the following formula: ; The value range of the uniformity index U is [0, 1], and a U value closer to 1 indicates more uniform exposure. In this embodiment, the exposure uniformity threshold U0 is set to 0.65. If U < U0, it is determined that the frame has non-uniform exposure (with locally overexposed or underexposed areas) and is rejected. If more than 15% of the pixels in the grayscale histogram are concentrated at grayscale level 0 (pure black) or 255 (pure white), the exposure is also determined to be abnormal.

[0037] The acquisition stability of the frame image is judged based on the attitude angle in the spatial position information, and the attitude angle offset θd is defined as the Euclidean distance between the attitude angle of the current frame and the attitude angle of the route design: ; Wherein, Δpitch, Δroll, and Δyaw are the deviation values of the pitch angle, roll angle, and yaw angle respectively, and k0=0.5 is the weight reduction coefficient of the yaw angle (since the deviation of the yaw angle has a relatively small influence on the geometric distortion of the image). In this embodiment, the allowable offset range θ0 is set to 5.0°, and if θd > θ0, it is determined that the attitude is unstable during the acquisition of this frame, and the frame is rejected.

[0038] The goal of key frame selection is to filter out a frame subset with the largest amount of information and the least redundancy from the applied frame image sequence, so as to reduce the calculation amount of subsequent processing on the premise of maintaining complete scene coverage. In the key frame selection process, the spatial distance di between the current frame Fi and the overall coverage of the key frame set K is calculated according to the spatial position information. The spatial distance is defined as the Euclidean distance between the center point of the current frame and the center point of the nearest frame in the key frame set: ; Wherein, C(F) represents the center point coordinate of frame F, which is determined by the longitude and latitude coordinates in the spatial position information, represents the Euclidean distance between two points. Euclidean distance is the most basic and commonly used distance measurement method in mathematics. In layman's terms, it is the straight-line distance between two points. The frames to be processed are sorted by spatial distance from small to large, and the frame with the smallest spatial distance is preferentially selected as the frame to be compared.

[0039] Sort the frames to be processed by the spatial distance di from small to large, and preferentially select the frame with the smallest spatial distance as the frame to be compared. Calculate the image space coverage difference δi between the current frame and the frame to be compared: ; Wherein, A(F) represents the projected coverage area of frame F on the ground. If δi < δ0 (δ0=0.3 is the coverage threshold set in this embodiment), it indicates that there is sufficient viewing angle difference and new information between the current frame and the key frame set, and the current frame is added to the key frame set; otherwise, this frame is skipped. The above process is executed cyclically until the processing of the applied frame image sequence is completed; Semantic segmentation employs a lightweight convolutional neural network for real-time pixel-level classification of frames. The network takes a frame as input and outputs a pixel-level classification mask of the same resolution, classifying each pixel into one of six major landform types: buildings, roads, artificial structures, vegetation, water bodies, and sandy areas. After semantic segmentation, buildings, roads, and artificial structures are labeled as rigid sub-regions, while vegetation, water bodies, and sandy areas are labeled as flexible sub-regions. The semantic segmentation results are used as boundary conditions for subsequent registration processing in the form of binary masks. Pixels within the rigid sub-region mask are forced to conform to global homography, while pixels within the flexible sub-region mask are allowed to undergo elastic deformation under local smoothing constraints. Buildings, roads, and artificial structures are classified as rigid sub-regions because their geometry remains largely unchanged within the aerial photography timescale (typically on the order of minutes), and their inter-frame variations mainly originate from the movement of the drone's viewpoint, thus belonging to rigid deformation. The reason why vegetation, water bodies, and sandy land are classified as flexible sub-regions is that these features exhibit continuous dynamic deformation (wind-induced vegetation swaying, water surface ripples, and sand texture changes). Their inter-frame changes contain non-rigid motion components. Using semantic segmentation results as boundary conditions, global homography transformation constraints are applied to pixels in rigid sub-regions, while local smoothing constraints are applied to pixels in flexible sub-regions. The geometric fidelity of rigid sub-regions is not affected by the elastic deformation of flexible sub-regions, ensuring the vectorization accuracy of key features such as buildings and roads in the surveying results. At the same time, the elastic deformation of flexible sub-regions is naturally fitted through local smoothing constraints, reducing the ghosting and blurring phenomena produced by traditional global homography methods in vegetation and water areas. Local smoothing constraints use a non-rigid transformation model to smooth the pixel displacement field in flexible sub-regions, and temporal continuity constraints are based on the change amplitude of the deformation field between adjacent time points.

[0040] like Figure 3 As shown, taking a set of keyframes as input, a lightweight convolutional neural network is first used for semantic segmentation, outputting rigid sub-region masks (buildings, roads, man-made facilities) and flexible sub-region masks (vegetation, water, sand). The two branches are processed in parallel. The rigid sub-region branch is subject to global homography constraints (i.e., stiffness constraints) to ensure the geometric accuracy of key mapping features such as buildings and roads. The flexible sub-region branch is subject to local smoothing constraints and temporal continuity constraints (i.e., smoothing regularization) to absorb non-rigid displacements such as vegetation swaying and water ripples. The constraints of the two branches and the data fidelity terms together constitute a unified energy function. The pixel-level elastic deformation field is solved through iterative optimization. After obtaining the elastic deformation field, image coordinate transformation and resampling are performed in partitions. The rigid sub-regions are globally homography aligned, and the flexible sub-regions are pixel-by-pixel elastic offset and local smoothing are fused. Finally, a local stitched image is output.

[0041] The cross-frame motion trajectory of feature points simultaneously contains two different types of motion signals: rigid parallax (low-frequency trend) generated by the UAV platform's own motion, and non-rigid motion noise (high-frequency reciprocating) generated by environmental factors such as swaying vegetation and water ripples. A Fast Fourier Transform (FFT) is performed on the motion trajectory to convert the time-domain signal to the frequency domain. In the frequency domain, components above a preset frequency threshold are identified as high-frequency reciprocating components (corresponding to environmental noise) and assigned zero weights for removal. Components below or equal to the preset frequency threshold are identified as low-frequency trend components (corresponding to true parallax) and assigned standard weights for retention as the registration basis. This frequency domain analysis method allows the registration process to automatically adapt to the intensity of non-rigid motion in different scenarios.

[0042] Feature point extraction is performed separately in rigid and flexible sub-regions. For each pair of adjacent keyframes, at least 200 pairs of matching feature points are extracted in the rigid sub-region and at least 150 pairs of matching feature points are extracted in the flexible sub-region. The cross-frame motion trajectory of the feature points in each sub-region is tracked in the frame sequence. Let the coordinates of feature point p in frame t be (xt, yt)p. By tracking its position in subsequent frames, the cross-frame motion trajectory of the feature point is obtained as Tp = {(xt, yt)p | t = 1, 2, ..., Nt}. Perform a Discrete Fourier Transform on the motion trajectory Tp to convert the time-domain motion signal to the frequency domain for analysis: ; Where Nt is the number of sampling points for the motion trajectory, k is the frequency index, and j is the imaginary unit. High-frequency reciprocating components (frequency f>fH, where fH=2Hz is the high-low frequency boundary set in this embodiment) and low-frequency trend components (f≤fH) are extracted through frequency domain analysis. The high-frequency reciprocating components are weighted by defining a frequency weighting function w(f). ; ; Where α=2.0 is the attenuation coefficient. This weighting function applies exponential attenuation to the high-frequency components, effectively suppressing high-frequency reciprocating motion noise caused by airflow disturbances and body vibration.

[0043] The global homography transformation constraint is to calculate the global projection transformation matrix H based on the matching feature point pairs within the rigid subregion, ensuring that the mapping relationship of pixels within the rigid subregion between different images satisfies the global projection transformation matrix. The homography matrix H is a 3×3 homogeneous transformation matrix that describes the projection transformation relationship between two images. For a pair of matching points (x,y) and (x',y') within the rigid subregion Rrig, the following conditions are met: ; Where H = [h11h12h13;h21h22h23;h31h32h33], expanding it yields: ; ; Robustly estimate the homography matrix H from the matching feature point pairs in the rigid sub-region and eliminate mismatched points.

[0044] We construct an energy function E that includes a data fidelity term Edata, a stiffness constraint term Erig, and a smoothing regularization term Esm. Within a unified optimization framework, we jointly solve for the pixel-level elastic deformation field D=[u,v]T (where u and v are the displacements of the pixel in the horizontal and vertical directions, respectively): Wherein, λ1 and λ2 are the weight coefficients of the stiffness constraint term and the smoothing regularization term, respectively, and are adaptively adjusted according to the geometric consistency metric of feature point matching in each sub-region, as well as the high-frequency reciprocating component and the filtered displacement trajectory.

[0045] The data fidelity term Edata measures the consistency between the pixel-level elastic deformation field and the filtered displacement trajectory, ensuring that the obtained deformation field matches the true spatial displacement after frequency domain filtering. Where Ω is the image domain and ψ(p)∈{0,1} is the validity indicator function (excluding occluded areas and saturated pixels). Let be the displacement vector of the pixel-level elastic deformation field to be solved at pixel p. Let be the displacement vector at pixel p of the filtered displacement trajectory obtained through inverse Fourier transform. For pixels The displacement component in the horizontal direction (x-axis direction of the image coordinate system), that is, the amount of horizontal displacement of the pixel after elastic deformation. For pixels The displacement component in the vertical direction (y-axis direction of the image coordinate system), that is, the offset of the pixel in the vertical direction after elastic deformation.

[0046] The stiffness constraint term Erig applies a global homography transformation constraint to the rigid subregion, causing the deformation of the rigid region to approximate the global rigid body transformation. Where ωrig(p) is the geometric consistency weight of the rigid sub-region, which is adaptively calculated based on the residual distribution of feature point matching in the neighborhood of this pixel: Where r(p) is the average residual from the matched feature points in the neighborhood of pixel p to the homography transformation prediction position, and σs=5.0 is the smoothing parameter. This weighting function gives stronger rigid constraints to regions with high geometric consistency, while appropriately relaxing the constraints to regions with local anomalies.

[0047] The smoothing regularization term Esm applies local smoothing constraints and temporal continuity constraints to the flexible subregion, ensuring the natural continuity and temporal consistency of deformation in the flexible region. Where ωdef is the weight of the smoothing regularization term, adaptively allocated according to the spectral energy ratio ρ of the high-frequency reciprocating component and the low-frequency trend component: Where ρlow is the spectral energy of the low-frequency trend component, and ρhigh is the spectral energy of the high-frequency reciprocating component. When the high-frequency energy accounts for a large proportion, ωdef is increased to apply a stronger smoothing constraint and suppress local irregular deformation caused by noise.

[0048] By jointly optimizing the three energy terms mentioned above, the pixel-level elastic deformation field D that minimizes the total energy E(D) is solved within a unified framework. This embodiment employs an iterative optimization algorithm based on gradient descent (L-BFGS), with a maximum number of iterations set to 500 and a convergence threshold set to 10⁻⁻⁶. 6 After obtaining the elastic deformation field D, image coordinate transformation and resampling are performed on the overlapping area based on D: global homography alignment is performed on the rigid sub-region (D(p)≈H(p)-p), and local smoothing fusion is performed on the flexible sub-region (D(p) is pixel-level adaptive displacement). Through the partitioned differential constraints of stiffness constraint term and smoothing regularization term in the energy function, the homography transformation accuracy of the rigid sub-region is not affected by the elastic deformation of the flexible sub-region. At the same time, the elastic displacement of the flexible sub-region achieves continuous and natural transition through the smoothing regularization term, effectively avoiding the splicing discontinuity and tearing problems caused by independent mesh optimization. The temporal continuity constraint enables the elastic deformation field to be smoothly transmitted between frames, and the deformation gradient of adjacent frames is constrained within a reasonable range, suppressing the accumulation and amplification of frame-by-frame registration error to the global splicing image, and ensuring the global consistency and geometric accuracy of the splicing result in a large area of ​​space under long-term and large-area operation scenarios.

[0049] The motion of the drone platform (flight speed, attitude adjustment) is represented as a low-frequency signal in the feature point trajectory, while dynamic factors in the environment (wind-induced vegetation swaying, water surface ripples) are represented as high-frequency signals. By setting an appropriate cutoff frequency in the frequency domain, the two types of signals can be effectively separated, allowing registration to be based only on low-frequency components that reflect the real spatial transformation relationship. This avoids interference from non-rigid motion noise on registration accuracy. The data fidelity term calculates the registration residual between the pixel-level elastic deformation field and the filtered displacement trajectory. Its function is to transform the separated registration basis (the filtered displacement trajectory restored to the spatial domain by inverse Fourier transform) into a data-driven constraint in the energy function. This residual is calculated only between two physical quantities of the same dimension within the same mathematical space (spatial domain). The stiffness constraint term calculates the positional deviation of pixels in the rigid sub-region after global homography transformation. Its form is the L2 difference between the elastic deformation field and the homography transformation predicted displacement. The smaller the deviation, the higher the degree of rigidity constraint satisfaction. The smoothing regularization term calculates the difference between the deformation vectors of adjacent pixels in the flexible sub-region (spatial smoothness) and the difference between the deformation vectors of adjacent time points (temporal continuity). The smaller the difference, the smoother and more natural the deformation field.

[0050] Multiple drones continuously collect and generate timestamped frames. The system sequentially incorporates these frames into a frame set and performs real-time stitching based on the incremental addition of new frames. During stitching, the system performs basic screening on the newly added frames to exclude poor-quality frames, thereby ensuring the quality of the stitched images.

[0051] The individual local mosaic images are then merged into a global mosaic image. First, global registration is performed: coordinate transformation is used to align the individual local mosaic images to a unified global coordinate system, eliminating positional and orientation differences between them. Then, overlapping area fusion is performed: overlapping areas between adjacent local mosaic images are merged to ensure a natural transition at the mosaic boundaries.

[0052] like Figure 5As shown, 1,1 represents the first keyframe (initial keyframe) of drone 1, 1,2 represents the second keyframe of drone 1, 1,3 represents the third keyframe of drone 1, 2,1 represents the first keyframe (initial keyframe) of drone 2, 2,2 represents the second keyframe of drone 2, 3,1 represents the first keyframe (initial keyframe) of drone 3, and so on. The remaining keyframes 1,3, 2,3, and 3,3 can be deduced from the above. As subsequent drones continue to shoot, the number of keyframes will gradually increase. Frames 1,1 and 1,2 are stitched together to obtain a partial stitched image of drone 1, and frames 2,1 and 2,2 are stitched together to obtain... The partial mosaic images of UAV 2 are stitched together with 3,1 and 3,2 to obtain the partial mosaic image of UAV 3. Then, the partial mosaic images of UAV 1, UAV 2 and UAV 3 are merged. Subsequently, with the addition of 1,3, 2,3 and 3,3, 1,3, 2,3 and 3,3 are stitched together with the partial mosaic images of UAV 1, UAV 2 and UAV 3 respectively. After stitching, the partial mosaic images of the new UAV 1, the new UAV 2 and the new UAV 3 are merged. The key is the fusion between 1,3, 2,3 and 3,3. The same applies to 1,3, 2,4 and 3,4.

[0053] like Figure 4 As shown, the single-machine local stitching and multi-machine global fusion process demonstrates a distributed stitching architecture of "independent stitching by each drone first, followed by global fusion of multiple drones." Each drone uses its own set of keyframes as input, independently performs partition registration and incremental stitching, and outputs its own local stitched image in parallel. Figure 1 Partial splicing Figure 2 After all local mosaic images (N) are output, the ground data processing center aligns them to a unified global coordinate system based on the spatial location information of each local mosaic image through coordinate transformation. It then merges overlapping areas between adjacent local mosaic images to generate a global mosaic image covering the entire target area. The local mosaic tasks of each UAV are independent and can be executed in parallel, significantly reducing the overall processing time in multi-UAV collaborative scenarios. Simultaneously, each local mosaic image retains the geometric accuracy and texture details within its covered sub-region. The global fusion stage only performs fusion processing once at the boundary overlap of adjacent local mosaic images, avoiding the frame-by-frame error accumulation problem in traditional frame-by-frame global serial mosaicking. This ensures the global consistency and geometric accuracy of the final global mosaic image over a large spatial area, making it directly usable as mapping data.

[0054] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended embodiments and their equivalents.

Claims

1. A method for real-time stitching and processing of large-area surveying and mapping data based on multi-UAV collaboration, characterized in that, The real-time stitching and processing method includes: several drones continuously collecting frame images and recording spatial location information in their respective coverage sub-areas of the target area to obtain the real-time updated frame image sequence of each drone; filtering each frame image sequence through a quality assessment method to obtain the application frame image sequence of the drone; each drone selecting an initial key frame from its own application frame image sequence to establish a local spatial reference; and selecting key frames from the application frame image sequence to form a key frame set. Semantic segmentation is performed on each frame in the keyframe set of the UAV, dividing pixels into rigid and flexible sub-regions. The relative positional relationships and overlapping areas between keyframes are determined based on spatial location information. The cross-frame motion trajectory of feature points within each sub-region is tracked, and a spectral transform is performed on the motion trajectory to extract high-frequency reciprocating components and low-frequency trend components. The high-frequency reciprocating components are then weighted, and an inverse Fourier transform is performed on the weighted frequency domain signal to obtain the filtered displacement trajectory in the spatial domain. Using the filtered displacement trajectory as the registration basis and the semantic segmentation result as the boundary condition, a global homography transformation constraint is applied to the rigid sub-region, and a local homography transformation constraint is applied to the flexible sub-region. Smoothing constraints and temporal continuity constraints are used to construct an energy function containing data fidelity terms, stiffness constraints, and smoothing regularization terms. A pixel-level elastic deformation field is obtained by jointly solving this function within a unified optimization framework. Each UAV generates a local mosaic image based on its initial keyframe. Image coordinate transformation and resampling are performed on the overlapping areas of each UAV according to the pixel-level elastic deformation field. The fused overlapping areas are then integrated frame by frame into their respective local mosaic images. The local mosaic images of several UAVs are globally registered, and coordinate transformation is used to align each local mosaic image to a unified global coordinate system. The overlapping areas between adjacent local mosaic images are then fused to generate a global mosaic image covering the target area.

2. The method for real-time stitching and processing of large-area surveying data based on multi-UAV collaboration as described in claim 1, characterized in that: The first frame with the timestamp is selected from the application frame sequence as the initial keyframe, and the pose and orientation of the initial keyframe are recorded as the current coverage state. For each subsequent candidate frame, the spatial coverage difference between the candidate frame and the set of selected keyframes is calculated based on the candidate frame's own pose and orientation. The spatial coverage difference includes the viewpoint deviation and the region increment. If the viewpoint deviation exceeds a preset viewpoint threshold and the region increment exceeds a preset increment threshold, the candidate frame is determined to be an information complementary frame. The information complementary frame is added to the set of selected keyframes and the coverage state is updated. If the candidate frame falls within the existing coverage area, it is determined to be a redundant frame and is skipped. This process is repeated until the application frame sequence is processed.

3. The method for real-time stitching and processing of large-area surveying and mapping data based on multi-UAV collaboration as described in claim 2, characterized in that: The viewing angle deviation is determined by the azimuth difference between the field of view of the candidate frame and the field of view of the nearest frame in the selected keyframe set; the region increment is determined by the proportion of the newly added region of the ground coverage area of ​​the candidate frame relative to the current overall coverage area of ​​the selected keyframe set.

4. The method for real-time stitching and processing of large-area mapping data based on multi-UAV collaboration as described in claim 1, characterized in that: Semantic segmentation uses a lightweight convolutional neural network to perform real-time pixel-level classification of frames. Buildings, roads, and man-made facilities are labeled as rigid sub-regions, while vegetation, water bodies, and sandy areas are labeled as flexible sub-regions. Using the semantic segmentation results as boundary conditions, global homography transformation constraints are applied to pixels in rigid sub-regions, and local smoothing constraints are applied to pixels in flexible sub-regions to induce elastic deformation in the flexible sub-regions.

5. The method for real-time stitching and processing of large-area mapping data based on multi-UAV collaboration as described in claim 4, characterized in that: The global homography transformation constraint is calculated based on the matching feature points in the rigid sub-region and applied to the rigid sub-region. The local smoothing constraint is applied to the pixel displacement field in the flexible sub-region based on the non-rigid transformation model. The temporal continuity constraint is applied based on the change amplitude of the deformation field between adjacent time steps. The weight of the stiffness constraint term is adjusted based on the residual value of the feature point matching in each sub-region. The weight of the smoothing regularization term is allocated based on the ratio of the spectral energy of the high-frequency reciprocating component to the low-frequency trend component.

6. The method for real-time stitching and processing of large-area surveying data based on multi-UAV collaboration as described in claim 1, characterized in that: The spectrum transformation includes performing a fast Fourier transform on the time-domain signal of the cross-frame motion trajectory of the feature points, determining the components with frequencies higher than a preset frequency threshold as high-frequency reciprocating components and assigning them zero weights, determining the components with frequencies lower than the preset frequency threshold as low-frequency trend components and assigning them standard weights, and performing an inverse Fourier transform on the frequency-domain signal after weight reduction processing to obtain the filtered displacement trajectory in the spatial domain.

7. The method for real-time stitching and processing of large-area surveying data based on multi-UAV collaboration as described in claim 1, characterized in that: The energy function comprises a linear combination of a data fidelity term, a stiffness constraint term, and a smoothing regularization term. The data fidelity term calculates the registration residual between the pixel-level elastic deformation field and the filtered displacement trajectory. The stiffness constraint term calculates the positional deviation of pixels within a rigid sub-region after global homography transformation. The smoothing regularization term calculates the difference between the deformation vectors of adjacent pixels within a flexible sub-region and the difference between the deformation vectors at adjacent time points. The minimum value of the energy function is obtained by solving the Gauss-Newton iteration method to obtain the pixel-level elastic deformation field.

8. The method for real-time stitching and processing of large-area mapping data based on multi-UAV collaboration as described in claim 1, characterized in that: The quality assessment method includes calculating frame image sharpness based on the Laplacian operator, evaluating frame image exposure uniformity based on histogram statistics, and judging frame image acquisition stability based on attitude angles in spatial location information. Frame images with sharpness below a preset sharpness threshold, exposure uniformity below a preset uniformity threshold, and attitude angle offset exceeding the allowable offset range are removed from the frame image sequence, while frames that meet the quality requirements are retained to form the application frame image sequence.

9. A real-time stitching and processing system for large-area surveying data based on multi-UAV collaboration, employing the real-time stitching and processing method for large-area surveying data based on multi-UAV collaboration as described in any one of claims 1-8, characterized in that: The real-time stitching and processing system includes: a processing module: several drones continuously collect frame images and record spatial location information in their respective coverage sub-areas of the target area, obtain the real-time updated frame image sequence of each drone, filter each frame image sequence through a quality assessment method to obtain the application frame image sequence of the drone, and each drone selects an initial key frame from its own application frame image sequence to establish a local spatial reference, and selects key frames from the application frame image sequence to form a key frame set. The stitching module performs semantic segmentation on each frame in the keyframe set of the UAV, dividing pixels into rigid and flexible sub-regions. Based on spatial location information, it determines the relative positional relationships and overlapping areas between keyframes, tracks the cross-frame motion trajectories of feature points within each sub-region, performs spectral transformation on the motion trajectories, extracts high-frequency reciprocating components and low-frequency trend components, reduces the weight of the high-frequency reciprocating components, and uses the low-frequency trend components as the registration basis. Using the semantic segmentation results as boundary conditions, it applies global homography constraints to the rigid sub-regions and local smoothing and temporal continuity constraints to the flexible sub-regions, constructing a system containing... The energy functions of the data fidelity term, stiffness constraint term, and smoothing regularization term are jointly solved within a unified optimization framework to obtain the pixel-level elastic deformation field. The UAVs generate local mosaic images based on their respective initial keyframes. Based on the pixel-level elastic deformation field, the overlapping areas of each UAV are transformed and resampled. The fused overlapping areas are then integrated into their respective local mosaic images frame by frame. The local mosaic images of several UAVs are globally registered. Through coordinate transformation, each local mosaic image is aligned to a unified global coordinate system. The overlapping areas between adjacent local mosaic images are fused to generate a global mosaic image covering the target area.

Images