A method for fusing satellite and low-altitude UAV remote sensing data

Abstract

The present invention belongs to the field of image processing and provides a method for fusing satellite and low-altitude unmanned aerial vehicle remote sensing data. The method comprises the following steps: extracting linear object contour information from a satellite remote sensing image of a target area and dividing the linear object contour into a plurality of contour segments; extracting real-time linear object contour information from the unmanned aerial vehicle remote sensing image of the target area and dividing the real-time linear object contour into a plurality of contour segments with the same structural form as in the satellite remote sensing image; performing corresponding matching on the linear object contour segments in the satellite remote sensing image and the unmanned aerial vehicle remote sensing image to construct a stable contour segment pair; performing geometric registration processing on the satellite remote sensing image and the stable area in the unmanned aerial vehicle remote sensing image; adjusting the observation angle of the unmanned aerial vehicle image in the changed area based on the relative spatial relationship between the unmanned aerial vehicle flight trajectory and the linear object, and fusing the image after the angle adjustment with the satellite remote sensing image.

Description

Technical Field

[0001] The present invention belongs to the technical field of remote sensing data fusion, and in particular relates to a method for fusing satellite and low-altitude unmanned aerial vehicle remote sensing data. Background Art

[0002] With the continuous development of remote sensing technology, remote sensing data based on various platforms has been widely used in areas such as geographic information acquisition, environmental monitoring, disaster warning, and urban management. Satellite remote sensing, currently the primary means of wide-area observation, offers wide coverage, stable time series, and strong multispectral imaging capabilities, making it suitable for continuous monitoring of large-scale geographic phenomena. However, due to limitations in orbital altitude and sensor characteristics, satellite remote sensing imagery still has limitations in spatial resolution and temporal update frequency, making it difficult to meet the demand for high-precision and rapid response to changes in local details.

[0003] To offset the shortcomings of satellite remote sensing data, low-altitude unmanned aerial vehicle (UAV) remote sensing technology has rapidly developed in recent years. Equipped with high-resolution visible light, multispectral, or thermal infrared sensors, UAVs can flexibly acquire surface information over a specific area. Their high spatial resolution, rapid deployment, and low cost make them particularly suitable for disaster response, key area monitoring, and detailed object identification. While UAV imagery offers remarkable detail and real-time performance, it is limited by range, weather conditions, and altitude, making it difficult to independently undertake large-scale or multi-time series remote sensing missions.

[0004] To fully leverage the wide coverage advantages of satellite imagery and the high precision of drone imagery, technical approaches for multi-source remote sensing image fusion have been gradually proposed in recent years. Existing remote sensing image fusion methods primarily include feature point-based image registration, deep learning-based feature fusion, and orthorectification-based projection overlay. Registration methods are often based on local features such as SIFT, SURF, and ORB, using matching points to construct geometric transformation models for image alignment. Deep learning methods extract image features through end-to-end network models and perform image enhancement or detail compensation under specific loss function constraints. Orthorectification methods typically rely on high-precision DEMs and sensor parameter models to project remote sensing images into a standard geographic spatial reference system for fusion.

[0005] However, existing methods still suffer from insufficient matching robustness, structural distortion, and semantic inconsistency when fusion multi-source remote sensing images with significant differences in spatial resolution, inconsistent imaging perspectives, and severe ground feature mutations. This is particularly true in sudden disaster scenarios such as floods and mudslides, where satellite maps are not updated due to delays, while drone imagery is collected in real time. In areas where ground features have undergone drastic changes, effective feature point matching relationships are often impossible to establish, resulting in reduced fusion accuracy and even erroneous fusion results. Summary of the Invention

[0006] In order to solve the problems in the prior art, the present invention provides a method for fusing satellite and low-altitude UAV remote sensing data, comprising the following steps:

[0007] Step S10, extracting linear feature contour information from a satellite remote sensing image of a target area, wherein the linear feature is a linear structure with spatial extensibility and a length greater than a first preset threshold; dividing the linear feature contour into a plurality of contour segments;

[0008] Step S20, extracting real-time linear feature contour information from the UAV remote sensing image of the target area, and dividing the real-time linear feature contour into a plurality of contour segments having the same structural form as in the satellite remote sensing image based on the geographic location information and flight trajectory data of the UAV;

[0009] Step S30, performing corresponding matching on the linear feature contour segments in the satellite remote sensing image and the UAV remote sensing image based on the geometric shape similarity index and the geographical location matching relationship of the contour segments, to construct a stable contour segment pair;

[0010] Step S40, using the stable contour segment pair as an anchoring reference, constructing a local area grid registration model, and performing geometric registration processing on the stable areas in the satellite remote sensing image and the UAV remote sensing image based on the registration model;

[0011] Step S50: For image areas where stable contour segment pairs cannot be established, it is determined that the area is an area where the state of the ground object has changed; in the changed area, a non-aligned fusion strategy based on the UAV remote sensing image is adopted, and based on the relative spatial relationship between the UAV flight trajectory and the linear feature, the observation angle of the UAV image in the changed area is adjusted, and the image after the angle adjustment is fused with the satellite remote sensing image.

[0012] Furthermore, step S10 includes the following sub-steps:

[0013] Step S101, performing image enhancement processing on the original satellite remote sensing image, including using histogram equalization to improve the image contrast;

[0014] Step S102, using the Canny edge detection algorithm to extract edge areas, and using Hough transform to detect linear feature contours with spatial continuity;

[0015] Step S103: filtering out short line segments smaller than a first preset threshold based on the geometric skeleton or boundary length;

[0016] Step S104, dividing the remaining linear features into multiple contour segments by combining equidistant division and curvature change division;

[0017] Step S105 : generating spatial attributes for each contour segment, including start and end point coordinates, length, direction, curvature characteristics, and coding identifiers.

[0018] Furthermore, in step S104, the sampling interval of the equidistant division is set to 10 meters to 50 meters, and the identification of the curvature mutation point is determined by determining whether the local curvature difference is greater than a set threshold, thereby ensuring the continuity and comparability of the divided segments in spatial morphology.

[0019] Furthermore, step S20 includes the following sub-steps:

[0020] Step S201, performing image distortion correction, attitude normalization, and GNSS-assisted spatial positioning enhancement on the drone image;

[0021] Step S202, extracting directional linear feature contours by combining Canny edge detection with morphological closing operations;

[0022] Step S203, dividing the contour into multiple segments with consistent structures using a combination rule of equidistant length and curvature threshold;

[0023] Step S204 , adding a spatial attribute description to each contour segment, including an image number, a heading angle, a position code, and a geometric feature vector.

[0024] Furthermore, the spatial attributes of each segment in step S204 include track point timestamps, which are used to establish consistency judgment on the time series in subsequent registration and change identification.

[0025] Furthermore, step S30 includes the following sub-steps:

[0026] Step S301: extracting a set of satellite image segments and a set of drone image segments, and establishing a segment space index;

[0027] Step S302, screening candidate matches based on the geographic center point within a set radius;

[0028] Step S303, constructing a comprehensive similarity scoring function using direction difference, curvature matching, length ratio and shape similarity;

[0029] Step S304, eliminating false matching segment pairs based on segment adjacency consistency;

[0030] Step S305 : Record the high-scoring segment pairs as stable contour segment pairs and write them into the anchor point matching data table.

[0031] Furthermore, the weight coefficient of the comprehensive scoring function in step S303 can be optimized and set based on the training set to improve the matching accuracy and avoid mismatching of segments in complex terrain areas.

[0032] Furthermore, step S40 includes the following sub-steps:

[0033] Step S401: extracting a geometric midpoint from a pair of stable contour segments as a control anchor point, and recording its corresponding image coordinate pair;

[0034] Step S402: constructing a Delaunay triangulation mesh based on the control anchor points and establishing a mapping relationship between the source image and the target image;

[0035] Step S403: generating a local geometric transformation function for each grid cell using an affine or thin plate spline interpolation method, and performing pixel remapping on the local block of the satellite image;

[0036] Step S404: using image stitching and edge fusion methods to generate consistent and stable region image output.

[0037] Furthermore, step S50 includes the following sub-steps:

[0038] Step S501, identifying all regions where no stable segment pairs are established, and constructing their two-dimensional boundary masks;

[0039] Step S502, matching the changed area with the drone image through the spatial coverage relationship, and extracting the corresponding flight trajectory data and shooting posture;

[0040] Step S503: Based on the angle between the flight trajectory direction and the ground feature direction, determine whether to perform image perspective correction, and adjust the image angle using perspective transformation or affine transformation method;

[0041] Step S504 : Using dominant substitution, weighted fusion or semantic guided fusion, the adjusted image block is covered to the satellite image change area to complete the fusion operation.

[0042] Furthermore, when the images are fused in step S504 , the fusion boundary is subjected to multi-scale transition using a Laplacian pyramid fusion method to enhance the naturalness and visual consistency of the fusion region boundary.

[0043] The proposed method for fusing satellite and low-altitude UAV remote sensing data builds a set of compatible contour segments before image fusion based on the spatial scalability and structural stability of linear features. This method also achieves highly reliable contour segment matching through geometric similarity and geographic location, effectively improving the accuracy and robustness of multi-source image registration. By constructing a localized grid registration model using stable contour segment pairs, the method achieves high-precision geometric alignment of satellite and UAV images within structurally unchanged regions. The fusion results demonstrate excellent spatial consistency and image detail preservation.

[0044] After identifying areas where stable contour pairing relationships cannot be established, the present invention determines that these areas are areas where the state of the ground objects has changed. In these areas, the present invention actively adopts a non-aligned fusion strategy based on drone images, and adjusts the image observation angle according to the spatial relationship between the drone flight trajectory and the direction of the ground objects, effectively avoiding the fusion distortion and structural dislocation problems caused by forced alignment, making the fused image more realistic, clear and time-sensitive when expressing change information.

[0045] Through differentiated design of structural differentiation and fusion strategies, the present invention achieves the aligned fusion of structurally stable areas and the independent expression of changing areas, improving the practical ability of the fusion system in disaster response, anomaly monitoring and emergency response, and has broad engineering application prospects and promotion value. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0047] Figure 1 This is a comparison of satellite and low-altitude drone remote sensing data;

[0048] Figure 2 This is a flow chart of the method for fusing satellite and low-altitude UAV remote sensing data in this application. DETAILED DESCRIPTION

[0049] Below, the invention is preferably described with reference to the accompanying drawings and specific embodiments.

[0050] This embodiment solves the above problem through the following steps:

[0051] Satellite remote sensing data refers to surface observation data acquired by remote sensing sensors aboard orbiting satellite platforms. It typically features wide coverage, stable acquisition cycles, and a rich set of multispectral channels, making it widely used for global and regional geographic information monitoring. Satellite remote sensing imagery possesses strong holistic capabilities and the ability to express trends. However, due to limitations such as orbital altitude, sensor resolution, and revisit cycles, its spatial resolution is typically low to medium (typically 10 to 100 meters). Its update frequency is limited, typically taking days or even weeks, making it difficult to meet the demand for high-precision, fast-response information acquisition in local areas.

[0052] Low-altitude drone remote sensing data refers to remote sensing imagery acquired close to the Earth's surface by using remote sensing equipment (such as high-resolution visible light cameras and multispectral sensors) aboard low-altitude platforms (e.g., multi-rotor and fixed-wing drones). Compared to satellite remote sensing, drone remote sensing data offers advantages such as high spatial resolution, flexible acquisition, and a short operational cycle. It is particularly suitable for emergency monitoring after emergencies, detailed analysis of target areas, and contour detection of edge structures. However, its limited observation range, low flight altitude, and significant influence from environmental, flight path, and meteorological factors make it difficult to independently achieve large-scale data coverage and continuous time-series monitoring.

[0053] To leverage the respective strengths of satellite remote sensing and low-altitude unmanned aerial vehicle (UAV) remote sensing, a new approach is being proposed: information fusion. This involves aligning and jointly processing data from different platforms across spatial, temporal, or semantic dimensions through algorithms or models, ultimately generating remote sensing information products that combine macroscopic vision with local precision. This fusion inherits the global consistency and temporal sequence of satellite remote sensing data while also overlaying the high-precision, high-contrast image details of UAV data. This significantly enhances the perception and decision-making capabilities of remote sensing systems in multi-source scenarios.

[0054] However, if Figure 1 As shown in the figure, there are many differences between satellite remote sensing images and UAV images. Since satellite remote sensing images have the characteristics of long update cycle and relatively low spatial resolution, they cannot reflect the current true state of the surface in a timely manner when facing sudden changes in the state of land objects (such as mud and rock flows, floods, landslides, forest fires, etc.); although low-altitude UAV remote sensing images have the advantages of real-time and high resolution, they are limited by the small imaging range and large changes in viewing angles, making it difficult to establish accurate correspondence with existing satellite images at the geometric structure and semantic levels.

[0055] Against the above background, the present invention proposes a method for fusing satellite and low-altitude UAV remote sensing data. By performing feature extraction, matching, registration and reconstruction on satellite remote sensing images of the same target area and real-time remote sensing images acquired by low-altitude UAVs, detail enhancement and spatiotemporal consistency expression of local high-resolution change areas can be achieved while maintaining wide-area coverage and temporal continuity. This method is suitable for various geographic spatial intelligent perception scenarios such as disaster monitoring, refined urban management, and dynamic agricultural situation assessment.

[0056] In order to achieve the above object, the present invention realizes data fusion through the following steps:

[0057] like Figure 2 As shown, step S10 is to extract linear feature contour information from the satellite remote sensing image of the target area, wherein the linear feature is a linear structure with spatial extensibility and its length is greater than a first preset threshold; and the linear feature contour is divided into multiple contour segments.

[0058] To achieve precise alignment and local change identification between low-altitude UAV and satellite remote sensing images, a stable set of spatial reference structures must first be constructed as anchoring for fusion. Linear features, due to their strong spatial scalability, structural continuity, and robustness to local disturbances, can serve as the foundation for constructing a regional registration model between satellite and UAV imagery. By extracting the contours of linear features from satellite remote sensing images and dividing them into multiple contour segments with spatial order and morphological characteristics, a preliminary spatial anchoring framework can be formed, supporting subsequent segment matching and local registration operations.

[0059] Linear features are geographic entities with distinct directionality, continuity, and linear extension in geographic space. Typical linear features include roads, rivers, coastlines, border zones, power lines, irrigation ditches, and railway tracks. In remote sensing images, linear features typically appear as strips or ribbons with distinct grayscale boundaries and strong extension. Their length is typically much greater than their width, and they have a clear direction in geographic space.

[0060] Specifically, the process of extracting and dividing the linear feature contour information in satellite remote sensing images includes the following sub-steps:

[0061] In step S101, the original satellite remote sensing image is preprocessed. The preprocessing includes image enhancement, edge enhancement, and multi-scale filtering. Image enhancement can use histogram equalization to enhance contrast. Edge enhancement can use the Canny edge detection algorithm to extract preliminary edge candidate regions. Multi-scale filtering can achieve structure preservation at different scales through Gaussian pyramid.

[0062] Step S102: Based on the preprocessed image, a linear structure extraction operation is performed. The extraction operation can be achieved through one of the following implementation methods: Optional implementation schemes include using a line detection algorithm based on Hough transform to extract a continuous edge set with significant directional features, or using a deep learning model to perform linear feature segmentation, such as using a trained semantic segmentation network (such as DeepLabv3+ or UNet) to classify targets such as roads and rivers pixel by pixel and extract the corresponding boundary contours.

[0063] In step S103, the extracted linear feature contour information is subjected to length screening, and only linear features with a length greater than a first preset threshold are retained. The length can be obtained by calculating the spatial distance or geometric skeleton length of the boundary contour. The first preset threshold is set based on the scale of the target area and the expected accuracy of the drone image, and is usually between tens of meters and hundreds of meters.

[0064] In step S104, the linear feature contour that meets the length requirement is divided into multiple contour segments. The division method is based on equidistant sampling or curvature change segmentation. Equidistant sampling refers to setting a segmentation point at a fixed distance along the contour curve to form segments. Curvature change segmentation refers to setting segment boundaries at the sudden change point of the curvature change of the contour curve to ensure that each segment has relatively uniform geometric characteristics and stable directional properties.

[0065] Step S105 , generating a unique segment identifier and geometric description information for each contour segment, wherein the geometric description information includes segment start and end point coordinates, midline direction, length value, local curvature features and position coding information, for subsequent cross-image matching and registration operations.

[0066] The above processing flow can construct a set of linear feature contour segments with clear structure and semantics in satellite remote sensing images, providing a spatial anchoring basis for identifying stable areas and changing areas in the subsequent fusion step, while also improving the robustness and accuracy of geometric alignment between different remote sensing data.

[0067] In a specific example, linear feature extraction and segmentation operations were performed on a high-resolution satellite remote sensing image of a mountainous area, and a set of contour segments containing dozens of mountain roads and rivers was obtained. The main road A was divided into 8 segments of equal length, and the river B was divided into 10 segments of equal length.

[0068] Step S20: extracting real-time linear feature contour information from the UAV remote sensing image of the target area, and dividing the real-time linear feature contour into a plurality of contour segments having the same structural form as that in the satellite remote sensing image based on the geographic location information and flight trajectory data of the UAV.

[0069] To ensure that low-altitude UAV remote sensing images can establish an accurate structural semantic correspondence with satellite remote sensing images, it is necessary to perform contour extraction and structural segmentation operations on the linear features contained in the UAV images. The extraction results are then spatially calibrated and segmented based on the UAV's geographic location information and flight trajectory data. Because the viewing angle, resolution, and imaging time of UAV images differ significantly from those of satellite images, if linear features are not constrained by a unified spatial segment structure, subsequent structural matching and regional fusion will find it difficult to establish a stable mapping relationship. By extracting linear feature contour segments with the same structural form as those in the satellite images from UAV images and maintaining the consistency of the geometric segmentation strategy, the success rate of inter-segment matching can be improved, and an effective basis for change identification in the disaster area can be formed.

[0070] A drone's geolocation information refers to the GPS coordinates of the camera's center point during capture, typically including longitude, latitude, and altitude. Flight trajectory data refers to the sequence of path points formed during continuous flight, used to identify the shooting angle, position sequence, and spatial coverage of each area in the image.

[0071] Specifically, the process of extracting linear feature contour information from UAV remote sensing images and segmenting them includes the following steps:

[0072] Step S201 , performing preprocessing operations on the drone image to improve the geometric consistency and spatial availability of the image in subsequent contour extraction and structure segmentation. The preprocessing includes image distortion correction, perspective normalization, and spatial positioning enhancement.

[0073] Image distortion correction includes the correction of radial distortion and tangential distortion caused by the camera lens. Specifically, it can be achieved by obtaining the camera's intrinsic parameter matrix and distortion parameter set, and using the inverse mapping function in the pinhole imaging model to perform pixel remapping to eliminate nonlinear distortions such as image edge stretching and compression.

[0074] Perspective normalization refers to correcting the geometric distortion caused by the different pitch and yaw angles of the image due to the tilted flight or attitude changes of the drone, so that it is as consistent as possible with the vertical perspective of the satellite image. Optional implementation methods include calculating the heading angle, pitch angle and roll angle based on the attitude data collected by the IMU, and using the perspective transformation function of a single image to perform affine or perspective correction.

[0075] Spatial positioning enhancement refers to the use of GNSS coordinates and timestamp data recorded during the drone flight, combined with the shutter trigger time when the image was taken, to project the four corners of the image into the geographic coordinate system and construct an image spatial reference system for subsequent contour spatial encoding and matching.

[0076] Step S202 : extracting linear feature contour information from the pre-processed image. The linear feature contour information is a set of strip-shaped structure boundary lines with significant directional continuity in the image.

[0077] The basic operations for contour extraction include edge detection and continuity enhancement. For edge detection, the Canny edge detection algorithm can be used. This algorithm consists of four stages: Gaussian filtering, gradient calculation, non-maximum suppression, and double-threshold connection. It can effectively extract boundary lines with strong edge response and clear structure.

[0078] Continuity enhancement can use closing operations and thinning processing in image morphological operations to connect broken edges and shrink wide edges into single-pixel lines, thereby improving the topological availability of the contour.

[0079] Optional implementation solutions include using deep neural networks to perform semantic segmentation processing on drone images, outputting categories such as roads, rivers, and canals in the image in the form of masks, and performing boundary extraction operations on the masks to obtain a set of contour lines. The network model may include structures such as UNet, DeepLabv3+, SegFormer, etc. that have been trained in remote sensing tasks.

[0080] Step S203 : segmenting the linear feature contour based on the contour line set to unify the contour segment structure with that in the satellite image.

[0081] The core of segmentation is to divide the continuous curve into structurally balanced sub-curves, each of which has independent geometric description capabilities.

[0082] The partitioning method includes two basic strategies: equidistant partitioning and curvature-driven partitioning:

[0083] Equidistant segmentation means inserting segmentation points on the contour curve at fixed geographic intervals, with each segment having the same length. The spacing value can be set based on the average length of the satellite image contour segments to ensure that the scale of the two source image segments is consistent;

[0084] Curvature-driven segmentation refers to setting segmentation boundaries at the points where the contour curvature changes (i.e., inflection points or bending points). This method calculates the local curvature value at each pixel point. When the change exceeds a preset threshold, it is set as a segmentation point. This method can preserve polygonal geometric features in areas with complex structures.

[0085] To avoid the segmentation of fragments that are too short or too many, you can set the minimum fragment length threshold and the maximum allowed number of fragments.

[0086] Step S204: Generate a spatial attribute description for each contour segment for use in structural comparison and spatial alignment in subsequent matching and fusion operations.

[0087] Space attributes include but are not limited to the following:

[0088] Geographic coordinates of the start and end points (calculated from image projection and GNSS data);

[0089] The medial axis direction vector (used to describe the structural directional consistency);

[0090] fragment length (a measure of structure size);

[0091] Average grayscale value and texture gradient (used for image contrast-assisted matching);

[0092] The shooting angle and flight attitude of the image to which the clip belongs (used to compensate for differences in viewing angle);

[0093] Spatial coding tags, including image sequence number, flight trajectory timestamp, segment number, etc.

[0094] This information is stored in a structured data format, such as GeoJSON or Shapefile, with coordinate reference system encoding to support direct spatial matching operations across images.

[0095] After processing through the above sub-steps, linear features in the drone imagery are restored to a set of fragments with spatial location, geometric features, and structural labels. This segmented form maintains a consistent structure with the satellite imagery in geographic space, providing a data foundation for establishing stable anchor points and identifying changing regions in subsequent steps. This processing strategy not only significantly improves the structural consistency of multi-source remote sensing imagery but also enhances the robustness of registration and fusion in non-rigid terrain and areas of local change, making it suitable for complex scenarios such as disaster response, road rupture monitoring, and river overflow analysis.

[0096] In a specific example, continuing the example of the previous step, after structured processing of the drone image, the system identified a set of contour segments of the main road A, numbered A1 to A8, of which A1 to A3 and A6 to A8 are normal in geometry, while segments A4 and A5 are severely deformed due to being covered by debris flows.

[0097] Step S30 , based on the geometric shape similarity index and the geographical location matching relationship of the contour segments, performing corresponding matching on the linear feature contour segments in the satellite remote sensing image and the UAV remote sensing image to construct a stable contour segment pair.

[0098] To accurately fuse the spatial structure of satellite and UAV remote sensing images, it is necessary to establish correspondences between linear feature contour segments with the same geographic entity meaning in both images. Due to differences in resolution, viewpoint, and time between the two image types, the lack of explicit segment-level matching can lead to misuse of registration anchors, misidentification of changed regions, and distorted fusion results. By performing one-to-one structural matching based on geometric similarity and geographic location matching between contour segments, we can effectively construct spatially consistent and structurally stable segment pairs, which serve as the basis for subsequent regional grid registration and separation of changed regions.

[0099] Specifically, the matching process of constructing stable contour segment pairs includes the following detailed steps:

[0100] In step S301 , all divided contour segment sets in the satellite image and the drone image are extracted and sorted, and are recorded as a first segment set and a second segment set respectively.

[0101] In this step, the contour segment data with spatial attributes generated in the previous step is called, and a segment attribute index structure is established, including the unique identifier of each segment, start and end point coordinates, direction vector, center point, length, curvature sequence and timestamp, etc.

[0102] The fragment data is loaded into the matching module in a structured format (such as GeoJSON or vector layer).

[0103] Step S302 : Preliminary geographical location screening is performed on the two sets of segment sets to narrow down the range of candidate matching segments.

[0104] In this step, the center point of each satellite image outline segment is used as a reference to search within the set geographic search radius to see whether the center points of all drone image segments are within the range.

[0105] The search radius is set based on the resolution of the satellite image, the time interval between satellite imaging and the actual collection time of the drone, and is generally set between 50 meters and 200 meters.

[0106] Optional implementation solutions include: using spatial index structures (such as quadtrees or R-trees) to accelerate spatial queries.

[0107] Step S303 : performing geometric shape similarity analysis on the preliminarily screened segment pairs, calculating similarity index scores, and determining whether they are matching segment pairs based on a set threshold.

[0108] Specific indicators include the following:

[0109] The angle difference between the starting and ending points is used to determine whether the directions are consistent;

[0110] The comparison of curvature mean difference and change trend is used to analyze the segment bending morphology;

[0111] Length ratio, used to exclude non-matches with large scale differences;

[0112] The shape similarity score can be obtained by accumulating the Euclidean distance of equal-length sample point sequences.

[0113] Each indicator is assigned a corresponding weight to form a comprehensive matching scoring system.

[0114] Step S304 : performing geometric consistency verification on the segment pairs with higher comprehensive scores to further eliminate false matches caused by local deformation or damage.

[0115] In this step, a local grid consistency constraint is introduced, that is, among the adjacent fragments around a certain fragment pair, if more than a set proportion of adjacent fragments cannot be successfully matched, the current fragment pair will be marked as an unstable pair.

[0116] The adjacency relationship here can be calculated through a topological adjacency graph or a connection graph between the start and end points.

[0117] At the same time, for some areas near the curvature mutation points, a stricter matching tolerance can be set to avoid misjudging the broken line as a continuous structure.

[0118] In step S305 , the segment pairs that pass the geometric verification are marked as stable contour segment pairs and written into the fusion anchor point table for subsequent steps to call.

[0119] The fusion anchor table includes information such as the ID, spatial coordinate pair, direction pair, length value, confidence score and image number of each pair of segments.

[0120] The anchor point pairs provide structural support for grid registration and are used as geometric transformation benchmarks within the matching area.

[0121] Through these steps, we can filter out pairs of fragments with clear structural correspondences, similar geometric shapes, and close spatial locations from a large number of redundant contour fragments. These pairs serve as registration anchors between the satellite and drone images, ensuring that subsequent fusion operations are performed only within structurally stable regions. This also isolates inconsistent regions, improving the geometric consistency and temporal expressiveness of the overall fused image. Compared with traditional full-image feature point matching methods, this method is more structurally constrained and robust, making it suitable for processing multi-source remote sensing data in scenarios with sudden changes.

[0122] In a specific example, continuing with the example in the previous steps, when performing contour segment matching on the drone image and the satellite image, the system found that segments A1 to A3 and A6 to A8 of the main road A can be matched one by one with the segments in the satellite image in terms of direction, position and length, and the matching confidence is greater than the set threshold. However, segments A4 and A5 failed to match successfully due to severe road interruption and coverage. Therefore, the system defines A1A3 and A6A8 as stable contour segment pairs and writes them into the fusion anchor point table to provide a structural benchmark for subsequent alignment grid generation. At the same time, the A4 and A5 areas are marked as unstable areas to be processed using the non-alignment strategy in the subsequent fusion process.

[0123] Step S40 , using the stable contour segment pair as an anchoring reference, constructing a local area grid registration model, and performing geometric registration processing on the stable areas in the satellite remote sensing image and the UAV remote sensing image based on the registration model.

[0124] To achieve high-precision fusion of satellite and UAV remote sensing images within structurally stable regions, it is necessary to establish a spatially robust geometric registration model that describes the transformation relationships within local regions. Since a set of stable contour segment pairs with spatial correspondence and structural consistency has been obtained in the previous steps, these segments can be used as anchors to construct a local region grid. Within this grid structure, geometric transformations are then applied to the satellite and UAV images to achieve registration and alignment. This registration model effectively accommodates geometric differences at local scales and is particularly well-suited for compensating for local nonlinear deformations caused by oblique imaging and attitude disturbances in low-altitude UAV imagery.

[0125] In this step, the anchor reference refers to a pair of spatial control points with a clear correspondence formed by matching stable contour segments. These point pairs can be used as input constraints for the transformation model in geometric space. The local area grid refers to a grid structure formed by dividing the image to be registered with the spatial positions of multiple anchor segment pairs as nodes. This structure supports deformation modeling based on segmented regions. Geometric registration refers to aligning the positions of the same geographic entity in two images in different image coordinate systems through spatial transformation functions, so that the image content is consistent at the pixel level. Common geometric transformations include affine transformation, projective transformation, thin plate spline deformation (TPS), etc.

[0126] Specifically, the process of constructing a grid registration model based on anchor segment pairs and completing stable region registration includes the following sub-steps:

[0127] Step S401 : extracting a set of control points for registration anchoring from a pair of stable contour segments, as a spatial reference for a subsequent local grid registration model.

[0128] First, all the determined stable contour segment pairs are traversed, and for each pair of segments, the geometric center point is extracted as the anchor control point;

[0129] The geometric center point can be obtained by calculating the midpoint between the start and end points of the segment, or the actual geometric center point position can be obtained by the curve length integration method;

[0130] If the segment length is long, in order to improve the grid density, multiple equidistant sampling points can be selected on the segment with a fixed interval, and each point is used as a local anchor point;

[0131] At the same time, the pixel coordinates and geographic coordinates of each control point in the satellite image and the drone image are recorded respectively, and a one-to-one correspondence is established;

[0132] Each pair of control points is stored as an anchor point pair structure in the form of (satellite image coordinates, corresponding drone image coordinates), forming a set of control point pairs;

[0133] The collection is stored in the form of a structured array and can be optionally written into GeoJSON format or Shapefile vector data format for subsequent spatial modeling.

[0134] Step S402: constructing a local area grid structure based on the set of control point pairs to support area division and local registration model fitting.

[0135] The satellite image coordinates of all control points are used as node inputs, and the spatial grid structure is constructed based on their two-dimensional plane positions;

[0136] Optional grid construction methods include but are not limited to:

[0137] The control points are triangulated using the Delaunay triangulation method to obtain a triangular mesh with stable topology and no sharp angles.

[0138] Using the regular grid partitioning method, a rectangular grid with fixed side length is constructed within the distribution range of the control points, and the control points are mapped to the grid nodes;

[0139] Create a local index for each grid cell (whether it is a triangle or a rectangle), recording the control points, spatial boundaries and region numbers it contains;

[0140] At the same time, the corresponding control point pairs in the UAV image are used as target nodes to establish a transformation target grid in the target image space;

[0141] Establish a "source grid-target grid" mapping relationship between the starting image and the target image, and each unit has a unique registration correspondence;

[0142] The grid structure is stored in a topological graph structure, supporting node updating, edge weight assignment and local interpolation.

[0143] Step S403: establishing a local geometric transformation model in each grid unit, and performing geometric registration processing on the stable region image blocks.

[0144] For each grid cell, take all the control point pairs it contains and construct a local geometric transformation function based on the spatial coordinate difference between the source grid and the target grid;

[0145] Optional implementations of the transformation function include:

[0146] Affine transformation model: Applicable to small linear deformation areas by calculating the affine matrix between control points;

[0147] Perspective transformation model: applicable to the projection distortion caused by low-altitude oblique imaging;

[0148] Thin plate spline interpolation model: suitable for areas with slight nonlinear distortion and has strong local deformation expression capabilities;

[0149] Before performing the geometric transformation, the corresponding original image blocks in the satellite image are first cropped according to the grid cell range;

[0150] Perform pixel-level remapping on the cropped image blocks and output new image blocks that are consistent with the spatial structure of the drone image.

[0151] During the remapping process, bilinear interpolation is used to maintain image details, or Lanczos interpolation is used to improve image edge accuracy;

[0152] After each image block is transformed, its transformation parameters and image position index are recorded for full image reconstruction.

[0153] Step S404: performing a fusion reconstruction operation on all registered image blocks to form a geometrically consistent image output result of the stable region.

[0154] Rearrange all registered image blocks to the corresponding positions of the output image according to their original grid numbers;

[0155] Perform edge smoothing and stitching operations between grid cell boundaries to avoid image breaks caused by transformation differences;

[0156] Smoothing methods include Gaussian mixture, edge blurring, and weighted average fusion of transition regions;

[0157] Verify the coordinates of the reconstructed area to ensure the consistency of the output image with the original image in the geographic reference system;

[0158] The output image is exported in GeoTIFF format or other geocoded image formats, with coordinate reference information (such as EPSG code and affine matrix);

[0159] The stable region fusion result will serve as the background layer for the subsequent step of generating the changing region expression and the overall fusion map.

[0160] Through the above steps, using control points formed by stable contour segments as registration anchors, a highly consistent image mesh transformation model was constructed within the structurally stable region, achieving high-precision geometric alignment between UAV remote sensing images and satellite images. This method offers advantages such as strong scalability, strong local deformation representation, and high fusion accuracy. It is particularly suitable for multi-source remote sensing image fusion tasks in non-rigid and non-globally symmetric scenarios, effectively avoiding full-image registration failures caused by sudden changes in disasters.

[0161] For example, continuing with the aforementioned mountain landslide scenario, the system constructed a local Delaunay mesh structure consisting of 18 triangular elements based on the anchor points generated for stable road sections A1 to A3 and A6 to A8. The system then fitted an affine transformation matrix within each element, aligning the local area of ​​the satellite image with the corresponding area in the drone image. Once all elements were complete, a satellite reference basemap consistent with the drone structure was reconstructed through edge fusion, providing a baseline for representing changes in landslide sections A4 and A5.

[0162] Step S50: For image areas where stable contour segment pairs cannot be established, it is determined that the area is an area where the state of the ground object has changed; in the changed area, a non-aligned fusion strategy based on the UAV remote sensing image is adopted, and based on the relative spatial relationship between the UAV flight trajectory and the linear feature, the observation angle of the UAV image in the changed area is adjusted, and the image after the angle adjustment is fused with the satellite remote sensing image.

[0163] When fusing satellite remote sensing imagery with low-altitude drone remote sensing imagery, some areas experience significant changes in surface morphology, making it impossible to establish effective contour segment matching between the two source images. These areas are typically affected by disasters or emergencies, such as debris flows that have destroyed roads or landslides that have covered rivers. In these situations, using registration or alignment strategies for fusion can introduce spatial distortions or semantic errors. To address this, a non-aligned fusion strategy based primarily on drone imagery is employed. This strategy combines the spatial relationship between drone flight trajectory information and surface feature structure, adjusts image angles, and enhances the realism and semantic continuity of the fusion result, ensuring that the fusion result accurately reflects the actual state of the area where the surface features have changed.

[0164] In this step, the changed area refers to the image area in which linear features of the same geographical location in the satellite image and the drone image cannot establish an effective pairing relationship. The image content of this area may be significantly different due to landform changes, occlusion or the generation of new structures. The non-aligned fusion strategy means that when the image content changes drastically and the registration relationship fails, the geometric registration operation is abandoned, and the drone image is directly used as the main image, and it is fused into the full image in a form with an optimized observation angle. The flight trajectory refers to the continuous spatial coordinate sequence recorded by the drone during the shooting mission, usually generated by the GNSS positioning system, which can be used to determine the shooting position and attitude direction corresponding to each frame of the image. Observation angle adjustment refers to correcting the projection angle of the drone image based on the relative spatial relationship between the trajectory point and the ground object, so as to improve the structural expression ability and visual consistency of the fused image.

[0165] Specifically, the non-aligned fusion operation for processing the changed region includes the following sub-steps:

[0166] Step S501 : identifying image regions where stable contour segment pairs cannot be established, and marking them as regions where ground feature states have changed.

[0167] Traverse all drone silhouette segment records and find segments that have not been successfully paired with any satellite segments in the silhouette segment matching step.

[0168] For each unmatched segment, its boundary coordinate information is obtained, and a minimum bounding rectangle or polygonal outline is constructed to describe the image area covered by the segment.

[0169] Multiple adjacent or continuous unmatched segments are spatially merged, and their contour boundaries are united using topological merging rules to form one or more complete change region masks.

[0170] The change region mask is a two-dimensional space mask layer, which is saved in a bitmap format or a vector format and is used to identify a target region in an image that needs to be subjected to non-aligned fusion processing.

[0171] At the same time, each changed region is assigned a unique identification code and attribute information, including the associated unmatched fragment number, estimated degree of change (based on grayscale difference or texture mutation), region area, center coordinates, etc.

[0172] Step S502: locate the corresponding drone image source in the changed area and extract the flight trajectory data when shooting the area.

[0173] According to the spatial boundary coordinates of the changed area, the image number containing the area is retrieved from the UAV image database. The retrieval method includes spatial index query and image coverage relationship matching;

[0174] The geographical coverage of a drone image can be defined by a polygon consisting of the geographical coordinates of the four corners of the image;

[0175] After confirming the image containing the changed area, the flight trajectory data at the time the image was taken is extracted from the flight log;

[0176] The trajectory data includes latitude and longitude coordinates, altitude, heading angle, pitch angle, roll angle, flight speed, image frame number, etc., which are usually recorded jointly by the GNSS and IMU modules in the format of standard CSV or dedicated binary flight control log;

[0177] At the same time, the distance and direction between the center point of the image and the center point of the change area are calculated for subsequent perspective adjustment calculations;

[0178] All track-image matching relationships are recorded in the image metadata table and bound to the changed area mask.

[0179] Step S503 : Based on the spatial relationship between the flight trajectory and the direction of the linear feature, the observation angle of the changed area is calculated, and the angle of the UAV image is adjusted.

[0180] First, the expected strike direction of the features in the change region is extracted. This direction can be estimated from the direction vectors of adjacent registered segments or obtained through a region boundary fitting algorithm.

[0181] Calculate the angle between the shooting direction (i.e., heading angle) of the drone image and the direction of the ground feature. If the angle is greater than a set threshold (e.g., 30 degrees), perform perspective adjustment.

[0182] Optional image perspective adjustment implementations include:

[0183] Perspective transformation method: Based on the geographic coordinates of the four corners of the image and the desired overhead transformation matrix, the projection mapping of the image in two-dimensional space is calculated, which is suitable for correcting slightly tilted shooting;

[0184] Affine transformation method: remap the image to the target direction coordinate system through linear translation, scaling and rotation;

[0185] 3D modeling and orthographic projection method: When the flight posture is complex or the regional height difference is large, 3D surface reconstruction based on DSM or point cloud data is used to generate standard orthographic images;

[0186] During the adjustment process, the image spatial resolution should be kept unchanged, and the edge area should be filled with zero or an edge extrapolation algorithm should be used;

[0187] The adjusted image is regenerated with image index and spatial reference information and saved as an independent image data block.

[0188] Step S504 : performing a non-aligned image fusion operation in the changed area to generate a locally updated fused image result.

[0189] For each changed area, according to its spatial mask, the image block of the corresponding area in the adjusted UAV image is intercepted;

[0190] The corresponding satellite image blocks are extracted from the original fusion background image as a fusion reference;

[0191] Different integration strategies can be selected based on business needs. Optional implementation solutions include:

[0192] Dominant substitution fusion: completely cover the original satellite image area with drone image patches;

[0193] Weighted fusion: UAV images and satellite images are weighted averaged at the pixel level, with weights dynamically adjusted based on texture complexity or grayscale mean;

[0194] Semantic fusion: Guided by the semantic segmentation results, only the target category areas (such as roads and water bodies) are replaced, and the rest are retained;

[0195] At the fusion boundary, image block fusion and stitching processing is performed, and edge gradient, convolution blur, Laplacian pyramid and other technologies are used to smooth the transition area;

[0196] Embed the fusion result into the full image to ensure coordinate consistency and pixel continuity;

[0197] Finally, a fused image is generated, and additional data such as the changed area location layer and fusion quality assessment report are output.

[0198] Through the above steps, the system implemented a non-aligned fusion method based on drone imagery in areas of drastic ground changes. This method corrected the image pose using the spatial relationship between flight trajectory and ground object orientation, and provided an alternative representation of the changed areas based on high-resolution, low-latency imagery. This method effectively addressed the failure of conventional registration models in catastrophic areas, improving the image fusion system's adaptability, expressiveness, and engineering practicality in complex and changing scenarios.

[0199] In a specific example, continuing with the aforementioned mountain landslide example, the area containing segments A4 and A5 was identified as a change region. The system retrieved the corresponding drone image number, IMG_0785, and extracted its flight heading angle as southeast. Analyzing that this direction is at an angle exceeding 45 degrees with the road's direction, the system invoked the perspective transformation module to perform projection correction on the image. The system then performed dominant substitution fusion on the change region, overlaying the landslide-covered road section onto the fused image with clear, high-resolution drone image blocks. Laplacian pyramid fusion was used for boundary transitions at the edges. The resulting image fully captured the spatial change details, such as road fractures and landslide coverage, meeting the needs of disaster emergency analysis and subsequent patch recognition.

[0200] The prior art mentioned in the above background technology section and specific embodiments section of the present invention can be regarded as part of the present invention and used to understand the meaning of some technical features or parameters.

Claims

1. A method for fusing satellite and low-altitude UAV remote sensing data, characterized in that: The method comprises the following steps: Step S10, extracting linear feature contour information from a satellite remote sensing image of a target area, wherein the linear feature is a linear structure with spatial extensibility and a length greater than a first preset threshold; Dividing the linear feature contour into a plurality of contour segments; Step S20, extracting real-time linear feature contour information from the UAV remote sensing image of the target area, and dividing the real-time linear feature contour into a plurality of contour segments having the same structural form as in the satellite remote sensing image based on the geographic location information and flight trajectory data of the UAV; Step S30, performing corresponding matching on the linear feature contour segments in the satellite remote sensing image and the UAV remote sensing image based on the geometric shape similarity index and the geographical location matching relationship of the contour segments, to construct a stable contour segment pair; Step S40, using the stable contour segment pair as an anchoring reference, constructing a local area grid registration model, and performing geometric registration processing on the stable areas in the satellite remote sensing image and the UAV remote sensing image based on the registration model; Step S50: For image areas where stable contour segment pairs cannot be established, it is determined that the area is an area where the state of the ground object has changed; in the changed area, a non-aligned fusion strategy based on the UAV remote sensing image is adopted, and based on the relative spatial relationship between the UAV flight trajectory and the linear feature, the observation angle of the UAV image in the changed area is adjusted, and the image after the angle adjustment is fused with the satellite remote sensing image.

2. The method according to claim 1, characterized in that The step S10 includes the following sub-steps: Step S101, performing image enhancement processing on the original satellite remote sensing image, including using histogram equalization to improve the image contrast; Step S102, using the Canny edge detection algorithm to extract edge areas, and using Hough transform to detect linear feature contours with spatial continuity; Step S103: filtering out short line segments smaller than a first preset threshold based on the geometric skeleton or boundary length; Step S104, dividing the remaining linear features into multiple contour segments by combining equidistant division and curvature change division; Step S105 : generating spatial attributes for each contour segment, including start and end point coordinates, length, direction, curvature characteristics, and coding identifiers.

3. The method according to claim 2, characterized in that In step S104, the sampling interval of the equal-distance division is set to 10 meters to 50 meters, and the identification of the curvature mutation point is determined by determining whether the local curvature difference is greater than a set threshold, thereby ensuring the continuity and comparability of the divided segments in spatial morphology.

4. The method according to claim 1, wherein The step S20 includes the following sub-steps: Step S201, performing image distortion correction, attitude normalization, and GNSS-assisted spatial positioning enhancement on the drone image; Step S202, extracting directional linear feature contours by combining Canny edge detection with morphological closing operations; Step S203, dividing the contour into multiple segments with consistent structures using a combination rule of equidistant length and curvature threshold; Step S204 : adding a spatial attribute description to each contour segment, including an image number, a heading angle, a position code, and a geometric feature vector.

5. The method according to claim 4, characterized in that The spatial attributes of each segment in step S204 include a track point timestamp, which is used to establish consistency judgment on the time series in subsequent registration and change identification.

6. The method according to claim 1, characterized in that The step S30 includes the following sub-steps: Step S301: extracting a set of satellite image segments and a set of drone image segments, and establishing a segment space index; Step S302, screening candidate matches based on the geographic center point within a set radius; Step S303, constructing a comprehensive similarity scoring function using direction difference, curvature matching, length ratio and shape similarity; Step S304, eliminating false matching segment pairs based on segment adjacency consistency; Step S305 : Record the high-scoring segment pairs as stable contour segment pairs and write them into the anchor point matching data table.

7. The method according to claim 6, characterized in that The weight coefficient of the comprehensive scoring function in step S303 is optimized and set based on the training set to improve the matching accuracy and avoid mismatching of segments in complex terrain areas.

8. The method according to claim 1, characterized in that The step S40 includes the following sub-steps: Step S401: extracting a geometric midpoint from a pair of stable contour segments as a control anchor point, and recording its corresponding image coordinate pair; Step S402: constructing a Delaunay triangulation mesh based on the control anchor points and establishing a mapping relationship between the source image and the target image; Step S403: generating a local geometric transformation function for each grid cell using an affine or thin plate spline interpolation method, and performing pixel remapping on the local block of the satellite image; Step S404: using image stitching and edge fusion methods to generate consistent and stable region image output.

9. The method according to claim 1, characterized in that The step S50 includes the following sub-steps: Step S501, identifying all regions where no stable segment pairs are established, and constructing their two-dimensional boundary masks; Step S502, matching the changed area with the drone image through the spatial coverage relationship, and extracting the corresponding flight trajectory data and shooting posture; Step S503: Based on the angle between the flight trajectory direction and the ground feature direction, determine whether to perform image perspective correction, and adjust the image angle using perspective transformation or affine transformation method; Step S504 : Using dominant substitution, weighted fusion or semantic guided fusion, the adjusted image block is covered to the satellite image change area to complete the fusion operation.

10. The method according to claim 9, characterized in that When the images are fused in step S504 , the fusion boundary is subjected to multi-scale transition using the Laplacian pyramid fusion method to enhance the naturalness and visual consistency of the fusion region boundary.

Images