Scene relocalization and target regression method and system in dynamic scene of unmanned aerial vehicle

Abstract

This invention relates to the field of unmanned aerial vehicle (UAV) technology, and particularly to a method and system for scene relocalization and target regression in dynamic UAV scenarios. The method includes: acquiring a local scene image and spatiotemporal attribute information currently observed by the UAV; determining the retrieval granularity; performing hierarchical retrieval according to the retrieval granularity; determining a target reference scene matching the current local scene; performing target detection on the local scene image to obtain the pixel position of the target in the image; performing feature matching between the local scene image and the target reference scene to obtain a homography transformation matrix; transforming the pixel position to the global map based on the pre-stored mapping relationship between the target reference scene and the global map and the homography transformation matrix to obtain an initial mapped position; verifying the trajectory consistency of multiple initial mapped positions of the same target in multiple consecutive frames; correcting the initial mapped positions based on the verification results; and completing target regression. This invention improves the accuracy of dynamic scene relocalization and the robustness of target regression.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) technology, and in particular to a method and system for scene relocalization and target regression in dynamic UAV scenarios. Background Technology

[0002] With the rapid development of the low-altitude economy, drones have evolved from simple "aerial cameras" to "aerial intelligent agents" with autonomous perception capabilities. In scenarios such as urban security, traffic patrol, and emergency disaster relief, it is necessary to map dynamic observation scenes and targets onto the global geospatial space in real time to support command personnel in accurately assessing the situation. However, traditional solutions have significant limitations:

[0003] On the one hand, mainstream digital twin drone control systems only focus on 3D scene modeling, static display and interaction, and have not opened up the business loop of "real-time observation-scene positioning-target regression". They cannot map dynamically captured targets (such as vehicles and personnel) in flight to the global map in real time, resulting in spatial information gaps where "the video stream has images, but there is no location on the map".

[0004] On the other hand, existing scene relocation technologies mostly rely on single GPS coarse positioning or static feature matching. When faced with high-speed drone flight, changing perspectives, and dynamic target interference, they are prone to problems such as low matching accuracy and high response delay, making it difficult to meet the millisecond-level real-time requirements in dynamic scenarios.

[0005] For example, patent application publication number CN121120947A (classification number G06T) provides a method and device for dynamic scene 3D reconstruction based on hydrogen-powered UAV reconnaissance; patent application publication number CN112509115A (classification number G06T) provides a method and system for time-varying unconstrained 3D reconstruction of dynamic scenes from sequential images; and patent application publication number CN121600022A (classification number G06T) provides a method for multi-target UAV tracking based on CNN-Transformer-Mamba network and spatiotemporal Mamba motion model.

[0006] Therefore, there is an urgent need for a method and system for scene relocalization and target regression in dynamic drone scenarios. Summary of the Invention

[0007] (a) Technical problems to be solved

[0008] In view of the above-mentioned shortcomings and deficiencies of the prior art, the present invention provides a method and system for scene relocation and target regression in dynamic scenarios of unmanned aerial vehicles (UAVs). It solves the technical problem that existing UAV systems cannot use local observation scenes and effective targets in them to quickly relocate and regress their positions in the global map in real time and accurately during dynamic flight, which leads to the disconnect between "what is seen" and "where it is" and the lack of real-time business closed loop in digital twin systems.

[0009] (II) Technical Solution

[0010] To achieve the above objectives, the main technical solutions adopted by the present invention include:

[0011] This invention provides a method for scene relocalization and target regression in dynamic UAV scenarios, including:

[0012] The system acquires local scene images and spatiotemporal attribute information observed by the UAV, dynamically determines the retrieval granularity based on the spatiotemporal attribute information, and performs hierarchical retrieval in a pre-constructed reference scene database according to the retrieval granularity to determine the target reference scene that matches the current local scene; wherein, each reference scene in the reference scene database pre-stores the mapping relationship between the scene and the global map;

[0013] The hierarchical retrieval includes a preliminary screening based on spatial constraints using spatiotemporal attribute information and a fine screening based on temporal constraints using a flight state prediction model.

[0014] Target detection is performed on the local scene image to obtain the pixel position of at least one target in the image; and feature matching is performed between the local scene image and the target reference scene to obtain the homography transformation matrix between the two.

[0015] Based on the mapping relationship between the target reference scene and the global map pre-stored and the homography transformation matrix, the pixel position is transformed into the global map to obtain the initial mapping position; and the trajectory consistency of multiple initial mapping positions of the same target in multiple consecutive frames is verified to obtain the verification result.

[0016] The initial mapping position is corrected based on the verification results to complete the target regression.

[0017] Optionally, in some embodiments of this application, the spatiotemporal attribute information includes: data calculated by receiving satellite signals through a global navigation satellite system onboard an unmanned aerial vehicle;

[0018] The data includes: the drone's current geographic location coordinates, flight speed, timestamp, and flight altitude.

[0019] Optionally, in some embodiments of this application, obtaining the local scene image and spatiotemporal attribute information currently observed by the UAV, and dynamically determining the retrieval granularity based on the spatiotemporal attribute information specifically includes:

[0020] The coverage area of ​​the local scene image in the global map is obtained based on the flight altitude, and the scene change rate is calculated based on the flight speed;

[0021] Based on the ratio of the coverage area to a preset range threshold and the ratio of the scene change rate to a preset rate threshold, a sampling step size adjustment coefficient is calculated by weighting. The candidate reference scene sampling step size is obtained by multiplying the adjustment coefficient by the benchmark sampling step size.

[0022] The retrieval granularity is determined based on the sampling step size of the candidate reference scene.

[0023] Optionally, in some embodiments of this application, performing hierarchical retrieval in a pre-built reference scene database according to the retrieval granularity to determine the target reference scene matching the current local scene specifically includes:

[0024] Based on the geographic location coordinates in the spatiotemporal attribute information and according to the retrieval granularity, candidate reference scenes are extracted from the reference scene database to form a first candidate set;

[0025] The geographical coordinates of each reference scene in the first candidate set are input into the flight state prediction model. Combined with the timestamp and flight speed in the spatiotemporal attribute information, the expected reference scene index range and confidence level corresponding to the current moment are predicted. The width of the index range is dynamically adjusted according to the confidence level. Reference scenes falling within the adjusted index range are selected from the first candidate set to form the second candidate set.

[0026] The current local scene image is matched with the reference scenes in the second candidate set, and the reference scene with the highest matching degree is selected as the target reference scene.

[0027] Optionally, in some embodiments of this application, the flight state prediction model includes:

[0028] The state transition layer is used to obtain the UAV's geographical location coordinates and timestamps corresponding to the reference scenarios of the previous N successful matches, construct the state transition equation based on the UAV's kinematic model, and obtain the expected geographical location and prediction covariance at the current moment; the kinematic model includes constraints on the rate of change of heading angle and the amount of change of velocity.

[0029] An uncertainty propagation layer is used to project the expected geographical location and prediction covariance onto the index space of the reference scene database to obtain the expected reference scene index position and index range width, and calculate the confidence level based on the index range width.

[0030] An adaptive adjustment layer is used to compare the confidence level with a preset threshold, obtain the comparison result, and obtain the index range based on the comparison result.

[0031] Optionally, in some embodiments of this application, feature matching is performed between the current local scene image and the reference scenes in the second candidate set, and the reference scene with the highest matching degree is selected as the target reference scene, including:

[0032] Obtain the scene complexity index of the current local scene image; the scene complexity index is a weighted sum of image information entropy, edge density, and gray-level variance;

[0033] The matching strategy is selected based on the scenario complexity index: when the scenario complexity index is lower than the first threshold, the second matching strategy is selected; otherwise, the first matching strategy is selected.

[0034] Based on the selected matching strategy, the current local scene image is matched with each reference scene in the second candidate set to calculate the matching score; the reference scene with the highest matching score is selected as the target reference scene.

[0035] Optionally, in some embodiments of this application, the first matching strategy employs FAST feature point extraction, SIFT feature description, forced matching combined with RANSAC filtering; the second matching strategy employs SuperPoint feature point extraction and LightGlue feature matching.

[0036] Optionally, in some embodiments of this application, the pixel position is transformed into the global map based on the mapping relationship between the target reference scene and the global map pre-stored and the homography transformation matrix to obtain the initial mapping position, including:

[0037] Obtain the first mapping matrix from the target reference scene to the global map, and the homography transformation matrix from the current local scene to the target reference scene as the second mapping matrix;

[0038] Calculate the inlier ratio when performing feature matching between the current local scene and the target reference scene, and determine the confidence weight of the second mapping matrix based on the inlier ratio;

[0039] Based on the confidence weight, the first mapping matrix and the second mapping matrix are chained together to obtain the combined mapping matrix from the current local scene to the global map;

[0040] The pixel position of the target in the local scene image is transformed to the global map coordinate system through the combined mapping matrix to obtain the initial mapping position.

[0041] Optionally, in some embodiments of this application, trajectory consistency verification is performed on multiple initial mapping positions of the same target in consecutive frames, and the verification results are obtained by:

[0042] Obtain the initial mapping positions of the same target in M ​​consecutive frames to form a position sequence; calculate the position jump variables between adjacent frames to obtain a jump sequence;

[0043] Calculate the mean and standard deviation of the jump sequence, and compare the standard deviation with a preset jump threshold; if the standard deviation is less than the preset jump threshold, the verification result is passed; if the standard deviation is greater than or equal to the preset jump threshold, identify abnormal frames in the jump sequence that exceed the sum of the jump mean and a preset multiple of the standard deviation, and mark the verification result as containing abnormal frames.

[0044] Optionally, in some embodiments of this application, when the verification result is passed, the initial mapping position of the current frame is used as the target regression position;

[0045] When the verification result contains abnormal frames, the initial mapping position corresponding to the abnormal frame is corrected: interpolation is performed based on the initial mapping positions of the normal frames before and after the abnormal frame, and the interpolated result is used to replace the initial mapping position of the abnormal frame as the target regression position.

[0046] Secondly, the present invention provides a scene relocalization and target regression system for dynamic UAV scenarios, comprising:

[0047] The target reference scene determination module is used to acquire the local scene image and spatiotemporal attribute information currently observed by the UAV, dynamically determine the retrieval granularity based on the spatiotemporal attribute information, and perform hierarchical retrieval in a pre-constructed reference scene database according to the retrieval granularity to determine the target reference scene that matches the current local scene; wherein, each reference scene in the reference scene database pre-stores the mapping relationship between the scene and the global map;

[0048] The hierarchical retrieval includes a preliminary screening based on spatial constraints using spatiotemporal attribute information and a fine screening based on temporal constraints using a flight state prediction model.

[0049] The feature matching module is used to perform target detection on the local scene image to obtain the pixel position of at least one target in the image; and to perform feature matching between the local scene image and the target reference scene to obtain the homography transformation matrix between the two.

[0050] The verification module is used to convert the pixel position to the global map based on the mapping relationship between the target reference scene and the global map and the homography transformation matrix to obtain the initial mapping position; and to perform trajectory consistency verification on multiple initial mapping positions of the same target in multiple consecutive frames to obtain the verification result.

[0051] The correction module is used to correct the initial mapping position based on the verification results, thereby completing the target regression.

[0052] (III) Beneficial Effects

[0053] The beneficial effects of this invention are as follows: The scene relocalization and target regression method and system for UAV dynamic scenarios of this invention significantly improves the relocalization speed and matching accuracy from local scene to global map in UAV dynamic scenarios by integrating spatiotemporal attribute constraints and flight state prediction model hierarchical retrieval strategy; at the same time, by using the homography transformation matrix obtained by feature matching and the pre-stored mapping relationship, the target pixel position is efficiently converted to the global map, and correction is made by combining trajectory consistency verification of consecutive multiple frames, which effectively suppresses the positioning deviation caused by single-frame mismatch or occlusion. Thus, while ensuring real-time performance, the robustness of target regression and the accuracy of geographic coordinates are greatly improved, providing a reliable business closed loop of "what you see is where you are" for UAV digital twins. Attached Figure Description

[0054] Figure 1 This is a flowchart illustrating a method for scene relocalization and target regression in a dynamic scene using an unmanned aerial vehicle (UAV) according to an embodiment of this application.

[0055] Figure 2 A flowchart illustrating the process of determining the target reference scene based on the retrieval granularity according to one embodiment of this application;

[0056] Figure 3 A schematic diagram of a process for feature matching between a current local scene image and a reference scene of a second candidate set according to an embodiment of this application;

[0057] Figure 4 A flowchart illustrating the process of obtaining verification results according to an embodiment of this application;

[0058] Figure 5 This is a schematic diagram illustrating the regression of a current scene target point to a global map according to an embodiment of this application;

[0059] Figure 6 This is a schematic diagram of the structure of a scene relocalization and target regression system for a drone in a dynamic scene according to an embodiment of this application. Detailed Implementation

[0060] To better explain and facilitate understanding of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0061] In current UAV visual positioning and target regression technologies, common practices mainly rely on single GPS information for coarse scene localization, or directly perform global feature matching between the current UAV image and a pre-established global map database. For example, some systems use the latitude and longitude coordinates transmitted by the UAV as an index to search for geographically similar scenes in a reference scene library as candidates, and then perform image matching using manual features such as SIFT and ORB to determine the location of the current scene in the global map. For target regression, the homography matrix obtained from single-frame matching is usually used to directly map the target pixel coordinates in the image to the geographic coordinate system.

[0062] However, the aforementioned existing technologies have the following prominent problems: First, relying solely on GPS data is affected by signal quality and multipath effects, resulting in positioning errors of several meters or even tens of meters. This leads to an excessively large scene retrieval range and long matching time, making it difficult to meet the real-time operation requirements of UAVs. Second, when using global feature matching alone, the computational complexity increases linearly with the database size, and matching delays or failures are prone to occur during dynamic flight. Third, the target regression process relies solely on the matching results of a single frame image, ignoring the temporal and spatial continuity of UAV flight. Once a mismatch occurs in a frame (such as a sudden change in viewpoint, partial occlusion, or changes in illumination), it will cause the target mapping position to jump or become incorrect, lacking an effective self-correction mechanism.

[0063] To address the core technical challenge of achieving rapid relocalization from a local scene to a global map while maintaining real-time performance, accuracy, and robustness in dynamic UAV flight scenarios, and accurately and stably reverting observed targets to the geographic coordinate system, this invention provides a method and system for scene relocalization and target regression in dynamic UAV scenarios. The method first acquires the current local scene image and its spatiotemporal attribute information from the UAV. Based on this information, the retrieval granularity is dynamically determined, and a hierarchical retrieval is performed in a pre-constructed reference scene database according to this granularity: first, spatial constraints are used for initial screening to form a first candidate set; then, temporal constraints based on a flight state prediction model (combined with previous successful matching results to predict the current index range) are used for fine-tuning, thereby rapidly narrowing the candidate scene range. Finally, the target reference scene is obtained through feature matching. Next, target detection is performed on the local scene image to obtain the target's pixel position, and the homography transformation matrix between the current image and the target reference scene is calculated. Combining the pre-stored mapping relationship from the target reference scene to the global map, the pixel position is converted into an initial mapping position in the global map. Based on this, trajectory consistency verification is performed on multiple initial mapping positions of the same target in consecutive frames, and the initial mapping positions are corrected according to the verification results.

[0064] This invention significantly reduces the computational overhead of scene matching through hierarchical retrieval, enabling relocation to have millisecond-level response capabilities. Through trajectory consistency verification and correction, it effectively suppresses positioning jumps caused by single-frame mismatches, making the target's relocation position smoother, more stable in time and with higher geographical accuracy. Thus, it provides UAV digital twin systems with real-time, accurate, and robust "what you see is where you are" holographic mapping capabilities.

[0065] To better understand the above technical solutions, exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the present invention can be understood more clearly and thoroughly, and that the scope of the present invention can be fully conveyed to those skilled in the art.

[0066] Figure 1 This is a flowchart illustrating a scene relocalization and target regression method for a UAV in a dynamic scenario according to Embodiment 1 of the present invention. Figure 1 As shown, the scene relocalization and target regression methods include:

[0067] Step S000: Perform a pre-capture aerial scan of the target area using a drone to acquire a continuous view image sequence covering the target area, and simultaneously record the GPS positioning data and flight attitude data corresponding to each frame of the image acquisition; perform feature extraction and matching based on adjacent images in the continuous view image sequence to restore the relative pose relationship between adjacent viewpoints, and jointly optimize all poses through bundle adjustment to generate a global orthophoto map of the target area as the global map; during the generation of the global map, store each frame of the image sequence as a reference scene in the reference scene database, and store the positioning data and flight attitude data corresponding to each frame of the reference scene together; at the same time, restore the projection mapping relationship between each reference scene and the global map through feature matching, and verify the projection mapping relationship—after projecting the known feature points in the reference scene onto the global map through the projection mapping relationship, calculate the error between the projected position and the actual position of the corresponding feature point in the global map. If the error is less than a preset verification threshold, the projection mapping relationship is confirmed to pass verification and pre-stored in the reference scene database; if the error is greater than or equal to the verification threshold, the reference scene is re-matched and the projection relationship is recalculated until it passes verification. Therefore, each reference scenario in the reference scenario database has a pre-stored verified mapping relationship between that scenario and the global map.

[0068] In the specific implementation process, in step S000, a global map and reference scene database are constructed by pre-aerial scanning, so that subsequent scene relocalization has a complete reference benchmark; the projection mapping relationship is verified before being stored, ensuring that the mapping relationship between each reference scene and the global map is accurate and reliable, avoiding the amplification of unverified mapping errors in subsequent chain projections, and ensuring the accuracy of target regression from the source.

[0069] Based on this, perform the following steps:

[0070] Step S100: Obtain the local scene image and spatiotemporal attribute information currently observed by the UAV, dynamically determine the retrieval granularity based on the spatiotemporal attribute information, perform hierarchical retrieval in the pre-constructed reference scene database according to the retrieval granularity, and determine the target reference scene that matches the current local scene; wherein, each reference scene in the reference scene database pre-stores the mapping relationship between the scene and the global map.

[0071] The hierarchical retrieval includes initial screening based on spatial constraints using spatiotemporal attribute information and fine screening based on temporal constraints using flight state prediction models.

[0072] The spatiotemporal attribute information includes: data obtained by receiving satellite signals through the UAV's onboard global navigation satellite system; the data includes: the UAV's current geographical coordinates, flight speed, timestamp, and flight altitude.

[0073] In the specific implementation process, spatiotemporal attribute information also includes:

[0074] The attitude angle information calculated in real time by the UAV's onboard inertial measurement unit includes roll, pitch, and yaw angles. By introducing attitude angle information, image perspective distortion caused by the UAV's own tilt or rotation can be effectively compensated, thereby improving the tolerance for viewpoint differences during subsequent feature matching. Specifically, when estimating the ground coverage area of ​​a local scene image based on flight altitude, the coverage area needs to be corrected by combining the current pitch and roll angles: when the UAV is in pitch or roll, the effective ground projection area of ​​the camera will exhibit trapezoidal distortion. This embodiment adopts the principle of digital orthorectification, using the attitude angles to construct a rotation matrix, correcting the current image to an approximately vertically downward-looking equivalent image before estimating its ground coverage area, thereby improving the accuracy of retrieval granularity calculation.

[0075] In addition, spatiotemporal attribute information can also include real-time wind speed and direction data collected by airborne meteorological sensors. In the flight state prediction model, the state transition layer introduces wind speed and direction as external disturbances into the state transition equation. That is, when recursively extrapolating the expected geographical location, a drift correction term caused by the wind field is superimposed. Specifically, the wind vector is inversely solved based on the difference between the airspeed and ground speed vectors of the UAV. This wind vector is then used to compensate for and predict the flight trajectory in the short term, thereby reducing the positioning prediction deviation caused by wind interference and further improving the confidence of the expected reference scene index range during the fine screening of time-series constraints.

[0076] By expanding the aforementioned multidimensional spatiotemporal attribute information, this invention can more precisely characterize the dynamic flight state and imaging environment of UAVs, providing richer prior constraints for hierarchical retrieval, feature matching, and projection transformation, and further enhancing the accuracy and reliability of scene relocalization and target regression in complex environments.

[0077] Further, in step S100, see... Figure 2 The process involves acquiring local scene images and spatiotemporal attribute information currently observed by the UAV, and dynamically determining the retrieval granularity based on the spatiotemporal attribute information. Specifically, this includes:

[0078] Step S110: Obtain the coverage area of ​​the local scene image in the global map based on the flight altitude, and calculate the scene change rate based on the flight speed.

[0079] In practice, obtaining the coverage area of ​​a local scene image within the global map based on flight altitude primarily relies on the known field-of-view parameters of the UAV's onboard camera. By reading the camera's horizontal and vertical field-of-view angles and combining them with the current altitude (i.e., flight altitude), the actual width and height of the ground corresponding to the image can be calculated using the tangent relationship: the coverage width equals twice the height multiplied by the tangent of half the horizontal field-of-view angle. If the camera is pitching or rolling, trapezoidal correction can be further performed using attitude angles to obtain the equivalent vertical downward-looking coverage area. The scene change rate is calculated based on the ratio of the current flight speed to the aforementioned coverage width, i.e., the flight distance per second divided by the ground coverage width. Its physical meaning is the proportion of new scenes entering the field of view per unit time. The higher this ratio, the lower the overlap between adjacent frames, requiring corresponding adjustments to the retrieval granularity to match faster scene updates. These two parameters together serve as the basis for dynamically adjusting the retrieval granularity, enabling the sampling step size of candidate reference scenes to adapt to changes in the UAV's altitude and speed.

[0080] Furthermore, this step introduces a scene content richness index as an auxiliary adjustment factor to avoid the predicament of excessively fine-grained retrieval without effective matching in low-texture or repetitive texture areas due to feature sparsity. Specifically, the texture richness of the current local scene image is quantitatively evaluated: first, the image is converted to a grayscale image, and the contrast and correlation features of the gray-level co-occurrence matrix are calculated. At the same time, the average value of the image gradient magnitude and the proportion of edge pixels are combined to obtain a scene content richness score. The higher the score, the richer the texture and structural information in the scene, and the more suitable it is for fine-grained feature matching. Conversely, if the content richness is low (e.g., large areas of water, snow, sand, or highly repetitive building facades), excessively fine-grained retrieval may increase the risk of mismatch due to the high similarity in appearance between candidate scenes. Based on this, this embodiment calculates the baseline retrieval granularity adjustment coefficient from the coverage and rate of change, and then corrects this coefficient according to the content richness score: when the content richness is higher than the high threshold, the retrieval granularity is maintained or slightly refined to fully utilize texture information for accurate matching; when the content richness is lower than the low threshold, the retrieval granularity is actively coarsened, i.e., the sampling step size is increased, and the requirement for the number of matching points in subsequent feature matching is reduced, relying more on spatiotemporal continuity and GPS constraints. By using content richness as a dynamic adjustment factor, it can adapt to the differences in visual characteristics of flight scenarios, smoothly switching retrieval strategies between complex urban areas and monotonous outdoor environments, thereby further improving the overall efficiency and robustness of hierarchical retrieval.

[0081] Step S120: Based on the ratio of the coverage area to the preset range threshold and the ratio of the scene change rate to the preset rate threshold, the sampling step size adjustment coefficient is calculated by weighting. The adjustment coefficient is multiplied by the baseline sampling step size to obtain the candidate reference scene sampling step size.

[0082] The ratio of the coverage area to the preset range threshold reflects the degree of deviation of the current flight altitude from the reference altitude. A larger ratio indicates a wider field of view, allowing for a larger step size to skip geographically adjacent redundant reference scenes. The ratio of the scene change rate to the preset rate threshold reflects the degree of deviation of the flight speed from the reference speed. A larger ratio indicates faster scene changes, requiring a larger step size to cover the possible scene range. The two ratios are weighted and summed according to preset weights to obtain the sampling step size adjustment coefficient. Simultaneously, when scene content richness-assisted adjustment is enabled, if the texture richness is lower than the preset richness threshold, a compensation coefficient greater than 1 is multiplied by the weighted summation result to expand the search range.

[0083] The following example illustrates the calculation process in detail. Assume a drone is flying over an urban area. Under baseline conditions, the preset standard coverage range is 80 meters, and the standard scene change rate is 0.2 (i.e., updating 20% ​​of the image content per second). The current actual flight altitude is low, with a measured ground coverage width of 40 meters; the flight speed is high, with a ground speed of 20 meters per second. Therefore, the current scene change rate is calculated as 20 divided by 40, which equals 0.5. The weight of the preset coverage ratio is 0.4, and the weight of the change rate ratio is 0.6. First, the coverage ratio is calculated as the standard coverage range divided by the current coverage range, i.e., 80 / 40 = 2.0. This ratio is greater than 1, indicating that the current field of view is narrower than the baseline, and the sampling step size should be increased. The change rate ratio is calculated as the current change rate divided by the standard change rate, i.e., 0.5 / 0.2 = 2.5. Similarly, this is greater than 1, indicating that the scene changes rapidly, and the step size should also be increased. The weighted summation yields the adjustment coefficient: 0.4 × 2.0 + 0.6 × 2.5 = 0.8 + 1.5 = 2.3. Assuming a baseline sampling step size of 10, the candidate reference scene sampling step size is 10 × 2.3 = 23, meaning one candidate is extracted every 23 reference scenes, resulting in a coarser retrieval granularity than the baseline. If scene content richness-assisted adjustment is also enabled, and the current image is a large, uniform lawn with a texture richness score below a preset low threshold (e.g., 0.3), then the adjustment coefficient 2.3 will be multiplied by a compensation coefficient greater than 1 (e.g., 1.5), resulting in a compensated adjustment coefficient of 2.3 × 1.5 = 3.45. The sampling step size becomes 10 × 3.45 = 34.5, rounded down to 35. By further expanding the sampling step size, the retrieval density in texture-degraded areas is actively reduced, relying more on GPS and temporal prediction to maintain positioning reliability. This example illustrates how to dynamically adjust the sampling step size based on coverage, rate of change, and texture richness, so that the retrieval granularity adapts to flight status and environmental characteristics.

[0084] Step S130: Determine the retrieval granularity based on the sampling step size of the candidate reference scene.

[0085] The sampling step size of the candidate reference scene is mapped to the index interval in the reference scene database. That is, one candidate is extracted for every number of reference scenes corresponding to this step size. This index interval is the retrieval granularity of the current frame. The retrieval granularity changes in real time with the flight status, which enables the rapid location of candidate regions with less computational overhead during the high-speed cruise phase, and ensures matching accuracy with denser sampling during the low-speed fine-tuning phase.

[0086] In the specific implementation process, hierarchical retrieval is performed in the pre-built reference scene database according to the retrieval granularity to determine the target reference scenes that match the current local scene, including:

[0087] Step S140: Based on the geographical location coordinates in the spatiotemporal attribute information and according to the retrieval granularity, extract candidate reference scenes from the reference scene database to form the first candidate set.

[0088] Specifically, centered on the UAV's current geographic location coordinates, and according to the index intervals determined by the retrieval granularity, a certain number of reference scenes are extracted from the reference scene database along the index direction to both sides, forming the first candidate set. Simultaneously, considering the drift error inherent in Global Navigation Satellite System (GNSS) positioning data, this step adds a spatial buffer radius to the geographic location coordinates. This spatial buffer radius is dynamically determined based on satellite signal strength—when the satellite signal strength is below a preset threshold, the spatial buffer radius is increased to cover a larger geographical area, preventing real matching scenes from being excluded from the candidate set due to positioning drift; when the satellite signal strength is above the preset threshold, the default spatial buffer radius is maintained. This spatial buffer mechanism significantly improves the robustness to positioning uncertainty in the initial screening stage, avoiding missed detections due to GPS drift.

[0089] Step S150: Input the geographical coordinates of each reference scene in the first candidate set into the flight state prediction model, combine the timestamp and flight speed in the spatiotemporal attribute information, predict the expected reference scene index range and confidence level corresponding to the current moment, and dynamically adjust the width of the index range according to the confidence level. Select reference scenes that fall within the adjusted index range from the first candidate set to form the second candidate set.

[0090] Specifically, the flight state prediction model consists of three working sub-layers: the state transition layer, the uncertainty propagation layer, and the adaptive adjustment layer. The specific implementation of each layer is explained in detail below.

[0091] The state transition layer is responsible for predicting the current location of the UAV using historical successful matching information. Specifically, it obtains the UAV's geographical coordinates and timestamps corresponding to the reference scenes from the previous N successful matches (e.g., the first 3 to 5). A state transition equation based on the UAV's kinematic model is constructed to obtain the expected geographical location and prediction covariance at the current moment. The kinematic model includes constraints on the rate of change of heading angle and the amount of change of velocity, making the position prediction not a simple linear extrapolation, but a prediction driven by flight dynamics—when the rate of change of heading angle is large, the predicted position deviates along a curve rather than extending along a straight line; when the amount of change of velocity is large, the forward distance of the predicted position is adjusted accordingly. The prediction covariance reflects the uncertainty of the state transition equation's own prediction result.

[0092] In the specific implementation process, considering the flight dynamics characteristics of the UAV (such as maximum turning angular velocity and maximum acceleration / deceleration), this layer constructs a constrained kinematic model, rather than a simple uniform linear extrapolation. The core of this model lies in including explicit constraints on the rate of change of heading angle and the amount of velocity change: when the rate of change of heading angle is large in historical data, the state transition equation will cause the predicted position to deviate along a curved trajectory (such as an arc or spiral) instead of extending along a straight line; when the amount of velocity change is large, the predicted forward distance will be adjusted accordingly based on acceleration information. In this way, the prediction results better reflect the actual maneuvering behavior of the UAV. Simultaneously, the state transition layer also calculates the uncertainty of this prediction, i.e., the prediction covariance. This covariance consists of two parts: first, the process noise of the state transition model itself (e.g., the possibility of the UAV being suddenly disturbed by gusts of wind or abrupt command changes); and second, the measurement noise of the previous N successfully matched positions. The larger the covariance value, the more uncertain the system is about the current prediction result, and the subsequent index range needs to be widened accordingly.

[0093] The uncertainty propagation layer receives the expected geographic location and prediction covariance output from the state transition layer and projects them onto the index space of the reference scene database. The reference scene database typically uses a one-dimensional or two-dimensional index (e.g., a linear index or spatial grid index sorted by latitude and longitude) to represent geographic locations. This layer first maps the expected geographic location to the expected index position in the database (equivalent to a scalar sequence number). Then, based on the magnitude and spatial distribution characteristics of the prediction covariance, it calculates the width of the index range corresponding to the prediction: the larger the prediction covariance, the wider the index range; conversely, the narrower the covariance. Furthermore, this layer calculates a confidence score based on the index range width. The basic logic is that a smaller width indicates a more concentrated and reliable prediction, with a confidence score closer to 1; a larger width indicates a more dispersed and less reliable prediction, with a confidence score approaching 0. To give the confidence score physical meaning, this embodiment pre-sets a minimum reference width and a maximum reference width. When the actual width is less than the minimum width, the confidence score is set to 1; when it is greater than the maximum width, the confidence score is set to 0, with a linear or exponential decay relationship in between.

[0094] The adaptive adjustment layer compares the confidence level with preset thresholds, obtains the comparison result, and determines the index range based on the result. This layer compares the calculated confidence level with multiple preset thresholds. When the confidence level is higher than a high threshold (e.g., 0.8), it indicates that the current prediction is very reliable. In this case, the original index range output by the uncertainty propagation layer can be directly used (or even appropriately narrowed) to minimize the number of candidate reference scenes and improve matching speed. When the confidence level is between a low and high threshold (e.g., 0.3 to 0.8), the original index range remains unchanged. When the confidence level is lower than a low threshold (e.g., 0.3), it indicates that the current prediction is extremely unreliable (possibly due to severe drone maneuvers, abnormal GPS signals, or flying out of the mapping area). If a narrow index range is still used in this case, it is highly likely that correct reference scenes will be missed. Therefore, the adaptive adjustment layer actively expands the width of the index range by a factor proportional to the prediction covariance (e.g., 1.5 times, 2 times, etc.), incorporating more candidate reference scenes into the second candidate set, sacrificing a small amount of computational overhead for reliability without missed detections. Furthermore, to prevent drastic fluctuations in the index range within a short period, a smoothing mechanism is introduced in this layer: an exponentially weighted moving average is applied to the index range widths of adjacent frames to avoid abrupt changes caused by single-frame anomalies. After the above dynamic adjustment, reference scenes whose index positions fall within the adjusted index range are selected from the first candidate set, thus forming the final "second candidate set". At this point, the temporal constraint screening in the hierarchical retrieval is completed, and subsequent steps will perform feature matching on the second candidate set.

[0095] The flight state prediction model in this embodiment upgrades simple spatial location filtering to predictive filtering with spatiotemporal constraints. This ensures that the candidate set not only includes scenes near the current GPS coordinates but also considers scenes indicated by the drone's motion trend, accurately locking onto candidate regions even in non-linear flight scenarios such as drone turns and speed changes. Simultaneously, the adaptive adjustment layer allows the index range to flexibly expand and contract based on prediction reliability. When predictions are reliable, the search range is narrowed to improve efficiency; when predictions are unreliable, the search range is expanded to ensure recall.

[0096] Step S160: Perform feature matching between the current local scene image and the reference scenes in the second candidate set, and select the reference scene with the highest matching degree as the target reference scene.

[0097] Specifically, see Figure 3 Step S160 includes:

[0098] Step S161: Obtain the scene complexity index of the current local scene image; the scene complexity index is the weighted sum of image information entropy, edge density and gray-level variance.

[0099] In this step, image information entropy is used to measure the degree of disorder in the gray-level distribution of an image, reflecting the richness of texture. Specifically, the current image is first converted to a grayscale image, and the probability of each gray level (0 to 255) appearing in the image is calculated (i.e., the number of pixels at that gray level divided by the total number of pixels). Then, the probability of each gray level is multiplied by its logarithm according to the information entropy formula, summed, and the negative value is taken to obtain the final entropy value. A higher entropy value indicates a more dispersed gray-level distribution and richer texture information; a lower entropy value suggests the image may present a uniform and flat area. Edge density is used to characterize the richness of structure and contours in an image. Sobel or Canny operators can be used to perform edge detection on the grayscale image, obtaining a binarized edge image. Then, the proportion of edge pixels to the total number of pixels in the entire image is calculated; this proportion is the edge density. A higher edge density indicates that the image contains more structural information and has higher scene complexity; conversely, sparse edges indicate a more monotonous scene. Gray-level variance directly reflects the dispersion of image pixel gray-level values, and is obtained by calculating the variance of the gray-level values ​​of all pixels in the entire image. A large gray-level variance means that the contrast between light and dark areas in the image is strong, and it has a large dynamic range; a small variance indicates that the overall gray level of the image is uniform and the contrast is low. After calculating the information entropy, edge density, and gray-level variance, these three indicators are normalized (e.g., using maximum-minimum normalization or Z-score normalization) to eliminate dimensional differences. Then, they are multiplied by preset weight coefficients (e.g., information entropy weight 0.4, edge density weight 0.3, gray-level variance weight 0.3) and summed to obtain a comprehensive scene complexity score in the range of 0 to 1. This score will be used for the adaptive selection of subsequent matching strategies. In this embodiment, the parameters required to calculate image information entropy, edge density, and gray-level variance are all stored in the spatiotemporal attribute information.

[0100] When the weighted sum of the three factors is high, the scene being represented has rich texture and structural features, making it suitable for lightweight feature matching to obtain stable results; when the weighted sum of the three factors is low, the scene being represented has sparse texture and simple structure, making it easy for lightweight matching methods to produce mismatches, requiring the use of more robust matching methods.

[0101] Step S162: Select a matching strategy based on the scene complexity index: when the scene complexity index is lower than the first threshold, select the second matching strategy; otherwise, select the first matching strategy.

[0102] The first matching strategy uses FAST feature point extraction, SIFT feature description, forced matching combined with RANSAC filtering; the second matching strategy uses SuperPoint feature point extraction and LightGlue feature matching.

[0103] In the specific implementation process, the first matching strategy employs FAST feature point extraction, SIFT feature description, forced matching combined with RANSAC filtering. This approach is suitable for conditions with limited computing resources or high scene complexity, controlling computational overhead while ensuring matching accuracy. The second matching strategy combines deep learning feature point extraction with attention-based feature matching. It exhibits strong adaptability to changes in illumination, seasons, weather, significant viewpoint differences, and weak texture environments, making it suitable for conditions with low scene complexity where traditional methods are prone to failure. By adaptively selecting the matching strategy based on scene complexity, the system maintains stable matching performance under different scene conditions, avoiding the problem of a sudden drop in accuracy under specific conditions with a single matching strategy.

[0104] Step S163: According to the selected matching strategy, perform feature matching between the current local scene image and each reference scene in the second candidate set, and calculate the matching score; select the reference scene with the highest matching score as the target reference scene.

[0105] The matching score is calculated based on three dimensions: The first dimension is the feature point matching degree score, which is the ratio of the number of matching point pairs to the total number of feature points involved in the matching. A matching degree threshold is set, and only candidates with a matching degree higher than this threshold are retained. The second dimension is the pixel distance consistency of the matching point pairs, meaning that the relative pixel distance between each matching point pair in the two images should be consistent; matching point pairs with abnormal pixel distances are deleted. The third dimension is the relative position consistency of the matching point pairs, meaning that the relative spatial arrangement of each matching point pair in the two images should be consistent; matching point pairs with abnormal relative positions are deleted. After filtering based on these three dimensions, the number of finally retained matching point pairs is used as the matching score for each candidate reference scene, and the reference scene with the highest matching score is selected as the target reference scene.

[0106] For example, assuming the first matching strategy is adopted, the specific steps are as follows: First, the FAST algorithm is used to extract feature points of the current image and the candidate reference scene, and SIFT descriptors are calculated to obtain initial matching pairs through forced matching. Then, the Random Sample Consensus Algorithm (RANSAC) is used to filter out mismatches. Assuming there are 300 initial matching pairs, RANSAC retains 75 pairs of in-between points. Then, the matching score is calculated from three dimensions: The first dimension calculates the matching degree score, that is, 75 divided by the total number of feature points involved in the matching (set to 400), which is 0.1875, higher than the threshold of 0.15, so it passes; The second dimension evaluates pixel distance consistency, and point pairs with abnormal distances are removed (for example, 5 pairs are deleted), leaving 70 pairs; The third dimension evaluates relative position consistency, and point pairs with abnormal arrangement relationships are removed (another 2 pairs are deleted), finally retaining 68 valid matching point pairs, and the matching score of this candidate scene is 68. The above process is repeated for each scene in the second candidate set, and the highest score is selected as the target reference scene. If the highest score is still lower than the preset success threshold (such as 30 pairs), the matching failure processing is triggered. Through comprehensive screening across these three dimensions, the system can effectively eliminate false matches and ensure the accuracy of target reference scene selection.

[0107] Furthermore, when the matching scores of all candidate reference scenes in step S163 are lower than the minimum matching threshold, the current frame is determined to have failed to match. At this time, the target reference scene determined in the previous frame is used as the target reference scene for the current frame, and the geographic location coordinates, timestamp, and number of consecutive matching failures for the current frame are recorded. If the number of consecutive matching failures does not reach a preset abnormal threshold, hierarchical retrieval continues in the next frame; if the number of consecutive matching failures reaches the preset abnormal threshold, it is determined that the current flight scene has deviated from the global map coverage area, scene relocation and target regression are stopped, and a deviation alarm is reported to the business system. Simultaneously, this embodiment automatically lowers the retrieval granularity by one level to expand the candidate range when matching fails, providing a larger search space when resuming matching and increasing the probability of successful rematch.

[0108] In step S160 of this embodiment, a scene complexity adaptive matching strategy is adopted. When the texture is rich, a lightweight FAST+SIFT+RANSAC is used to save computing power. When the texture is sparse or the lighting changes, it switches to SuperPoint+LightGlue to ensure matching robustness, thus balancing efficiency and accuracy. Multi-dimensional matching scores (matching degree, pixel distance consistency, relative position consistency) are used to eliminate mismatches step by step, which significantly improves the accuracy of target reference scene selection. When matching fails, the reference scene of the previous frame is used and the retrieval granularity is dynamically reduced to expand the subsequent search space. Combined with failure counting and alarm mechanisms, the continuity and self-recovery capability of the system under brief interference or scene deviation are effectively improved.

[0109] Furthermore, the scene relocalization and target regression method from the drone's perspective in this embodiment also includes:

[0110] Step S200: Perform target detection on the local scene image to obtain the pixel position of at least one target in the image; and perform feature matching between the local scene image and the target reference scene to obtain the homography transformation matrix between the two.

[0111] In the specific implementation process, step S200 includes the following sub-steps:

[0112] Step S210: Perform object detection on the local scene image to obtain the pixel location of at least one object in the image and its corresponding detection confidence. A pre-trained object detection model is used to detect the current local scene image, obtaining the category label, pixel location bounding box, and detection confidence for each object in the image. The detection confidence reflects the model's trustworthiness of the detection result and ranges from 0 to 1.

[0113] For targets with a detection confidence level below a preset confidence threshold, they are not directly eliminated but marked as targets awaiting confirmation. For targets awaiting confirmation, a temporal consistency check is performed: it checks whether the target was detected with a confidence level above the threshold in previous consecutive frames. If the target was successfully detected in all M previous consecutive frames and the pixel position changes continuously, the target awaiting confirmation in the current frame is upgraded to a confirmed target and retained. If there are no continuous detection records for the target in previous frames, the target awaiting confirmation in the current frame is eliminated. This temporal consistency check mechanism ensures that brief occlusions or false detections in a single frame do not lead to target loss or the introduction of false targets, while also avoiding the omission of valid targets due to excessively high confidence thresholds.

[0114] Step S220: For each confirmed target, extract the center point of its pixel location bounding box as the target's pixel location in the image, and record the width and height of the bounding box as the target's scale information. If the pixel location of the same target changes little in adjacent frames but the scale changes drastically, it may indicate a target identity switch rather than real motion. In this case, the mapping result for the target should be less reliable.

[0115] Step S230: Perform feature extraction and matching on the local scene image and the target reference scene to obtain a set of matching point pairs. Based on the matching strategy selected in step S162, feature points are extracted and described on both the local scene image and the target reference scene. Preliminary matching is then performed using a similarity metric between feature descriptions to obtain an initial set of matching point pairs. Next, a bidirectional consistency check is performed on the initial set of matching point pairs: for each feature point in the local scene image, after finding the best matching point in the target reference scene, a reverse search is conducted using that best matching point as the starting point to find its best matching point in the local scene image. If the reverse search result matches the original feature point, the matching point pair is retained; otherwise, it is discarded. This bidirectional consistency check effectively filters out ambiguous and mismatched matches in unidirectional matching, improving the reliability of the set of matching point pairs.

[0116] Step S240: Based on the set of matching point pairs, estimate the homography matrix between the local scene image and the target reference scene using the random sampling consistency method. During the iteration of the random sampling consistency method, four sets of matching point pairs are randomly selected each time to calculate the candidate homography matrix. The remaining matching point pairs are then verified by forward and backward projection using the candidate homography matrix, and the number of inliers with projection errors less than a preset error threshold is counted. After the iteration, the candidate homography matrix with the largest number of inliers is used as the initial homography matrix, and the initial homography matrix is ​​refitted using all inliers to obtain the final homography matrix.

[0117] Furthermore, this embodiment introduces a geometric constraint pre-screening mechanism based on the random sampling consistency method: before randomly selecting four sets of matching point pairs each time, collinearity and scale consistency checks are performed on the set of matching point pairs. The collinearity check excludes sampling combinations with three or more matching point pairs on the same straight line, because collinear point pairs cannot provide sufficient geometric constraints to estimate the homography transformation matrix; the scale consistency check calculates the local scaling ratio reflected by each matching point pair and excludes abnormal matching point pairs whose local scaling ratio deviates from the overall median scaling ratio by more than a preset deviation multiple, because such matching point pairs are likely from incorrect matching or objects located on different planes. Through geometric constraint pre-screening, unreasonable sampling combinations are eliminated, the number of invalid iterations is reduced, and the estimation accuracy of the final homography transformation matrix is ​​improved.

[0118] Step S250: Verify the validity of the homography transformation matrix. Calculate the determinant value corresponding to the homography transformation matrix. When the determinant value is close to zero, it indicates that the homography transformation matrix is ​​close to degradation and cannot reliably describe the mapping relationship between the two scenes. In this case, the homography transformation matrix is ​​determined to be invalid. Simultaneously, the ratio of the number of inliers to the total number of matched point pairs in step S240 is used as the inlier rate. When the inlier rate is lower than a preset inlier rate threshold, it indicates that the mismatch ratio in the matched point pairs is too high, and the homography transformation matrix is ​​determined to be invalid.

[0119] When the homography transformation matrix is ​​deemed invalid, a downgraded matching strategy is implemented: the set of matching point pairs obtained in step S230 is sorted in ascending order of matching descriptive distance; a preset number of matching point pairs before sorting are selected as a high-confidence subset; affine transformation estimation is performed on this high-confidence subset; and the affine transformation matrix is ​​used to replace the homography transformation matrix for subsequent target regression. Compared to homography, affine transformation has fewer degrees of freedom and lower requirements on the quantity and quality of matching point pairs. It can still provide a usable approximate mapping relationship in scenarios with unsatisfactory matching conditions, ensuring that the target regression process is not interrupted due to homography transformation failure.

[0120] Step S200 effectively reduces the interference of false detections and missed detections on subsequent target regression by performing temporal consistency verification during the target detection stage, and scale change monitoring provides an early warning mechanism for target identity switching; bidirectional consistency verification and geometric constraint pre-screening in the feature matching stage improve the matching quality from the source and reduce invalid iterations; validity verification of the homography transformation matrix and degradation matching strategy ensure that usable mapping relationships can still be provided when matching conditions deteriorate, enhancing the robustness of the system in complex flight scenarios.

[0121] Step S300: Based on the mapping relationship and homography transformation matrix between the target reference scene and the global map, the pixel position is transformed into the global map to obtain the initial mapping position; and the trajectory consistency of multiple initial mapping positions of the same target in multiple consecutive frames is verified to obtain the verification result.

[0122] See Figure 4 The above step S300 specifically includes:

[0123] Step S310: Obtain the first mapping matrix from the target reference scene to the global map, and the homography transformation matrix from the current local scene to the target reference scene as the second mapping matrix.

[0124] The first mapping matrix is ​​pre-stored after verification during the mapping stage in step S000, and its accuracy is guaranteed. The second mapping matrix is ​​calculated in real time during step S200, and its accuracy is affected by the matching quality of the current frame, thus exhibiting uncertainty. Therefore, the reliability of the second mapping matrix needs to be quantitatively evaluated before chaining.

[0125] Step S320: Calculate the inlier ratio when performing feature matching between the current local scene and the target reference scene, and determine the confidence weight of the second mapping matrix based on the inlier ratio.

[0126] The inlier ratio is the ratio of the number of inliers finally confirmed by the random sampling consensus method in step S240 to the total number of matching point pairs. A higher inlier ratio indicates a more reliable estimation of the second mapping matrix, and a larger confidence weight; a lower inlier ratio indicates greater matching noise, higher uncertainty of the second mapping matrix, and a smaller confidence weight. Specifically, the inlier ratio is compared with preset lower and upper limits: when the inlier ratio reaches the upper limit, the confidence weight is set to its maximum value of 1; when the inlier ratio is lower than the lower limit, the confidence weight is set to its preset minimum value; when the inlier ratio is between the two, the confidence weight is determined by linear interpolation.

[0127] Furthermore, this embodiment introduces an inter-frame mapping continuity factor to correct the confidence weights. The difference between the second mapping matrix of the previous frame and the second mapping matrix of the current frame is measured. When this difference exceeds a preset difference threshold, it indicates a non-smooth jump in the mapping relationship between the two frames, possibly due to a matching error or a sudden scene change. In this case, the confidence weight is reduced by multiplying it by a penalty coefficient less than 1. When the difference does not exceed the preset difference threshold, the confidence weight remains unchanged. This correction mechanism automatically reduces the weight of frames with abnormally changing mapping relationships in subsequent combination and verification, preventing abnormal mapping from contaminating the target regression results.

[0128] Step S330: Based on the confidence weight, the first mapping matrix and the second mapping matrix are chained together to obtain the combined mapping matrix from the current local scene to the global map.

[0129] Chaining involves multiplying the second mapping matrix by the first mapping matrix sequentially to obtain a direct mapping relationship from the current local scene to the global map. The role of the confidence weight is reflected in subsequent steps: when the confidence weight corresponding to the combined mapping matrix of a frame is low, the initial mapping position obtained by that frame through the combined mapping matrix is ​​assigned a lower confidence weight in trajectory consistency verification. This makes the verification process more tolerant of anomalies in low-confidence frames, preventing normal fluctuations in low-confidence frames from being misjudged as abnormal jumps.

[0130] Step S340: Transform the pixel position of the target in the local scene image to the global map coordinate system through a combined mapping matrix to obtain the initial mapping position.

[0131] Step S350: Obtain the initial mapping positions of the same target in M ​​consecutive frames to form a position sequence; calculate the position jump variables between adjacent frames to obtain a jump sequence.

[0132] The position jump variable is the Euclidean distance between the same target in two adjacent frames in the global map coordinate system. For frames marked as low-confidence mapping results, their corresponding position jump variables are multiplied by the corresponding confidence weight in subsequent statistics, so that the jumps generated by low-confidence frames are statistically weakened, avoiding false anomaly judgments caused by decreased mapping accuracy.

[0133] Step S360: Calculate the mean and standard deviation of the jump sequence, and compare the standard deviation with the preset jump threshold; if the standard deviation is less than the preset jump threshold, the verification result is passed; if the standard deviation is greater than or equal to the preset jump threshold, identify abnormal frames in the jump sequence that exceed the sum of the jump mean and the preset multiple standard deviation, and mark the verification result as containing abnormal frames.

[0134] When the verification result is passed, the initial mapping position of the current frame is used as the target regression position;

[0135] When the verification result contains abnormal frames, the initial mapping position corresponding to the abnormal frame is corrected: interpolation is performed based on the initial mapping positions of the normal frames before and after the abnormal frame, and the interpolated result is used to replace the initial mapping position of the abnormal frame as the target regression position.

[0136] Step S400: Correct the initial mapping position based on the verification results to complete the target regression.

[0137] Specifically, when the verification result contains isolated abnormal frames, interpolation is performed based on the initial mapping positions of the normal frames before and after the abnormal frame, and the interpolated result is used to replace the initial mapping position of the abnormal frame as the target regression position.

[0138] When the verification result contains continuously abnormal frames, the mapping position of the target within the abnormal frame segment is not interpolated and corrected. Instead, the target's regression position within the abnormal frame segment is marked as awaiting manual confirmation, and the service linkage triggering for the target in the corresponding frame segment is suspended to avoid making incorrect decisions based on unreliable location information. When the target recovers to normal in subsequent frames and the number of consecutive normal frames reaches the preset recovery frame number, the awaiting manual confirmation status is automatically lifted, and the normal target regression output is restored.

[0139] Steps S300 and S400 quantify the reliability of the chain-combined mapping results through confidence weights, so that low-confidence frames are automatically downweighted in the verification process instead of being simply excluded. The trajectory consistency verification distinguishes between isolated anomalies and persistent anomalies. It performs interpolation correction on isolated anomalies and isolates and suspends business linkage on persistent anomalies, which not only ensures the regression continuity under occasional disturbances, but also avoids erroneous decisions caused by systematic deviations.

[0140] The scene relocalization and target regression method for UAV dynamic scenarios described in this embodiment has the following overall advantages:

[0141] First, by pre-building a global map and verifying and storing the projection mapping relationship between the reference scene and the global map, the risk of unverified mapping errors being amplified step by step in chain projection is eliminated from the source, providing a reliable benchmark mapping relationship for target regression.

[0142] Second, the retrieval granularity is dynamically determined based on flight altitude, flight speed, and the richness of scene content, so as to achieve an adaptive balance between retrieval efficiency and matching accuracy. Hierarchical retrieval quickly narrows the candidate range through initial screening with spatial constraints, and then performs fine screening with temporal constraints through flight state prediction model. The flight state prediction model replaces simple linear extrapolation with nonlinear prediction driven by flight dynamics, and realizes the elastic expansion and contraction of the index range based on the propagation of prediction uncertainty, thus taking into account both retrieval speed and accuracy.

[0143] Third, the feature matching stage adaptively selects the matching strategy based on the complexity of the scene and performs a multi-dimensional comprehensive evaluation of the matching results, ensuring the robustness of matching under different scene conditions; the temporal consistency verification in the target detection stage reduces the interference of false detection and missed detection, and the scale change monitoring provides early warning for target identity switching.

[0144] Fourth, confidence weights and inter-frame mapping continuity corrections are introduced during chain-based combination mapping to quantify the reliability of the second mapping matrix, so that low-confidence frames are automatically downweighted rather than simply excluded in trajectory consistency verification. Trajectory consistency verification distinguishes between isolated anomalies and persistent anomalies. Interpolation correction is applied to isolated anomalies, while isolation and suspension of service linkage are applied to persistent anomalies. This ensures regression continuity under occasional disturbances and avoids erroneous decisions caused by systemic deviations.

[0145] Fifth, when a match fails, the previous frame reference scene is used and the retrieval granularity is dynamically reduced. Combined with the continuous failure counting and deviation alarm mechanism, the system's self-recovery capability and operational safety are effectively improved in the event of brief interference or scene deviation.

[0146] In summary, as Figure 5As shown, this embodiment combines hierarchical retrieval, adaptive matching, confidence quantification, and temporal verification to achieve robust optimization of the entire link from local scene relocation to target regression in dynamic UAV scenarios. Even in complex environments (lighting changes, weak textures, occlusion, and maneuvering flight), it can still complete the accurate and smooth mapping of the target to the global map with millisecond-level latency, providing reliable geographic registration and target tracking capabilities for digital twin systems.

[0147] In addition, such as Figure 6 As shown, this embodiment of the invention also provides a scene relocalization and target regression system for dynamic UAV scenarios, including:

[0148] The target reference scene determination module is used to acquire the local scene image and spatiotemporal attribute information currently observed by the UAV, dynamically determine the retrieval granularity based on the spatiotemporal attribute information, and perform hierarchical retrieval in a pre-constructed reference scene database according to the retrieval granularity to determine the target reference scene that matches the current local scene; wherein, each reference scene in the reference scene database pre-stores the mapping relationship between the scene and the global map;

[0149] The hierarchical retrieval includes a preliminary screening based on spatial constraints using spatiotemporal attribute information and a fine screening based on temporal constraints using a flight state prediction model.

[0150] The feature matching module is used to perform target detection on the local scene image to obtain the pixel position of at least one target in the image; and to perform feature matching between the local scene image and the target reference scene to obtain the homography transformation matrix between the two.

[0151] The verification module is used to convert the pixel position to the global map based on the mapping relationship between the target reference scene and the global map and the homography transformation matrix to obtain the initial mapping position; and to perform trajectory consistency verification on multiple initial mapping positions of the same target in multiple consecutive frames to obtain the verification result.

[0152] The correction module is used to correct the initial mapping position based on the verification results, thereby completing the target regression.

[0153] The scene relocalization and target regression system for UAVs in dynamic scenarios in this embodiment significantly improves the efficiency and accuracy of scene relocalization for UAVs in complex dynamic scenarios by dynamically determining the retrieval granularity and combining it with spatiotemporal attribute information for hierarchical retrieval. By utilizing homography transformation and trajectory consistency verification, it effectively filters out mismatches and observation noise, stably maps the local target pixel positions to the global map, and makes corrections based on the verification results, thereby achieving highly robust and low-drift target regression, meeting the practical needs of UAV real-time positioning and continuous tracking.

[0154] In the description of this invention, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0155] In this invention, unless otherwise explicitly specified and limited, the terms "installation," "connection," "linking," and "fixing," etc., should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral part; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium; they can refer to the internal communication of two components or the interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0156] In this invention, unless otherwise explicitly specified and limited, "above" or "below" the second feature can mean that the first and second features are in direct contact, or that they are in indirect contact through an intermediate medium. Furthermore, "above," "over," or "on top" the second feature can mean that the first feature is directly above or diagonally above the second feature, or simply indicates that the first feature is at a higher horizontal level than the second feature. "Below," "below," or "beneath" the second feature can mean that the first feature is directly below or diagonally below the second feature, or simply indicates that the first feature is at a lower horizontal level than the second feature.

[0157] In the description of this specification, the terms "one embodiment," "some embodiments," "embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0158] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make modifications, alterations, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A scene relocalization and target regression method in a dynamic scene of a UAV, characterized in that, include: The system acquires local scene images and spatiotemporal attribute information observed by the UAV, dynamically determines the retrieval granularity based on the spatiotemporal attribute information, and performs hierarchical retrieval in a pre-constructed reference scene database according to the retrieval granularity to determine the target reference scene that matches the current local scene; wherein, each reference scene in the reference scene database pre-stores the mapping relationship between the scene and the global map; The hierarchical retrieval includes a preliminary screening based on spatial constraints using spatiotemporal attribute information and a fine screening based on temporal constraints using a flight state prediction model. Target detection is performed on the local scene image to obtain the pixel position of at least one target in the image; and feature matching is performed between the local scene image and the target reference scene to obtain the homography transformation matrix between the two. Based on the mapping relationship between the target reference scene and the global map pre-stored and the homography transformation matrix, the pixel position is transformed into the global map to obtain the initial mapping position; and the trajectory consistency of multiple initial mapping positions of the same target in multiple consecutive frames is verified to obtain the verification result. The initial mapping position is corrected based on the verification results to complete the target regression.

2. The method of claim 1, wherein, The spatiotemporal attribute information includes: data obtained by receiving satellite signals through the UAV's onboard global navigation satellite system; The data includes: the drone's current geographic location coordinates, flight speed, timestamp, and flight altitude.

3. The scene relocalization and target regression method for UAVs in dynamic scenarios according to claim 2, characterized in that, Acquiring the local scene image and spatiotemporal attribute information currently observed by the UAV, and dynamically determining the retrieval granularity based on the spatiotemporal attribute information specifically includes: The coverage area of ​​the local scene image in the global map is obtained based on the flight altitude, and the scene change rate is calculated based on the flight speed; Based on the ratio of the coverage area to the preset range threshold and the ratio of the scene change rate to the preset rate threshold, a sampling step size adjustment coefficient is calculated by weighting. The candidate reference scene sampling step size is obtained by multiplying the adjustment coefficient by the benchmark sampling step size. The retrieval granularity is determined based on the sampling step size of the candidate reference scene.

4. The scene relocalization and target regression method for UAVs in dynamic scenarios according to claim 1, characterized in that, According to the retrieval granularity, a hierarchical search is performed in the pre-built reference scene database to determine the target reference scene that matches the current local scene. Specifically, this includes: Based on the geographic location coordinates in the spatiotemporal attribute information and according to the retrieval granularity, candidate reference scenes are extracted from the reference scene database to form a first candidate set; The geographical coordinates of each reference scene in the first candidate set are input into the flight state prediction model. Combined with the timestamp and flight speed in the spatiotemporal attribute information, the expected reference scene index range and confidence level corresponding to the current moment are predicted. The width of the index range is dynamically adjusted according to the confidence level. Reference scenes falling within the adjusted index range are selected from the first candidate set to form the second candidate set. The current local scene image is matched with the reference scenes in the second candidate set, and the reference scene with the highest matching degree is selected as the target reference scene.

5. The scene relocalization and target regression method for UAVs in dynamic scenarios according to claim 4, characterized in that, The flight state prediction model includes: The state transition layer is used to obtain the UAV's geographical location coordinates and timestamps corresponding to the reference scenarios of the previous N successful matches, construct the state transition equation based on the UAV's kinematic model, and obtain the expected geographical location and prediction covariance at the current moment; the kinematic model includes constraints on the rate of change of heading angle and the amount of change of velocity. An uncertainty propagation layer is used to project the expected geographical location and prediction covariance onto the index space of the reference scene database to obtain the expected reference scene index position and index range width, and calculate the confidence level based on the index range width. An adaptive adjustment layer is used to compare the confidence level with a preset threshold, obtain the comparison result, and obtain the index range based on the comparison result.

6. The scene relocalization and target regression method for UAVs in dynamic scenarios according to claim 4, characterized in that, The current local scene image is matched with the reference scenes in the second candidate set, and the reference scene with the highest matching degree is selected as the target reference scene, including: Obtain the scene complexity index of the current local scene image; the scene complexity index is a weighted sum of image information entropy, edge density, and gray-level variance; The matching strategy is selected based on the scenario complexity index: when the scenario complexity index is lower than the first threshold, the second matching strategy is selected; otherwise, the first matching strategy is selected. Based on the selected matching strategy, the current local scene image is matched with each reference scene in the second candidate set to calculate the matching score; the reference scene with the highest matching score is selected as the target reference scene.

7. The method for scene relocalization and target regression in dynamic UAV scenarios according to claim 6, characterized in that, The first matching strategy uses FAST feature point extraction, SIFT feature description, forced matching combined with RANSAC filtering; the second matching strategy uses SuperPoint feature point extraction and LightGlue feature matching.

8. The method for scene relocalization and target regression in dynamic UAV scenarios according to claim 1, characterized in that, Based on the mapping relationship between the target reference scene and the global map pre-stored and the homography transformation matrix, the pixel position is transformed into the global map to obtain the initial mapping position, including: Obtain the first mapping matrix from the target reference scene to the global map, and the homography transformation matrix from the current local scene to the target reference scene as the second mapping matrix; Calculate the inlier ratio when performing feature matching between the current local scene and the target reference scene, and determine the confidence weight of the second mapping matrix based on the inlier ratio; Based on the confidence weight, the first mapping matrix and the second mapping matrix are chained together to obtain the combined mapping matrix from the current local scene to the global map; The pixel position of the target in the local scene image is transformed to the global map coordinate system through the combined mapping matrix to obtain the initial mapping position.

9. The method for scene relocalization and target regression in dynamic UAV scenarios according to claim 1, characterized in that, The trajectory consistency of the same target at multiple initial mapping positions in consecutive frames is verified, and the verification results are obtained as follows: Obtain the initial mapping positions of the same target in M ​​consecutive frames to form a position sequence; calculate the position jump variables between adjacent frames to obtain a jump sequence; Calculate the mean and standard deviation of the jump sequence, and compare the standard deviation with a preset jump threshold; if the standard deviation is less than the preset jump threshold, the verification result is passed; if the standard deviation is greater than or equal to the preset jump threshold, identify abnormal frames in the jump sequence that exceed the sum of the jump mean and a preset multiple of the standard deviation, and mark the verification result as containing abnormal frames.

10. The scene relocalization and target regression method for UAVs in dynamic scenarios according to claim 9, characterized in that, When the verification result is passed, the initial mapping position of the current frame is used as the target regression position; When the verification result contains abnormal frames, the initial mapping position corresponding to the abnormal frame is corrected: interpolation is performed based on the initial mapping positions of the normal frames before and after the abnormal frame, and the interpolated result is used to replace the initial mapping position of the abnormal frame as the target regression position.

11. A scene relocalization and target regression system for dynamic UAV scenarios, characterized in that it includes: The target reference scene determination module is used to acquire the local scene image and spatiotemporal attribute information currently observed by the UAV, dynamically determine the retrieval granularity based on the spatiotemporal attribute information, and perform hierarchical retrieval in a pre-constructed reference scene database according to the retrieval granularity to determine the target reference scene that matches the current local scene; wherein, each reference scene in the reference scene database pre-stores the mapping relationship between the scene and the global map; The hierarchical retrieval includes a preliminary screening based on spatial constraints using spatiotemporal attribute information and a fine screening based on temporal constraints using a flight state prediction model. The feature matching module is used to perform target detection on the local scene image to obtain the pixel position of at least one target in the image; and to perform feature matching between the local scene image and the target reference scene to obtain the homography transformation matrix between the two. The verification module is used to convert the pixel position to the global map based on the mapping relationship between the target reference scene and the global map and the homography transformation matrix to obtain the initial mapping position; and to perform trajectory consistency verification on multiple initial mapping positions of the same target in multiple consecutive frames to obtain the verification result. The correction module is used to correct the initial mapping position based on the verification results, thereby completing the target regression.

Images