Fusion positioning method of low-altitude three-dimensional navigation map

Abstract

The application relates to the technical field of navigation positioning, and provides a fusion positioning method of a low-altitude three-dimensional navigation map, which fuses point cloud data and image data, extracts stable semantic features by combining semantic segmentation and instance recognition, and gives prior stable scores to construct a low-altitude three-dimensional navigation map, so as to provide a more accurate reference datum for subsequent positioning; the stable semantic features in a field of view range in the map are called in real time, real-time semantic features are recognized through a convolutional neural network, accurate association between map features and real-time observation is realized, high-quality feature data is provided for subsequent matching; the features are matched through a PNP algorithm, a semantic observation vector is generated, a fusion weight is determined, satellite and attitude data are fused to generate a positioning result, and the accuracy and stability of the positioning result are significantly improved; based on the positioning result, a re-projection residual error is calculated, unstable features with low confidence are removed, and a map update is triggered, adaptive optimization of the map is realized, and the reliability of long-term positioning is ensured.

Description

Technical Field

[0001] This application relates to the field of navigation and positioning technology, and in particular to a fusion positioning method for low-altitude three-dimensional navigation maps. Background Technology

[0002] With the rapid expansion of the low-altitude economy, the demand for high-precision and high-reliability positioning technologies in low-altitude scenarios is becoming increasingly urgent, driving the evolution of low-altitude positioning from a single mode to a multi-technology integration approach. Early low-altitude positioning mainly relied on a combination of satellite positioning and inertial measurement units (IMUs). Satellite positioning provided absolute position reference, while IMUs temporarily filled the positioning gap when satellite signals were briefly interrupted. This approach was widely used in low-altitude, low-speed flight scenarios, such as the daily inspection tasks of small drones.

[0003] Subsequently, with the advancement of 3D modeling and computer vision technologies, the fusion of point clouds and images has become a new trend. Point clouds are collected by LiDAR and can accurately restore spatial geometry, while images are acquired by high-definition cameras and contain rich texture and semantic information. Existing technologies can achieve registration of point clouds and images through ICP algorithms to construct low-altitude 3D maps and achieve relative positioning in areas with weak satellite signals, breaking through the scenario limitations of traditional satellite positioning and further expanding the applicable scope of low-altitude positioning.

[0004] In recent years, the rise of deep learning technology has promoted the deep integration of semantic recognition and localization technologies. Some solutions use convolutional neural networks to extract semantic features from images and use these semantic features as a localization reference. Compared with pure geometric features, semantic features are less affected by environmental changes and can effectively improve the robustness of localization. They have already been initially applied in fields such as low-altitude surveying and precision drone operations.

[0005] Although low-altitude positioning technology has achieved multi-source data fusion and semantic assistance, there are still many key defects in the complex and dynamically changing low-altitude scenarios, which restrict the further improvement of positioning performance. Existing low-altitude 3D maps are mostly built based on static data collected in a single instance. However, there are a large number of dynamically changing elements in low-altitude scenarios. The semantic features corresponding to these elements will lose stability over time, but existing maps cannot recognize such changes, leading to feature matching errors in the subsequent positioning process and significantly reducing positioning accuracy.

[0006] Existing technologies, when fusing different data for location identification, generally adopt a fixed weight model, failing to consider the varying reliability of different semantic features and thus unable to enhance their supporting role in the location results through weight optimization, ultimately affecting the overall accuracy of fused positioning. Considering the shortcomings of the existing technologies, the core technical problem this application aims to solve is how to construct a dynamically updatable low-altitude 3D navigation map to improve the fused positioning accuracy in complex low-altitude scenarios. Summary of the Invention

[0007] To address the shortcomings of existing technologies, this application provides a fusion positioning method for low-altitude three-dimensional navigation maps. The method includes: acquiring and fusing point cloud data and image data of a low-altitude region, performing semantic segmentation and instance recognition based on the fused data to extract stable semantic features, and determining a prior stability score for each stable semantic feature to generate a low-altitude three-dimensional navigation map.

[0008] Real-time acquisition of observation data from airborne sensors; retrieval of stable semantic features and prior stability scores within the current field of view from the low-altitude 3D navigation map; and processing of visual images through a convolutional neural network to identify real-time semantic features.

[0009] The PNP algorithm is used to match real-time semantic features with stable semantic features to generate semantic observation vectors and calculate matching confidence. The fusion weight of the semantic observation vectors is determined based on the product of prior stability score and matching confidence. The weighted semantic observation vectors are then fused with satellite data and attitude data to generate positioning results.

[0010] Based on the localization results, the reprojection residuals of stable semantic features are calculated. The confidence of stable semantic features is dynamically updated by Bayesian update based on the reprojection residuals. When the confidence of stable semantic features is less than the confidence threshold, they are recorded as unstable features. The prior stability score of unstable features is set to zero and unstable features are removed. At the same time, a map update signal is generated.

[0011] As an optional implementation, determining the prior stability score includes:

[0012] The evaluation criteria are determined by the physical properties, physical size, historical change frequency, and frequency of identification of stable semantic features in low-altitude observation scenarios.

[0013] By comparing the positioning effects of different weight combinations, the weight proportions of different evaluation criteria are determined, and the weighted sum is calculated according to the weight proportions to obtain the initial prior stable score.

[0014] The initial prior stability score is corrected according to the type of stable semantic feature to determine the prior stability score of each stable semantic feature, and the stable semantic features are combined to generate a low-altitude 3D navigation map.

[0015] As an optional implementation, the extraction of stable semantic features includes:

[0016] Acquire point cloud data and image data of the low-altitude region at at least two time points, combine pre-classified semantic labels to perform spatial registration and fusion of point cloud data and image data, compare the fused data at different time points, identify pixel change areas and mark them as dynamic areas;

[0017] Based on the fused data, three-dimensional regions with the same semantic attributes are segmented from point cloud data and geometric features are extracted. Texture and spectral features are extracted from image data. The geometric features are weighted and fused with texture and spectral features to identify instances and obtain an instance feature set.

[0018] By combining spatial and temporal dimensions, the stability of the instance feature set is screened to generate candidate stable features. The anti-interference ability of the candidate stable features is evaluated to select and extract stable semantic features.

[0019] As an optional implementation, the generation of semantic observation vectors includes:

[0020] From the low-altitude 3D navigation map, stable semantic features of the same type as the real-time semantic features and located within the current field of view are called to construct a candidate matching set and obtain 3D coordinates and prior stability scores.

[0021] Using the three-dimensional coordinates of stable semantic features and the two-dimensional coordinates of real-time semantic features as input, the PNP algorithm is used to calculate the initial pose value, and the matching pair is output based on the physical size as a constraint.

[0022] Semantic observation vectors are generated based on matching pairs. These vectors include stable semantic features of successful matches, coordinate deviations of matching pairs, size matching degree, and the proportion of matching pairs.

[0023] As an optional implementation, the identification of real-time semantic features includes:

[0024] Real-time acquisition of observation data from airborne sensors, including visual images, satellite data, and attitude data; preprocessing of visual images, including image contrast enhancement and motion blur suppression.

[0025] The preprocessed visual image is input into a convolutional neural network to extract global scene features, edge contours, and texture changes within the current field of view.

[0026] Semantic anchors are constructed by combining the types of stable semantic features. These anchors guide the convolutional neural network to focus on regions of the same type as the stable semantic features in order to identify real-time semantic features.

[0027] As an optional implementation, determining the fusion weights of the semantic observation vector includes:

[0028] Match confidence is generated based on the coordinate deviation, scale matching degree, and proportion of matching pairs.

[0029] The total number of real-time semantic features within the current field of view is counted. If the total number of features is greater than or equal to the number threshold, the prior stability score is normalized and corrected.

[0030] The fusion weights of the semantic observation vectors are determined by the product of the prior stability score and the matching confidence, and the fusion weights are dynamically adjusted according to the spatial distribution entropy of real-time semantic features in the visual image.

[0031] As an optional implementation, the generation of positioning results includes:

[0032] Satellite data is filtered by signal strength through sliding window filtering, while global coordinates are preserved. Satellite weights are assigned based on signal strength. At the same time, attitude data is smoothed to obtain motion state, and attitude weights are assigned based on smoothing error.

[0033] The weighted semantic observation vector is used as the main positioning constraint and fused with satellite data to obtain intermediate positioning results including global coordinates. The intermediate positioning results are then fused with attitude data to generate the initial positioning results.

[0034] The initial positioning result is compared with the positioning deviation of a single data source. If the positioning deviation is greater than or equal to the deviation threshold, the process is reversed to adjust the fusion weights of the matching pair and the semantic observation vector to generate the positioning result.

[0035] As an optional implementation, the generation of map update signals includes:

[0036] The dynamically updated confidence level is compared with the preset confidence threshold to filter out unstable features with a confidence level lower than the confidence threshold.

[0037] Set the prior stability score of unstable features to zero, remove unstable features from the low-altitude 3D navigation map, and record the removal time and location range of unstable features.

[0038] The map update signal, which includes both local and global update signals, is determined by the proportion of the number of unstable features removed to the number of stable semantic features in the current field of view.

[0039] As an optional implementation, the calculation of the reprojection residuals of stable semantic features includes:

[0040] Extract the real-time pose of the current airborne equipment from the positioning results, and call the three-dimensional coordinates of stable semantic features that overlap with the current field of view in the low-altitude three-dimensional navigation map;

[0041] Based on real-time pose, the three-dimensional coordinates of stable semantic features are projected onto the two-dimensional plane of the visual image to obtain the theoretical projection coordinates, and the deviation of the theoretical projection coordinates is corrected based on the physical dimensions during the projection process.

[0042] The distance between the theoretical projection coordinates and the two-dimensional coordinates of the real-time semantic features is calculated to obtain the basic residual value. The residual correction parameter is set according to the type of stable semantic features. The basic residual value is multiplied by the residual correction parameter to obtain the reprojection residual of the stable semantic features.

[0043] As an optional implementation, the confidence level of the dynamically updated stable semantic features includes:

[0044] The prior stable score of stable semantic features is used as the initial prior probability of Bayesian update, and the matching confidence is used as the initial weight of the likelihood function.

[0045] The likelihood function is dynamically adjusted based on the magnitude of the reprojection residual. The temporary confidence of the stable semantic features is calculated by combining the adjusted likelihood function with the initial prior probability using Bayes' theorem.

[0046] Based on the proportion of matching pairs in the semantic observation vector, the temporary confidence is decayed and corrected to obtain the confidence of the dynamically updated stable semantic features.

[0047] Compared with existing technologies, the beneficial effects of this application are as follows: By fusing point cloud data and image data, combining semantic segmentation and instance recognition to extract stable semantic features and assign prior stability scores, a low-altitude 3D navigation map is constructed, providing a more accurate reference benchmark for subsequent positioning and reducing the interference of unstable features on subsequent positioning; stable semantic features within the field of view of the map are called in real time, and real-time semantic features are identified through convolutional neural networks, realizing accurate correlation between map features and real-time observations, providing high-quality feature data for subsequent matching; features are matched and semantic observation vectors are generated through the PNP algorithm, and fusion weights are determined, and then satellite and attitude data are fused to generate positioning results, significantly improving the accuracy and stability of positioning results; reprojection residuals are calculated based on positioning results, and confidence is dynamically adjusted through Bayesian updates, eliminating unstable features with low confidence and triggering map updates, realizing adaptive optimization of the map, ensuring consistency between the map and the actual scene, and guaranteeing the reliability of long-term positioning. Attached Figure Description

[0048] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0049] Figure 1 A flowchart illustrating the low-altitude three-dimensional navigation map fusion positioning method provided in this application embodiment;

[0050] Figure 2The flowchart illustrates the generation of semantic observation vectors in the low-altitude three-dimensional navigation map fusion positioning method provided in this embodiment of the application.

[0051] Figure 3 The flowchart below shows the logic of calculating the reprojection residual of stable semantic features in the low-altitude three-dimensional navigation map fusion positioning method provided in this embodiment of the application. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of the embodiments of this application more apparent and understandable, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments.

[0053] like Figure 1 The diagram shows a flowchart of a method for fusing and locating low-altitude three-dimensional navigation maps, as provided in this application embodiment. The method includes:

[0054] This implementation takes the scenario of urban low-altitude drone inspection and logistics navigation as an example.

[0055] S1. Acquire and fuse point cloud data and image data of low-altitude areas, and perform semantic segmentation and instance recognition based on the fused data to extract stable semantic features, and determine a prior stability score for each stable semantic feature to generate a low-altitude 3D navigation map.

[0056] Furthermore, extracting stable semantic features includes:

[0057] Acquire point cloud data and image data of the low-altitude region at at least two time points, combine pre-classified semantic labels to perform spatial registration and fusion of point cloud data and image data, compare the fused data at different time points, identify pixel change areas and mark them as dynamic areas;

[0058] Based on the fused data, three-dimensional regions with the same semantic attributes are segmented from point cloud data and geometric features are extracted. Texture and spectral features are extracted from image data. The geometric features are weighted and fused with texture and spectral features to identify instances and obtain an instance feature set.

[0059] By combining spatial and temporal dimensions, the stability of the instance feature set is screened to generate candidate stable features. The anti-interference ability of the candidate stable features is evaluated to select and extract stable semantic features.

[0060] Drone navigation for logistics inspections needs to avoid temporary interference, retaining only long-term, fixed navigation reference features. However, seasonal environmental changes, such as vegetation growth and weather effects, necessitate precise differentiation between temporary interference and natural variations through multi-time-point data comparison to proactively eliminate unstable factors. Data sources are selected from two typical seasons in the target logistics low-altitude area: the rainy season and the dry season. During the rainy season, vegetation is dense and water accumulation creates reflections, while during the dry season, vegetation is sparse and ground texture is clear. Point cloud data refers to LiDAR point clouds, and image data refers to multispectral imagery. LiDAR point clouds cover the logistics take-off and landing points and the surrounding areas of major flight routes, while multispectral imagery contains... Visible light and near-infrared bands are used to adapt to feature recognition under different weather conditions. The pre-classified semantic labels are graded according to navigation importance, including core labels, basic labels, auxiliary labels, and exclusion labels. Core labels are logistics warehouses and delivery stations, which are key navigation references. Basic labels are main roads and overpasses to support route planning. Auxiliary labels are green belts and fixed guardrails to assist in positioning correction. Exclusion labels are temporary parking lots and construction areas, which are sources of interference to be avoided. The low-altitude area usually refers to the space below 1,000 meters above the Earth's surface, which is the main carrier of low-altitude economic activities such as drone logistics, urban inspection, and general aviation.

[0061] When performing spatial registration and fusion, the core label is used as the benchmark. For example, a logistics warehouse presents a regular and high-density 3D structure in point cloud data, and appears as continuous grayscale blocks in image data. First, the spatial boundaries of the two are accurately aligned. Then, based on this alignment accuracy, the spatial positions of the basic label and auxiliary label are adjusted to ensure the overall data spatial consistency. The identification of dynamic regions is achieved by comparing the data after fusion of two seasons. If a certain area has no obvious features in the rainy season image data, but new pixel areas with irregular shapes appear in the dry season, and the 3D structure of the corresponding location in the point cloud data has no complete outline, such as no clear building walls and road textures, it is marked as a dynamic region. At the same time, the spatial range and interference type of the dynamic region are recorded, including temporary promotional sheds and road maintenance facilities. Thus, semantic labels are hierarchically classified according to navigation importance to ensure the registration accuracy of core features. The comparison of multi-season data covers changes in the natural environment, avoiding misjudging temporary water accumulation in the rainy season as dynamic interference and improving the accuracy of dynamic region identification.

[0062] Logistics navigation requires precise differentiation of characteristics between different entities within the same category, such as logistics warehouse A and logistics warehouse B. A single data modality has limitations; point cloud data can reflect three-dimensional form but cannot identify wall markings, while image data can capture texture but struggles to determine the actual function of the building. Multimodal features need to be integrated to achieve accurate instance identification. Point cloud data processing focuses on structural integrity, segmenting three-dimensional areas with continuous walls, fixed roof outlines, and no obvious openings, excluding loosely structured objects like temporary sheds. Geometric features of these three-dimensional areas are extracted, including the overall building shape, wall tilt, and roof style. The presence of dedicated loading and unloading platforms is also recorded; these platforms are characterized by regular three-dimensional structures extending outwards from the building's edges. Image processing expands across the segmented point cloud data areas, extracting texture and spectral features for the corresponding three-dimensional regions. Texture features include the texture of the company logo on the warehouse exterior walls and the anti-slip stripes on the loading and unloading platforms. Spectral features include the high reflectivity of the metal roof and the low reflectivity of the concrete walls. The presence of indicator lights required for nighttime navigation is also noted; these lights are highlighted in specific spectral bands.

[0063] Feature fusion requires scenario adaptation. In logistics warehouse areas, geometric and texture features are prioritized. Geometric features include the structure of loading and unloading platforms, while texture features include company logos, used to distinguish different warehouses. In road areas, geometric and spectral features are prioritized. Geometric features include road width and the presence of median strips, while spectral features include differences in road surface reflectance, used to distinguish different road types. Instance recognition is based on the uniqueness of feature combinations. For example, a regular rectangular building, vertical walls, flat roof, and the presence of logistics company logos on the exterior walls with a metal roof are identified as logistics warehouse A. A polygonal building, sloping walls, a pitched roof, and the absence of logos with concrete walls are identified as a surrounding factory building. This ultimately forms an instance feature set containing instance name, geometric features, texture features, spectral features, and navigation function attributes. The navigation function attributes include the presence of loading and unloading platforms and navigation lights. This ensures that features match navigation requirements. Instance recognition, combined with functional attributes, ensures that the instance feature set not only contains basic information but also reflects practical value for logistics navigation, providing a basis for judging functional stability for subsequent screening.

[0064] If there are spatial misalignments or functional changes in the instance feature set, such as a mismatch between the location of the loading and unloading platform recorded in the point cloud and the actual location in the image, causing the drone to be unable to land accurately, or a warehouse loading and unloading platform being temporarily modified and losing its navigation function, it is necessary to perform spatiotemporal consistency screening to retain features with consistent spatial location and long-term stable function. Among them, the spatial dimension focuses on functional matching when performing stability screening. The geometric features of the instance feature are projected onto the corresponding area of ​​the image data, and the overlap between the projection result and the actual functional area in the image data is checked. For example, the overlap between the spatial location of the loading and unloading platform projected onto the corresponding area of ​​the image data and the texture range of the loading and unloading platform in the image data. If the overlap is low, that is, the platform location in the projected data corresponds to the wall in the image, it is judged as a spatial misalignment and is removed. At the same time, the spatial correlation between the instance feature and the surrounding fixed features is checked. For example, the warehouse should be directly connected to the road to avoid unreasonable features where there is no passage between the warehouse and the road.

[0065] When performing stability screening in the time dimension, the focus is on core functions. First, instance features are compared with dynamic area markers. If an instance is located within a dynamic area, it is directly removed. Then, the core functional attributes of instance features in the two seasons are compared. If a warehouse has a fixed loading and unloading platform and navigation lights during the rainy season but loses these functions during the dry season, and there are no markers in the image data and no platform structure in the point cloud data, it is judged as a functional change and removed. If only non-functional features differ and the core functions remain unchanged, they are retained as candidate stable features. Non-functional features include wall stains and vegetation coverage. Spatial screening combines functional matching and surrounding association to avoid positioning deviations caused by data errors. Temporal screening focuses on core functions to avoid misjudging non-functional changes as feature instability, thus improving screening accuracy.

[0066] Drone logistics navigation often faces complex environmental interference. During the day, strong light and backlight can cause blurred markings; cloudy weather brings variable lighting; at night, it is necessary to distinguish the navigation light from surrounding light sources; and low-altitude flight may encounter obstructions from trees and power lines. It is necessary to evaluate the identifiability and functional availability of candidate stable features in these scenarios to ensure stable positioning. The anti-interference capability of candidate stable features must be evaluated, including anti-interference under lighting and obstruction conditions, as well as functional availability. Lighting conditions include strong light and backlight, cloudy shadows, and low light at night. In strong light and backlight, the light source is behind the feature; in cloudy shadows, the feature is partially in shadow; and in low light at night, only the navigation light or ambient light is relied upon. In backlight scenarios, the texture edges of candidate stable features must be checked to ensure that the texture does not disappear completely. In shadow scenarios, the structural outline of candidate stable features must be checked to ensure that it can be identified through shadow contrast. In nighttime scenarios, spectral feature differences must be used to check whether the navigation light can be distinguished from surrounding streetlights to avoid misidentification.

[0067] The assessment of occlusion resistance is based on point cloud data simulating the view of a drone flying along a major logistics route. Objects obstructing candidate stable features, such as trees on both sides of the road and power lines crossing the route, are marked. Then, the occlusion of key identification points of candidate stable features, including warehouse signs and navigation lights, is evaluated. If, in most locations along the route, key identification points are unobstructed or only partially obstructed but do not affect overall recognition (e.g., a sign is partially obstructed but the category can still be identified), then the occlusion resistance capability meets the standard. The assessment of functional usability focuses on navigation functions. For example, in backlit scenarios, it assesses whether the anti-slip stripes of the loading and unloading platform can still be identified to ensure accurate boundary positioning when the drone lands. In nighttime scenarios, it assesses whether the flashing characteristics of the navigation lights can still be identified. The data is captured by sensors to avoid confusion with other light sources. Simultaneously, candidate stable features are evaluated through three categories, ultimately determining stable semantic features. This results in stable semantic features including instance name, 3D coordinates, key identification point locations, and anti-interference scene adaptability. Anti-interference scene adaptability includes adaptability to nighttime navigation and backlight scenarios. The anti-interference assessment balances identifiability and functional usability, aligning with actual logistics navigation needs. Scene adaptability labeling provides a basis for subsequent prior stability scoring, avoiding treating features applicable to a single scene as equivalent to those applicable to all scenes. The anti-interference scene adaptability and key identification point locations of the stable semantic features become core references for the observation reliability in prior stability scoring.

[0068] Specifically, determining the prior stability score includes:

[0069] The evaluation criteria are determined by the physical properties, physical size, historical change frequency, and frequency of identification of stable semantic features in low-altitude observation scenarios.

[0070] By comparing the positioning effects of different weight combinations, the weight proportions of different evaluation criteria are determined, and the weighted sum is calculated according to the weight proportions to obtain the initial prior stable score.

[0071] The initial prior stability score is corrected according to the type of stable semantic feature to determine the prior stability score of each stable semantic feature, and the stable semantic features are combined to generate a low-altitude 3D navigation map.

[0072] The requirements for feature stability in logistics navigation are multifaceted, including physical stability (no alteration of buildings and no damage to roads), navigation function stability (no removal of loading / unloading platforms and normal operation of navigation lights), and stability in all-scene observation (recognition in both day and night, and in all weather conditions). A comprehensive evaluation from multiple dimensions is necessary to ensure that the score reflects the overall navigation value of the features. Physical attributes focus on structure and irreplaceability. Structural durability is judged based on materials and construction standards, including reinforced concrete, brick, and metal. Construction standards include whether it is a formal building and whether it has a fixed foundation. Irreplaceability is judged based on whether it is the only reference for the route; for example, a reinforced concrete warehouse has better physical properties than a temporary brick or wood building, and a unique reference warehouse is more irreplaceable than multiple similar buildings in the vicinity. Physical dimensions focus on saliency and navigation coverage. Saliency is judged based on the relative size of stable semantic features in the surrounding environment and the presence of prominent structures. Relative size includes whether it is significantly larger than surrounding objects, and prominent structures include towering navigation beacons and extended loading / unloading platforms. Navigation coverage is judged based on the distance at which the features can be recognized; for example, a large warehouse with a beacon is more saliency-recognizable than a small warehouse without prominent structures.

[0073] Historical change frequency focuses on morphological and functional changes. Morphological changes are determined based on past records of expansion, alteration, and demolition, while functional changes are determined based on records of additions or removals of navigation functions. Navigation functions include loading / unloading platforms and navigation lights. For example, warehouses with no morphological changes and consistent function over many years have a lower change frequency than buildings with frequent alterations. Low-altitude observation frequency focuses on full-scene recognition, based on the successful recognition of stable semantic features in different past scenarios and the flight distance at which recognition occurs. Different scenarios include sunny days, cloudy days, nighttime, and backlighting. Flight distances include near, medium, and far. For example, a warehouse that can be recognized in all scenarios and at all distances... Kubi's observation frequency for signs that are only identifiable at close range on clear days is better; the navigation function needs to be added to adapt to the logistics scenario, focusing on the necessity of the function, and judging based on whether it is a take-off and landing point or a loading and unloading point, whether there are night navigation facilities, and whether it is located at a key turning point of the route. For example, the navigation function of logistics take-off and landing point warehouses is more important than that of surrounding non-logistics buildings. This makes the evaluation more in line with the logistics scenario, taking into account both form and function in each dimension, avoiding focusing only on physical form while ignoring functional stability, and ensuring that the score fully reflects the practical value. The navigation function and the observation information of the whole scenario provide a scenario adaptation reference for dynamic weight allocation, ensuring that the weight prioritizes the core needs of logistics navigation.

[0074] Different navigation scenarios have different dimensional requirements. Around takeoff and landing points, priority should be given to ensuring functional availability and stable close-range recognition. Long-haul routes require priority to ensure long-range recognition and long-term stability. Complex business districts require priority to ensure interference-resistant recognition and physical stability. Fixed weights cannot adapt to scenario differences and need to be dynamically allocated. Different weight combinations are configured according to the scenario. In takeoff and landing point scenarios, navigation function has the highest weight, prioritizing loading / unloading and takeoff / landing functions. Next is the frequency of recognition in low-altitude observation scenarios to ensure stable close-range recognition. The weights of physical attributes, physical dimensions, and historical change frequency decrease in that order. In long-haul route scenarios, physical dimensions have the highest weight to ensure easy long-range recognition. Next is the historical change frequency to ensure long-term stability. The weights of physical attributes, the frequency of recognition in low-altitude observation scenarios, and navigation function decrease in that order. In complex business district scenarios, the frequency of recognition in low-altitude observation scenarios has the highest weight to ensure interference-resistant recognition. Next is physical attributes to ensure physical stability. The weights of historical change frequency, navigation function, and physical dimensions decrease in that order.

[0075] The weight combinations are validated using actual navigation data. In take-off and landing scenarios, if a weight combination enables the UAV to accurately locate the loading and unloading platform with minimal positioning deviation, it is determined as the weight proportion for that scenario. In long-haul scenarios, if a weight combination enables the UAV to continuously correct its flight path using long-range features with minimal flight path deviation, it is determined as the weight proportion for that scenario. In the initial calculation of the prior stability score, the main scenario to which each stable semantic feature is adapted is first determined. For example, logistics warehouses are adapted to take-off and landing scenarios, and overpasses are adapted to long-haul scenarios. Then, according to the weight proportion of the corresponding scenario, the performance of the stable semantic features in each dimension is graded and scored to divide them into multiple levels, including three levels from poor to excellent. The initial prior stability score is obtained by weighting the weights according to their respective weight proportions. Scenario-based weight allocation avoids a one-size-fits-all approach, allowing the prior stability score to adapt to different scenario requirements. Validation based on actual navigation data ensures that the weight combinations are practical and reliable.

[0076] Within the same dimension, the stability standards and scenario values ​​of different types of stable semantic features vary. For example, vegetation features, although subject to seasonal variations, can assist in obstacle avoidance in commercial areas. Signage features, though small in size, are core navigation references at night. The prior stability score needs to be adjusted based on the type of stable semantic feature and scenario requirements to ensure accurate matching of navigation value. The types of stable semantic features include core function features, route reference features, and environmental aid features. Core function features include logistics warehouses, take-off and landing platforms, and navigation beacons, directly supporting core navigation needs. Route reference features include overpasses, main roads, and fixed medians, supporting route planning and correction. Environmental aid features include fixed green belts and large sculptures, assisting in positioning and obstacle avoidance. The prior stability score is adjusted according to the type. For core function features, in take-off and landing scenarios, a higher score for navigation functionality is given, as take-off and landing are highly dependent on functionality. In nighttime scenarios, the presence of nighttime navigation facilities further enhances the score, as nighttime functionality is dependent on features. If a brief functional failure has occurred but has been repaired and remains stable long-term, the score is not lowered to avoid negating the overall value due to occasional malfunctions.

[0077] Secondly, for route reference, in long-haul scenarios, a better score in physical size (i.e., easy identification at a distance) results in an additional score. In complex commercial district scenarios, a better score in frequency of identification in low-altitude observation scenarios (i.e., strong anti-interference) results in an additional score. If new fixed buildings are added nearby but do not affect its identification and navigation function, the score will not be lowered to avoid misjudging the feature value due to changes in the surrounding environment. Finally, for environmental assistance, in complex commercial district scenarios, if it can assist in obstacle avoidance (i.e., clearly define the route boundaries), then even if the historical change frequency is slightly high (i.e., seasonal leaf drop), the score will not be lowered. Around takeoff and landing points, if it can assist in position correction (i.e., serve as a reference landmark for warehouses), then... Further score enhancement: After correcting the initial prior stability score according to the above rules, the final prior stability score of each stable semantic feature is obtained. This score is then associated with the stable semantic features to generate a low-altitude 3D navigation map that includes the stable semantic features, prior stability scores, and scene adaptability. The correction rules, which combine typification and scenario-based approaches, avoid judging feature value by a single standard. This ensures that the score accurately reflects the navigation role of different stable semantic features in different scenarios. The final prior stability score provides core parameters for the subsequent calculation of the fusion weights of semantic observation vectors. An accurate prior stability score ensures that the fusion weights match the actual value of the stable semantic features, thereby improving positioning accuracy.

[0078] S2. Real-time acquisition of observation data from airborne sensors, retrieval of stable semantic features and prior stability scores within the current field of view from the low-altitude 3D navigation map, processing of visual images through a convolutional neural network to identify real-time semantic features, matching real-time semantic features with stable semantic features using the PNP algorithm to generate semantic observation vectors and calculate matching confidence, determining the fusion weight of semantic observation vectors based on the product of prior stability scores and matching confidence, and fusing the weighted semantic observation vectors with satellite data and attitude data to generate positioning results.

[0079] Furthermore, identifying real-time semantic features includes:

[0080] Real-time acquisition of observation data from airborne sensors, including visual images, satellite data, and attitude data; preprocessing of visual images, including image contrast enhancement and motion blur suppression.

[0081] The preprocessed visual image is input into a convolutional neural network to extract global scene features, edge contours, and texture changes within the current field of view.

[0082] Semantic anchors are constructed by combining the types of stable semantic features. These anchors guide the convolutional neural network to focus on regions of the same type as the stable semantic features in order to identify real-time semantic features.

[0083] Drone inspections are susceptible to interference from low-altitude environments, such as flight vibrations causing image blurring and strong light or shadows causing image contrast imbalances. Directly using raw images for feature recognition can easily lead to errors, necessitating image quality optimization to improve subsequent recognition accuracy. Real-time acquisition of observation data from onboard sensors, including visual images, satellite data, and attitude data, is crucial. Visual images are extracted and preprocessed to suit the inspection scene's characteristics. A contrast enhancement algorithm adjusts the brightness of dark areas caused by shadows and overly bright areas caused by strong light, ensuring clear feature textures. A motion blur suppression algorithm, based on the drone's attitude data, determines the direction and amplitude of vibration, performs pixel compensation on blurred areas, and restores feature edge details, ensuring the preprocessed visual image accurately reflects the scene features within the current field of view. Attitude data includes angular velocity and acceleration. Contrast enhancement addresses the issue of invisible features due to uneven lighting, while motion blur suppression eliminates the impact of flight vibrations on image quality, providing a high-quality image data source for subsequent feature extraction.

[0084] Inspection navigation needs to simultaneously consider both the global scene layout and local feature details. The global scene layout includes determining whether the current location is a main urban road or an industrial park to avoid route deviation. Local feature details include identifying inspection targets such as power transmission towers and traffic lights. Single-dimensional feature extraction cannot cover both aspects; therefore, dual-dimensional feature extraction is required to achieve comprehensive scene perception. The pre-processed visual image is input into a convolutional neural network for the extraction of both global scene features and local detail features. Global scene feature extraction uses large-size convolutional kernels to capture the overall layout of the visual image, including the distribution density of features and terrain contours within the current field of view. For example, it can identify whether there are continuous building clusters and open green spaces, and output descriptive information of global scene features; while the extraction of local detail features is achieved by using small-sized convolution kernels and pooling operations to capture the edge contours and texture changes of features. The edge contours include the triangular structure of the transmission tower and the circular contour of the traffic lights, and the texture changes include the brick joint texture of the building walls and the marking texture of the road surface, outputting local feature vectors. The extraction of both types of features not only grasps the global scene features and avoids scene misjudgment caused by isolated local features, but also accurately captures local details, providing detailed support for the identification of inspection targets and improving the comprehensiveness and accuracy of feature extraction.

[0085] In inspection scenarios, the number of features within the field of view is large, such as the simultaneous presence of buildings, trees, streetlights, and transmission towers. If all features are identified indiscriminately, irrelevant features can easily interfere, such as trees obscuring the identification of critical transmission towers. Therefore, semantic anchors are needed to focus on key features to improve recognition efficiency and accuracy. From a low-altitude 3D navigation map, the types of stable semantic features preset for the current inspection route are obtained, including transmission towers, substation buildings, and traffic lights. These types are used as semantic anchors, which are then associated with the global scene features and local detail features output by the convolutional neural network. An attention mechanism is used to strengthen the convolutional neural network's understanding of semantic features. The weights of stable semantic features of the same type as the anchor point are determined. For example, when the semantic anchor point is a power transmission tower, the network prioritizes the recognition of features with triangular contours and metallic textures, while weakening the weights of non-target features such as trees and ordinary streetlights. The final output includes real-time semantic features that include type labels and the two-dimensional coordinates of the features in the visual image, ensuring that the real-time semantic features are dominated by the features of the inspection target. The semantic anchor point guides the network to avoid interference from irrelevant features and improve the recognition efficiency of target features. At the same time, the weights of the attention mechanism are strengthened to solve the problem of recognition difficulties caused by key features being occluded or similar to other features in complex scenes, ensuring the accuracy and relevance of real-time feature recognition.

[0086] Specifically, such as Figure 2 As shown, the generated semantic observation vector includes:

[0087] From the low-altitude 3D navigation map, stable semantic features of the same type as the real-time semantic features and located within the current field of view are called to construct a candidate matching set and obtain 3D coordinates and prior stability scores.

[0088] Using the three-dimensional coordinates of stable semantic features and the two-dimensional coordinates of real-time semantic features as input, the PNP algorithm is used to calculate the initial pose value, and the matching pair is output based on the physical size as a constraint.

[0089] Semantic observation vectors are generated based on matching pairs. These vectors include stable semantic features of successful matches, coordinate deviations of matching pairs, size matching degree, and the proportion of matching pairs.

[0090] Blindly matching real-time semantic features with all stable semantic features in the map increases computational load and easily leads to cross-regional mismatches. For example, matching power transmission towers in region A with power transmission towers in region B requires first filtering out stable semantic features with high correlation to the real-time semantic features to narrow the matching range and improve matching efficiency and accuracy. From the low-altitude 3D navigation map, matching is first filtered by type, retaining only stable semantic features that are consistent with the type of the real-time semantic features. For example, if the real-time semantic feature is power transmission towers, only stable semantic features of power transmission towers in the map are retained. Then, matching is filtered by field of view. Combining the current pose of the UAV with the 3D coordinates of the stable semantic features, it is calculated whether the stable semantic features are within the field of view of the current UAV visual image. Features outside the field of view are eliminated. Stable semantic features that simultaneously satisfy type and field of view matching are integrated to construct a candidate matching set. The 3D coordinates and prior stability score of each stable semantic feature are extracted and associated with the candidate matching set. Thus, through dual filtering by type and field of view, the matching range is significantly narrowed, the computational load of invalid matching is reduced, and cross-type and cross-field of view mismatches are avoided, laying the foundation for subsequent accurate matching.

[0091] A precise correspondence must be established between the two-dimensional coordinates of real-time semantic features and the three-dimensional coordinates of stable semantic features to achieve pose association. However, simple geometric matching is prone to matching deviations due to feature similarity, so physical size constraints need to be incorporated to improve matching accuracy. Using the three-dimensional coordinates of stable semantic features and the two-dimensional coordinates of real-time semantic features in the candidate matching set as input data for the PNP algorithm, the algorithm is started to calculate the initial value of the UAV's current pose, i.e., the initial pose value. During the matching process, the physical size of the stable semantic features is introduced as a constraint; for example, the physical height of a stable transmission tower is recorded, and the pixel height of the corresponding real-time semantic feature in the visual image must conform to this physical height. Based on the proportional relationship with the current shooting distance, if the actual physical size corresponding to the pixel height of the real-time semantic feature deviates too much from the physical size of the stable semantic feature, the matching combination is excluded. After filtering by physical size constraints, the PNP algorithm outputs matching pairs of real-time semantic features and stable semantic features that meet the constraints. Each matching pair includes the identifier of the corresponding real-time semantic feature, the identifier of the stable semantic feature, and the coordinate correspondence. The PNP algorithm realizes the association between two-dimensional coordinates and three-dimensional coordinates, providing a basis for pose calculation, while the physical size constraint solves the problem of mismatch of similar features, ensures the geometric consistency of the matching pairs, and improves the reliability of the matching results.

[0092] Matching pairs only contain basic feature correspondences. However, localization fusion requires information such as matching quality and coverage. Matching quality includes the magnitude of coordinate deviation and size matching degree, while matching coverage includes the proportion of matching pairs. This information needs to be integrated into a structured semantic observation vector to provide a quantitative basis for subsequent weight calculation and fusion. Using matching pairs as the core data foundation, a semantic observation vector is constructed. This vector first includes the identifier of the successfully matched stable semantic feature, used to associate prior information of that stable semantic feature in the map. Secondly, it includes the coordinate deviation of the matching pair, i.e., the theoretical coordinates of the stable semantic feature projected onto the visual image versus the real-time semantic feature coordinates. The description of the deviation in actual two-dimensional coordinates also includes size matching degree, which describes the degree of fit between the pixel size of the real-time semantic features converted to actual size and the physical size of the stable semantic features. Finally, it includes the proportion of matching pairs, which is the ratio of the number of currently successfully matched pairs to the total number of candidate matching pairs, reflecting the coverage and reliability of the current matching. This information is integrated in a structured format to form the final semantic observation vector. The semantic observation vector expands the matching results from a simple correspondence to a multi-dimensional information set including matching quality and matching coverage, providing a comprehensive quantitative basis for subsequent fusion weight calculation and avoiding weight judgment bias caused by relying on only a single matching relationship.

[0093] Furthermore, determining the fusion weights for the semantic observation vectors includes:

[0094] Based on the coordinate deviation, scale matching degree, and proportion of matching pairs, a standardized matching confidence score is generated.

[0095] The total number of real-time semantic features within the current field of view is counted. If the total number of features is greater than or equal to the number threshold, the prior stability score is normalized and corrected.

[0096] The fusion weights of the semantic observation vectors are determined by the product of the prior stability score and the matching confidence, and the fusion weights are dynamically adjusted according to the spatial distribution entropy of real-time semantic features in the visual image.

[0097] The fusion weights need to be determined based on the reliability of the matching results. However, the coordinate deviation, size matching degree, and quantity ratio of the matching pairs cannot intuitively reflect the overall reliability when they exist alone. For example, the coordinate deviation may be small but the quantity ratio may be low, or the quantity ratio may be high but the size matching degree may be poor. Therefore, the three factors need to be integrated into a unified confidence index to quantify the matching reliability. For the matching pair information in the semantic observation vector, the reliability of coordinate deviation, size matching degree, and quantity ratio are evaluated separately. The smaller the coordinate deviation, the higher the corresponding reliability score; the more consistent the size matching degree, the higher the corresponding reliability score; and the higher the quantity ratio, the higher the corresponding reliability score. The reliability scores of the three dimensions are weighted according to the degree of matching core. Among them, coordinate deviation and size matching degree directly affect the matching accuracy and have a higher weight than quantity ratio. After weighting, a standardized matching confidence score is obtained, which can comprehensively reflect the overall reliability of the current matching results. The standardized matching confidence score integrates multi-dimensional matching information into a single quantitative index, avoiding the one-sidedness of single-dimensional information, making the judgment of matching reliability more intuitive and comprehensive, and providing a clear reliability basis for the fusion weights.

[0098] When there are too many real-time semantic features of a certain type within the current field of view, such as densely distributed streetlights, directly using the prior stability score to calculate the fusion weight can easily lead to the superposition of weights for similar features. For example, the cumulative weight of multiple streetlight features may cause their influence to exceed that of transmission tower features. It is necessary to correct and balance the weight ratio of similar features to avoid weight imbalance. The total number of real-time semantic features within the current field of view should be counted, especially the total number of real-time semantic features that match stable semantic features of the same type. If the total number of features of a certain type of real-time semantic features is greater than or equal to the preset number threshold for the inspection scenario, the prior stability score of the stable semantic features corresponding to that type of real-time semantic features should be normalized and corrected. The prior stability scores of all features of that type should be scaled proportionally so that the weight ratio of that type of stable semantic features after correction does not exceed the preset upper limit, ensuring that the weight ratio of key features is not diluted by dense secondary features. Normalization correction solves the problem of the weakening of the weight ratio of key features caused by the superposition of weights of dense secondary features, balances the weight ratio of different types of features, ensures that inspection features can play a dominant role during fusion, and avoids secondary features interfering with positioning accuracy.

[0099] The distribution of real-time semantic features in visual images is uneven; for example, some regions have dense features while others have sparse features. If a fixed weight is used, the concentrated weights in dense regions can lead to localization errors. Therefore, the fusion weights need to be dynamically adjusted based on the feature distribution to improve their scene adaptability. First, the fusion weights of the semantic observation vectors are calculated based on the product of the corrected prior stable score and the matching confidence. Then, the spatial distribution of real-time semantic features in the visual image is analyzed. If a region has dense features, the fusion weight corresponding to that region's vector is appropriately reduced to avoid excessive concentration of fusion weights. If a region only contains key features, the corresponding weight is appropriately increased. The fusion weights of the vectors enhance their positioning reference role. Through spatial distribution analysis, the fusion weights are dynamically adjusted to obtain the fusion weights of the semantic observation vectors. This dynamic adjustment allows the fusion weights to adapt to the spatial distribution differences of real-time semantic features, avoiding excessive influence from features in dense areas and insufficient weights for sparse key features. This ensures that the fusion weight of each semantic fusion vector matches its actual reference value in positioning, improving the rationality and accuracy of fusion weight allocation. Ultimately, the fusion weights take into account historical stability, real-time matching reliability, and spatial distribution rationality, providing support for subsequent multi-source data fusion and ensuring that the positioning results can prioritize key and reliable feature information.

[0100] Specifically, the generated location results include:

[0101] Satellite data is filtered by signal strength through sliding window filtering, while global coordinates are preserved. Satellite weights are assigned based on signal strength. At the same time, attitude data is smoothed to obtain motion state, and attitude weights are assigned based on smoothing error.

[0102] The weighted semantic observation vector is used as the main positioning constraint and fused with satellite data to obtain intermediate positioning results including global coordinates. The intermediate positioning results are then fused with attitude data to generate the initial positioning results.

[0103] The initial positioning result is compared with the positioning deviation of a single data source. If the positioning deviation is greater than or equal to the deviation threshold, the process is reversed to adjust the fusion weights of the matching pair and the semantic observation vector to generate the positioning result.

[0104] Satellite data is susceptible to signal strength fluctuations due to low-altitude obstruction, while attitude data is prone to high-frequency noise due to flight jitter. Directly fusing raw data would affect positioning accuracy. Furthermore, the two types of data have different reliability levels; for example, strong satellite signals result in high reliability, while weak signals result in low reliability. Therefore, preprocessing and assigning corresponding weights based on reliability are necessary to lay the foundation for fusion. A sliding window filtering process is used to process satellite data, using multiple consecutive frames as a window to filter out valid satellite data with signal strength meeting the requirements of the inspection scenario, eliminating abnormal data with excessively low signal strength, and retaining valid global coordinate information. Then, satellite weights are assigned based on signal strength; the stronger the signal, the higher the satellite weight, ensuring that strong signal satellite data can be used more effectively. It provides a more reliable global coordinate reference; the attitude data is smoothed and filtered to eliminate high-frequency noise caused by flight jitter, resulting in a stable short-term motion state of the UAV. Then, attitude weights are assigned based on the smoothing error of the attitude data. The smaller the smoothing error, the more stable the motion state, and the higher the attitude weight, ensuring that stable attitude data can provide a reliable short-term motion reference. Satellite data preprocessing removes outliers, and weight allocation makes its reliability match the satellite weights. Attitude data smoothing eliminates noise, and weight allocation makes its stability correlated with the attitude weights. The reliability of both types of data is improved after preprocessing, and the corresponding weight allocation is more reasonable, providing high-quality and highly adaptable data support for multi-source fusion.

[0105] Semantic observation vectors, satellite data, and attitude data play different roles in localization. Semantic observation vectors provide precise local positioning through feature matching, satellite data provides a global coordinate reference, and attitude data provides short-term motion continuity. Direct fusion can easily lead to overlapping or cancellation of effects; therefore, layered fusion is necessary to fully leverage the advantages of each data type. The approach involves fusing semantic observation vectors with satellite data, and then further fusing them with attitude data. First, the weighted semantic observation vector is used as the primary localization constraint because it focuses on inspection features and provides precise local positioning. Then, it is fused with satellite data, where the semantic observation vector provides local position correction and the satellite data provides a global coordinate reference. The combination of both yields a localized position including global coordinates. Intermediate localization results address the issues of easy drift in local localization of semantic observation vectors and occlusion in satellite data-based localization. Then, intermediate localization results are fused with attitude data. Attitude data provides short-term UAV motion trends, including flight direction and speed, allowing for dynamic fine-tuning of the intermediate localization results. This compensates for localization interruptions that may occur during feature matching gaps, generating continuous and stable initial localization results. Layered fusion fully leverages the advantages of the three types of data: the local accuracy of semantic observation vectors, the global benchmark of satellite data, and the short-term continuity of attitude data. This avoids the limitations of single data sources, while the master-slave constraint relationship ensures the dominant role of inspection features in localization, improving the accuracy and continuity of the initial localization results.

[0106] The initial positioning result may deviate significantly from the positioning of a single data source due to certain data anomalies. These anomalies include sudden weakening of satellite signals and occasional deviations in semantic matching. Directly outputting this result would affect the accuracy of the inspection navigation. Therefore, the reliability of the result needs to be verified, and the initial positioning result with excessive deviation needs to be corrected. The initial positioning result is compared with the individual positioning results of semantic observation vectors, satellite data, and attitude data, respectively. The deviation between the initial positioning result and these three types of individual results is calculated. If the deviation is within the preset deviation threshold range of the inspection scenario, the initial positioning result is considered reliable and is directly output as the final positioning result. If any deviation is greater than or equal to the deviation threshold range, such as the initial positioning result deviating significantly from the semantic observation vector, the result is considered unreliable. If the positioning result of the observation vector has too large a deviation, the problem is traced back. First, semantic matching is checked, low-confidence matching pairs are re-verified, abnormal matching pairs are removed, and the fusion weight of the semantic observation vector is recalculated. Then, the positioning result is regenerated according to the above process until the deviation meets the requirements, and the final positioning result is output. Consistency verification ensures the reliability of the positioning result and avoids positioning deviation caused by a single data anomaly. The correction by backtracking accurately locates the root cause of the problem. By rematching and adjusting the fusion weight, the deviation can be corrected without overall rework. This ensures positioning accuracy, reduces invalid calculations, and improves inspection efficiency. At the same time, backtracking enables the positioning process to have self-correction capabilities, adapt to the complex and ever-changing environmental interference in low-altitude inspections, and ensure the stability of continuous inspections.

[0107] S3. Based on the positioning results, calculate the reprojection residuals of stable semantic features. Based on the reprojection residuals, use Bayesian updates to dynamically update the confidence of stable semantic features. When the confidence of a stable semantic feature is less than the confidence threshold, it is recorded as an unstable feature. Set the prior stability score of the unstable feature to zero and remove the unstable feature. At the same time, generate a map update signal.

[0108] Furthermore, such as Figure 3 As shown, calculating the reprojection residuals of stable semantic features includes:

[0109] Extract the real-time pose of the current airborne equipment from the positioning results, and call the three-dimensional coordinates of stable semantic features that overlap with the current field of view in the low-altitude three-dimensional navigation map;

[0110] Based on real-time pose, the three-dimensional coordinates of stable semantic features are projected onto the two-dimensional plane of the visual image to obtain the theoretical projection coordinates, and the deviation of the theoretical projection coordinates is corrected based on the physical dimensions during the projection process.

[0111] The distance between the theoretical projection coordinates and the two-dimensional coordinates of the real-time semantic features is calculated to obtain the basic residual value. The residual correction parameter is set according to the type of stable semantic features. The basic residual value is multiplied by the residual correction parameter to obtain the reprojection residual of the stable semantic features.

[0112] The calculation of reprojection residuals requires a correspondence between the current position and attitude of the UAV and the 3D coordinates of stable semantic features in the map. If the two are not precisely correlated, subsequent projection calculations will have a reference deviation, causing the residuals to lose their reference value. It is necessary to establish a precise correlation between the two to ensure the reliability of the calculation reference. From the positioning results, the real-time pose of the current UAV, including position coordinates and attitude angles, is extracted to reflect the current flight direction and tilt state of the UAV. Then, the UAV's positioning system is correlated with the index of the low-altitude 3D navigation map, and the 3D coordinates of stable semantic features that overlap with the current UAV's field of view in the low-altitude 3D navigation map are called. This ensures that each called stable semantic feature is within the current observable range, forming a correspondence between the real-time pose and the 3D coordinates of the stable semantic features. This clarifies the two core references for the calculation of reprojection residuals, avoiding calculation deviations caused by unclear references. At the same time, the field of view is filtered to ensure that the called stable semantic features have a basis in actual observation, eliminating the interference of invalid features on the calculation.

[0113] The real-time pose of a drone exhibits slight tilts, such as lateral tilt and pitch, which can cause offsets when the 3D coordinates of stable semantic features are directly projected onto the visual image. Without correcting this offset, the deviation between the theoretical projection coordinates and the actual observed coordinates will include pose tilt errors, failing to accurately reflect the stability of the features themselves. Therefore, it is necessary to correct the projection deviation by incorporating physical dimensions. Based on the real-time pose, the 3D coordinates of the stable semantic features are mapped to the 2D plane of the drone's visual image using a spatial projection algorithm, obtaining initial theoretical projection coordinates. Referring to the physical dimensions of the stable semantic features, including the height of the logistics warehouse and the side length of the navigation markers, the corresponding image pixel size at the current projection distance is calculated. If the pixel size corresponding to the theoretical projection coordinates does not match the pixel size converted from the physical dimensions, the projection coordinates are adjusted according to the size deviation to correct the projection offset caused by pose tilt, resulting in the final theoretical projection coordinates. This approach corrects the projection deviation caused by pose tilt through physical size constraints, making the theoretical projection coordinates more closely match the actual imaging patterns of the stable semantic features. It avoids interference from factors other than the features themselves in the calculation of the reprojection residuals, ensuring that the reprojection residuals accurately reflect the matching accuracy and stability of the features.

[0114] Different types of stable semantic features are affected by environmental interference to varying degrees. For example, warehouse outlines are not easily obscured, while signs are easily obscured by trees. If a uniform standard is used to calculate the reprojection residuals, it will lead to deviations in the stability judgment of different stable semantic features. It is necessary to adjust the residuals according to the type of stable semantic feature to ensure that the reprojection residuals match the actual characteristics of the feature. The distance between the theoretical projection coordinates and the two-dimensional coordinates of the real-time semantic feature is calculated to obtain the basic residual value. The residual correction parameter is set according to the type of stable semantic feature. For large and well-defined stable semantic features, a smaller residual correction parameter is set because these features are imaged stably, and even small deviations can reflect stability issues. For stable semantic features that are easily affected by interference, a larger residual correction parameter is set because these features are prone to occlusion and bias. The judgment criteria need to be relaxed, and the basic residual value is multiplied by the corresponding residual correction parameter to obtain the reprojection residual of the final stable semantic feature. The reprojection residual is corrected according to type to avoid the judgment bias caused by a one-size-fits-all calculation method. This allows the reprojection residual to more accurately reflect the actual stable state of different types of stable semantic features, improving the effectiveness of the reprojection residual as a basis for stability judgment. Finally, the reprojection residual accurately associates the feature type with the actual stable state, providing a quantitative indicator for the subsequent Bayesian update confidence, ensuring that the confidence update can fit the actual characteristics of stable semantic features.

[0115] Furthermore, the confidence level of dynamically updated stable semantic features includes:

[0116] The prior stable score of stable semantic features is used as the initial prior probability of Bayesian update, and the matching confidence is used as the initial weight of the likelihood function.

[0117] The likelihood function is dynamically adjusted based on the magnitude of the reprojection residual. The temporary confidence of the stable semantic features is calculated by combining the adjusted likelihood function with the initial prior probability using Bayes' theorem.

[0118] Based on the proportion of matching pairs in the semantic observation vector, the temporary confidence is decayed and corrected to obtain the confidence of the dynamically updated stable semantic features.

[0119] Confidence updates require combining the historical stable state of stable semantic features with current observations. The historical stable state serves as prior information, while the current observations represent likelihood information. Relying on only one type of information can lead to a biased assessment of semantic feature stability; for example, a historically stable semantic feature might be misjudged as unstable due to a single accidental deviation. The prior stability score of the stable semantic feature is used as the initial prior probability for the Bayesian update. This initial prior probability reflects the long-term stable state of the stable semantic feature; for example, stable semantic features with no historical changes and high observation frequency have a high initial prior probability. The matching confidence score is used as the initial weight of the Bayesian update likelihood function. This initial weight reflects the matching reliability between the current stable semantic feature and the real-time semantic feature; for example, stable semantic features with small coordinate deviations and a high proportion of occurrences have a high initial weight. This completes the initial parameter configuration for the Bayesian update. The initial parameters simultaneously cover both historical stable states and current observations, avoiding biased judgments caused by a single type of information. This provides a comprehensive and balanced initial basis for confidence updates, ensuring that the updated confidence score considers both long-term and short-term stability.

[0120] The reprojection residuals directly reflect the matching accuracy and stability of the current stable semantic features. Small residuals indicate accurate matching and stable features, while large residuals suggest feature changes or matching errors. The reprojection residuals need to be correlated with the likelihood function to dynamically adjust the initial weights of the current observations, ensuring that confidence updates respond to the latest observations. The initial weights of the likelihood function are adjusted based on the magnitude of the reprojection residuals. Small reprojection residuals indicate that the current observations support stable semantic features, so the initial weights of the likelihood function are increased. Large reprojection residuals indicate that the current observations suggest unstable semantic features, so the initial weights of the likelihood function are decreased. The initial weights of the numbers are determined, and the greater the deviation of the reprojection residuals from a reasonable range, the greater the reduction in the initial weights. The adjusted likelihood function and the initial prior probability are substituted into the formula of Bayes' theorem to calculate the temporary confidence of the stable semantic features. This temporary confidence initially integrates historical prior information and current observation information. The likelihood function is dynamically adjusted through the reprojection residuals so that the confidence update can respond to the current observation results in real time, avoiding the judgment lag caused by overly solidified historical prior information. For example, the stable semantic features may have changed, but the historical prior information still maintains a high confidence, thus improving the sensitivity of the confidence to the current state of the stable semantic features.

[0121] Temporary confidence only considers the matching accuracy reflected by the reprojection residual, without considering matching coverage. For example, a stable semantic feature with only one matching pair and a small reprojection residual has a different stability confidence than a stable semantic feature with multiple matching pairs and small reprojection residuals. Ignoring matching coverage can lead to overly optimistic judgments about the stability of semantic features with low coverage. Therefore, temporary confidence needs to be adjusted in conjunction with matching coverage. The percentage of matching pairs recorded in the semantic observation vector is extracted. If this percentage is high, meaning most candidate features are matched successfully, it indicates comprehensive semantic feature observation coverage, and temporary confidence is not attenuated or only slightly attenuated. If this percentage is low, meaning only a few candidate features are matched successfully, it indicates poor feature observation coverage. Insufficient coverage leads to unobserved changes. Temporary confidence levels are attenuated based on the percentage of unobserved changes, with a larger attenuation rate for lower percentages, resulting in the final dynamically updated confidence level of stable semantic features. This is then combined with matching coverage to adjust the confidence level, avoiding misjudgments of stability due to insufficient matching coverage. This ensures that stable semantic features with high coverage and high-precision matching receive high confidence levels, while the confidence levels of stable semantic features with low coverage matching are appropriately lowered, improving the comprehensiveness and reliability of confidence assessment. Ultimately, the confidence level comprehensively reflects the historical stability state, current matching accuracy, and coverage of stable semantic features, providing accurate judgment criteria for subsequent unstable feature screening and map update signal generation.

[0122] Specifically, the generation of map update signals includes:

[0123] The dynamically updated confidence level is compared with the preset confidence threshold to filter out unstable features with a confidence level lower than the confidence threshold.

[0124] Set the prior stability score of unstable features to zero, remove unstable features from the low-altitude 3D navigation map, and record the removal time and location range of unstable features.

[0125] The map update signal, which includes both local and global update signals, is determined by the proportion of the number of unstable features removed to the number of stable semantic features in the current field of view.

[0126] A clear confidence threshold is needed to determine whether the confidence level is below an acceptable stability level. If the confidence level is too low, it indicates that the stable semantic features can no longer reliably support positioning. These unstable features need to be screened out to prevent them from continuing to participate in subsequent positioning and causing errors, while also providing a clear target for map updates. Based on the positioning accuracy requirements of logistics and delivery navigation scenarios, a pre-set confidence threshold is used. A higher confidence threshold is set for core positioning features, including logistics warehouses and delivery stations, because these features have high positioning dependence and must be strictly ensured to be stable. For auxiliary positioning features, such as fixed roadside signs, a lower confidence threshold is set, because these features only assist in correction and the standard can be appropriately relaxed. The dynamically updated confidence level is compared with the corresponding confidence threshold, and stable semantic features with a confidence level lower than the confidence threshold are screened out and marked as unstable features. Confidence thresholds are set differently according to feature importance to avoid the mistaken deletion of core features or the omission of auxiliary features due to a one-size-fits-all screening, ensuring that the screened unstable features are all objects that cannot support positioning, while also clarifying the specific target of map updates.

[0127] If unstable features remain in the low-altitude 3D navigation map, they will lead to misuse during subsequent positioning, affecting positioning accuracy. Therefore, they need to be handled promptly, and the processing information recorded to provide clues for subsequent map updates, ensuring the update process is traceable and accurately executed. The prior stability score of the unstable feature is reset to zero to prevent it from being misused in semantic matching and fusion weight calculations. Then, all associated information of the unstable feature, including 3D coordinates, physical dimensions, and anti-interference attributes, is removed from the low-altitude 3D navigation map. Simultaneously, the processing information of the unstable feature is recorded, including the removal time and location range. The time needs to be accurate to the moment of processing, and the location range refers to the 3D coordinates of the unstable feature and its surrounding associated area, forming a processing record for the unstable feature. This timely zeroing of the prior stability score and removal of the unstable feature prevents interference with subsequent positioning. Detailed recording of processing information provides accurate update clues for subsequent map updates, avoiding regional omissions or feature misjudgments during updates, ensuring the update process is efficient and accurate.

[0128] Different numbers of unstable features reflect different map update needs. For example, a small number of scattered unstable features only require local updates, while a large number of concentrated unstable features require global updates. If a uniform update signal is used, it will lead to waste of resources or incomplete updates. That is, local problems should be addressed with global updates or global problems with local updates. Different map update signals should be generated according to the proportion of unstable features. The proportion of the number of unstable features removed to the number of stable semantic features in the current field of view should be statistically analyzed. If this proportion is low, it means that only a few semantic features are unstable and scattered in different areas. This indicates that only a local area of ​​the map needs to be updated, and a local update signal should be generated. The local update signal includes the local area range to be updated and the feature types for which data needs to be acquired. The local area range is determined based on the location range of the unstable features, while the feature types are determined based on the removed feature types.

[0129] If this proportion is high, meaning a large number of features are unstable and concentrated in a certain area, it indicates that there have been significant changes in that area of ​​the map, such as the expansion of a logistics park or road reconstruction. In this case, a global update signal is generated. In addition to the information from the local update signal, the global update signal adds suggestions on the area where data acquisition and fusion need to be re-executed, as well as the priority ranking of the original stable semantic features, so that core features can be restored first during the update. The generated map update signal is sent to the map management system to trigger the subsequent update process of the low-altitude 3D navigation map. Differentiated update signals are generated according to the proportion of unstable features to achieve on-demand updates and avoid resource waste and incomplete updates. Local signals focus on small-scale precise updates, while global update signals cover large-scale in-depth updates, ensuring that the low-altitude 3D navigation map can efficiently and accurately adapt to changes in the actual environment. The map update signal provides clear guidance for the subsequent data source supplementation and feature reconstruction of the low-altitude 3D navigation map, ensuring that the low-altitude 3D navigation map can continue to maintain accuracy and timeliness, and providing reliable map support for the subsequent logistics delivery navigation of UAVs.

Claims

1. A fusion positioning method for low-altitude three-dimensional navigation maps, characterized in that, include: The system acquires and fuses point cloud data and image data from low-altitude areas, performs semantic segmentation and instance recognition based on the fused data to extract stable semantic features, and determines a prior stability score for each stable semantic feature to generate a low-altitude 3D navigation map. The system acquires observation data from airborne sensors in real time, retrieves stable semantic features and prior stability scores from the low-altitude three-dimensional navigation map within the current field of view, and processes visual images through a convolutional neural network to identify real-time semantic features. The observation data includes visual images, satellite data, and attitude data. The PNP algorithm is used to match real-time semantic features with stable semantic features to generate semantic observation vectors and calculate matching confidence. The fusion weight of the semantic observation vectors is determined based on the product of prior stability score and matching confidence. The weighted semantic observation vectors are then fused with satellite data and attitude data to generate positioning results. Based on the localization results, the reprojection residuals of stable semantic features are calculated. The confidence of stable semantic features is dynamically updated by Bayesian update based on the reprojection residuals. When the confidence of stable semantic features is less than the confidence threshold, they are recorded as unstable features. The prior stability score of unstable features is set to zero and unstable features are removed. At the same time, a map update signal is generated.

2. The fusion positioning method for low-altitude three-dimensional navigation maps as described in claim 1, characterized in that, The determination of the prior stability score includes: The evaluation criteria are determined by the physical properties, physical size, historical change frequency, and frequency of identification of stable semantic features in low-altitude observation scenarios. By comparing the positioning effects of different weight combinations, the weight proportions of different evaluation criteria are determined, and the weighted sum is calculated according to the weight proportions to obtain the initial prior stable score. The initial prior stability score is corrected according to the type of stable semantic feature to determine the prior stability score of each stable semantic feature, and the stable semantic features are combined to generate a low-altitude 3D navigation map.

3. The fusion positioning method for low-altitude three-dimensional navigation maps as described in claim 2, characterized in that, The extraction of stable semantic features includes: Acquire point cloud data and image data of the low-altitude region at at least two time points, combine pre-classified semantic labels to perform spatial registration and fusion of point cloud data and image data, compare the fused data at different time points, identify pixel change areas and mark them as dynamic areas; Based on the fused data, three-dimensional regions with the same semantic attributes are segmented from point cloud data and geometric features are extracted. Texture and spectral features are extracted from image data. The geometric features are weighted and fused with texture and spectral features to identify instances and obtain an instance feature set. By combining spatial and temporal dimensions, the stability of the instance feature set is screened to generate candidate stable features. The anti-interference ability of the candidate stable features is evaluated to select and extract stable semantic features.

4. The fusion positioning method for low-altitude three-dimensional navigation maps as described in claim 3, characterized in that, The generated semantic observation vector includes: From the low-altitude 3D navigation map, stable semantic features of the same type as the real-time semantic features and located within the current field of view are called to construct a candidate matching set and obtain 3D coordinates and prior stability scores. Using the three-dimensional coordinates of stable semantic features and the two-dimensional coordinates of real-time semantic features as input, the PNP algorithm is used to calculate the initial pose value, and the matching pair is output based on the physical size as a constraint. Semantic observation vectors are generated based on matching pairs. These vectors include stable semantic features of successful matches, coordinate deviations of matching pairs, size matching degree, and the proportion of matching pairs.

5. The fusion positioning method for low-altitude three-dimensional navigation maps as described in claim 4, characterized in that, The real-time semantic features to be identified include: Real-time acquisition of observation data from airborne sensors, including visual images, satellite data, and attitude data; preprocessing of visual images, including image contrast enhancement and motion blur suppression. The preprocessed visual image is input into a convolutional neural network to extract global scene features, edge contours, and texture changes within the current field of view. Semantic anchors are constructed by combining the types of stable semantic features. These anchors guide the convolutional neural network to focus on regions of the same type as the stable semantic features in order to identify real-time semantic features.

6. The fusion positioning method for low-altitude three-dimensional navigation maps as described in claim 5, characterized in that, The fusion weights for determining the semantic observation vectors include: Match confidence is generated based on the coordinate deviation, scale matching degree, and proportion of matching pairs. The total number of real-time semantic features within the current field of view is counted. If the total number of features is greater than or equal to the number threshold, the prior stability score is normalized and corrected. The fusion weights of the semantic observation vectors are determined by the product of the prior stability score and the matching confidence, and the fusion weights are dynamically adjusted according to the spatial distribution entropy of real-time semantic features in the visual image.

7. The fusion positioning method for low-altitude three-dimensional navigation maps as described in claim 6, characterized in that, The generated location results include: Satellite data is filtered by signal strength through sliding window filtering, while global coordinates are preserved. Satellite weights are assigned based on signal strength. At the same time, attitude data is smoothed to obtain motion state, and attitude weights are assigned based on smoothing error. The weighted semantic observation vector is used as the main positioning constraint and fused with satellite data to obtain intermediate positioning results including global coordinates. The intermediate positioning results are then fused with attitude data to generate the initial positioning results. The initial positioning result is compared with the positioning deviation of a single data source. If the positioning deviation is greater than or equal to the deviation threshold, the process is reversed to adjust the fusion weights of the matching pair and the semantic observation vector to generate the positioning result.

8. The fusion positioning method for low-altitude three-dimensional navigation maps as described in claim 7, characterized in that, The generated map update signal includes: The dynamically updated confidence level is compared with the preset confidence threshold to filter out unstable features with a confidence level lower than the confidence threshold. Set the prior stability score of unstable features to zero, remove unstable features from the low-altitude 3D navigation map, and record the removal time and location range of unstable features. The map update signal, which includes both local and global update signals, is determined by the proportion of the number of unstable features removed to the number of stable semantic features in the current field of view.

9. The fusion positioning method for low-altitude three-dimensional navigation maps as described in claim 8, characterized in that, The reprojection residuals used to calculate stable semantic features include: Extract the real-time pose of the current airborne equipment from the positioning results, and call the three-dimensional coordinates of stable semantic features that overlap with the current field of view in the low-altitude three-dimensional navigation map; Based on real-time pose, the three-dimensional coordinates of stable semantic features are projected onto the two-dimensional plane of the visual image to obtain the theoretical projection coordinates, and the deviation of the theoretical projection coordinates is corrected based on the physical dimensions during the projection process. The distance between the theoretical projection coordinates and the two-dimensional coordinates of the real-time semantic features is calculated to obtain the basic residual value. The residual correction parameter is set according to the type of stable semantic features. The basic residual value is multiplied by the residual correction parameter to obtain the reprojection residual of the stable semantic features.

10. The fusion positioning method for low-altitude three-dimensional navigation maps as described in claim 9, characterized in that, The confidence level of the dynamically updated stable semantic features includes: The prior stable score of stable semantic features is used as the initial prior probability of Bayesian update, and the matching confidence is used as the initial weight of the likelihood function. The likelihood function is dynamically adjusted based on the magnitude of the reprojection residual. The temporary confidence of the stable semantic features is calculated by combining the adjusted likelihood function with the initial prior probability using Bayes' theorem. Based on the proportion of matching pairs in the semantic observation vector, the temporary confidence is decayed and corrected to obtain the confidence of the dynamically updated stable semantic features.

Images