Unmanned platform accurate positioning method based on image semantic feature extraction

Abstract

The invention discloses an unmanned platform accurate positioning method based on image semantic feature extraction, and relates to the technical field of unmanned platform positioning, and the method comprises the steps: analyzing an environment image through a lightweight semantic segmentation network, and generating a semantic stability graph containing four types of stability regions; generating a global reliability graph in combination with the local angular point response graph and extracting high-reliability key points; generating a feature descriptor fused with semantic information by adopting a lightweight network subjected to semantic enhancement training; establishing a high-confidence 2D-3D feature point corresponding relation through bidirectional nearest neighbor matching and semantic consistency verification; and finally, solving the six-degree-of-freedom pose of the unmanned platform based on a PnP algorithm under an RANSAC framework. Key points are extracted from a long-term stable region through semantic-guided feature detection, and matching is performed in combination with semantic-enhanced feature descriptors, so that interference of environmental dynamics and long-term appearance changes is effectively overcome, and the positioning precision, robustness and real-time performance in a complex large-scale scene are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of unmanned platform positioning technology, and in particular to a method for precise positioning of unmanned platforms based on image semantic feature extraction. Background Technology

[0002] In real-time pose perception systems such as autonomous driving, robot navigation, and augmented reality, the accuracy and efficiency of visual positioning directly determine the reliability of upper-level decisions. Currently, the industry commonly adopts a two-stage approach: offline mapping followed by online localization. The offline stage utilizes structure-of-motion (SOM) technology to construct a sparse 3D point cloud global map; the online stage extracts local image features, establishes the correspondence between 2D image features and 3D map points, and finally solves the camera pose using the PnP algorithm combined with a random sampling consistency optimization framework. This approach places three core requirements on local features: long-term appearance invariance, spatial uniformity, and controllable quantity.

[0003] However, existing technologies struggle to simultaneously meet these requirements. Traditional manually designed features, such as SIFT and ORB, rely on local gradient statistics. Under long-term appearance changes due to factors like lighting, seasons, and weather, the repeatability and discriminative power of these features significantly degrade. While subsequent end-to-end feature learning based on convolutional neural networks, such as SuperPoint and D2Net, have improved the ability to discriminate local textures, their training objectives typically focus only on the reliability of local textures, lacking consideration for the semantic stability of feature points. This results in a large concentration of extracted feature points in semantically unstable regions such as the sky, tree canopy, pedestrians, and vehicles—regions that are easily matched in the short term but whose appearance is prone to drastic changes in the long term. Consequently, the accuracy of the localization system experiences a precipitous decline over long-term operation.

[0004] To mitigate the negative impact of unstable features, some existing solutions attempt to introduce explicit semantic segmentation networks during the testing phase, filtering feature points using manually set stability weights. This approach not only increases the cascaded complexity and processing latency of the system, but more importantly, its filtering effect is highly sensitive to the accuracy of semantic segmentation. In low-visibility scenarios such as nighttime, rain, and fog, segmentation errors directly lead to a sharp decline in localization robustness. Furthermore, while recently proposed advanced matchers based on graph neural networks or dense correlations can improve matching accuracy, their computational complexity increases quadratically with the number of features. This enormous computational overhead makes it difficult to achieve real-time operation on resource-constrained automotive-grade or airborne embedded platforms.

[0005] In summary, the existing "local feature extraction + subsequent matching" approach faces an inherent contradiction in complex, large-scale real-world environments: it is difficult to balance long-term robustness, low processing latency, and lightweight system design. Summary of the Invention

[0006] The purpose of this invention is to provide a precise positioning method for unmanned platforms based on image semantic feature extraction. By extracting key points from long-term stable regions through semantically guided feature detection and matching them with semantically enhanced feature descriptors, this method solves the problems of poor feature reliability and low computational efficiency caused by relying on additional semantic segmentation networks in existing visual positioning methods under dynamic environmental changes and long-term appearance changes. This improves the accuracy, robustness and real-time performance of unmanned platforms in complex large-scale scenes.

[0007] To address the aforementioned technical problems, a first aspect of this invention provides a method for precise localization of unmanned platforms based on image semantic feature extraction, comprising the following steps: Acquire real-time environmental images collected by the visual sensors of the unmanned platform during the current detection cycle; The real-time environment image is parsed based on a lightweight semantic segmentation network to generate pixel-level semantic segmentation results, and the pixel-level semantic segmentation results are mapped into a semantic stability map containing four types of stability regions. Calculate the local corner response map of the real-time environment image, combine the local corner response map with the semantic stability map to generate a global reliability map, and extract a preset number of high reliability key points from the global reliability map; Based on a lightweight feature description network, a feature descriptor with fused semantic information is generated for each of the high-reliability key points. The feature descriptors are matched with the descriptor benchmark library, and the matching results are checked for bidirectional nearest neighbor and semantic consistency to obtain a high-confidence 2D-3D feature point correspondence. Based on the high-confidence 2D-3D feature point correspondence, the 6-DOF pose of the unmanned platform is calculated, and the precise positioning information of the unmanned platform is obtained based on the 6-DOF pose.

[0008] Furthermore, the real-time environment image is parsed using a lightweight semantic segmentation network to generate pixel-level semantic segmentation results, and these results are mapped to a semantic stability map containing four types of stable regions, including: The lightweight semantic segmentation network is used to parse the real-time environment image, outputting a semantic label for each pixel in the real-time environment image belonging to a preset category set, and generating a segmentation mask containing pixel-level semantic information. The segmentation mask constitutes the pixel-level semantic segmentation result. Based on the preset mapping rules, each semantic label in the pixel-level semantic segmentation result is classified into a stable region, and respectively classified into an unstable region, a dynamic region, a short-term localization region, or a long-term localization region. Based on the stability region class result corresponding to each pixel, and combined with the stability weight corresponding to the stability region, a semantic stability map with the same spatial dimension as the real-time environment image is generated. The value of each pixel in the semantic stability map is the stability weight corresponding to its category.

[0009] Furthermore, before generating a feature descriptor that fuses semantic information for each of the high-reliability key points based on a lightweight feature description network, the method further includes: Acquire historical environmental images collected by the visual sensor and construct the training dataset for the lightweight feature description network; Several historical environment images are obtained from the training dataset. Training triples are constructed based on the pixel-level semantic segmentation results. The training triples include anchor samples, positive samples, and negative samples, where positive samples and anchor samples belong to the same semantic category, and negative samples and anchor samples belong to different semantic categories. Construct an inter-class loss and use a triplet loss function with hard margin to process the training triplets, so that the distance between feature descriptors of different semantic categories in the feature embedding space is not less than the hard margin. Construct an intra-class loss and use a soft ranking loss function based on average precision to process feature descriptors belonging to the same semantic category, and optimize the relative distance ranking of similar features in the feature embedding space; The inter-class loss and the intra-class loss are weighted and summed according to preset dynamic adjustment weights to obtain the joint loss function; Based on the joint loss function, the trainable parameters of the lightweight feature description network are updated using the backpropagation algorithm until the model converges.

[0010] Further, the step of constructing training triples based on the pixel-level semantic segmentation results includes: From the current training batch, sample pairs with the highest feature similarity across categories are selected as difficult negative samples, and sample pairs with the lowest feature similarity within the same category are selected as difficult positive samples, which are used to construct the training triplet.

[0011] Further, updating the trainable parameters of the lightweight feature description network based on the joint loss function using the backpropagation algorithm includes: When the number of training iterations is less than a first preset threshold, the weight of the inter-class loss is configured to be greater than the weight of the intra-class loss. When the number of training iterations reaches the first preset threshold, the weight adjustment phase is initiated, and the weight of the intra-class loss is linearly increased according to the preset adjustment rate, while the weight of the inter-class loss is linearly decreased accordingly. When the number of training iterations reaches the second preset threshold, the weight adjustment phase ends, and the weight of the intra-class loss is fixed to a target value that is higher than the weight of the inter-class loss.

[0012] Furthermore, before updating the trainable parameters of the lightweight feature description network based on the joint loss function using the backpropagation algorithm, the method further includes: Using a pre-defined semantic segmentation network as the teacher network, historical environment images from the training dataset are input into the teacher network, and the output of a pre-defined intermediate layer is extracted as teacher features. Extract the intermediate layer output of the lightweight feature description network encoder that corresponds to the specified intermediate layer of the teacher network as the student feature; Calculate the L1 norm between the teacher features and the student features, and use it as the feature consistency loss; The feature consistency loss is added to the joint loss function to form the overall training loss, which is used instead of the joint loss function for backpropagation and parameter update of the lightweight feature description network.

[0013] Further, the step of calculating the local corner response map of the real-time environment image and combining the local corner response map with the semantic stability map to generate a global reliability map includes: The initial corner response of the real-time environment image is calculated based on the corner detection algorithm, and an initial local response map is generated. The initial corner response map is transformed and optimized using a lightweight convolutional neural network to obtain the local corner response map; The global reliability map is obtained by performing pixel-by-pixel element-wise multiplication between the local corner response map and the semantic stability map.

[0014] Furthermore, before performing pixel-by-pixel element-wise multiplication on the local corner response map and the semantic stability map, the method further includes: Boundary region analysis is performed on the semantic stability map to identify the semantic region contours belonging to dynamic objects; Based on the semantic region contour of the dynamic object, calculate the Euclidean distance from each pixel to the nearest contour point and generate a distance mapping map. Based on a Gaussian function with a preset standard deviation parameter, each distance value in the distance mapping map is mapped to a corresponding weight value, generating a Gaussian decay weight map. The weight value decays as the corresponding distance value increases in a Gaussian function relationship. The semantic stability map is multiplied pixel by pixel by the Gaussian decay weight map to generate the optimized semantic stability map.

[0015] Further, the step of matching the feature descriptor with a descriptor benchmark library and performing bidirectional nearest neighbor and semantic consistency checks on the matching results to obtain a high-confidence 2D-3D feature point correspondence includes: Perform bidirectional nearest neighbor matching. For each feature descriptor in the real-time environment image, search for the reference descriptor with the closest Euclidean distance in the descriptor reference library, and verify whether the nearest neighbor of the reference descriptor in the real-time environment image is the feature descriptor used as the query item. Feature point pairs that satisfy the bidirectional nearest neighbor condition are formed into preliminary candidate matching pairs. Geometric discrimination is used to screen the preliminary candidate matching pairs. The ratio of the nearest neighbor distance to the second nearest neighbor distance in each candidate matching pair is calculated, and candidate matching pairs with a ratio exceeding a preset threshold are removed. Semantic consistency verification is performed on the candidate matching pairs filtered by geometric discrimination. The semantic categories of the two-dimensional feature points from the real-time environment image and the three-dimensional feature points from the descriptor benchmark library in the candidate matching pairs are consistent. Candidate matching pairs with inconsistent semantic categories are eliminated. Based on the values ​​corresponding to the high reliability key points in the global reliability map, the candidate matching pairs that pass the semantic consistency check are sorted, and the candidate matching pairs corresponding to the high values ​​are selected to establish the correspondence between the high-confidence 2D-3D feature points.

[0016] Furthermore, calculating the 6-DOF pose of the unmanned platform includes: In the high-confidence 2D-3D feature point correspondence, the RANSAC algorithm framework is used for iterative sampling and verification. In each iteration, a minimum subset of feature point correspondence is randomly selected, and the candidate pose is calculated by combining the PnP algorithm with the real-time state parameters of the vision sensor. For each candidate pose, determine the reprojection error of all 2D-3D feature point correspondences under the candidate pose, and mark the feature point correspondences with reprojection errors less than a preset threshold as interior points. After a preset number of iterations, the set of interior points corresponding to the candidate pose with the most interior points is selected as the set of interior points with the greatest consistency. Based on the maximum consistent interior point set, the 6-DOF pose of the unmanned platform is calculated.

[0017] Accordingly, a second aspect of the present invention provides a precise positioning system for an unmanned platform based on image semantic feature extraction, comprising: The data acquisition module is used to acquire real-time environmental images collected by the visual sensors of the unmanned platform during the current detection cycle; The image parsing module is used to parse the real-time environment image based on a lightweight semantic segmentation network, generate pixel-level semantic segmentation results, and map the pixel-level semantic segmentation results into a semantic stability map containing four types of stability regions. The key point extraction module is used to calculate the local corner response map of the real-time environment image, combine the local corner response map with the semantic stability map to generate a global reliability map, and extract a preset number of high reliability key points from the global reliability map. The feature extraction module is used to generate a feature descriptor that integrates semantic information for each of the high-reliability key points based on a lightweight feature description network. The data matching module is used to match the feature descriptors with the descriptor benchmark library, and to perform bidirectional nearest neighbor and semantic consistency verification on the matching results to obtain a high-confidence 2D-3D feature point correspondence. The positioning calculation module is used to calculate the 6-DOF pose of the unmanned platform based on the high-confidence 2D-3D feature point correspondence, and obtain the accurate positioning information of the unmanned platform based on the 6-DOF pose.

[0018] Accordingly, a third aspect of the present invention provides an electronic device, including: at least one processor; and a memory connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to cause the at least one processor to perform the above-described method for precise positioning of unmanned platforms based on image semantic feature extraction.

[0019] Accordingly, a fourth aspect of the present invention provides a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the above-described method for precise positioning of unmanned platforms based on image semantic feature extraction.

[0020] The above-described technical solutions of the embodiments of the present invention have the following beneficial technical effects: 1. By transforming pixel-level semantic information of real-time environmental images into stability weights and using these weights to guide the extraction and selection of feature points, the problem of traditional visual features being prone to failure under seasonal, lighting changes and dynamic object interference is fundamentally solved. Feature points are actively concentrated in long-term stable areas such as buildings and traffic lights, while extraction in volatile areas such as the sky and vehicles is suppressed, thereby ensuring the repeatability and stability of feature points in the time dimension. This enables unmanned platforms to obtain continuous and reliable positioning capabilities even when experiencing long-term environmental changes or facing instantaneous dynamic interference. 2. By introducing a feature description network jointly trained with inter-class and intra-class losses, an enhanced descriptor incorporating semantic information is generated. This descriptor ensures clear separation between features of different categories of objects in the feature embedding space, while optimizing the relative distribution of features within the same category of objects, thus giving the feature descriptor stronger semantic discriminative ability. This significantly improves the accuracy and uniqueness of feature matching, effectively reduces the false matching rate, provides a data foundation with higher confidence for subsequent pose solving, and directly improves the absolute accuracy of the localization results. 3. By embedding a semantic guidance mechanism into the feature extraction front end and employing a lightweight network structure for feature detection and description, a highly integrated, end-to-end localization solution is constructed. While ensuring performance, it eliminates the need to call independent semantic segmentation networks or complex graph matchers during the inference stage, greatly reducing the computational complexity of the system and the latency caused by inter-module coupling. This enables the entire precise localization method to achieve stable and efficient real-time operation on resource-constrained embedded computing units such as vehicle-mounted and airborne systems, demonstrating strong engineering application value. Attached Figure Description

[0021] Figure 1 This is a flowchart of the precise positioning method for unmanned platforms based on image semantic feature extraction provided in this embodiment of the invention; Figure 2 This is a flowchart of feature point extraction provided in an embodiment of the present invention; Figure 3 This is a feature training flowchart provided in an embodiment of the present invention; Figure 4 This is a flowchart of descriptor generation provided in an embodiment of the present invention; Figure 5 This is a flowchart of pose calculation provided in an embodiment of the present invention. Detailed Implementation

[0022] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments and the accompanying drawings. It should be understood that these descriptions are merely exemplary and not intended to limit the scope of the invention. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0023] Please refer to Figure 1 The first aspect of this invention provides a method for precise localization of an unmanned platform based on image semantic feature extraction, comprising the following steps: Step S100: Obtain real-time environmental images collected by the unmanned platform's visual sensors during the current detection cycle.

[0024] In practice, the global shutter industrial camera mounted on the front end of the unmanned platform continuously acquires RGB images with a resolution of 1280×960 at a frame rate of no less than 30Hz. After the images are acquired, they are immediately sent to the preprocessing module, where size normalization to a uniform resolution, brightness enhancement based on histogram equalization, and lens distortion correction according to the camera calibration parameters are performed in sequence to eliminate optical errors and provide standardized image input for subsequent processing.

[0025] Step S200: The real-time environment image is parsed based on a lightweight semantic segmentation network to generate pixel-level semantic segmentation results, and the pixel-level semantic segmentation results are mapped into a semantic stability map containing four types of stability regions.

[0026] The UpperNet-ConvNeXt lightweight network with only 7M parameters is used as the semantic segmentation engine. This network has been pre-trained on the ADE20K dataset. After performing forward inference on the input image, the network outputs 150 pixel-level semantic labels to form the initial segmentation mask. Then, through a pre-defined fixed mapping table, the 150 fine categories are classified into four macroscopic stability regions: transient texture regions such as sky and water are classified as unstable regions and assigned a stability weight of 0.1; moving targets such as vehicles and pedestrians are classified as dynamic regions and assigned a weight of 0.2; seasonally changing objects such as vegetation and grass are classified as short-term localization regions and assigned a weight of 0.7; and structurally stable objects such as buildings and traffic lights are classified as long-term localization regions and assigned the highest weight of 1.0. Finally, a semantic stability map with the same resolution as the input image is generated, and the value of each pixel in the map is the weight value of the stability category to which the corresponding position belongs.

[0027] Step S300: Calculate the local corner response map of the real-time environment image, combine the local corner response map with the semantic stability map to generate a global reliability map, and extract a preset number of high reliability key points from the global reliability map.

[0028] First, the SuperPoint corner detection algorithm is run to obtain initial corner responses, forming an initial local response map. Then, a lightweight convolutional neural network containing three residual modules is used to deeply optimize this response map, enhancing the response values ​​of textured regions through feature transformation while suppressing the response values ​​of repetitive textured regions, resulting in an optimized local corner response map. This local corner response map is then multiplied pixel-wise with the semantic stability map. During this process, a Gaussian decay constraint with a standard deviation σ=5 is applied to the boundary regions of dynamic objects, generating a global reliability map through element-wise multiplication. Finally, a non-maximum suppression algorithm is executed on this map, and the top K highest score points are retained as high reliability key points, where K is set to 1024 or 2048 depending on the computing resources.

[0029] Step S400: Based on a lightweight feature description network, generate a feature descriptor that integrates semantic information for each high-reliability key point.

[0030] A lightweight network structure based on residual blocks is adopted. This network takes the feature map downsampled by 8 times as input, performs feature transformation through 3 residual blocks, and outputs a 128-dimensional feature descriptor. The network acquires semantic awareness through a special training mechanism: during the training phase, hard-margin triplet loss is used to force descriptors of different semantic categories to separate in the feature embedding space, and soft ranking loss based on average precision is combined to optimize the internal relative distance of similar features. In addition, a knowledge distillation mechanism is introduced to align the features of the intermediate layer of the network with the corresponding layer features of the teacher network according to the L1 norm, which further enhances the semantic expressive ability. After training, the network can generate feature descriptors with both texture discrimination and semantic consistency for each key point.

[0031] Step S500: Match the feature descriptor with the descriptor benchmark library, and perform bidirectional nearest neighbor and semantic consistency verification on the matching results to obtain a high-confidence 2D-3D feature point correspondence.

[0032] First, bidirectional nearest neighbor matching is performed: for each feature descriptor in the current frame, the nearest Euclidean descriptor is found in the descriptor benchmark library, and the nearest neighbor of the benchmark descriptor in the current frame is verified in reverse to see if it is the original query descriptor. Only bidirectional consistent matching pairs are retained. Then, geometric discrimination filtering is performed, and the ratio of the farthest neighbor distance to the second nearest neighbor distance of each matching pair is calculated. Fuzzy matches with a ratio exceeding the empirical threshold of 0.8 are removed. Next, semantic consistency verification is performed, and the semantic category labels of the feature points in the current frame and the 3D feature points in the benchmark library are compared. Matching pairs with inconsistent categories are removed. Finally, the remaining matching pairs are sorted according to the keypoint scores in the global reliability graph, and matching pairs with high reliability scores are selected for subsequent calculations.

[0033] Step S600: Based on the high-confidence 2D-3D feature point correspondence, calculate the 6-DOF pose of the unmanned platform, and obtain the accurate positioning information of the unmanned platform based on the 6-DOF pose.

[0034] First, iterative sampling is performed within the RANSAC framework: in each iteration, four pairs of 2D-3D feature point correspondences are randomly selected as the minimum subset, and candidate poses are calculated using the EPnP algorithm combined with camera intrinsic parameters. For each candidate pose, the reprojection error of all feature point correspondences under that pose is calculated, and correspondences with errors less than 2 pixels are marked as inliers. After a preset 200 iterations, the set of inliers corresponding to the candidate pose with the most inliers is selected as the maximum consistency inlier set. Finally, based on this inlier set, a nonlinear optimization method is used to accurately solve the 6-DOF pose of the unmanned platform, including the three-dimensional position coordinates and three-axis attitude angles. The pose sequence is then temporally smoothed using a Kalman filter, ultimately outputting stable and reliable accurate positioning information.

[0035] In complex, large-scale environments, the semantically guided feature detection mechanism of this invention effectively suppresses feature extraction in dynamic and transient regions, significantly improving the long-term stability of feature points. The semantically enhanced feature descriptor, trained through joint loss, achieves stronger class discrimination capabilities, greatly reducing the false matching rate. The lightweight network design and end-to-end processing flow significantly reduce computational complexity while ensuring positioning accuracy, enabling the entire system to achieve real-time processing at over 30Hz on embedded platforms. This provides reliable technical support for the stable operation of unmanned platforms in urban scenarios, suburban environments, and even low-light conditions at night.

[0036] like Figure 2 As shown, in step S200, a lightweight semantic segmentation network is used to parse the real-time environment image, generate pixel-level semantic segmentation results, and map the pixel-level semantic segmentation results into a semantic stability map containing four types of stable regions, including: Step S210: The real-time environment image is parsed using a lightweight semantic segmentation network, and the semantic label of each pixel in the real-time environment image belonging to the preset category set is output. A segmentation mask containing pixel-level semantic information is generated, and the segmentation mask constitutes the pixel-level semantic segmentation result.

[0037] The UpperNet-ConvNeXt network, pre-trained on the ADE20K dataset, is used as the core of semantic parsing. This network extracts image features layer by layer through an encoder-decoder architecture and restores spatial resolution through upsampling during the decoding stage. Finally, a classification layer assigns a specific semantic label from 150 predefined categories to each pixel. After the network completes forward inference, it outputs a two-dimensional matrix with the same size as the input image. This matrix is ​​the segmentation mask, where each element stores the semantic category number of the corresponding pixel. This segmentation mask constitutes the pixel-level semantic segmentation result required for subsequent processing.

[0038] Step S220: Based on the preset mapping rules, each semantic label in the pixel-level semantic segmentation result is classified into a stable region, and then classified into an unstable region, a dynamic region, a short-term localization region, or a long-term localization region.

[0039] Category mapping is achieved through a lookup table, which is pre-set and embedded in the program during system initialization. The 150 fine-grained categories from the ADE20K dataset are categorized according to their spatiotemporal stability characteristics in long-term visual localization tasks: categories with rapidly changing appearances, such as clouds and water surfaces, are mapped as unstable regions; categories with moving characteristics, such as pedestrians and cars, are mapped as dynamic regions; categories with seasonal appearance changes, such as trees and shrubs, are mapped as short-term localization regions; and categories with stable structures, such as building facades, traffic signs, and streetlights, are mapped as long-term localization regions. This classification process is completed by performing a lookup operation on each semantic label in the segmentation mask, generating a region classification map of the same size as the input image.

[0040] Step S230: Based on the stability region class result corresponding to each pixel, and combined with the stability weight corresponding to the stability region, a semantic stability map with the same spatial dimension as the real-time environment image is generated. The value of each pixel in the semantic stability map is the stability weight corresponding to its category.

[0041] Four types of stable regions are assigned quantized stability weight values: long-term localization region weight is set to 1.0, short-term localization region weight is set to 0.7, dynamic region weight is set to 0.2, and unstable region weight is set to 0.1. When generating the semantic stability map, each pixel in the region classification map obtained in step S220 is mapped to its corresponding stability weight value. The resulting semantic stability map is a single-channel floating-point matrix with the same resolution as the input image. The value of each element in the matrix is ​​the stability weight of the pixel at that location. This weight matrix will serve as an important basis for keypoint selection in the subsequent feature extraction process.

[0042] The above steps transform the original image into a semantic stability map with clear physical meaning. This map retains pixel-level spatial accuracy and incorporates prior knowledge of stability based on semantic understanding, providing reliable semantic guidance for subsequent feature point selection. This ensures the stability and reliability of feature point extraction from the source and lays a solid foundation for the long-term robustness of the entire localization system.

[0043] Further, in step S300, the calculation of the local corner response map of the real-time environment image, combined with the semantic stability map, generates a global reliability map, including: Step S310: Calculate the initial corner response of the real-time environment image based on the corner detection algorithm, and generate an initial local response map.

[0044] The SuperPoint corner detection algorithm is used as the basic detector. This algorithm directly predicts the probability of corners at various locations in the image through a convolutional neural network with an encoder-decoder structure. The network first extracts features from the input image using a VGG-style encoder, and then predicts the corner positions and descriptors through two independent decoder heads. In the corner prediction branch, the network outputs a heatmap of the same size as the input image, where each pixel value represents the confidence score that the position is a corner. This output serves as the initial corner response map, reflecting the distribution of corners in the image based on local texture features, and provides basic data for subsequent optimization.

[0045] Step S320: Perform feature transformation and optimization on the initial corner response map based on a lightweight convolutional neural network to obtain a local corner response map.

[0046] A specially designed lightweight optimized network is employed, which contains three cascaded residual modules, each consisting of two 3×3 convolutional layers and skip connections. The network receives an initial corner response map as input and learns the spatial context of corner responses through multi-layer convolutional operations. During feature transformation, it enhances the response values ​​of salient texture regions while suppressing the responses of repetitive textures and low-quality corner regions. The local corner response map output after network optimization significantly improves the discrimination ability of corner quality while maintaining the spatial distribution of corners, enabling truly salient corners to obtain higher response scores.

[0047] Step S330: Perform pixel-by-pixel element-wise multiplication on the local corner response map and the semantic stability map to obtain the global reliability map.

[0048] First, the two images are spatially aligned and resolution matched to ensure a one-to-one correspondence between each pixel position. During the multiplication operation, the local corner response map provides a corner quality score based on texture saliency, while the semantic stability map provides a region stability prior based on semantic understanding. After multiplication, corners located in long-term stable semantic regions (such as building facades) will retain their high response values, while the response values ​​of corners located in dynamic or unstable semantic regions (such as pedestrians and vehicles) will be suppressed. In particular, for dynamic object boundary regions, a Gaussian decay constraint with a standard deviation of σ=5 is applied to further smooth the response value changes in transition regions. The final generated global reliability map integrates texture features and semantic information, providing a more comprehensive and reliable evaluation basis for key point selection.

[0049] Furthermore, such as Figure 3 As shown, before performing pixel-by-pixel element-wise multiplication of the local corner response map and the semantic stability map in step S330, the following steps are also included: Step S330a: Perform boundary region analysis on the semantic stability map to identify the semantic region contours belonging to dynamic objects.

[0050] First, the semantic stability map is divided into regions, and all connected components marked as dynamic regions are extracted. Then, an edge detection algorithm is performed on each dynamic region component, using the Canny edge detector combined with morphological closing operations to obtain the complete region boundary contour. For adjacent dynamic regions, their spatial relationship is calculated, and if the distance is less than a set threshold, contour merging is performed. The final output dynamic object semantic region contour is a closed polygon composed of a series of continuous pixels. These contours accurately define the precise boundary positions of all dynamic objects (such as vehicles, pedestrians, etc.) in the image.

[0051] Step S330b: Based on the semantic region contour of the dynamic object, calculate the Euclidean distance from each pixel to the nearest contour point and generate a distance mapping map.

[0052] A distance transformation algorithm is used to process the contour data of dynamic objects. First, the contour points are marked as zero-distance points. Then, a fast distance transformation algorithm based on vector propagation is used to calculate the precise Euclidean distance from each pixel in the image to the nearest contour point. During the calculation, negative distance values ​​are assigned to pixels inside the contour, and positive distance values ​​are assigned to pixels outside the contour, thus establishing a signed distance field. The generated distance map has the same spatial dimension as the original image. The value of each pixel in the map represents the geometric distance from that location to the nearest dynamic region edge. This distance information provides a spatial basis for subsequent Gaussian decay.

[0053] Step S330c: Based on the Gaussian function with preset standard deviation parameter, each distance value in the distance mapping map is mapped to a corresponding weight value, generating a Gaussian decay weight map. The weight value decays with the corresponding distance value in a Gaussian function relationship.

[0054] A Gaussian kernel function with a standard deviation of σ=5 is used to perform a nonlinear transformation on the distance map, mapping the distance value of each pixel to a decay weight between 0 and 1 through the Gaussian function. In the specific calculation, the boundary of the dynamic region (the position where the distance is zero) is used as the reference point, with a weight value of 1. As the absolute value of the distance increases, the weight value decays according to the Gaussian distribution curve. For a pixel with a distance value of d in the distance map, its corresponding decay weight is calculated as exp(-d² / 2σ²). The final Gaussian decay weight map forms a smooth transition region near the boundary of the dynamic object, avoiding abrupt changes in the weight value.

[0055] Step S330d involves performing a pixel-by-pixel multiplication operation between the semantic stability map and the Gaussian decay weight map to generate an optimized semantic stability map.

[0056] First, ensure that the two input images have the same spatial dimensions and pixel alignment, then perform pixel-by-pixel multiplication. During the operation, the stability weights near the boundaries of dynamic regions in the original semantic stability image are modulated by Gaussian decay weights, with the decay being greater the closer to the boundary. After modulation, the stability weights of the dynamic object boundary regions gradually transition from the original value of 0.2 along the boundary normal to unaffected region weight values, forming a smooth weight gradient. This processing method effectively eliminates the weight abrupt change problem caused by inaccurate semantic segmentation edges, making the extraction of feature points in these transition regions more reasonable.

[0057] The semantic stability map described above, while maintaining the original semantic partitioning, adds a special processing mechanism for the sidebands of dynamic regions. This enables the subsequent feature point extraction process to more accurately avoid the influence of dynamic objects, while ensuring a smooth transition of stability weight changes. This provides more accurate and robust semantic prior information for the generation of the global reliability map, thereby significantly improving the rationality of feature point selection and the adaptability of the entire localization system in dynamic environments.

[0058] Furthermore, such as Figure 4 As shown, before generating a feature descriptor with fused semantic information for each high-reliability key point using a lightweight feature description network in step S400, the following steps are also included: Step S401: Obtain historical environmental images collected by the visual sensor and construct a training dataset for a lightweight feature description network.

[0059] By systematically collecting a large number of historical environmental images acquired by the visual sensors of the unmanned platform during actual operation, these images cover scenes from different seasons, weather conditions, lighting environments, and geographical regions. All acquired images undergo rigorous quality screening, eliminating low-quality images that are blurry, overexposed, or underexposed, and retaining effective samples with clear textures and rich semantic information. The screened images are then subjected to standardized preprocessing, including scaling to a uniform resolution of 640×480, image channel normalization, and histogram equalization enhancement. Finally, a training dataset of sufficient scale and diverse scenes is constructed, providing ample data support for subsequent network training.

[0060] Step S402: Obtain several historical environment images from the training dataset, and construct training triples based on pixel-level semantic segmentation results. The training triples include anchor samples, positive samples, and negative samples, where positive samples and anchor samples belong to the same semantic category, and negative samples and anchor samples belong to different semantic categories.

[0061] A batch of images are randomly sampled from the training dataset. For each image, pixel-level semantic segmentation results are obtained through a pre-trained semantic segmentation network. Based on semantic label information, multiple key points are randomly selected from the same semantic category to construct positive sample pairs, and key points are selected from different semantic categories to construct negative sample pairs. Special attention is paid to the balance of sample difficulty during the construction process, including both easy-to-distinguish simple samples and difficult samples that are similar in appearance but different in semantics. The final training triplet contains anchor samples, positive samples, and negative samples. Positive samples and anchor samples share the same semantic category label, while negative samples come from different semantic categories, providing effective training samples for subsequent comparative learning.

[0062] Step S403: Construct inter-class loss and use a triplet loss function with hard margin to process the training triplets, so that the feature descriptors of different semantic categories maintain a distance of not less than the hard margin in the feature embedding space.

[0063] The triplet loss function with hard margin is used, which requires that the distance between feature descriptors of different semantic categories in the embedding space must be greater than a set hard margin value. Specifically, for each training triplet (anchor a, positive sample p, negative sample n), the loss function is calculated as max(0, D(a,p) - D(a,n) + margin), where D represents the Euclidean distance and the margin is set to 0.2. By optimizing this loss, the feature descriptors learned by the network are forced to clearly distinguish between different semantic categories, and sufficient feature distance can be maintained even when they are similar in appearance.

[0064] Step S404: Construct intra-class loss and use a soft ranking loss function based on average precision to process feature descriptors belonging to the same semantic category, and optimize the relative distance ranking of similar features in the feature embedding space.

[0065] A soft-ranking loss function based on average precision is adopted. This loss does not force similar features to cluster in a single cluster center, but optimizes the relative ranking relationship of similar features in the feature space. When calculating the loss function, for each anchor sample, its distance ranking with all other samples of the same type is calculated. The optimization objective is to rank positive samples with high similarity before samples with low similarity. By introducing a softmax function with a temperature coefficient to soften the ranking index, the loss function can provide a smoother gradient signal, promoting the compact distribution of similar features while maintaining a reasonable internal structure.

[0066] Step S405: The inter-class loss and intra-class loss are weighted and summed according to preset dynamic adjustment weights to obtain the joint loss function.

[0067] A dynamic weight adjustment strategy was designed. In the early stage of training, a higher weight (0.7) was assigned to the inter-class loss to ensure the separability of features of different classes. As the training progressed, the weight ratio of the intra-class loss was gradually increased to achieve a balance between the two types of losses (0.5:0.5) in the middle stage of training. In the later stage of training, the weight of the intra-class loss was further increased to 0.7 to optimize the discrimination ability of features of the same class. This dynamic weight scheduling mechanism is controlled by a preset threshold to ensure that the network can focus on the most critical optimization goal in different training stages and achieve globally optimal feature learning.

[0068] Step S406: Based on the joint loss function, update the trainable parameters of the lightweight feature description network through the backpropagation algorithm until the model converges.

[0069] The Adam optimizer is used to update network parameters, with an initial learning rate of 0.001 and a cosine annealing strategy for learning rate decay. In each training iteration, the feature descriptors obtained from forward propagation are calculated first, followed by the inter-class and intra-class losses, and then the joint loss is obtained by weighted summation according to dynamic weights. The gradient of the loss function with respect to the parameters of each layer of the network is calculated using the backpropagation algorithm, and gradient clipping is used to prevent gradient explosion. Finally, all trainable parameters of the network are updated. An early stopping mechanism is introduced during training, and training is terminated when the validation set loss no longer decreases for several consecutive epochs to ensure that a fully trained and overfitted final model is obtained.

[0070] Through the complete training process described above, the lightweight feature description network can learn feature representations that possess both strong discriminative power and semantic consistency. The generated descriptors not only maintain feature similarity among objects of the same type but also effectively distinguish objects of different semantic categories, providing a high-quality feature description foundation for subsequent feature matching and significantly improving the matching accuracy and robustness of the entire localization system. In particular, the dynamic weight adjustment strategy and hard sample mining mechanism introduced during training enable the network to adaptively balance different learning objectives, maintaining superior performance even in complex scenarios.

[0071] Further, step S402, which involves constructing training triples based on pixel-level semantic segmentation results, includes: Step S4021: From the current training batch, select sample pairs with the highest feature similarity across categories as difficult negative samples, and select sample pairs with the lowest feature similarity within the same category as difficult positive samples, to construct training triples.

[0072] First, initial feature descriptors are extracted from all samples in the current training batch, and a feature similarity matrix for the entire batch is constructed based on these feature descriptors. During the selection of difficult negative samples, each anchor sample in the batch is traversed, and the top K samples with the highest feature cosine similarity among its non-class samples are selected as candidate difficult negative samples. K is dynamically adjusted according to the batch size, typically set to 5%-10% of the batch size. During the selection of difficult positive samples, for each anchor sample, the M samples with the greatest feature Euclidean distance among its class samples are selected as candidate difficult positive samples. The value of M is balanced with the number of negative samples. In practice, the deep feature distance of all sample pairs is calculated, and an optimized k-nearest neighbor algorithm is used to quickly retrieve the nearest neighbors. Parallel computation is used to accelerate the construction of the similarity matrix. Finally, in the constructed training triplet, each anchor sample is combined with multiple difficult positive and difficult negative samples to form reinforced training sample pairs. These sample pairs effectively improve the model's ability to distinguish boundary cases.

[0073] By implementing a hard sample mining strategy, the training process can automatically focus on the most challenging sample combinations, forcing the feature description network to learn more discriminative feature representations. This mechanism significantly improves the model's feature discrimination ability in complex scenes, enabling the generated descriptors to maintain strong discriminativeness when faced with objects that are similar in appearance but different in meaning. At the same time, it enhances the feature stability of similar objects under different viewpoints and lighting conditions, laying a solid foundation for subsequent feature matching, thereby improving the robustness and accuracy of the localization system in real complex environments.

[0074] Further, in step S406, the trainable parameters of the lightweight feature description network are updated using the backpropagation algorithm based on the joint loss function, including: Step S4061: When the number of training iterations is less than the first preset threshold, configure the weight of the inter-class loss to be greater than the weight of the intra-class loss.

[0075] In the initial training phase, a first preset threshold of 50 training epochs was set. During this phase, the weight of the inter-class loss was set to 0.7, and the weight of the intra-class loss was set to 0.3. This weight allocation strategy is based on the fundamental principles of deep metric learning and is used to establish clear inter-class decision boundaries in the early stages of training to prevent confusion between features of different semantic categories in the embedding space. In practice, the training progress is monitored in real time through a training epoch counter. When the counter value is less than 50, the inter-class loss is always given dominance to ensure that the network first learns the basic ability to distinguish between different semantic categories, laying the foundation for subsequent fine-grained optimization.

[0076] Step S4062: When the number of training iterations reaches the first preset threshold, the weight adjustment phase is started. The weight of intra-class loss is linearly increased according to the preset adjustment rate, and the weight of inter-class loss is linearly decreased accordingly.

[0077] After 50 training epochs, the weight adjustment mechanism is activated, with an adjustment rate of 0.002 linearly per epoch. During this phase, the intra-class loss weight increases by 0.002 per training epoch from an initial value of 0.3, while the inter-class loss weight decreases by 0.002 per training epoch from an initial value of 0.7. This gradual adjustment strategy ensures the stability of the training process and avoids training oscillations caused by sudden changes in loss weights. The real-time values ​​of the two loss weights are maintained through precise floating-point arithmetic, and the weight parameters are updated before the start of each training batch to ensure that the latest weight configuration is used during forward and backward propagation.

[0078] Step S4063: When the number of training iterations reaches the second preset threshold, the weight adjustment phase ends, and the weight of the intra-class loss is fixed to a target value that is higher than the weight of the inter-class loss.

[0079] The second preset threshold is set to 150 training epochs. When the training epochs reach this threshold, the weight adjustment process is terminated. At this point, the intra-class loss weight has gradually increased to 0.5, and the inter-class loss weight has decreased to 0.5 accordingly. Subsequently, these two weight values ​​are fixed and maintained until the end of training to ensure that the compactness of intra-class features and the separability of inter-class features can be optimized equally in the later stages of training. This weight scheduling scheme achieves a training strategy that smoothly transitions the training focus from macro-level category separation to micro-level feature discrimination through precise threshold control and linear change mechanism.

[0080] By implementing dynamic weight scheduling, the training process can adaptively adjust and optimize the focus according to the characteristics of different stages, ensuring stability in the early stage of training and enabling fine-tuning in the later stage. This mechanism enables the feature description network to learn feature representations step by step, first establishing clear boundaries between categories, then optimizing the feature distribution within categories, and finally obtaining feature descriptors with both strong discriminative power and good generalization ability, providing important technical support for improving the performance of the entire localization system.

[0081] Furthermore, before updating the trainable parameters of the lightweight feature description network based on the joint loss function using the backpropagation algorithm in step S406, the following steps are also included: Step S406a: Using a preset semantic segmentation network as the teacher network, input historical environment images from the training dataset into the teacher network, and extract the output of the preset intermediate layer as teacher features.

[0082] The ConvNeXt network, which achieves state-of-the-art performance in scene segmentation tasks, is used as the teacher network with fixed weights. This network possesses strong semantic understanding capabilities and rich feature representation capabilities. In the specific implementation, historical environment images from the training dataset are input into the teacher network. During the forward propagation process, the intermediate layer outputs of the third and fourth stages of the network are accurately extracted as teacher features. These intermediate layer features contain rich spatial detail information and incorporate high-level semantic context information. To ensure feature quality, the extracted teacher features are standardized to have a distribution characteristic of zero mean and unit variance, providing standardized input for subsequent feature alignment.

[0083] Step S406b: Extract the intermediate layer output from the lightweight feature description network encoder that corresponds to the intermediate layer specified by the teacher network as the student feature.

[0084] During the forward propagation of the lightweight feature description network, the intermediate layer outputs corresponding to the encoder part and the teacher network are extracted simultaneously. To ensure the alignment of the feature space, the number of channels of the student network is first adjusted to the same dimension as the teacher feature through a 1×1 convolutional layer, and then the spatial resolution is adjusted to a consistent size using bilinear interpolation. During feature extraction, special attention is paid to intermediate layers that contain rich semantic information. These layers retain sufficient spatial details and have a certain semantic abstraction ability, enabling them to effectively learn from and compare with the teacher features.

[0085] Step S406c: Calculate the L1 norm between teacher features and student features, as the feature consistency loss.

[0086] First, the teacher and student features are flattened into feature vectors. Then, the absolute difference between the two is calculated element by element. Finally, the L1 norm loss is obtained by summing the absolute differences of all elements. During the calculation, the spatial dimension of the feature map is subjected to average pooling to reduce computational complexity while maintaining the global statistical properties of the features. To enhance the stability of training, a gradient truncation mechanism is introduced in the loss calculation to prevent gradient explosion caused by excessive feature differences. The final feature consistency loss reflects the degree of difference between the student network and the teacher network in semantic feature representation.

[0087] In step S406d, the feature consistency loss is added to the joint loss function to form the overall training loss, which is used to replace the joint loss function for backpropagation and parameter update of the lightweight feature description network.

[0088] The feature consistency loss is weighted and summed with the original joint loss function by a weight coefficient of 0.3 to form the final overall training loss. During backpropagation, the overall training loss simultaneously provides the feature description network with supervision signals from the task itself and semantic guidance signals from the teacher network. Through this joint optimization mechanism, the student network can learn the specific feature description task requirements and inherit the teacher network's strong semantic understanding ability. During the parameter update phase, only the parameters of the student network are updated, while the parameters of the teacher network remain fixed, ensuring the stability of the knowledge distillation process.

[0089] By implementing the knowledge distillation framework, the lightweight feature description network significantly improves its semantic feature expression capabilities while maintaining efficient computation. This enables the generated feature descriptors to not only have good texture discrimination but also incorporate deep semantic understanding, thereby exhibiting stronger robustness and generalization ability in complex real-world application scenarios and providing important technical support for the performance optimization of the entire positioning system.

[0090] Furthermore, such as Figure 5 As shown, step S500 involves matching the feature descriptors with the descriptor benchmark library and performing bidirectional nearest neighbor and semantic consistency checks on the matching results to obtain high-confidence 2D-3D feature point correspondences, including: Step S510: Perform bidirectional nearest neighbor matching. For each feature descriptor in the real-time environment image, search for the reference descriptor with the closest Euclidean distance in the descriptor reference library, and verify whether the nearest neighbor of the reference descriptor in the real-time environment image is the feature descriptor used as the query item. The feature point pairs that meet the bidirectional nearest neighbor condition are formed into preliminary candidate matching pairs.

[0091] First, a kd-tree index structure for the descriptor benchmark library is constructed to accelerate the nearest neighbor search process. For each 128-dimensional feature descriptor in the current frame, the benchmark descriptor with the smallest Euclidean distance is found in the benchmark library, and its corresponding 3D map point coordinates are recorded. Then, reverse verification is performed. The found benchmark descriptor is used as the query term, and its nearest neighbor is searched among all feature descriptors in the current frame. The match is considered valid only when the result obtained by the reverse search is consistent with the original query descriptor. This two-way verification mechanism can effectively eliminate ambiguous matches caused by feature similarity and ensure that each matching pair is consistent in both directions, thereby forming a reliable preliminary candidate matching set.

[0092] Step S520: Perform geometric discrimination screening on the preliminary candidate matching pairs, calculate the ratio of the nearest neighbor distance to the second nearest neighbor distance in each candidate matching pair, and remove candidate matching pairs whose ratio exceeds a preset threshold.

[0093] Using Lowe's ratio test principle, for each preliminary candidate matching pair, not only its nearest neighbor distance but also its second nearest neighbor distance is recorded, and the ratio of the two is calculated as a geometric discriminant index of matching quality. In practice, an empirical threshold of 0.8 is set for the ratio. When the ratio of the nearest neighbor distance to the second nearest neighbor distance of a matching pair exceeds this threshold, the matching is considered not geometrically significant enough and there may be multiple similar candidates, so it is eliminated. This screening process significantly reduces the number of fuzzy matches and retains those reliable matching pairs that are highly unique in the feature space.

[0094] Step S530: Perform semantic consistency verification on the candidate matching pairs filtered by geometric discrimination, check whether the semantic categories to which the two-dimensional feature points from the real-time environment image and the three-dimensional feature points from the descriptor benchmark library belong are consistent, and remove candidate matching pairs with inconsistent semantic categories.

[0095] Based on the pixel-level semantic labels provided by the semantic segmentation network, the semantic category information at both ends of each matching pair is compared. In practice, the semantic label corresponding to the feature point of the current frame is accessed, and the semantic category associated with the 3D feature point stored in the descriptor benchmark library is queried. When the two belong to the same semantic category, the match is considered to be semantically consistent; otherwise, it is discarded. This semantic-level verification can effectively identify and eliminate erroneous matches that have similar geometric features but actually belong to different semantic objects, significantly improving the semantic rationality of the matching results.

[0096] Step S540: Based on the values ​​corresponding to the high reliability key points in the global reliability map, sort the candidate matching pairs that have passed the semantic consistency check, and select the candidate matching pairs corresponding to the high values ​​to establish a high-confidence 2D-3D feature point correspondence.

[0097] First, the reliability score corresponding to each keypoint is obtained from the global reliability map. These scores are distributed between 0 and 1, reflecting the comprehensive quality of the keypoint in terms of semantic stability and texture saliency. Then, the candidate matching pairs that have passed the above screening are sorted in descending order according to these scores, and the matching pairs with the highest reliability scores are selected to participate in the subsequent pose calculation. In practice, a dynamic selection mechanism is set up to adaptively adjust the number of matching pairs participating in the pose solution according to the scene complexity. Typically, the top 200-300 matching pairs with the highest scores are selected in complex scenes, and the top 100-150 matching pairs are selected in simple scenes, to ensure that the accuracy is guaranteed while optimizing the computational efficiency.

[0098] Through the feature matching process, a progressive screening process from initial matching to high-confidence correspondence was achieved. The evaluation criteria were combined with three dimensions: geometric feature similarity, semantic consistency, and key point reliability. The final 2D-3D feature point correspondence has extremely high accuracy and robustness, providing high-quality input data for subsequent pose solving and ensuring that the entire positioning system can maintain stable and reliable performance in various complex environments.

[0099] Further, the calculation of the 6-DOF pose of the unmanned platform in step S600 includes: Step S610: In the high-confidence 2D-3D feature point correspondence, the RANSAC algorithm framework is used for iterative sampling and verification. In each iteration, a minimum subset of feature point correspondence is randomly selected, and the candidate pose is calculated by combining the PnP algorithm with the real-time state parameters of the visual sensor.

[0100] First, the maximum number of iterations of the RANSAC algorithm is set to 200. In each iteration, 6 pairs of matching points are randomly selected from the high-confidence 2D-3D feature point correspondences that have passed the previous screening to form a minimum subset. During the selection process, a uniform sampling strategy is adopted to ensure the diversity of spatial distribution and avoid sampling bias. For each sampling subset, the EPnP algorithm is called in combination with the camera intrinsic parameter matrix to calculate the candidate pose, where the camera intrinsic parameters include pre-calibrated parameters such as focal length, principal point coordinates, and distortion coefficients. In each iteration, the rotation matrix and translation vector of the calculated candidate pose are recorded to provide a basis for subsequent interior point verification.

[0101] Step S620: For each candidate pose, determine the reprojection error of all 2D-3D feature point correspondences under the candidate pose, and mark the feature point correspondences with reprojection errors less than a preset threshold as interior points.

[0102] For each candidate pose, it is applied to all 2D-3D feature point correspondences. The 3D map points are projected onto the 2D image plane through projection transformation. The Euclidean distance between the projected points and the actual feature points is calculated as the reprojection error. A threshold of 2 pixels is set for the reprojection error. When the reprojection error corresponding to a feature point is less than this threshold, it is marked as an inlier of the current candidate pose. During the error calculation process, a bidirectional error evaluation mechanism is adopted, considering both forward projection error and backward reprojection error to ensure the comprehensiveness and accuracy of the evaluation. Finally, the number of inliers and the quality score are calculated for each candidate pose.

[0103] Step S620: After a preset number of iterations, select the set of interior points corresponding to the candidate pose with the most interior points as the set of interior points with the greatest consistency.

[0104] After completing all 200 preset iterations, the statistical results of interior points for all candidate poses are comprehensively analyzed. This analysis considers not only the number of interior points but also the spatial distribution uniformity and geometric consistency of the interior point set. The interior point set corresponding to the candidate pose with the largest number of interior points and a reasonable spatial distribution is selected as the maximum consistency interior point set. If multiple candidate poses have similar numbers of interior points, the interior point quality scores are further compared, and the one with the highest comprehensive score is selected as the final result. This selection process ensures that the final interior point set has the best geometric consistency and statistical significance.

[0105] Step S630: Calculate the 6-DOF pose of the unmanned platform based on the maximum consistent inlier set.

[0106] The iterative reweighted least squares method is used to finely optimize the maximum consistency inlier set. First, the initial pose estimate is calculated using the EPnP algorithm with the selected inlier set. Then, based on the initial estimate, the Levenberg-Marquardt algorithm is used for nonlinear optimization to minimize the sum of reprojection errors of all inliers. During the optimization process, each inlier is assigned an adaptive weight based on its reprojection error. The inlier with the larger the error has, the smaller the weight, in order to reduce the interference of outliers on the final result. The final 6-DOF pose includes three translational degrees of freedom (X, Y, Z coordinates) and three rotational degrees of freedom (roll, pitch, yaw). This complete pose information accurately describes the spatial position and attitude of the unmanned platform relative to the world coordinate system at the current moment.

[0107] Based on the above process, the accurate 6-DOF pose can be robustly estimated from feature-matched data that may contain noise and outliers. The RANSAC framework effectively eliminates outlier interference, and nonlinear optimization ensures the accuracy of the final result. The entire pose solution process has good real-time performance while ensuring accuracy, providing reliable spatial state information for navigation decision-making and control of unmanned platforms. It is a key technology for achieving precise positioning.

[0108] Accordingly, a second aspect of the present invention provides a precise positioning system for an unmanned platform based on image semantic feature extraction, comprising: The data acquisition module is used to acquire real-time environmental images collected by the visual sensors of the unmanned platform during the current detection cycle; The image parsing module is used to parse real-time environmental images based on a lightweight semantic segmentation network, generate pixel-level semantic segmentation results, and map the pixel-level semantic segmentation results into a semantic stability map containing four types of stability regions. The key point extraction module is used to calculate the local corner response map of the real-time environment image, combine the local corner response map with the semantic stability map to generate a global reliability map, and extract a preset number of high reliability key points from the global reliability map. The feature extraction module is used to generate a feature descriptor that integrates semantic information for each high-reliability key point based on a lightweight feature description network. The data matching module is used to match feature descriptors with a descriptor benchmark library and perform bidirectional nearest neighbor and semantic consistency checks on the matching results to obtain high-confidence 2D-3D feature point correspondences. The positioning calculation module is used to calculate the 6-DOF pose of the unmanned platform based on the high-confidence 2D-3D feature point correspondence, and obtain the accurate positioning information of the unmanned platform based on the 6-DOF pose.

[0109] Accordingly, a third aspect of the present invention provides an electronic device, including: at least one processor and a memory connected to the at least one processor. The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, cause the at least one processor to perform the aforementioned method for precise localization of an unmanned platform based on image semantic feature extraction.

[0110] Accordingly, a fourth aspect of the present invention provides a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the above-described method for precise positioning of unmanned platforms based on image semantic feature extraction.

[0111] The embodiments of this invention aim to protect a method for precise localization of unmanned platforms based on image semantic feature extraction, which has the following effects: 1. By transforming pixel-level semantic information of real-time environmental images into stability weights and using these weights to guide the extraction and selection of feature points, the problem of traditional visual features being prone to failure under seasonal, lighting changes and dynamic object interference is fundamentally solved. Feature points are actively concentrated in long-term stable areas such as buildings and traffic lights, while extraction in volatile areas such as the sky and vehicles is suppressed, thereby ensuring the repeatability and stability of feature points in the time dimension. This enables unmanned platforms to obtain continuous and reliable positioning capabilities even when experiencing long-term environmental changes or facing instantaneous dynamic interference. 2. By introducing a feature description network jointly trained with inter-class and intra-class losses, an enhanced descriptor incorporating semantic information is generated. This descriptor ensures clear separation between features of different categories of objects in the feature embedding space, while optimizing the relative distribution of features within the same category of objects, thus giving the feature descriptor stronger semantic discriminative ability. This significantly improves the accuracy and uniqueness of feature matching, effectively reduces the false matching rate, provides a data foundation with higher confidence for subsequent pose solving, and directly improves the absolute accuracy of the localization results. 3. By embedding a semantic guidance mechanism into the feature extraction front end and employing a lightweight network structure for feature detection and description, a highly integrated, end-to-end localization solution is constructed. While ensuring performance, it eliminates the need to call independent semantic segmentation networks or complex graph matchers during the inference stage, greatly reducing the computational complexity of the system and the latency caused by inter-module coupling. This enables the entire precise localization method to achieve stable and efficient real-time operation on resource-constrained embedded computing units such as vehicle-mounted and airborne systems, demonstrating strong engineering application value.

[0112] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0113] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0114] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0115] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1The steps of the function specified in one or more boxes.

[0116] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for precise positioning of an unmanned platform based on image semantic feature extraction, characterized in that, The method comprises the following steps: acquiring a real-time environment image collected by a visual sensor of an unmanned platform in a current detection period; analyzing the real-time environment image based on a lightweight semantic segmentation network, generating a pixel-level semantic segmentation result, and mapping the pixel-level semantic segmentation result into a semantic stability map containing four types of stability regions; calculating a local corner point response map of the real-time environment image, combining the local corner point response map with the semantic stability map to generate a global reliability map, and extracting a preset number of high-reliability key points in the global reliability map; generating a feature descriptor with fused semantic information for each high-reliability key point based on a lightweight feature description network; matching the feature descriptor with a descriptor reference library, and performing bidirectional nearest neighbor and semantic consistency verification on the matching result to obtain a high-confidence 2D-3D feature point correspondence relationship; calculating a 6-degree-of-freedom pose of the unmanned platform based on the high-confidence 2D-3D feature point correspondence relationship, and obtaining accurate positioning information of the unmanned platform based on the 6-degree-of-freedom pose. 2.The method of claim 1, wherein, The analyzing the real-time environment image based on the lightweight semantic segmentation network, generating a pixel-level semantic segmentation result, and mapping the pixel-level semantic segmentation result into a semantic stability map containing four types of stability regions comprises: analyzing the real-time environment image using the lightweight semantic segmentation network, outputting a semantic label of each pixel point in the real-time environment image belonging to a preset category set, generating a segmentation mask containing pixel-level semantic information, and the segmentation mask constitutes the pixel-level semantic segmentation result; based on a preset mapping rule, classifying each semantic label in the pixel-level semantic segmentation result into a stability region, and respectively classifying into an unstable region, a dynamic region, a short-term positioning region, or a long-term positioning region; based on the corresponding stability region class result of each pixel point, combining the stability weight corresponding to the stability region, generating a semantic stability map with the same spatial dimension as the real-time environment image, and the value of each pixel point in the semantic stability map is the stability weight corresponding to its category.

3. The method of claim 2, wherein, Before the generating a feature descriptor with fused semantic information for each high-reliability key point based on a lightweight feature description network, the method further comprises: acquiring a historical environment image collected by the visual sensor, and constructing a training data set of the lightweight feature description network; acquiring a plurality of historical environment images in the training data set, constructing a training triple according to the pixel-level semantic segmentation result, the training triple containing an anchor sample, a positive sample and a negative sample, wherein the positive sample and the anchor sample belong to the same semantic category, and the negative sample and the anchor sample belong to different semantic categories; constructing an inter-class loss, processing the training triple by using a triple loss function with a hard margin, and keeping the distance between feature descriptors of different semantic categories in a feature embedding space not less than the hard margin; The intra-class loss is constructed, and a soft ranking loss function based on average precision is used to process feature descriptors belonging to the same semantic class, so as to optimize the relative distance ranking of the same class features in the feature embedding space; The intra-class loss and the inter-class loss are weighted and summed according to a preset dynamic adjustment weight to obtain a joint loss function; Based on the joint loss function, the trainable parameters of the lightweight feature description network are updated through a back propagation algorithm until the model converges.

4. The method of claim 3, wherein, The training triplets are constructed according to the pixel-level semantic segmentation result, including: From the current training batch, the sample pair with the highest feature similarity across classes is selected as a difficult negative sample, and the sample pair with the lowest feature similarity within the same class is selected as a difficult positive sample, which are used to construct the training triplets.

5. The method of claim 3, wherein, The joint loss function is based on the joint loss function, and the trainable parameters of the lightweight feature description network are updated through a back propagation algorithm, including: When the number of training iterations is less than a first preset threshold, the weight of the inter-class loss is greater than the weight of the intra-class loss; When the number of training iterations reaches the first preset threshold, a weight adjustment phase is started, the weight of the intra-class loss is linearly increased according to a preset adjustment rate, and the weight of the inter-class loss is correspondingly linearly reduced; When the number of training iterations reaches a second preset threshold, the weight adjustment phase ends, and the weight of the intra-class loss is fixed as a target value higher than the weight of the inter-class loss. 6.The method of claim 3, wherein, Before the joint loss function is based on the joint loss function, the trainable parameters of the lightweight feature description network are updated through a back propagation algorithm, further including: Using a preset semantic segmentation network as a teacher network, inputting historical environment images in the training data set into the teacher network, and extracting the output of a preset intermediate layer as a teacher feature; Extracting the output of the intermediate layer corresponding to the specified intermediate layer of the teacher network in the encoder of the lightweight feature description network as a student feature; Calculating the L1 norm between the teacher feature and the student feature as a feature consistency loss; Adding the feature consistency loss and the joint loss function to form a total training loss, which replaces the joint loss function for the back propagation and parameter update of the lightweight feature description network.

7. The method of claim 1, wherein, The local corner response map of the real-time environment image is calculated, and the global reliability map is generated by combining the local corner response map and the semantic stability map, including: Based on a corner detection algorithm, the initial corner response of the real-time environment image is calculated to generate an initial local response map; Based on a lightweight convolutional neural network, the initial corner response map is subjected to feature transformation and optimization to obtain the local corner response map; The local corner response map and the semantic stability map are subjected to element-level multiplication operation pixel by pixel to obtain the global reliability map. 8.The method of claim 7, wherein, Before the local corner response map and the semantic stability map are subjected to element-level multiplication operation pixel by pixel, further including: Performing boundary region analysis on the semantic stability map to identify the semantic region contour of the dynamic object; Based on the dynamic object semantic region contour, the Euclidean distance of each pixel to the nearest contour point is calculated to generate a distance map; mapping each distance value in the distance map into a corresponding weight value which is attenuated in a Gaussian function relationship with the increase of the corresponding distance value, to generate a Gaussian attenuation weight map based on a Gaussian function with a preset standard deviation parameter; performing pixel-by-pixel multiplication operation on the semantic stability map and the Gaussian attenuation weight map to generate an optimized semantic stability map. 9.The method of claim 1, wherein, The matching of the feature descriptor with the descriptor reference library and the bidirectional nearest neighbor and semantic consistency verification of the matching result to obtain a high-confidence 2D-3D feature point correspondence relationship, including: Performing bidirectional nearest neighbor matching, for each feature descriptor of the real-time environment image, finding the reference descriptor with the closest Euclidean distance in the descriptor reference library, and verifying whether the nearest neighbor of the reference descriptor in the real-time environment image is the feature descriptor as the query item, forming a preliminary candidate matching pair of feature points that meet the bidirectional nearest neighbor condition; Geometric discriminant screening of the preliminary candidate matching pair, calculating the ratio of the nearest neighbor distance to the second nearest neighbor distance in each candidate matching pair, and eliminating the candidate matching pair with a ratio exceeding a preset threshold; Semantic consistency verification of the candidate matching pair screened by the geometric discriminant, checking whether the semantic categories to which the two-dimensional feature points from the real-time environment image and the three-dimensional feature points from the descriptor reference library in the candidate matching pair belong are consistent, and eliminating the candidate matching pair with inconsistent semantic categories; According to the value corresponding to the high-reliability key point in the global reliability map, sorting the candidate matching pair that passes the semantic consistency verification, and selecting the candidate matching pair corresponding to the high value for establishing the high-confidence 2D-3D feature point correspondence relationship. 10.The method of claim 1-9, wherein, The calculation of the 6-degree-of-freedom pose of the unmanned platform, including: In the high-confidence 2D-3D feature point correspondence relationship, using the RANSAC algorithm framework for iterative sampling and verification, randomly selecting a minimum subset of feature point correspondence relationships in each iteration, and calculating a candidate pose using the PnP algorithm combined with the real-time state parameters of the visual sensor; For each candidate pose, determine the re-projection error of all 2D-3D feature point correspondence relationships under the candidate pose, and mark the feature point correspondence relationship with a re-projection error less than a preset threshold as an inlier; After a preset number of iterations, select the inlier set corresponding to the candidate pose with the most inliers as the maximum consistency inlier set; Based on the maximum consistency inlier set, calculate the 6-degree-of-freedom pose of the unmanned platform.

Images