An aircraft visual positioning method and system fusing semantic features and spatial geometric constraints

Abstract

The application discloses a kind of aircraft visual positioning method and system fusing semantic features and spatial geometric constraints, specifically: obtain the aerial image that aircraft camera gathers, the aerial image is carried out semantic segmentation, generates corresponding semantic segmentation single-channel mask image;While map is taken as reference map, corresponding semantic segmentation single-channel mask image is generated;Two semantic segmentation single-channel mask images are converted into several independent binary layers according to category;Based on this, find the matching area with the highest matching degree in reference map and aerial image;Obtain the attitude information and flight height information of aircraft, calculate the projection deviation caused by body tilt, realize the solution of aircraft absolute position.The application has good engineering practical value and popularization prospect.

Description

Technical Field

[0001] This invention belongs to the field of computer vision technology, and in particular relates to a method and system for visual localization of aircraft that integrates semantic features and spatial geometric constraints. Background Technology

[0002] Visual positioning technology, as a core technology for autonomous navigation and precise control of unmanned aerial vehicles (UAVs), has been widely applied in military and civilian fields such as aerial remote sensing, disaster monitoring, target reconnaissance, and logistics distribution in recent years. Its basic principle is to acquire real-time aerial images using an onboard camera and match them with a pre-built reference map (satellite remote sensing image or orthophoto map) to calculate the UAV's real-time geographical location. Compared to global satellite navigation systems, visual positioning technology has significant advantages such as strong autonomy, good anti-interference capabilities, and low cost. Especially in complex environments where satellite signals are denied or interfered with, visual positioning has become an important technical approach to ensure the autonomous navigation capabilities of UAVs.

[0003] The existing mainstream visual positioning methods can be broadly categorized into the following two types: (I) Visual localization method based on local feature point matching These methods extract salient local feature points from aerial images and baseline maps, construct feature descriptors, and perform feature matching. They then utilize the geometric transformation relationships between matched point pairs to achieve localization. Representative methods include scale-invariant feature transformation, accelerated robust feature mapping, and the Superpoint feature point detection and matching method based on deep learning, which has been used in recent years. The core technology of these methods relies on the gray-level gradient changes in local regions of the image—feature point detection and descriptor construction are both based on pixel-level gray-level gradient statistics.

[0004] However, this type of method has the following inherent technical drawbacks in practical engineering applications: First, the mathematical premise of the local feature point method heavily relies on rich and stable texture information in the image. When an aircraft flies in a complex low-altitude environment, factors such as drastic changes in lighting conditions causing shadow occlusion, degradation of ground surface texture, and repetitive texture scenes will severely damage local gray-level gradient information or generate a large number of ambiguous features. Under these conditions, the number of detected feature points decreases sharply or the distinguishability of feature descriptors decreases significantly, leading to matching failures or the generation of a large number of mismatched point pairs, resulting in a sharp increase in localization error or even failure to complete localization.

[0005] Secondly, the multi-scale matching capability is insufficient. There are significant resolution differences between aerial images acquired by the aircraft at different flight altitudes and the reference map. When dealing with large-scale changes, the scale invariance of the local feature point method is limited, making it difficult to maintain stable matching performance under cross-scale conditions.

[0006] Third, the uneven distribution of feature points leads to poor localization robustness. Feature point methods tend to cluster feature points in areas with rich texture, while feature points are sparse or even missing in areas with weak texture. This causes the matching results to be overly dependent on specific regions, and the overall localization performance drops sharply when local features are missing.

[0007] (II) Visual localization method based on template matching This type of method uses aerial images as templates and calculates the similarity between the template and candidate regions on a baseline map using a sliding window approach, selecting the location with the highest similarity as the matching result. This method directly utilizes the original pixel information of the image for similarity measurement, avoiding the information loss introduced during feature point extraction and descriptor construction.

[0008] Traditional template matching methods typically calculate similarity directly based on the grayscale or RGB color space of the original image. However, these methods are still severely limited by the underlying pixel features of the image: changes in illumination directly alter the distribution of pixel grayscale values, causing pixels belonging to the same ground feature to exhibit completely different grayscale characteristics under different lighting conditions, resulting in matching failure; the grayscale difference between shaded and unshaded areas can be several times, distorting the matching similarity score; in addition, template matching based directly on pixel grayscale is computationally intensive, and performing a sliding window search across the entire image requires significant computational overhead, making it difficult to meet the application requirements of real-time aircraft positioning.

[0009] In recent years, some researchers have attempted to combine semantic segmentation techniques with visual localization, converting aerial images into semantic category maps for matching, in order to overcome the impact of illumination variations and texture degradation on matching performance. However, existing semantic-based matching methods still have the following shortcomings: Firstly, in terms of semantic matching similarity calculation, existing methods typically treat multi-category semantic maps as a whole for similarity measurement, without fully considering the orthogonality of features between different semantic categories. Since different categories are not completely independent in spatial distribution, directly calculating the overall similarity can lead to cross-interference between dissimilar semantic features—for example, the response of the building category may contaminate the matching score of the adjacent road category, thereby introducing matching bias and reducing localization accuracy.

[0010] Secondly, most existing semantic matching methods only focus on the matching accuracy at the image level, neglecting the impact of aircraft attitude changes on positioning calculations. In actual flight, influenced by factors such as airflow disturbances and maneuvering, aircraft inevitably experience attitude changes such as roll and pitch, causing the camera's optical axis to deviate from the vertical direction, resulting in a projection deviation between the center point of the aerial image and the aircraft's ground position. When the flight altitude is high or the attitude angle is large, this deviation can reach tens or even hundreds of meters. Existing methods directly use the center point of the matching region as the basis for aircraft position calculation, ignoring this geometric deviation, leading to a significant decrease in positioning accuracy.

[0011] In summary, existing visual localization methods face several technical bottlenecks in complex low-altitude environments, including feature instability, poor scale adaptability, cross-interference from dissimilar semantic features, and a lack of attitude geometric compensation. These limitations make it difficult to meet the practical requirements of high-precision and robust autonomous navigation and localization for aircraft. Therefore, there is an urgent need for an aircraft visual localization method that can integrate high-level semantic features and spatial geometric constraints. This method would fundamentally overcome the dependence of traditional methods on low-level pixel gradient information and establish a complete attitude compensation mechanism to achieve high-precision and high-reliability localization in complex environments. Summary of the Invention

[0012] Purpose of the invention: In order to solve the problems existing in the prior art, the present invention provides a method and system for aircraft visual localization that integrates semantic features and spatial geometric constraints.

[0013] Technical solution: This invention provides a visual localization method for aircraft that integrates semantic features and spatial geometric constraints, including:

[0014] Aerial images captured by the aircraft's camera are acquired, and semantic segmentation is performed on these images to generate a single-channel semantic segmentation mask image. Simultaneously, a map is used as a reference image, and semantic segmentation is performed on the reference image to generate a single-channel semantic segmentation mask image of the reference image.

[0015] The single-channel mask image for semantic segmentation of aerial images is transformed into C independent binary layers of aerial images according to categories, where C is the total number of categories. The single-channel mask image for semantic segmentation of the baseline image is also transformed into several independent baseline images according to categories. Figure 2 Value-enhancing layers; extracting several baselines Figure 2 The baseline of the corresponding category in the binarized layer of the aerial image. Figure 2 Value-enhanced layers;

[0016] Calculate the similarity between C pairs of binarized layers, and find the region in the baseline image that has the highest matching degree with the aerial image as the matching region;

[0017] The system acquires the attitude and altitude information of the aircraft, calculates the projection deviation caused by the aircraft tilt, converts the pixel coordinates of the matching area into the geographic coordinates of the aircraft's ground point, and performs projection deviation compensation to achieve the calculation of the aircraft's absolute position.

[0018] Furthermore, the YOLO11-seg network is used for semantic segmentation of the image.

[0019] Furthermore, for the c-th category, if a pixel in the semantic segmentation single-channel mask image belongs to that category, the value of that pixel is set to 1; otherwise, it is set to 0, thus obtaining the binarized layer for the c-th category.

[0020] Furthermore, the size of the single-channel mask image for semantic segmentation of the aerial image is w×h, where w is the width and h is the height; from the top left corner of the single-channel mask image for semantic segmentation of the baseline image... To begin, select the matching region:

[0021] ;

[0022] in, This indicates that for category c, the value calculated using NCC is based on the baseline. Figure 2 Value-based layer coordinates Normalized correlation between the location and the corresponding location in the binarized layer of the aerial image; Represents the coordinates in the single-channel mask image for semantic segmentation of the baseline image. Pixel value at that location, Represents the coordinates in the single-channel mask image for semantic segmentation of aerial images. Pixel value at; Represents the coordinates of a single-channel mask image for semantic segmentation of the baseline image. The matching score is calculated between the location and the corresponding position in the single-channel mask image of the aerial image semantic segmentation. The matching scores of all coordinates in a region are added together to obtain the overall matching score for that region. The region with the highest overall matching score is selected as the matching region.

[0023] Furthermore, the expression for the projection bias is as follows:

[0024] ;

[0025] Where h is the altitude of the aircraft. This is the equivalent attitude angle.

[0026] Furthermore, the projection bias is expressed as the projection offset caused by the roll and pitch angles in the geographic coordinate system:

[0027] ;

[0028] in, and These represent the projection offsets caused by the roll and pitch angles in the geographic coordinate system, respectively.

[0029] ;

[0030] in, R represents the camera's height in the geographic frame, and R is the rotation matrix. Let be the optical axis vector of the camera coordinate system on the aircraft. and These represent the roll angle and pitch angle, respectively.

[0031] Calculate the absolute position of the aircraft:

[0032] ;

[0033] in, and The north and east coordinates of the aircraft at the coordinates of its ground point are respectively given. and Match the north and east coordinates of the center point of the region in the map coordinate system. For heading angle, .

[0034] An aircraft visual positioning system that integrates semantic features and spatial geometric constraints includes:

[0035] Image acquisition and processing module: used to acquire aerial images captured by the aircraft camera, perform semantic segmentation on the aerial images, and generate a single-channel mask image for semantic segmentation of the aerial images; at the same time, using the map as a reference map, performing semantic segmentation on the reference map, and generating a single-channel mask image for semantic segmentation of the reference map.

[0036] The binarization module is used to transform the single-channel mask image for semantic segmentation of aerial images into C independent aerial image binarization layers according to categories, where C is the total number of categories. It also transforms the single-channel mask image for semantic segmentation of the baseline image into several independent baseline images according to categories. Figure 2 Value-enhancing layers; extracting several baselines Figure 2 The baseline of the corresponding category in the binarized layer of the aerial image. Figure 2 Value-enhanced layers;

[0037] Matching Region Module: Used to calculate the similarity between C pairs of binarized layers, and to find the region with the highest matching degree with the aerial image in the baseline image as the matching region;

[0038] The position calculation module is used to acquire the attitude information and flight altitude information of the aircraft, calculate the projection deviation, convert the pixel coordinates of the matching area position into the geographic coordinates of the aircraft's ground point, and perform projection deviation compensation to realize the calculation of the aircraft's absolute position.

[0039] A computer device includes a memory, a processor, and a computer program stored in the memory and capable of running on the processor, wherein the processor executes the computer program to implement the steps of the aforementioned aircraft visual localization method that integrates semantic features and spatial geometric constraints.

[0040] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the aforementioned aircraft visual localization method that integrates semantic features and spatial geometric constraints.

[0041] Beneficial Effects: This invention fundamentally overcomes the dependence of traditional visual localization methods on low-level pixel gradient information through semantic feature extraction, significantly improving feature stability in complex environments. Existing visual localization methods rely heavily on stable and abundant local gray-level gradient information in images. When aircraft encounter uneven lighting, shadow occlusion, or texture degradation (such as weakly textured areas like water surfaces, snowfields, or uniform farmland) during low-altitude flight, the effectiveness of feature extraction drops sharply, and the matching failure rate increases significantly. This invention employs deep learning semantic segmentation networks such as YOLO11-seg to perform pixel-level semantic annotation on land cover categories such as roads, buildings, water systems, vegetation, and residential areas in aerial images, transforming low-level pixel gray-level information into high-level semantic category information. Since semantic features correspond to the geometric structure and category attributes of land cover, they remain stable under conditions of lighting changes and shadow occlusion, fundamentally overcoming the inherent defects of traditional feature point methods.

[0042] This invention proposes a multi-layer Normalized Cross-Correlation (NCC) strategy and a semantic binary decoupling mechanism, effectively eliminating cross-interference between dissimilar semantic features and significantly improving the reliability of semantic matching. The invention innovatively constructs a multi-layer NCC strategy, spatially decoupling a single-channel semantic map into multiple mutually orthogonal binary layers—independently constructing a binary mask for each semantic category (pixels belonging to that category are set to 1, otherwise to 0), and calculating NCC similarity on each independent dimension. This mechanism mathematically guarantees complete orthogonality between features of different categories, thoroughly eliminating cross-interference between dissimilar semantics, enabling the matching similarity score to truly reflect the degree of overlap between the template and the candidate region in the same semantic category, significantly enhancing the reliability of semantic image matching.

[0043] This invention constructs a spatial ray projection geometric model and utilizes the aircraft's inertial attitude and altitude information to compensate for projection deviations, achieving high-precision absolute position calculation. Most existing semantic matching methods only focus on image-level matching accuracy, neglecting the impact of aircraft attitude changes on the calculation of geographic coordinates of the sub-aircraft point. This invention further constructs a spatial ray projection geometric model based on this. Experimental data shows that when the flight altitude is 1000 meters and the equivalent rotation angle is 15°, the geometric deviation caused by fuselage tilt exceeds 200 meters. Directly ignoring this deviation would result in severe positioning errors. This invention utilizes real-time roll, pitch, and yaw angles and flight altitude information obtained from inertial devices, and uses the spatial ray projection model to inversely solve the pixel coordinates of the matching area into precise geographic coordinates, effectively compensating for the projection deviation. Experimental results show that the positioning error after geometric deviation compensation is concentrated within a small range, significantly better than positioning methods that ignore this deviation.

[0044] The method of this invention does not rely on global satellite navigation system signals and is applicable to autonomous positioning and navigation tasks of aircraft in environments where satellite navigation signals are limited or denied. It can perform navigation tasks well in complex low-altitude scenarios with uneven lighting, shadow occlusion, and texture degradation, and has good engineering practical value and promotion prospects. Attached Figure Description

[0045] Figure 1 This is a flowchart of the method of the present invention;

[0046] Figure 2 A schematic diagram illustrating the principle of multi-layer semantic NCC matching provided for the content of this invention;

[0047] Figure 3 This is an example diagram illustrating the matching effect under uneven lighting conditions provided in an embodiment of the present invention.

[0048] Figure 4 A flowchart of the overall positioning algorithm provided in an embodiment of the present invention;

[0049] Figure 5 This is a schematic diagram showing the effect of camera attitude angle on the projection position, where (a) is a diagram showing the positional relationship between the spatial ray and the geographic plane; and (b) is a diagram showing the projection relationship of the spatial ray in the geographic coordinate system.

[0050] Figure 6 The cumulative positioning error function curves of this method and the two existing methods are shown.

[0051] Figure 7 The diagrams show the matching results of the proposed method and two existing methods. (a) shows the matching results of the SURF feature point matching method, (b) shows the matching results of the Superpoint feature point matching method, and (c) shows the matching results of the proposed method. Detailed Implementation

[0052] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0053] like Figure 1 or Figure 4 As shown, this invention provides a visual localization method for aircraft that integrates semantic features and spatial geometric constraints, comprising the following steps:

[0054] Step 1: Acquire aerial images captured by the aircraft camera, and use a semantic segmentation network to extract the semantic features of ground features from the aerial images to generate a single-channel mask image for semantic segmentation of the aerial images;

[0055] The semantic segmentation network is a deep learning semantic segmentation or instance segmentation network, used to perform pixel-level semantic annotation of land cover categories such as roads, buildings, water systems, vegetation and residential areas, so as to enhance feature stability in scenarios with uneven lighting, shadow occlusion and texture degradation.

[0056] In this embodiment, a baseline map was acquired using orthophoto mapping. The map covers an area of ​​approximately 5 km × 1 km, with an image resolution of 49380 × 14746 pixels. The baseline map was cropped to a 750 × 750 pixel image using random cropping, and typical terrain features were marked in the image. The corresponding single-channel mask image for semantic segmentation of the baseline map was extracted using the same method.

[0057] This embodiment selects the YOLO11-seg network as the semantic segmentation network. To enable the YOLO11-seg network to accurately extract semantic features of ground objects, random flipping and brightness adjustment techniques are used during training to expand sample diversity. The training loss function includes a category classification loss based on binary cross-entropy loss (BCE Loss), a bounding box regression loss based on CIOU and DFL, and a semantic segmentation mask loss based on pixel-level BCE loss. These three losses are weighted and summed to obtain the final loss function. SGD is selected as the optimizer, and a cosine annealing strategy is used to dynamically adjust the learning rate until the network loss function converges, obtaining the optimal network weight model. The model of this invention effectively overcomes the influence of uneven illumination, shadow occlusion, and texture degradation in complex low-altitude environments on matching stability, significantly improving the success rate of image matching and the robustness of localization.

[0058] Step 2: Construct a multi-level Normalized Cross-Correlation (NCC) strategy. This strategy spatially decouples the single-channel semantic map into multiple mutually orthogonal binary layers for similarity calculation. This significantly enhances the reliability of semantic image matching while completely eliminating interference from cross-features of different types. For example... Figure 2 As shown, NCC similarity is calculated on each independent dimension to shield against cross-interference of heterogeneous semantics; sliding window matching is performed on templates of each scale in the semantic segmentation image corresponding to the base map, and the similarity score of multi-layer NCC evaluates the similarity between the template and the candidate region; thus, the candidate region with the highest similarity is selected as the final matching result to achieve stable cross-scale matching between aerial images and base maps.

[0059] The specific binarization method is as follows: for the c-th category, if a pixel in the semantic segmentation single-channel mask image belongs to that category, then the value of that pixel is set to 1; otherwise, it is set to 0, thus obtaining the binarized layer of the c-th category.

[0060] The single-channel mask image for semantic segmentation of aerial images is transformed into C independent binary layers of aerial images according to categories, where C is the total number of categories in the aerial images. The single-channel mask image for semantic segmentation of the baseline image is also transformed into several independent baseline images according to categories. Figure 2 Value-enhancing layers; extracting several baselines Figure 2 The baseline of the corresponding category in the binarized layer of the aerial image. Figure 2 Value-enhanced layers.

[0061] Let the single-channel mask image for semantic segmentation in the baseline image be I(x,y), and the template (single-channel mask image for semantic segmentation in aerial image) be T(x,y). The size of the template is w×h (w represents the width, and h represents the height). The region in the baseline image with (x,y) as the upper left corner is compared with the template. The multi-layer NCC calculation formula is as follows:

[0062] ;

[0063] in, This indicates that for category c, the value calculated using NCC is based on the baseline. Figure 2 Value-based layer coordinates Normalized correlation between the location and the corresponding location in the binarized layer of the aerial image; Represents the coordinates in the single-channel mask image for semantic segmentation of the baseline image. Pixel value at that location, Represents the coordinates in the single-channel mask image for semantic segmentation of aerial images. Pixel value at; Represents the coordinates of a single-channel mask image for semantic segmentation of the baseline image. The matching score is calculated between the location and the corresponding position in the single-channel mask image for semantic segmentation of the aerial image. The matching scores of all coordinates within a region are summed to obtain the overall matching score for that region. A higher score indicates a higher degree of matching between the location and the template. The region with the highest overall matching score is selected as the matching region. The matching region in this embodiment is as follows: Figure 3 As shown.

[0064] Step 3: Obtain the inertial attitude information and flight altitude information of the aircraft, construct a spatial ray projection geometric model, and solve the pixel coordinates of the matching area position into the geographic coordinates of the aircraft's ground point to realize the absolute position calculation of the aircraft.

[0065] To investigate the influence of flight altitude h and attitude angle on geometric deviation L, this embodiment uses a uniform random sampling method to generate experimental samples. Among them, attitude angle... Quantized using the equivalent rotation angle, the calculation formula is as follows:

[0066]

[0067] In the formula, , Let x be the rotation matrix about the roll axis and y be the rotation matrix about the pitch axis.

[0068] Assuming the aircraft's flight altitude does not exceed 1000 meters, the equivalent rotation angle does not exceed 15°, and the ground surface is approximately a plane, the formula for calculating the geometric deviation is:

[0069]

[0070] Geometric deviation L and equivalent attitude angle The geometric deviation is positively correlated with the flight altitude h, and exceeds 200 meters at a flight altitude of 1000 meters. Ignoring this deviation directly would lead to severe positioning errors. To correct this deviation, this invention utilizes real-time attitude angle and altitude information obtained from inertial devices to construct a spatial ray projection model. This model converts the pixel coordinates obtained from image matching into precise geographic coordinates, thereby achieving high-precision positioning estimation.

[0071] like Figure 5 As shown in (a) and (b), given the camera attitude information (including roll angle, pitch angle, and yaw angle), flight altitude, and the position information of the optical center projection point, the position information of the ground point can be deduced using spatial rays and geometric corrections. Let the optical axis vector of the camera coordinate system be... After rotation matrix Transforming to a geographic coordinate system, the derived formula is as follows:

[0072] ;

[0073] In the formula, The camera's altitude in the geographic system. and Let the projected offsets be the roll angle and the pitch angle, respectively, then the total projected offset is... The derivation formula based on the geographic coordinate system is as follows:

[0074] ;

[0075] In the formula, N UAV E UAV N IMG E IMG These are the north and east coordinates of the machine's ground point and the north and east coordinates of the optical center projection point (i.e., the center point of the matching region), respectively, in meters.

[0076] This implementation compares the YOLO11-seg network with mainstream semantic segmentation networks, including U-Net, DeepLabv3+, SegFormer, and YOLO8-seg. The model training results will be evaluated using the mean Intersection over Union (mIoU), which measures the accuracy of the model's inference. Specifically, it calculates the overlap between the predicted mask and the ground truth label mask, using the following formula:

[0077]

[0078] In the formula, k is the number of label categories, p ij This represents the number of cases where category i is inferred to be category j. From the changes in the relevant functions during the training process of each network, it can be concluded that the performance of the YOLO-based segmentation network is significantly better than other semantic segmentation networks. The matching effect under uneven lighting conditions is as follows... Figure 3 As shown.

[0079] like Figure 4 As shown in the overall localization algorithm flowchart, the results of the YOLO-based segmentation network mentioned above are evaluated using a normalized cross-correlation coefficient template to select matching regions. Subsequently, the spatial ray geometry model is substituted to compensate for the projection deviation (i.e., geometric deviation) caused by the aircraft tilt, and the latitude and longitude of the ground point corresponding to each frame of the image are calculated.

[0080] The experiment uses the statistical distribution of positioning error as an evaluation index to comprehensively analyze the positioning performance of the algorithm in actual flight trajectories, with satellite positioning results used as the ground truth for comparison. Statistical analysis of the performance indicators of various visual positioning methods shows that the positioning errors of existing mainstream visual matching algorithms in different scenarios typically range from several meters to over ten meters. Therefore, error tolerance thresholds of 5 meters and 10 meters are set to verify matching accuracy, respectively. Simultaneously, results with positioning errors greater than 20 meters are considered as mismatches for auxiliary verification, to validate the matching accuracy and effectiveness of the algorithm under existing technical standards. Experimental results show that the present invention can achieve good matching results in low-altitude scenarios that effectively handle uneven lighting, shadow occlusion, and texture degradation.

[0081] Figure 6 The cumulative error function curves of our proposed method and two existing methods (SURF feature point matching and Superpoint feature point matching) are recorded. The horizontal axis represents the positioning error, and the vertical axis represents the number of experimental samples within the error threshold, i.e., the positioning success rate. The graph clearly shows that our method's cumulative error curve is consistently located at the top left across all error thresholds, indicating that under the same error tolerance, our method consistently locates more samples correctly than the comparison methods. Particularly in the low-error region (error less than 0.5m), our method's cumulative success rate is significantly better than SURF and Superpoint feature point matching, reflecting its outstanding advantage in high-precision positioning scenarios. Furthermore, when the error threshold is increased to its maximum value, our method achieves a 100% success rate, meaning all experimental samples are correctly located. In contrast, SURF and Superpoint feature point matching still fail to locate some samples under the same conditions, indicating insufficient adaptability to complex scenarios. From the curve shape, the cumulative distribution curve of this method rises rapidly and tends to saturate within a small error range, indicating that its positioning error distribution is concentrated and its stability is good. In contrast, the curves of SURF feature point matching and Superpoint feature point matching rise relatively slowly, with more dispersed error distribution and larger fluctuations in positioning accuracy. The above results show that this method is significantly better than traditional feature point matching methods that rely on local gray-level gradients in terms of positioning accuracy, success rate, and stability.

[0082] Figure 7 This diagram illustrates the matching results of our method compared to two existing methods (SURF feature point matching and Superpoint feature point matching). Figure 7 (a) in the diagram shows the matching effect of the SURF feature point matching method. Figure 7 (b) in the diagram illustrates the matching effect of the Superpoint feature point matching method. Figure 7Image (c) in the diagram illustrates the matching effect of the method of this invention. A comparative observation of the three sets of sub-images shows that both SURF feature point matching and Superpoint feature point matching methods exhibit varying degrees of mismatch problems during the matching process. For example... Figure 7 As shown in (a), the SURF method relies on the Hessian matrix to detect local extrema. Its matching results contain a large number of mismatched point pairs that deviate from the true correspondence. The distribution of matching points is relatively scattered, especially in areas where the image texture changes are not significant, where the mismatch phenomenon is more prominent. Figure 7 Figure (b) demonstrates the matching performance of the Superpoint method. Although this method uses deep learning technology to extract adaptive feature points, improving the matching quality compared to SURF, it still suffers from a certain percentage of mismatches when faced with complex conditions such as varying illumination and repetitive textures. The accuracy and consistency of its matched point pairs still need improvement. In contrast, Figure 7 The method of the present invention shown in (c) demonstrates significantly better matching performance than the two methods mentioned above under the same conditions. The matched point pairs accurately fall into their corresponding real positions, the matching lines are neat and orderly, and no obvious mismatches occur. The above comparison results fully illustrate that feature point matching methods such as SURF and Superpoint feature point matching are essentially based on pixel-level local gray-level gradient changes for feature extraction and matching. Their mathematical premise requires that the image has stable and rich local texture information. When there are uneven lighting or texture repetition in the scene, this mathematical premise is violated, leading to a decrease in the distinguishability of feature descriptors and thus causing matching failure. In contrast, the method of the present invention does not rely on local gradient features, but establishes matching relationships based on global structural information or semantic-level constraints, effectively avoiding the impact of local texture changes on matching performance. Therefore, it can still achieve accurate and stable matching results in complex scenes.

[0083] Based on the same inventive concept, this application provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the aforementioned method for acquiring point cloud data based on a simulation scanning BIM model.

[0084] Based on the same inventive concept, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps of the aforementioned method for acquiring point cloud data based on a simulated scanning BIM model.

[0085] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0086] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0087] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0088] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0089] It should also be noted that the various specific technical features described in the above embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, the present invention will not describe the various possible combinations separately.

Claims

1. A visual localization method for aircraft that integrates semantic features and spatial geometric constraints, characterized in that, include: Aerial images captured by the aircraft's camera are acquired, semantic segmentation is performed on the aerial images, and a single-channel mask image for semantic segmentation of the aerial images is generated. Simultaneously, the map is used as the reference map, and semantic segmentation is performed on the reference map to generate a single-channel mask image of the reference map semantic segmentation. The single-channel mask image for semantic segmentation of aerial images is transformed into C independent binary images of aerial images according to categories, where C is the total number of categories. The single-channel mask image for semantic segmentation of the reference image is transformed into several independent binary images of the reference image according to categories. The reference image binary images of the corresponding categories of the aerial image binary images are extracted from the several binary images of the reference images. Calculate the similarity between C pairs of binarized layers, and find the region in the baseline image that has the highest matching degree with the aerial image as the matching region; The system acquires the attitude and altitude information of the aircraft, calculates the projection deviation caused by the aircraft tilt, converts the pixel coordinates of the matching area into the geographic coordinates of the aircraft's ground point, and performs projection deviation compensation to achieve the calculation of the aircraft's absolute position.

2. The aircraft visual localization method integrating semantic features and spatial geometric constraints according to claim 1, characterized in that, The YOLO11-seg network is used for semantic segmentation of images.

3. The aircraft visual localization method integrating semantic features and spatial geometric constraints according to claim 1, characterized in that, For the c-th category, if a pixel in the semantic segmentation single-channel mask image belongs to that category, the value of that pixel is set to 1; otherwise, it is set to 0, thus obtaining the binarized layer for the c-th category.

4. The aircraft visual localization method integrating semantic features and spatial geometric constraints according to claim 1, characterized in that, The single-channel mask image for semantic segmentation of the aerial image is w×h, where w is the width and h is the height; the top left corner of the single-channel mask image for semantic segmentation of the baseline image is used as the model. To begin, select the matching region: ； in, This represents the coordinates in the binarized layer of the baseline map calculated using NCC for the c-th category. Normalized correlation between the location and the corresponding location in the binarized layer of the aerial image; Represents the coordinates in the single-channel mask image for semantic segmentation of the baseline image. Pixel value at that location, Represents the coordinates in the single-channel mask image for semantic segmentation of aerial images. Pixel value at; Represents the coordinates of a single-channel mask image for semantic segmentation of the baseline image. The matching score is calculated between the location and the corresponding position in the single-channel mask image of the aerial image semantic segmentation. The matching scores of all coordinates in a region are added together to obtain the overall matching score for that region. The region with the highest overall matching score is selected as the matching region.

5. The aircraft visual localization method integrating semantic features and spatial geometric constraints according to claim 1, characterized in that, The expression for projection bias is as follows: ； Where h is the altitude of the aircraft. This is the equivalent attitude angle.

6. The aircraft visual localization method integrating semantic features and spatial geometric constraints according to claim 5, characterized in that, Projection bias is expressed as the projection offset caused by roll and pitch angles in the geographic coordinate system: ； in, and These represent the projection offsets caused by the roll and pitch angles in the geographic coordinate system, respectively. ； in, The camera's altitude in the geographic system. Let be a rotation matrix. Let be the optical axis vector of the camera coordinate system on the aircraft. and These represent the roll angle and pitch angle, respectively. Calculate the absolute position of the aircraft: ； in, and The north and east coordinates of the aircraft at the coordinates of its ground point are respectively given. and Match the north and east coordinates of the center point of the region in the map coordinate system. For heading angle, .

7. An aircraft visual positioning system integrating semantic features and spatial geometric constraints, characterized in that, include: Image acquisition and processing module: used to acquire aerial images captured by the aircraft camera, perform semantic segmentation on the aerial images, and generate a single-channel mask image for semantic segmentation of the aerial images; Simultaneously, the map is used as the reference map, and semantic segmentation is performed on the reference map to generate a single-channel mask image of the reference map semantic segmentation. The binarization processing module is used to convert the single-channel mask image of the aerial image semantic segmentation into C independent aerial image binarization layers according to the category, where C is the total number of categories. It also converts the single-channel mask image of the reference image semantic segmentation into several independent reference image binarization layers according to the category. Finally, it extracts the reference image binarization layer of the corresponding category from the several reference image binarization layers. Matching Region Module: Used to calculate the similarity between C pairs of binarized layers, and to find the region with the highest matching degree with the aerial image in the baseline image as the matching region; The position calculation module is used to acquire the attitude information and flight altitude information of the aircraft, calculate the projection deviation, convert the pixel coordinates of the matching area position into the geographic coordinates of the aircraft's ground point, and perform projection deviation compensation to realize the calculation of the aircraft's absolute position.

8. A computer device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the aircraft visual localization method that integrates semantic features and spatial geometric constraints as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the aircraft visual localization method that integrates semantic features and spatial geometric constraints as described in any one of claims 1 to 6.

Images