An absolute position sensing method and system under GPS denial conditions

Abstract

This invention discloses an absolute position perception method and system under GPS-denied conditions. The method includes the following steps: Step A: Images captured by rotating a camera at a specific angle are stitched together in a 360° circular pattern to create a panoramic image; Step B: Based on a semantic segmentation method improved by Deeplabv3+, the skyline is extracted from the natural panoramic image formed in Step A; Step C: OpenGL is used to visualize DEM data to form a rendering image, and the Canny operator is used to perform edge detection on the rendering image to output the skyline; Step D: By using VGG16 as a skyline feature encoder for the natural image and the DEM image, and combining it with a Siamese neural network to output the similarity between the two types of skylines, the self-position is perceived. This invention addresses the challenge of perceiving one's own position on the ground in situations such as military applications under GPS-denied conditions, and conducts research on determining one's own position to achieve the goal of absolute positioning that does not rely on GPS and initial position and whose error does not increase over time.

Description

Technical Field

[0001] This invention belongs to the field of image information processing, and specifically relates to an absolute position sensing method and system under GPS rejection conditions. Background Technology

[0002] Current positioning generally uses satellite positioning technologies such as GPS and BeiDou. However, these technologies are unusable in certain situations. For example, in wartime, satellites and other electronic equipment are primary targets. Once destroyed, positioning becomes impossible, and all weapons, equipment, aircraft, and vehicles will be unable to determine their location. In the field, interference can weaken satellite signals, rendering GPS and other satellite technologies unusable and preventing positioning. Therefore, inventing a self-positioning technology that does not rely on GPS is of paramount importance.

[0003] Without relying on GPS or other satellite positioning technologies, Li Yinya et al. (patent publication number CN109579841A) disclosed a high-precision positioning method for vehicle-mounted fire-fighting high-load rotary-wing UAVs under GPS denial conditions. This method uses a cable-type displacement sensor and an axis angle encoder to achieve high-precision positioning of the vehicle-mounted UAV. However, this positioning only determines the altitude of the UAV, not its position. Wang Weiping et al. (patent publication number CN110068335A) disclosed a real-time positioning method and system for UAV swarms under GPS denial conditions. This method detects and matches the salient features between captured images, triangulates the common features, forms a sparse reconstruction, and generates a global map accessible to all UAVs. However, this method can only determine the relative positions between UAVs, not their absolute positions.

[0004] In recent years, absolute positioning under GPS denial conditions has mostly relied on skylines because skylines in nature possess uniqueness and stability. They remain largely unchanged over long periods and vary significantly from place to place, making them a good feature for positioning. Currently, Tzeng et al. proposed an image-based visual positioning system in desert environments (published in The 18th International Conference on Information Fusion, 2013, Vol. 3) based on DEMs and skylines. This system does not require any metadata such as GPS readings, camera focal length, or field of view; it only utilizes publicly available digital elevation models (DEMs) to quickly and accurately locate photographs in the desert, although the accuracy is not high.

[0005] Against this backdrop, it is particularly important to research a method for absolute positioning that is highly accurate, does not rely on GPS information, and relies solely on information provided by the natural environment, such as the skyline. Summary of the Invention

[0006] The technical problem to be solved by the present invention is to provide an absolute position sensing method and system under GPS rejection conditions. By using the inherent skyline features of nature and combining them with DEM (Digital Elevation Data), the method performs skyline retrieval and matching, thereby realizing absolute position positioning under GPS rejection conditions. The method is then integrated into a complete system.

[0007] The technical solution adopted in this invention is as follows:

[0008] An absolute position sensing method under GPS denied conditions, the method comprising the following steps:

[0009] Step A: Stitch the discrete natural images captured by the vehicle-mounted camera into a 360° panoramic natural image.

[0010] Step B: Use a semantic segmentation network based on Deeplabv3+ to extract the natural panoramic skyline from the natural panoramic image;

[0011] Step C: Based on the remote sensing center, obtain the DEM data of the area to be sensed, use OpenGL to form a rendering image of the DEM data, use the Canny operator to perform edge detection on the rendering image, and output the panoramic skyline of the DEM image.

[0012] Step D: Using the transfer learning method, VGG16 is used as the feature encoder of the Siamese neural network. The similarity between the panoramic skyline of the natural image and the panoramic skyline of the DEM image is used as the matching result, thereby perceiving its own position based on the skyline matching result.

[0013] Furthermore, the specific processing procedure of step A is as follows:

[0014] Step A1: Using a spherical camera on the reconnaissance vehicle at a fixed viewpoint, take pictures at 40° intervals, rotate it one full circle, and obtain 9 natural images;

[0015] Step A2: Project the natural images to be stitched onto the same coordinate system. The formula for cylindrical projection is as follows:

[0016]

[0017] Where (x,y) represents the original coordinates of the pixel in the natural image, (x、,y、) represents the coordinates after cylindrical projection, (widht,height) represents the width and height of the natural image respectively, and f represents the camera focal length.

[0018] To ensure visual consistency after stitching, the images to be stitched need to be projected onto a standard coordinate system.

[0019] Since the transformation of cylindrical coordinates is relatively simple and the projected image is independent of its position on the cylindrical surface, the cylindrical panoramic image described by it can satisfy 360-degree surround view in the horizontal direction and has a good visual effect. Therefore, cylindrical projection can be used to complete image stitching.

[0020] Step A3: Use the SIFT operator to identify feature points between adjacent natural images, and then perform feature matching on the nine acquired natural images. Use the image after feature matching as the image to be stitched together.

[0021] By using the SIFT algorithm, this method is insensitive to changes in the order, orientation, scale, and brightness of the image inputs used for feature matching, and is also insensitive to noisy images that are not panoramic. It can quickly identify feature points between multiple natural images and complete accurate matching between feature points of natural images.

[0022] Step A4: By constructing the Laplacian pyramid, the image to be stitched is decomposed into different spatial frequency bands, and then merged and fused at each spatial frequency layer to obtain a natural panoramic image.

[0023] By utilizing the Laplacian pyramid, features and details across various frequency bands are preserved and integrated, resulting in a natural image stitching effect.

[0024] Furthermore, by constructing a Laplacian pyramid, the images to be stitched are decomposed into different spatial frequency bands, and merging and fusing are performed at each spatial frequency layer to obtain a natural panoramic image. The specific steps are as follows:

[0025] Step A41: Construct the Gaussian pyramid of the images to be stitched together;

[0026] Step A42: Using each layer of the Gaussian pyramid image, subtract the expanded image of the previous layer after upsampling and Gaussian convolution to obtain LP. Then, merge the LPs of the same layer in the overlapping area using a weighted average method. LP represents the Laplacian pyramid.

[0027] Step A43: Expand the merged LP starting from the top layer, and add the expanded image to the merged LP of the next layer to obtain the fused image of the next layer;

[0028] Step A44: Complete image fusion layer by layer to obtain preliminary natural image stitching results;

[0029] Step A45: Separate the preliminary natural panoramic image obtained by stitching in step A44 from the middle to form two images p1 and p2;

[0030] Step A46: Set p2 as the beginning of the second stitching and p1 as the end. Use the SIFT operator to obtain the feature elements of the overlapping part of the p2 and p1 images, perform matching, and obtain a new image after feature matching, which is used as the image to be stitched.

[0031] Step A47: By constructing the Laplacian pyramid, the images to be stitched are decomposed into different spatial frequency bands, and merging and fusing are performed on each spatial frequency layer to obtain a natural panoramic image. This achieves seamless fusion of images p2 and p1, removes the overlap between the beginning and end of the images, and thus obtains the final natural image stitching result.

[0032] By linearly scanning the leftmost and rightmost columns of the stitched panoramic image, the four corner points—top left, top right, bottom left, and bottom right—are identified, and perspective transformation is performed using these corner points to correct the image.

[0033] The panoramic image obtained by stitching has the problem of misalignment at the left and right pixel level. Therefore, a perspective transformation mechanism is introduced to realize the automatic straightening of the image and solve the problem of alignment of left and right pixels.

[0034] Further, the specific processing procedure of step B is as follows: Step B1: Replace the backbone network ResNet-101 in the Deeplabv3+ semantic segmentation network with a GhostNet network. The GhostNet network consists of 101 GhostBottleNeck modules stacked together. Input the natural panoramic image and process it through the semantic segmentation network improved based on Deeplabv3+ to obtain the segmentation feature map F of the natural image. nature ;

[0035] Compared to ResNet-101, GhostNet employs depthwise separable convolutions, significantly reducing parameters and computational cost while having minimal impact on skyline detection performance. The GhostNet network is constructed by stacking GhostModules to form GhostBottleNeck modules, which are then further stacked to create the GhostNet network. The GhostModule first performs a regular convolution operation on the input to obtain the intrinsic feature map. Then, it performs a linear depthwise separable convolution operation on each channel of the intrinsic feature map to obtain the Ghost feature map. Finally, it fuses the intrinsic and Ghost feature maps to obtain the final feature map output by the GhostModule. The advantage of this approach is that it reduces redundant information generated by regular convolution, reducing the number of parameters and computational cost while ensuring relatively effective feature extraction. The GhostBottleNeck module first performs a GhostModule operation on the input feature map F to obtain feature map F1. Then, it performs batch normalization and ReLU activation on feature map F1, and then passes it through another GhostModule to obtain feature map F2. After another batch normalization, it obtains feature map F3. Finally, it concatenates feature map F3 with feature map F to obtain the final output feature map F of the GhostBottleNeck module. Gout ;

[0036] The DeepLabV3+ semantic segmentation network uses ResNet-101 as its backbone, but ResNet-101 has significant parameter redundancy and computational cost. Therefore, this system proposes using GhostNet as the feature extraction network in the DeepLabV3+ semantic segmentation network. Compared to ResNet-101, GhostNet significantly reduces both parameters and computational cost, and has minimal impact on skyline detection performance, thus creating a lightweight DeepLabV3+ semantic segmentation network.

[0037] Similar to residual networks, GhostNet is composed of stacked GhostModules forming GhostBottleNeck modules, which are then further stacked to form the GhostNet network. Each GhostModule first performs a regular convolution on the input image to obtain an intrinsic feature map. Then, it performs a linear depthwise separable convolution on each channel of the intrinsic feature map to obtain Ghost features. Finally, it fuses the intrinsic and Ghost features to obtain the final feature map. This approach reduces redundant information generated by regular convolution, minimizing the number of parameters and computational cost while ensuring relatively effective feature extraction.

[0038] Step B2: Feature enhancement;

[0039] Using the CCAM module to segment feature map F nature Information augmentation is performed to obtain the augmented feature map FM. Out ;

[0040] The CCAM (Convolutional Chanel Attention Modules) module is used to segment the feature map F. nature The feature information of each channel is compressed into a single feature value through global average pooling and global max pooling, respectively, thus obtaining the over-pooled feature map FM. GA and global max pooling feature map FM GM The feature map FM is obtained by splicing the features according to the channels. s ;

[0041] Then feature map FM s The input is fed into a Conv-BN-ReLU structure, where Conv uses convolutions to perform FM on the input feature map. s Dimensionality reduction and feature extraction are performed; dimensionality increase is then achieved using a linear layer to obtain the feature map FM. Ex and FM feature map Ex First, decompose the feature map into a linear global average pooling feature map (FM) according to the channel. EGA and linear global max pooling feature map FM EGM FM EGA and FM EGM The final channel feature map (FM) is obtained by adding pixels one by one and then activating it with a sigmoid function. Channel Finally, the channel feature map is FM Channel and segmentation feature map F nature The enhanced feature map FM is obtained by multiplying the channels. Out .

[0042] Furthermore, the region attention module is used to enhance the feature map FM.Out After noise removal, pooling and convolution operations are performed to obtain the initial feature map FM of the skyline. Ini ;

[0043] Existing skyline detection algorithms extract skyline feature maps with significant noise, thus affecting the effectiveness of skyline feature extraction using dynamic programming. To address this issue, a Convolution Area Attention Module (CAAM) is employed, with the following module structure: Figure 8 As shown, this is used to enhance the features of the skyline area.

[0044] The region attention module consists of a mask branch and a trunk branch; the output of the mask branch and the output of the trunk branch are added together.

[0045] The trunk branch directly transmits the input image to the output.

[0046] The mask branch adopts a U-Net structure, which means that the encoder first downsamples the input image, and then the decoder upsamples the feature map to gradually restore the feature scale.

[0047] The specific process is as follows:

[0048] Step B3: FM enhancement of feature maps Out First, after one downsampling step using Maxpooling and BottleNeck, feature map F1 is obtained. Then, after another downsampling step using Maxpooling and BottleNeck, feature map F2 is obtained.

[0049] Step B4: Upsample feature map F2 using BottleNeck once to obtain feature map F3;

[0050] Step B5: After concatenating feature map F1 and feature map F3 by channel, perform two upsampling operations using BottleNeck and bilinear interpolation to obtain feature map FM. Up ;

[0051] Feature Map FM Up Scale and Enhanced Feature Map FM Out The same scale;

[0052] Step B6: FM feature map Up After convolution to reduce the feature dimension, a single-channel region probability feature map (FM) is output. SingleAnd use sigmoid to normalize the probability values ​​to the range [0,1]; FM the single-channel region probability feature map Single The region attention information is first multiplied by the Trunk branch channel by channel and then added together. The output is the feature map FM after region enhancement. Area The specific calculation formula is as follows:

[0053] H(x) = (x + f(I(x))) × I(x)

[0054] In the formula, H(x) is the probability feature map output by the CCAM module, f(I(x)) is the output feature map of the Mask branch, and I(x) is the feature map input by the CCAM module;

[0055] Step B7: After passing through the GhostNet network, CCAM module, and CAAM attention mechanism, the initial feature map FM of the skyline can be obtained through convolution and pooling operations. Ini .

[0056] Furthermore, the SCPA module is used to preprocess the acquired natural images to obtain feature maps FM. Binary and utilize feature map FM Binary With the initial feature map FM Ini The fusion is performed, and the fused result is subjected to two RCU operations and a 1*1 Conv convolution operation, and then upsampled to obtain the final skyline semantic segmentation feature map.

[0057] The threshold of the final skyline semantic segmentation feature map is set to 0.95, that is, points with a probability value greater than 0.95 are set to 1, and points with a probability value less than 0.95 are set to 0, thus obtaining the final output natural panoramic skyline image.

[0058] The RCU consists of four concatenated convolutional kernels with sizes of 3*1, 1*3, 3*1, and 1*3, respectively. The convolution operations of the first three kernels are normalized using Batch Normalization (BN) and then activated using the ReLU function. The convolution operation of the last kernel is normalized using BN and then superimposed with the input of the RCU to obtain the output.

[0059] The present invention utilizes the SCPA module, which can be found in the paper "A skyline detection algorithm for use in different weather and environmental conditions" (published at the IEEE International Conference on Electro Information Technology, 2016). The technical solution employs two parts of the SCPA module: preprocessing and searching for local maxima of the gradient.

[0060] Preprocessing the acquired natural images using the SCPA module first converts the input color image into a grayscale image. Then, taking advantage of the characteristic that the grayscale values ​​of distant areas in mountainous images are greater than those of nearby areas, each sampled column in the grayscale image is processed to obtain the pixel gradient of the corresponding column.

[0061] For each point in the sampling column, determine whether the grayscale difference between the current point and its upper and lower halves (within a distance of 2) is greater than 0. If yes, mark the point as 1; otherwise, mark it as 0. Then, find the location of the maximum pixel gradient in the continuous region marked as 1 as a candidate point to suppress other non-maximum values. Finally, after processing all sampling columns, obtain the binary gradient information map FM. Binary ;

[0062] G m (y)=f(m·p,y+2)-f(m·p,y+2);

[0063]

[0064] (x m ,y k ) = arg max(G(y)); (x m ,y k )∈Area(x m ,k)

[0065] In the formula, f(m·p,y) represents the value at position (m·p,y) in the grayscale image, and p refers to the sampling interval. m (y) represents the grayscale gradient at the m-th sampling column position, g m (y) is the gradient label, (x) is the gradient label. m ,y k This represents the maximum value of the gradient in the region.

[0066] Furthermore, when training the semantic segmentation network based on Deeplabv3+, the natural panoramic images obtained by stitching in step A are used to create labels for the skyline in the panoramic images, resulting in a training set of 150 panoramic natural images with a size of 800*600.

[0067] The training cycle is 150 epochs, the batch size is set to 8, and the parameters of the semantic segmentation network based on Deeplabv3+ are optimized using the stochastic gradient descent algorithm. The initial learning rate is 0.001 and the learning rate is adjusted using cosine fire reduction. The momentum is set to 0.9 and the weight descent is set to 0.0005.

[0068] During training, the skyline feature map FM generated after passing through the GhostNet network and the CCAM attention mechanism is used. Out The weight parameters of the Mask branch are optimized by calculating the loss using the region label information. CAAM's Mask branch uses binary cross-entropy as the loss function, as shown in the following formula:

[0069]

[0070] In the formula, N is the total length of the region probability map after it is expanded into a vector, and y is the region label. This is the predicted value for the Mask branch.

[0071] Furthermore, the specific processing procedure of step C is as follows:

[0072] Step C1: Reading and converting DEM data;

[0073] First, the data needs to be read and stored in matrix form. Then, the GetGeoTransform() method in the GDALDataset class of GDAL is used to obtain the coordinate information of the DEM data. The obtained information is stored as an array. A set of key coefficients is obtained, and the matrix row and column numbers are converted to geographic coordinates using a formula.

[0074] X = gt[0] + col * gt[1] + row * gt[2]

[0075] Y = gt[3] + col*gt[4] + row*gt[5]

[0076] Where col and row represent column number and row number respectively; gt[i] represents the six key coefficients obtained; gt[0] and gt[3] represent the geographic coordinates at image coordinate (0,0); gt[1] and gt[5] represent the resolution of the image x-axis and y-axis respectively; gt[2] and gt[4] represent the image rotation coefficients; and XY represents the geographic coordinates.

[0077] Using these parameters, not only can DEM data be segmented into matrix form, but the actual geographic coordinates of any point in the matrix can also be obtained.

[0078] Step C2: Generate perspective view;

[0079] Step C21: Use the frustum function in the vmath library of OpenGL to generate a perspective projection matrix by setting the distance between the near plane rectangle, the far plane, and the near plane. Use the position information of the points on the model to replace the color information of the points on the model to generate a normal perspective view without depth information.

[0080] Step C22: Output the distance values ​​from the fragment to the viewpoint to the color buffer to obtain a two-dimensional projection of the ground distance field with the viewpoint as the reference point onto the screen, i.e., a distance map. The z-value after the projection transformation is shown in the following formula:

[0081]

[0082] Where -f and -n are the distances between the far and near planes, respectively.

[0083] It can be observed that this is a nonlinear function, and z is near the clipping plane. ndc The value varies considerably, but the accuracy is good; however, near the far clipping plane, within a certain distance, z... ndc Nearly equal, but with low accuracy.

[0084] Step C23: Set the distance z ndc and depth z eye Transforming into a linear relationship, we get:

[0085] To achieve this depth visualization, and later extract the scene's skyline and ridgeline information from the depth map, the distance z needs to be... ndc and depth z eye Transform it into a linear relationship;

[0086] Since the result is negative, it cannot be used directly as a grayscale value; otherwise, the entire scene would turn black. Therefore, we need to take its negative value as the depth value.

[0087]

[0088] Finally, normalizing the depth value obtained from the above formula to the range of [0,1] will yield a distance map in which depth and grayscale have a linear relationship.

[0089] Step C3: DEM skyline extraction:

[0090] Step C31: First, using the Canny algorithm, the Sobel operator is applied to the DEM image to calculate its first derivatives in the horizontal and vertical directions, thus obtaining the gradient maps of the DEM image in the horizontal and vertical directions. Finally, the gradients and directions of the boundaries of these two gradient maps can be obtained, as shown below:

[0091]

[0092]

[0093] G x and G y This represents the gradient in the x and y directions. Edge_Gradient and Angle represent the gradient and direction, respectively.

[0094] Step C32: Nonmaximum suppression.

[0095] By comparing the gradient of a pixel with the gradients of two points in the two directions before and after the gradient of this pixel, we can determine whether this pixel is a local maximum gradient point in its neighborhood. If it is not a local maximum gradient point, it is excluded.

[0096] Step C33: Using a dual-threshold detection and hysteresis boundary method, points above the upper threshold are identified as strong edges, and points below the lower threshold are identified as non-edges. Hysteresis tracking is performed on points between the upper and lower thresholds (which become weak edges). That is, weak edges connected to strong edges are identified as edges, and otherwise they are identified as non-edges.

[0097] Furthermore, the specific processing procedure of step D is as follows:

[0098] Step D1: Skyline feature extraction;

[0099] VGG16 is used as the feature encoder of the Siamese neural network. Two skyline images are input into the Siamese neural network. The VGG16 feature encoder maps the input to a new space to form the representation of the skyline image in the new space, that is, the encoded skyline features.

[0100] The main purpose of this operation is to increase the network depth while maintaining the same receptive field, thereby improving the performance of the neural network to some extent. For example, the stacking of three 3×3 convolutional kernels with a stride of 1 can be seen as a receptive field of size 7 (meaning three consecutive 3×3 convolutions are equivalent to a 7×7 convolution), with a total of 3×(9×C^2) parameters. If a 7×7 convolutional kernel is used directly, the total number of parameters is 49×C^2, where C refers to the number of input and output channels. Clearly, 27×C^2 is less than 49×C^2, meaning fewer parameters are used, and the 3×3 convolutional kernel helps to better preserve image properties. The VGG16 model uses the pre-trained parameter model from the official documentation.

[0101] Step D2: Establishing the DEM discrete feature database;

[0102] Each DEM image is resized to 224*224*3. Then, a Siamese neural network is used to encode the features of each panoramic image in the DEM to obtain a 7*7*512 feature map, which is then stored offline in a CSV file.

[0103] Step D3: Feature matching;

[0104] After resizing the natural panoramic image to a standard size of 224*224*3, it is input into the Siamese neural network to calculate its feature map F. Final The size is 7*7*512; traverse the DEM offline feature database and calculate the feature map F. Final The Euclidean distance between the point and each feature map in the DEM discrete feature database is calculated; the DEM point with the highest similarity (smallest Euclidean distance) is selected as the location point.

[0105] On the other hand, an absolute position sensing system under GPS rejection conditions includes:

[0106] Natural panoramic image acquisition and stitching unit: stitches discrete natural images acquired by the vehicle-mounted camera into a 360° circular image.

[0107] Natural Image Panoramic Skyline Extraction Unit: Employs a semantic segmentation network based on Deeplabv3+ to extract the natural panoramic skyline from natural panoramic images;

[0108] DEM Image Panoramic Skyline Extraction Unit: Based on the remote sensing center, DEM data of the area to be sensed is obtained, OpenGL is used to form a rendering image of the DEM data, and the Canny operator is used to perform edge detection on the rendering image to output the panoramic skyline of the DEM image.

[0109] Location-aware unit: Using transfer learning, VGG16 is used as the feature encoder of the Siamese neural network. The similarity between the panoramic skyline of the natural image and the panoramic skyline of the DEM image is used as the matching result, thereby perceiving its own location based on the skyline matching result.

[0110] The natural panoramic image acquisition and stitching unit, the natural image panoramic skyline extraction unit, the DEM image panoramic skyline extraction unit, and the position awareness unit use the aforementioned absolute position awareness method under GPS rejection conditions for data processing.

[0111] Beneficial effects

[0112] This invention provides an absolute position perception method and system under GPS rejection conditions. The method includes the following steps: Step A: Images captured by rotating a camera at a specific angle are stitched together in a 360° circular pattern to create a panoramic image; Step B: A semantic segmentation method based on Deeplabv3+ is used to extract the skyline from the natural panoramic image formed in Step A; Step C: OpenGL is used to visualize the DEM data to form a rendering image, and then the Canny operator is used to perform edge detection on the rendering image to output the skyline; Step D: A transfer learning method is used, employing VGG16 as an encoder for the skyline features of the natural image and the DEM image, and then combined with a Siamese neural network to output the similarity between the two types of skylines as the matching result, thereby perceiving the user's position based on the skyline; Step E: The various subsystems are integrated into a system, and the user interface is written using Qt to form a complete system. This invention proposes a complete system that, under GPS denial conditions, selects existing natural skylines as features and combines them with DEM data, applying deep learning to perform skyline matching and retrieval, thereby achieving self-localization of absolute position. This method is fast and accurate, and can be applied in military warfare when GPS and other satellites are destroyed, as well as in reconnaissance and other fields, determining absolute position through images, with broad application prospects. Attached Figure Description

[0113] Figure 1 This is a flowchart of the absolute position sensing method under GPS rejection conditions in an example of the present invention;

[0114] Figure 2 This is a schematic diagram of the DeepLabv3+ network used for skyline extraction from natural panoramic images in this invention example;

[0115] Figure 3 This is a schematic diagram of the twin neural network used in the skyline detection and matching in an example of the present invention;

[0116] Figure 4 The images are renderings and panoramic skyline images obtained using DEM data. (a) is the rendering of a point in the DEM data after OpenGL rendering, and (b) is the 360° panoramic skyline image extracted from the DEM.

[0117] Figure 5 The discrete images acquired by the camera in Example 1 are (a)-(i) rotated by 40°, 80°, 120°, 160°, 200°, 240°, 280°, 320°, and 360° respectively.

[0118] Figure 6 This is the stitched panoramic natural image from Example 1;

[0119] Figure 7 The skyline image is extracted from the panoramic natural image in Example 1;

[0120] Figure 8 The matching results from Example 1 are displayed in the form of a heatmap;

[0121] Figure 9 The discrete images acquired by the camera in Example 2 are (a)-(i) rotated by 40°, 80°, 120°, 160°, 200°, 240°, 280°, 320°, and 360° respectively.

[0122] Figure 10 This is the stitched panoramic natural image from Example 2;

[0123] Figure 11 The skyline image is extracted from the panoramic natural image in Example 2;

[0124] Figure 12 The matching results from Example 2 are displayed in the form of a heatmap;

[0125] Figure 13 The discrete images acquired by the camera in Example 3 are (a)-(i) rotated by 40°, 80°, 120°, 160°, 200°, 240°, 280°, 320°, and 360° respectively.

[0126] Figure 14 This is the stitched panoramic natural image from Example 3;

[0127] Figure 15 The skyline image is extracted from the panoramic natural image in Example 3;

[0128] Figure 16 The matching results in Example 3 are displayed in the form of a heatmap. Detailed Implementation

[0129] The present invention will be further described below with reference to the accompanying drawings:

[0130] Example 1:

[0131] This embodiment focuses on natural images captured by a vehicle-mounted camera in a field test area. The overall implementation process is as follows: Figure 1 As shown, its self-positioning assessment is performed according to the following steps:

[0132] Step A: Stitch the discrete images into a 360° panoramic image.

[0133] The vehicle-mounted camera rotates 40° every time, thus capturing 9 images in one revolution. Figure 5 As shown in (a)-(i), the latitude and longitude coordinates at this time are recorded. The latitude and longitude coordinates of the sampling point in this example are (113.271711°E, 28.372615°N). To ensure the stitching effect, there is a certain overlap area in each image. Next, the acquired images are cylindrically projected, and the feature points of each discrete image are found by the SIFT algorithm for image matching.

[0134] After image matching is completed, a Laplacian pyramid of the images to be stitched is constructed, decomposing the images into different spatial frequency bands. Merging and fusing are then performed at each spatial frequency level to obtain a natural panoramic image. However, the resulting natural panoramic image has some overlap at the head and tail. Therefore, a secondary stitching method is used to remove redundancy, and then the largest rectangular region is cropped to obtain the final natural panoramic image, as shown below. Figure 6 As shown.

[0135] Step B: After completing Step A, the natural panoramic image will be obtained. For the skyline portion of the image, use Photoshop to colorize the skyline separately to create labels. Then, train the lightweight DeepLabV3+ semantic segmentation network. Once the network's model parameters are obtained, the natural panoramic image to be matched can be input. The network model is as follows: Figure 2 As shown. Inputting the data into the network will produce the final skyline image, as shown. Figure 7 As shown.

[0136] Step C: The DEM data of the test area is read using the GDAL library, and then a 3D model of the DEM data is rendered using OpenGL. Next, the surface model is rendered to generate a rendered image. Finally, the skyline is extracted from the DEM using the Canny algorithm. To save image retrieval and matching time, the skyline extraction from the DEM is performed offline. The skyline extraction effect from the DEM is as follows: Figure 4 As shown.

[0137] Step D: Employ a Siamese neural network, using VGG16 as the encoder, to perform feature encoding on the skyline image. The network structure is as follows: Figure 3 As shown. The VGG16 model uses the officially trained parameters. First, the skyline image extracted from the DEM in step C is feature-encoded, and the encoded feature vector is stored for reuse in future tests. The stitched natural panoramic image from step A is also feature-encoded into a feature vector V using the same Siamese neural network. Finally, the Euclidean distance is calculated between the feature vector V from the natural panoramic image and the skyline feature vector from the DEM. The smaller the Euclidean distance, the greater the similarity between the two skylines, indicating that the location point is closest to that point. Simultaneously, a heat map of the location is drawn based on the Euclidean distance, as shown. Figure 8 As shown, a direct observation can be made. In this example, the latitude and longitude coordinates of the location obtained by the invented positioning method are (113.271500°E, 28.372300°N), and the error between the latitude and longitude of the location and the actual location is 40.65 meters, which meets the positioning requirements of relevant military applications under GPS denial conditions.

[0138] Example 2:

[0139] This embodiment focuses on natural images captured by a vehicle-mounted camera in a field test area. The overall implementation process is as follows: Figure 1 As shown, its self-positioning assessment is performed according to the following steps:

[0140] Step A: Stitch discrete images into a 360° panoramic image.

[0141] During the experiment, the vehicle-mounted camera rotated 40° every time, thus capturing 9 images in one revolution [e.g., Figure 9 As shown in (a)-(i), the latitude and longitude coordinates at this time are recorded. The latitude and longitude coordinates of the sampling point in this example are (113.242855°E, 28.359405°N). To ensure the stitching effect, there is a certain overlap area in each image. Next, the acquired images are cylindrically projected, and the feature points of each discrete image are found by the SIFT algorithm for image matching.

[0142] After image matching is completed, a Laplacian pyramid of the images to be stitched is constructed, decomposing the images into different spatial frequency bands. Merging and fusing are then performed at each spatial frequency level to obtain a natural panoramic image. However, the resulting natural panoramic image has some overlap at the head and tail. Therefore, a secondary stitching method is used to remove redundancy, and then the largest rectangular region is cropped to obtain the final natural panoramic image, as shown below. Figure 10As shown.

[0143] Step B: After completing Step A, the natural panoramic image will be obtained. For the skyline portion of the image, use Photoshop to colorize the skyline separately to create labels. Then, train the lightweight DeepLabV3+ semantic segmentation network. Once the network's model parameters are obtained, the natural panoramic image to be matched can be input. The network model is as follows: Figure 2 As shown. Inputting the data into the network will produce the final skyline image, as shown. Figure 11 As shown.

[0144] Step C: The DEM data of the test area is read using the GDAL library, and then a 3D model of the DEM data is rendered using OpenGL. Next, the surface model is rendered to generate a rendered image. Finally, the skyline is extracted from the DEM using the Canny algorithm. To save image retrieval and matching time, the skyline extraction from the DEM is performed offline. The skyline extraction effect from the DEM is as follows: Figure 4 As shown.

[0145] Step D: Employ a Siamese neural network, using VGG16 as the encoder, to perform feature encoding on the skyline image. The network structure is as follows: Figure 3 As shown. The VGG16 model uses the officially trained parameters. First, the skyline image extracted from the DEM in step C is feature-encoded, and the encoded feature vector is stored for reuse in future tests. The stitched natural panoramic image from step A is also feature-encoded into a feature vector V using the same Siamese neural network. Finally, the Euclidean distance is calculated between the feature vector V from the natural panoramic image and the skyline feature vector from the DEM. The smaller the Euclidean distance, the greater the similarity between the two skylines, indicating that the location point is closest to that point. Simultaneously, a heat map of the location is drawn based on the Euclidean distance, as shown. Figure 12 As shown, a direct observation can be made. In this example, the latitude and longitude coordinates of the location obtained by the invented positioning method are (113.243100°E, 28.360100°N), and the error between the latitude and longitude of the location and the actual location is 80.91 meters, which meets the positioning requirements of relevant military applications under GPS denial conditions.

[0146] Example 3:

[0147] This embodiment focuses on natural images captured by a vehicle-mounted camera in a field test area. The overall implementation process is as follows: Figure 1 As shown, its self-positioning assessment is performed according to the following steps:

[0148] Step A: Stitch discrete images into a 360° panoramic image.

[0149] During the experiment, the vehicle-mounted camera rotated 40° every time, thus capturing 9 images in one revolution [e.g., Figure 13 As shown in (a)-(i), the latitude and longitude coordinates at this time are recorded. The latitude and longitude coordinates of the sampling point in this example are (113.243975°E, 28.363742°N). To ensure the stitching effect, there is a certain overlap area in each image. Next, the acquired images are cylindrically projected, and the feature points of each discrete image are found using the SIFT algorithm for image matching.

[0150] After image matching is completed, a Laplacian pyramid of the images to be stitched is constructed, decomposing the images into different spatial frequency bands. Merging and fusing are then performed at each spatial frequency level to obtain a natural panoramic image. However, the resulting natural panoramic image has some overlap at the head and tail. Therefore, a secondary stitching method is used to remove redundancy, and then the largest rectangular region is cropped to obtain the final natural panoramic image, as shown below. Figure 14 As shown.

[0151] Step B: After completing Step A, the natural panoramic image will be obtained. For the skyline portion of the image, use Photoshop to colorize the skyline separately to create labels. Then, train the lightweight DeepLabV3+ semantic segmentation network. Once the network's model parameters are obtained, the natural panoramic image to be matched can be input. The network model is as follows: Figure 2 As shown. Inputting the data into the network will produce the final skyline image, as shown. Figure 15 As shown.

[0152] Step C: The DEM data of the test area is read using the GDAL library, and then a 3D model of the DEM data is rendered using OpenGL. Next, the surface model is rendered to generate a rendered image. Finally, the skyline is extracted from the DEM using the Canny algorithm. To save image retrieval and matching time, the skyline extraction from the DEM is performed offline. The skyline extraction effect from the DEM is as follows: Figure 4 As shown.

[0153] Step D: Employ a Siamese neural network, using VGG16 as the encoder, to perform feature encoding on the skyline image. The network structure is as follows: Figure 3As shown. The VGG16 model uses the officially trained parameters. First, the skyline image extracted from the DEM in step C is feature-encoded, and the encoded feature vector is stored for reuse in future tests. The stitched natural panoramic image from step A is also feature-encoded into a feature vector V using the same Siamese neural network. Finally, the Euclidean distance is calculated between the feature vector V from the natural panoramic image and the skyline feature vector from the DEM. A smaller Euclidean distance indicates a greater similarity between the two skylines, suggesting the location point is closest to that point. Simultaneously, a heat map of the location is drawn based on the Euclidean distance, as shown. Figure 16 As shown, a direct observation can be made. In this example, the latitude and longitude coordinates of the location obtained by the invented positioning method are (113.244100°E, 28.363900°N), and the error between the latitude and longitude of the location and the actual location is 21.41 meters, which meets the positioning requirements of relevant military applications under GPS denial conditions.

[0154] This invention also provides an absolute position sensing system under GPS rejection conditions, comprising:

[0155] Natural panoramic image acquisition and stitching unit: stitches discrete natural images acquired by the vehicle-mounted camera into a 360° circular image.

[0156] Natural Image Panoramic Skyline Extraction Unit: Employs a semantic segmentation network based on Deeplabv3+ to extract the natural panoramic skyline from natural panoramic images;

[0157] DEM Image Panoramic Skyline Extraction Unit: Based on the remote sensing center, DEM data of the area to be sensed is obtained, OpenGL is used to form a rendering image of the DEM data, and the Canny operator is used to perform edge detection on the rendering image to output the panoramic skyline of the DEM image.

[0158] Location-aware unit: Using transfer learning, VGG16 is used as the feature encoder of the Siamese neural network. The similarity between the panoramic skyline of the natural image and the panoramic skyline of the DEM image is used as the matching result, thereby perceiving its own location based on the skyline matching result.

[0159] The natural panoramic image acquisition and stitching unit, the natural image panoramic skyline extraction unit, the DEM image panoramic skyline extraction unit, and the position awareness unit use the aforementioned absolute position awareness method under GPS rejection conditions for data processing.

[0160] It should be understood that the functional unit modules in the various embodiments of the present invention can be concentrated in one processing unit, or each unit module can exist physically separately, or two or more unit modules can be integrated into one unit module, and can be implemented in hardware or software.

[0161] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0162] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0163] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0164] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0165] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. An absolute position sensing method under GPS denial conditions, characterized in that, The method includes the following steps: Step A: Stitch the discrete natural images captured by the vehicle-mounted camera into a 360° panoramic natural image. Step B: Use a semantic segmentation network based on Deeplabv3+ to extract the natural panoramic skyline from the natural panoramic image; Step C: Based on the remote sensing center, obtain the DEM data of the area to be sensed, use OpenGL to form a rendering image of the DEM data, use the Canny operator to perform edge detection on the rendering image, and output the panoramic skyline of the DEM image. Step D: Using the transfer learning method, VGG16 is used as the feature encoder of the Siamese neural network. The similarity between the panoramic skyline of the natural image and the panoramic skyline of the DEM image is used as the matching result, so as to perceive its own position based on the skyline matching result. The specific processing procedure for step D is as follows: Step D1: Skyline feature extraction; VGG16 is used as the feature encoder of the Siamese neural network. Two skyline images are input into the Siamese neural network. The VGG16 feature encoder maps the input to a new space to form the representation of the skyline image in the new space, that is, the encoded skyline features. Step D2: Establishing the DEM discrete feature database; Each DEM image is resized to 224*224*3. Then, a Siamese neural network is used to encode the features of each panoramic image in the DEM to obtain a 7*7*512 feature map, which is then stored offline in a CSV file. Step D3: Feature matching; After resizing the natural panoramic image to a standard size of 224*224*3, it is input into the Siamese neural network to calculate its feature map F. Final The size is 7*7*512; traverse the DEM offline feature database and calculate the feature map F. Final Calculate the Euclidean distance between the feature map and each feature map in the DEM discrete feature database; select the DEM point with the highest similarity as the location point.

2. The absolute position sensing method under GPS denial conditions according to claim 1, characterized in that, The specific processing procedure for step A is as follows: Step A1: Using a spherical camera on the reconnaissance vehicle at a fixed viewpoint, take pictures at 40° intervals, rotate it one full circle, and obtain 9 natural images; Step A2: Project the natural images to be stitched onto the same coordinate system. The formula for cylindrical projection is as follows: ； in, This represents the original coordinates of a pixel in a natural image. The coordinates are represented by the cylindrical projection, (width, height) represent the width and height of the natural image, respectively, and f represents the camera focal length; Step A3: Use the SIFT operator to identify feature points between adjacent natural images, and then perform feature matching on the nine acquired natural images. Use the image after feature matching as the image to be stitched together. Step A4: By constructing the Laplacian pyramid, the image to be stitched is decomposed into different spatial frequency bands, and then merged and fused at each spatial frequency layer to obtain a natural panoramic image.

3. The absolute position sensing method under GPS rejection conditions according to claim 2, characterized in that, By constructing a Laplacian pyramid, the images to be stitched are decomposed into different spatial frequency bands, and then merged and fused at each spatial frequency layer to obtain a natural panoramic image. The specific steps are as follows: Step A41: Construct the Gaussian pyramid of the images to be stitched together; Step A42: Using each layer of the Gaussian pyramid image, subtract the expanded image of the previous layer after upsampling and Gaussian convolution to obtain LP. Then, merge the LPs of the same layer in the overlapping area using a weighted average method. LP represents the Laplacian pyramid. Step A43: Expand the merged LP starting from the top layer, and add the expanded image to the merged LP of the next layer to obtain the fused image of the next layer; Step A44: Complete image fusion layer by layer to obtain a preliminary natural panoramic image; Step A45: Separate the preliminary natural panoramic image obtained from step A44 from the middle to form two images p1 and p2; Step A46: Set p2 as the beginning of the second stitching and p1 as the end. Use the SIFT operator to obtain the feature elements of the overlapping part of the p2 and p1 images, perform matching, and obtain a new image after feature matching, which is used as the image to be stitched. Step A47: By constructing the Laplacian pyramid, the images to be stitched are decomposed into different spatial frequency bands, and merging and fusing are performed on each spatial frequency layer to obtain a natural panoramic image. This achieves seamless fusion of images p2 and p1, removes the overlap between the beginning and end of the images, and thus obtains the final natural image stitching result.

4. The absolute position sensing method under GPS denial conditions according to claim 1, characterized in that, The specific processing procedure of step B is as follows: Step B1: Replace the ResNet-101 backbone network in the Deeplabv3+ semantic segmentation network with a GhostNet network. The GhostNet network consists of 101 GhostBottleNeck modules stacked together. Input the natural panoramic image and process it through the semantic segmentation network based on Deeplabv3+ to obtain the segmentation feature map F of the natural image. nature ; Step B2 Feature enhancement; Using the CCAM module to segment feature map F nature Information augmentation is performed to obtain the augmented feature map FM. Out ; The CCAM module is used to segment the feature map F. nature The feature information of each channel is compressed into a single feature value through global average pooling and global max pooling, respectively, thus obtaining the global average pooling feature map FM. GA and global max pooling feature map FM GM The feature map FM is obtained by splicing the features according to the channels. s ; Then feature map FM s The input is fed into a Conv-BN-ReLU structure, where Conv uses convolutions to perform FM on the input feature map. s Perform dimensionality reduction and feature extraction; Dimensionality increase using linear layers to obtain feature maps FM Ex and FM feature map Ex First, decompose the feature map into a linear global average pooling feature map (FM) according to the channel. EGA and linear global max pooling feature map FM EGM FM EGA and FM EGM The final channel feature map (FM) is obtained by adding pixels one by one and then activating it with a sigmoid function. Channel Finally, the channel feature map is FM Channel and segmentation feature map F nature The enhanced feature map FM is obtained by multiplying the channels. Out .

5. The absolute position sensing method under GPS denial conditions according to claim 4, characterized in that, FM of Enhanced Feature Maps using Region Attention Module Out After noise removal, pooling and convolution operations are performed to obtain the initial feature map FM of the skyline. Ini ; The region attention module consists of a mask branch and a trunk branch; the output of the mask branch and the output of the trunk branch are added together. The trunk branch directly transmits the input image to the output. The mask branch adopts a U-Net structure, that is, the encoder first downsamples the input image, and then the decoder upsamples the feature map to gradually restore the feature scale. The specific process is as follows: Step B3: FM enhancement of feature maps Out The feature map is obtained by first performing Maxpooling and BottleNeck downsampling. After another round of Maxpooling and BottleNeck downsampling, the feature map is obtained. ; Step B4: Process the feature map Perform an upsampling of BottleNeck to obtain the feature map. ; Step B5: Transfer the feature map With feature map After concatenating channels, BottleNeck and bilinear interpolation are used for two upsampling operations to obtain the feature map FM. Up ; Step B6: FM feature map Up After convolution to reduce the feature dimension, a single-channel region probability feature map (FM) is output. Single And use sigmoid to normalize the probability values ​​to the range [0,1]; FM the single-channel region probability feature map Single The region attention information is first multiplied by the Trunk branch channel by channel and then added together. The output is the feature map FM after region enhancement. Area The specific calculation formula is as follows: H(x) = (x + f(I(x))) × I(x); In the formula, H(x) is the probability feature map output by the CCAM module, f(I(x)) is the output feature map of the Mask branch, and I(x) is the feature map input by the CCAM module; Step B7: After passing through the GhostNet network, CCAM module, and CAAM attention mechanism, the initial feature map FM of the skyline can be obtained through convolution and pooling operations. Ini .

6. The absolute position sensing method under GPS denial conditions according to claim 5, characterized in that, The SCPA module was used to preprocess the acquired natural images to obtain feature maps FM. Binary and utilize feature map FM Binary With the initial feature map FM Ini The fusion is performed, and the fused result is subjected to two RCU operations and a 1*1 Conv convolution operation, and then upsampled to obtain the final skyline semantic segmentation feature map. The threshold of the final skyline semantic segmentation feature map is set to 0.95, that is, points with a probability value greater than 0.95 are set to 1, and points with a probability value less than 0.95 are set to 0, thus obtaining the final output natural panoramic skyline image. The RCU consists of four concatenated convolutional kernels with sizes of 3*1, 1*3, 3*1, and 1*3, respectively. The convolution operations of the first three kernels are normalized using BN (Batch Normalization) and then activated using the ReLU function. The convolution operation of the last kernel is normalized using BN and then superimposed with the input of the RCU to obtain the output. Preprocessing the acquired natural images using the SCPA module first converts the input color image into a grayscale image. Then, taking advantage of the characteristic that the grayscale values ​​of distant areas in mountainous images are greater than those of nearby areas, each sampled column in the grayscale image is processed to obtain the pixel gradient of the corresponding column. For each point in the sampling column, determine whether the grayscale difference between the current point and its upper and lower halves (within a distance of 2) is greater than 0. If yes, mark the point as 1; otherwise, mark it as 0. Then, find the location of the maximum pixel gradient in the continuous region marked as 1 as a candidate point to suppress other non-maximum values. Finally, after processing all sampling columns, obtain the binary gradient information map FM. Binary ; G m (y)=f(m·p,y+2)-f(m·p,y+2); ； (x m ,and k )=arg max(G(y));(x m ,and k )∈Area(x m ,k) In the formula, Indicates the position as In a grayscale image, p refers to the sampling interval; Let g be the grayscale gradient at the m-th sampling column position. m (y) is the gradient label, (x) is the gradient label. m ,y k This represents the maximum value of the gradient in the region.

7. The absolute position sensing method under GPS denial conditions according to claim 1, characterized in that, When training the semantic segmentation network based on Deeplabv3+, the natural panoramic images obtained by stitching in step A are used to create labels for the skyline in the panoramic images to obtain the training set. The training set consists of 150 panoramic natural images with a size of 800*600. The training cycle is 150 epochs, the batch size is set to 8, and the parameters of the semantic segmentation network based on Deeplabv3+ are optimized using the stochastic gradient descent algorithm. The initial learning rate is 0.001 and the learning rate is adjusted using cosine decrementing. The momentum is set to 0.9 and the weight descent is set to 0.0005. During training, the skyline feature map FM generated after passing through the GhostNet network and the CCAM attention mechanism is used. Out The weight parameters of the Mask branch are optimized by calculating the loss using the region label information; CAAM's Mask branch uses binary cross-entropy as the loss function, as shown in the following formula: ； In the formula, N is the total length of the region probability map after it is expanded into a vector, and y is the region label. This is the predicted value for the Mask branch.

8. The absolute position sensing method under GPS denial conditions according to claim 1, characterized in that, The specific processing procedure for step C is as follows: Step C1: Reading and converting DEM data; First, the data is read and stored in matrix form. The GetGeoTransform() method in the GDALDataset class of GDAL is used to obtain the coordinate information of the DEM data. The obtained information is stored in array form. A set of key coefficients is obtained, and the matrix row and column numbers are converted to geographic coordinates using a formula: X = gt[0] + col * gt[1] + row * gt[2] Y=gt[3]+col*gt[4]+row*gt[5]; Where col and row represent column number and row number respectively; gt[i] represents the six key coefficients obtained; gt[0] and gt[3] represent the geographic coordinates at image coordinate (0,0); gt[1] and gt[5] represent the resolution of the image x-axis and y-axis respectively; gt[2] and gt[4] represent the image rotation coefficients; and XY represents the geographic coordinates. Step C2: Generate perspective view; Step C21: Use the frustum function in the vmath library of OpenGL to generate a perspective projection matrix by setting the distance between the near plane rectangle, the far plane, and the near plane. Use the position information of the points on the model to replace the color information of the points on the model to generate a normal perspective view without depth information. Step C22: Output the distance values ​​from the fragment to the viewpoint to the color buffer to obtain a two-dimensional projection of the ground distance field with the viewpoint as the reference point onto the screen, i.e., a distance map. The z-value after the projection transformation is shown in the following formula: ； Where -f and -n are the far and near plane distances, respectively; Step C23: Set the distance z ndc and depth z eye Transforming into a linear relationship, we get: ； Step C3: DEM skyline extraction: Step C31: First, using the Canny algorithm, the Sobel operator is applied to the DEM image to calculate its first derivatives in the horizontal and vertical directions, thus obtaining the gradient maps of the DEM image in the horizontal and vertical directions. Finally, the gradients and directions of the boundaries of these two gradient maps can be obtained, as shown below: ； in, and This represents the gradient in the x and y directions; Edge_Gradient and Angle represent the gradient and direction, respectively. Step C32: Nonmaximum suppression By comparing the gradient of a pixel with the gradients of two points in the two directions before and after the gradient of this pixel, we can determine whether this pixel is a local maximum gradient point in its neighborhood. If it is not a local maximum gradient point, it is excluded. Step C33: Using a dual-threshold detection and hysteresis boundary method, points above the upper threshold are identified as strong edges, and points below the lower threshold are identified as not edges. Points between the upper and lower thresholds are identified as weak edges. Hysteresis tracking is then performed, meaning that weak edges connected to strong edges are identified as edges, and otherwise they are identified as non-edges.

9. An absolute position sensing system under GPS rejection conditions, characterized in that, include: Natural panoramic image acquisition and stitching unit: stitches discrete natural images acquired by the vehicle-mounted camera into a 360° circular image. Natural Image Panoramic Skyline Extraction Unit: Employs a semantic segmentation network based on Deeplabv3+ to extract the natural panoramic skyline from natural panoramic images; DEM Image Panoramic Skyline Extraction Unit: Based on the remote sensing center, DEM data of the area to be sensed is obtained, OpenGL is used to form a rendering image of the DEM data, and the Canny operator is used to perform edge detection on the rendering image to output the panoramic skyline of the DEM image. Location-aware unit: Using transfer learning, VGG16 is used as the feature encoder of the Siamese neural network. The similarity between the panoramic skyline of the natural image and the panoramic skyline of the DEM image is used as the matching result, thereby perceiving its own location based on the skyline matching result. The natural panoramic image acquisition and stitching unit, the natural image panoramic skyline extraction unit, the DEM image panoramic skyline extraction unit, and the position awareness unit use the absolute position awareness method under GPS rejection conditions as described in any one of claims 1-8 for data processing.

Images