Cross-region emergency positioning method under GNSS unavailable condition

Abstract

The present application belongs to the technical field of positioning and navigation, and particularly relates to a cross-region emergency positioning method under GNSS unavailable conditions. The method comprises the following steps: 1) in an offline state, collecting geographical environment information data, wherein the geographical environment includes three types of urban general environment, post-disaster environment and desert environment; 2) processing the geographical environment information data collected in step 1) by using an improved semantic segmentation algorithm, then synchronously storing the segmented image and geographical space information to construct a database; 3) collecting environment image data in an online positioning mode; 4) processing the environment image data collected in step 3) by using the same semantic segmentation algorithm as that in step 2), searching and matching the processed environment image data in the database established in step 2), calculating a rotation and translation matrix of the image, and finally calculating GNSS information of the current position and feeding back to the user. The method has the effects of high precision and fast positioning.

Description

Technical Field

[0001] This invention belongs to the field of positioning and navigation technology, and specifically relates to a cross-regional emergency positioning method under GNSS unavailability conditions. Background Technology

[0002] Global Navigation Satellite System (GNSS) is a relatively mature positioning technology that has been widely used in many fields. GNSS has advantages such as high positioning accuracy, continuous positioning over long periods, and low investment and maintenance costs, but it also has disadvantages such as low reliability, susceptibility to external environmental interference, and blind spots in positioning signals.

[0003] In practical applications of GNSS positioning, the weak signal strength and publicly available code structure make it susceptible to various forms of interference. Navigation signals are easily affected by complex electromagnetic interference, and malicious deception can easily lead to positioning failures. Even using small electromagnetic jammers to generate RF signals to overwhelm GNSS satellite electromagnetic signals can render receivers tens or even hundreds of kilometers away inoperable. Deceptive jamming techniques can cause users to receive incorrect pseudorange information, which is then interpreted as introducing errors in position calculations, thus deceiving the target. Inertial navigation is also a high-precision positioning method, typically used in conjunction with GNSS information to determine positioning information. However, the accuracy of this traditional passive positioning method gradually decreases with increasing navigation distance, requiring calibration using existing national geodetic control points.

[0004] In emergency situations where users need positioning despite GNSS signal interference, they cannot rely on GNSS for location information. Furthermore, in urban areas or disaster-stricken regions, damaged geodetic control points render inertial navigation equipment (INS) unusable, hindering effective calibration and making precise positioning difficult. In desert and Gobi environments, finding nearby geodetic control points is often challenging. Guiding data from existing control points tens or even hundreds of kilometers away via traverse surveying cannot meet timeliness requirements. Additionally, high-precision unification of location information within the same coordinate system is a crucial requirement in future network architectures. Traditional map-based emergency positioning methods are increasingly inadequate for meeting these high-precision demands. Summary of the Invention

[0005] This invention addresses the problem in existing technologies where it is difficult to achieve accurate and rapid positioning using satellite positioning equipment in cross-regional emergency positioning under GNSS unavailability conditions. It proposes a cross-regional emergency positioning method and system for GNSS unavailability conditions, applicable to use in ordinary urban conditions, post-disaster environments, and desert environments.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0007] A cross-regional emergency positioning method under GNSS unavailability conditions includes the following steps:

[0008] Step 1: Collect geographic environment information data in offline mode, that is, collect geographic environment information data during normal work and training. The geographic environment includes three types: urban general environment, post-disaster environment and desert environment.

[0009] Step 2: The geographic environment information data collected in Step 1 is processed using an improved semantic segmentation algorithm, and then the segmented images and geospatial information are stored synchronously to build a database.

[0010] Step 3: Online positioning and environmental image data acquisition;

[0011] Step 4: Process the environmental image data collected in Step 3 using the same semantic segmentation algorithm as in Step 2. Retrieve and match the environmental image data processed in Step 3 in the database established in Step 2, calculate the rotation and translation matrix of the image, and finally calculate the GNSS information of the current location and feed it back to the user.

[0012] Preferably, the semantic segmentation algorithm in step 2 is as follows:

[0013] Step 2.1: Select the geographic environment information data collected in Step 1 as the training set;

[0014] Step 2.2: Build the improved IRN neural network using PyCharm IDE;

[0015] Step 2.3: Use the images in the training set of Step 1 as input data for the IRN neural network constructed in Step 2, and train the IRN network constructed in Step 2 to obtain the weights for image segmentation;

[0016] Step 2.4: Use the IRN network weights trained in Step 3 to re-segment the input image.

[0017] Preferably, in step 4, for ordinary urban environments, the matching algorithm is an image matching algorithm; for post-disaster environments, the matching algorithm is a multi-frame image matching algorithm; for desert environments, the retrieved images are first extracted using the Canny edge detection operator to extract ridgelines, and then matched using the multi-frame image matching algorithm.

[0018] Preferably, in step 4, the image matching algorithm for matching in a typical urban environment is as follows:

[0019] Step 4.1.1: For the environmental image data retrieved from the database in Step 2, the FAST algorithm is used to detect feature points, specifically detecting the pixel values ​​around the candidate feature points.

[0020]

[0021] Where c(p) represents circle p, I(x) is the gray level of any point on the circumference, I(p) is the gray level of the center of the circle, and ε d The threshold for the difference in gray values: if N is greater than a given threshold, then p is considered a feature point;

[0022] Step 4.1.2: After obtaining feature point p, the BRIEF algorithm is used to calculate a descriptor for feature point p. Specifically, N pairs of points are selected around feature point p, and the comparison results of these N pairs of points are combined as a descriptor. When the similarity between two descriptors is greater than 90%, the two points are determined to be a matching pair.

[0023] Step 4.1.3: After obtaining N sets of matching pairs through steps 4.1.1 and 4.1.2, the RANSAC algorithm is used to optimize these N sets of matching pairs to prevent mismatches.

[0024] Preferably, in step 4, for post-disaster environments, the matching algorithm used is a multi-frame image matching algorithm. This algorithm simultaneously associates N images, one of which is the image to be matched (IA), and the other N-1 images are the top N-1 retrieval results with the highest similarity in the image retrieval process. Specifically:

[0025] Step 4.2.1: When a new image IA to be matched is obtained, the first N-1 images with the highest similarity are obtained through image retrieval;

[0026] Step 4.2.2: Using the same method as steps 4.1.1 and 4.1.2, extract feature points on IA and calculate their descriptors;

[0027] Step 4.2.3: Match IA with the N-1 images obtained from the previous image retrieval;

[0028] a) Use the KNN matching algorithm to extract matching pairs corresponding to feature points in IA. ;

[0029] b) Compare the SIFT matching between the nearest neighbor distance and the second nearest neighbor distance. That is, take a SIFT keypoint in one image and find the two keypoints in another image that are closest to it in terms of Euclidean distance. If the ratio obtained by dividing the nearest distance by the second nearest distance between these two keypoints is between 0.4 and 0.6, then accept this pair of matching points. If the ratio is less than 0.4, then there are not enough matching pairs, and return to the beginning of step b to start again.

[0030] c) Solve the matching problem for N images to obtain the final corresponding L N ;

[0031] Step 4.2.3.4: If this is not the last image to be matched, return to step 1; otherwise, stop.

[0032] Preferably, in step 4, matching is performed under three different environments. For the cumulative error generated during the positioning process, loop closure detection and correction are applied. Based on local loop closures, the RANSAC method is optimized to calculate the rotation and translation matrix. The main steps of loop closure detection and correction are as follows:

[0033] 1) Calculate the similarity between the image to be matched and the N-1 similar images retrieved, and take the minimum value among them. When detecting loop closure images, the similarity between the image to be matched and the image to be matched must be greater than this minimum value.

[0034] 2) Obtain candidate images for loop closure and save them in a list. The list will then be purged twice. First, candidate images with a similarity greater than 80% of their maximum value will be retained. Second, the sum of the similarities between the candidate images and their adjacent images in all retrieved images will be calculated, and candidate images with a sum greater than 75% of their maximum value will be retained.

[0035] 3) Further consistency checks are performed on the obtained candidate images. If the current frame passes the consistency check, no loop closure is detected; if it fails the consistency check, a loop closure is detected.

[0036] 4) After detecting a loopback frame, calculate the rotation and translation matrix between the current image and the loopback image;

[0037] 5) The calculated rotation and translation matrix can be used to adjust the pose of the current image. By utilizing the pose relationship between two previously known frames, all poses can be adjusted.

[0038] Preferably, in step 4, in a desert environment, the specific algorithm for extracting ridgelines using the Canny edge detection operator in step 4 is as follows:

[0039] Step 4.3.1: Calculate the gradient and its direction of the retrieved image;

[0040] Step 4.3.2: After calculating the gradient intensity and direction of each image in step 4.3.1, the non-maximum suppression edge thinning method is used to retain the points with the largest local gradient in order to obtain the thinned edge;

[0041] Step 4.3.3: Apply dual threshold detection to determine the real and potential ridge edges.

[0042] Compared with the prior art, the beneficial effects of the present invention are:

[0043] 1. Improvements and innovations in the application algorithm: An improved semantic segmentation algorithm was used to enhance the segmentation accuracy and effect for sparse feature regions such as urban post-disaster street scenes, deserts, and Gobi deserts.

[0044] 2. This invention improves and integrates algorithms for use in urban post-disaster areas and sparsely characterized areas such as deserts and Gobi. It includes semantic segmentation algorithms, image retrieval algorithms, feature extraction and description algorithms, image matching algorithms, and loop closure detection. Traditional algorithms are singular and easily affected by the environment, leading to inaccurate positioning. This invention integrates these technologies to provide a more convenient and accurate online positioning technology, achieving real-time precise positioning. Furthermore, the algorithms of this invention have been improved and adapted for three different environments, making them more targeted and adaptable, and achieving better positioning and pose calculation accuracy compared to the original technologies. Attached Figure Description

[0045] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0046] In the attached diagram:

[0047] Figure 1 This is a schematic diagram illustrating the operation of the present invention in an urban environment;

[0048] Figure 2 This is a technical roadmap of the method of the present invention;

[0049] Figure 3 This is a block diagram of the system of the present invention;

[0050] Figure 4 This is the overall system operation flow of the present invention;

[0051] Figure 5 This is a database configuration diagram for the present invention;

[0052] Figure 6 This is a flowchart illustrating the GNSS information calculation process of this invention.

[0053] Figure 7 This is a flowchart of the method of the present invention;

[0054] Figure 8 This is the structure of CBAM_A in this invention. Detailed Implementation

[0055] The preferred embodiments of the present invention will be described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0056] Figure 1 This is a schematic diagram of the operation of the method of the present invention. After the vehicle is equipped with an on-board graphics workstation, a binocular camera, and a combined inertial navigation system, it can perform routine data collection, capture images of the geographical environment, store the corresponding GNSS information, process the data to extract features, and store the images, the corresponding GNSS information, and the extracted features in the on-board graphics workstation.

[0057] The combined inertial navigation system is used to record the shooting position information corresponding to each image when acquiring data offline and online. When used offline, it can read the GNSS position information of similar images when matching data sets. At the same time, the inertial navigation system is an autonomous navigation system that does not rely on external information and does not radiate energy to the outside.

[0058] The basic working principle of inertial navigation is based on Newton's laws of motion. By measuring the acceleration of the carrier in the inertial reference frame, integrating it over time, and transforming it into the navigation coordinate system, information such as velocity, yaw angle, and position in the navigation coordinate system can be obtained. This information can be used to calculate offline GNSS information.

[0059] The vehicle-mounted graphics workstation mainly consists of a high-performance laptop. This research requires the collection and processing of a large dataset, while also needing to be easily portable within the vehicle. The workstation's functions include offline dataset collection and online image matching with a topline database based on currently captured images. Based on successfully matched images, the workstation calculates the latitude and longitude information of the current camera to correct errors accumulated by the inertial navigation system over time.

[0060] This invention requires the collection of a large dataset, the processing of the dataset, and its storage. For datasets stored in an offline database, the system needs to quickly retrieve the most similar image when it is online. Therefore, the tower-type deep learning graph second-hand workstation requires high-performance graphics processing and storage space to meet the real-time requirements of online real-time positioning.

[0061] In emergency positioning mode, images near the image acquisition site are captured using a binocular camera and input into the vehicle-mounted graphics workstation. The rotation transformation matrix of the image and the retrieved image is calculated through image semantic segmentation, image retrieval technology and image matching technology, and finally the GNSS information of the location to be located is calculated.

[0062] The technical approach and main research content of this invention are as follows: Figure 2 As shown. The main research contents are: damage-resistant positioning technology for urban areas, sparse feature positioning technology for Gobi and desert areas, geospatial feature information extraction and storage technology, and development and application of geospatial feature information software and hardware systems.

[0063] The technical modules of the present invention are as follows Figure 3 As shown, the modules are arranged from left to right as follows: the implementation process of the offline system and the online system; the cross-regional emergency positioning method and system under GNSS unavailability conditions; and the development and construction of the emergency positioning system and hardware system. The emergency positioning system includes both offline and online systems. The offline system includes: an offline data acquisition module, an offline image processing module, and an image storage module; the online system includes: an online data acquisition module, an online image processing module, and a real-time positioning module.

[0064] Offline data acquisition module: By building an onboard hardware system, this module primarily collects geographic environmental information during offline training, ensuring it does not interfere with the user's daily life and training. The onboard hardware system includes: a binocular camera, a combined inertial navigation system, an onboard graphics workstation, and a tower-type deep learning graphics workstation. The binocular camera is mounted on the vehicle roof using a camera mount. The roof-mounted binocular camera has an adjustable mounting platform, which is detachable and mounted on the roof. Both the mount and the platform are existing technologies and will not be described in detail here.

[0065] Offline Image Processing Module: This module uses machine vision methods to process the acquired offline data. To meet the high real-time requirements of users' online work, it is necessary to filter complex urban road information, removing useless information such as pedestrians, vehicles, and sky, reducing the complexity of subsequent feature extraction and shortening the time for image retrieval and feature matching. Specifically, it performs semantic segmentation on the images to generate urban road images with useful information removed, and then extracts features from these images to generate depth images.

[0066] Image Storage Module: In this database, to optimize retrieval efficiency, images are stored in the computer's file system. The database can access the stored images via links. This significantly reduces storage space requirements. Based on the GNSS information retrieved from the previous query, combined with the current speed of the navigation vehicle and the query interval, database space is allocated to optimize retrieval efficiency. Additionally, the depth images generated by the offline data processing module will define the search range according to region. Furthermore, the search range can also be defined by pre-planning routes. These coarse filtering operations can constrain the query within a certain range, reducing computer workload and accelerating retrieval efficiency.

[0067] Online data acquisition module: Through the hardware system installed in the vehicle, mainly binocular camera equipment, it acquires image data of the online environment.

[0068] Online image processing module: Based on the online data acquisition module, the images acquired by the online data acquisition module are preprocessed: stereo matching of left and right views is performed to generate a disparity map, and semantic segmentation of the images is performed to remove unimportant information, so as to facilitate subsequent image retrieval, matching and other operations by the real-time positioning module.

[0069] Real-time positioning module: Addressing the challenge of users being unable to quickly obtain accurate positioning information when GNSS becomes unavailable online, the real-time positioning module can rapidly acquire real-time geographic location and spatial information. Its working steps are as follows: After processing the image through the online image processing module, image retrieval is used to find the top N images (N=3) most similar to the currently captured image in the offline database. Multi-frame image matching is performed between the captured image and the retrieved images, and the rotation and translation matrix is ​​calculated. This allows the calculation of the rotation and translation relationship between the online real-time captured image and the successfully matched images in the offline database, obtaining the GNSS information of the real-time captured image, which is then fed back to the user.

[0070] Please see Figure 4The figure shows the overall flowchart of the system for this invention, illustrating the specific data processing methods of the offline and online systems. Offline: The binocular camera acquires binocular images and associates them with GNSS and timestamp information acquired by the integrated inertial navigation system. The backend preprocesses and semantically segments the acquired binocular images, removing invalid information. After these steps, feature extraction and stereo matching are performed on the binocular images to generate depth information. Finally, the images and depth information are stored in the offline database. Online: The backend enters the online system. Without the assistance of the integrated inertial navigation system, GNSS information cannot be acquired. The binocular camera captures binocular images. The backend preprocesses and semantically segments the images, extracts feature points from the processed images, and performs stereo matching to generate depth information. Then, the system performs image retrieval in the offline database, finding the multiple images most similar to the current image. Multi-frame matching is performed between the captured image and the retrieved images to calculate the rotation and translation matrix. The multi-frame pose optimization algorithm is used to optimize the rotation and translation matrix, and GNSS information is calculated in real time. The calculated GNSS information is transmitted to the front end, where the front-end map module displays the GNSS coordinates and vehicle route in real time, and optimizes the path to keep the vehicle on the road.

[0071] The following will refer to the appendix of this invention patent. Figure 5-7 The method described in this invention patent will be clearly and completely described.

[0072] Example 1:

[0073] A cross-regional emergency positioning method under GNSS unavailability conditions includes the following steps:

[0074] Step 1: In offline mode, use the hardware system of the cross-regional emergency positioning system to collect general urban environmental information data. That is, collect geographic environmental information data during normal work and training to ensure that it does not interfere with the user's daily life and training.

[0075] Step 2: The geographic environment information data collected in Step 1 is processed using an improved semantic segmentation algorithm. The segmented images and geospatial information are then synchronously stored to construct a database. Specifically, the semantic segmentation algorithm is as follows:

[0076] Step 2.1: Select the geographic environment information data collected in Step 1 as the training set;

[0077] In this invention, the training set uses 1500 training images for weight training of the semantic segmentation algorithm. Of these, 1000 images are from a publicly available dataset containing road information, and 500 are self-collected scene images of a road in Xi'an. The publicly available dataset used is Cityscapes. The dataset is divided into an 8:1:1 structure, with 80% used as the training set, 10% as the validation set, and 10% as the test set. For the self-collected images, LabelMe software is used for data annotation. Each image in the dataset is labeled according to the nineteen defined categories.

[0078] Step 2.2: Build the improved IRN neural network using PyCharm IDE;

[0079] The IRN neural network uses an encoder-decoder structure. The overall process is as follows: First, multi-scale pyramid convolutions are introduced into the backbone network. Second, the standard convolutions in the dilated spatial convolutional pooling pyramid are replaced with depthwise separable convolutions to reduce the number of parameters in the overall model. Finally, a multi-scale method is used in the decoding layer to capture the global background. The background features are then combined with shallow features and the dilated spatial pyramid pooling layer through an attention mechanism to enrich the semantic information of the fused shallow features.

[0080] The detailed process is as follows:

[0081] The encoder uses a residual network with pyramidal convolutions as its backbone. By introducing pyramidal convolutions into the residual blocks, different depths of convolution kernels can be used in different layers of the backbone network, dividing the input features into different groups and performing convolution calculations independently.

[0082] The improved pyramid convolution is introduced by dividing the input features into different groups and performing convolution calculations independently. When combined with the backbone network, considering the reduction in the spatial size of the feature maps, the number of branches gradually decreases. In the initial stage, the feature maps pass through four branches, and in the final stage, the feature maps pass through one branch. Compared with standard convolution, pyramid convolution can expand the receptive field of the kernel without increasing computational cost. It can also apply different types of kernels in parallel to process the input, with different spatial resolutions and depths. Therefore, the new residual blocks can capture more detailed information. In the first layer of the network, the pyramid convolution kernel divides the output channels into four groups, with each group using convolution kernel sizes of 3×3, 5×5, 7×7, and 9×9, and the number of output channels being 64, 64, 64, and 64, respectively. The corresponding number of grouped convolutional groups are 1, 4, 8, and 16, respectively. In the second layer, the output channels are divided into three groups, with each group using convolution kernel sizes of 3×3, 5×5, and 7×7, and the number of output channels being 128, 128, and 256, respectively. The corresponding number of grouped convolutional groups are 1, 4, and 8, respectively. The third layer divides the output channels into two groups, each using a convolution kernel of 3×3 and 5×5, with 512 output channels in each group, and 1 and 4 convolution groups in the corresponding groups. The fourth layer uses a 3×3 convolution kernel to convolve all channels, with 1 group.

[0083] In the decoding layer, depthwise separable convolutions are introduced into the dilated spatial pooling convolution pyramid. This can reduce the model size and the number of model parameters without significantly affecting the results, thereby accelerating training efficiency and even enabling the model to run on servers with lower configurations.

[0084] Depthwise separable convolution is a two-step process that significantly reduces the number of model parameters and improves training efficiency with only a small loss of accuracy. Depthwise separable convolution decomposes traditional convolution into a depthwise convolution and a 1×1 convolution.

[0085] The first step of depthwise separable convolution is layer-by-layer convolution, which performs convolution on each input channel with a single convolution kernel. Each convolution kernel only convolves on one channel of the input layer, so the number of convolution kernels is equal to the number of channels.

[0086] The second step is to increase the depth. Pointwise convolution is used to achieve pixel fusion between channels. A 1×1 convolution kernel is used to create a linear combination of depthwise convolutional layers. This second step involves performing a regular 1×1 convolution on the input, which is equivalent to collecting features from each point.

[0087] Traditional convolution operations consider both channels and regions simultaneously, while depthwise separable convolution considers only regions first, and then channels. This achieves separation between channels and regions.

[0088] In the dilated spatial pooling pyramid, ordinary convolutions are replaced with depthwise separable convolutions. First, a channel-wise 3×3 convolution is performed on the input feature map to separate the channels. Then, 1×1 convolutions are used for feature concatenation. The improved ASPP module... Figure 2 This is reflected in the ASPP module. Finally, the output feature maps are concatenated and their channel count is reduced by depthwise separable convolution, removing useless features.

[0089] Replacing the depthwise separable convolution module with the ASPP module can reduce model parameters and accelerate model training efficiency with minimal impact on model accuracy.

[0090] By using features learned at multiple scales to enrich the representation of shallow features, this approach helps encode both global and local contexts. This invention proposes a novel multi-scale attention mechanism combined network, which, after combination, is embedded into the original Deeplab v3+ decoding layer through an improved attention mechanism, CBAM_A.

[0091] Specifically, in the configuration of this invention, features at multiple scales are represented as F. S , where S represents the layer number of the feature map. Since the features of each layer have different resolutions, they are upsampled to the same resolution using bilinear interpolation, resulting in an enlarged feature map represented as F′. S The corresponding S represents the layer number corresponding to the feature map. Then, without changing the original network structure, F1 is output through a 1×1 convolution, and subsequent F′2, F′3, and F′4 are concatenated to form a tensor. This tensor is then convolved to obtain a multi-scale feature map, which is then processed through the CBAM_A module to obtain F. MS Among them, F MS As shown in equation (1):

[0092] F MS =CBAM_A(Conv(F2′+F3′+F4′)) (1)

[0093] Therefore, F MS Encode low-level features from shallow layers and high-level semantic features from deeper layers, and increase the weight layers of features through an attention mechanism so that the model can ultimately perform better features.

[0094] The structure of CBAM_A is as follows Figure 8 As shown, CBAM_A primarily replaces ordinary convolutions in the structure with depthwise separable convolutions, reducing the number of computational parameters. The main purpose of introducing CBAM_A is to introduce attention mechanisms in both the channel and spatial dimensions, making the extracted features more refined and improving the model's performance.

[0095] CBAM_A introduces an attention mechanism in both channel and spatial dimensions, making the extracted features more refined and improving the model's performance.

[0096] The channel attention module is characterized by maintaining the channel dimension while compressing the spatial dimension, primarily focusing on meaningful information in the input feature map. The specific operation is as follows: First, the input feature map undergoes two parallel max pooling and average pooling operations, transforming the C×H×W feature map into a C×1×1 feature map. Then, it passes through a SharedMLP (Conv / Linear, ReLU, Conv / Linear) to compress the number of channels to C / r, where r (reduction) is typically set to 16. The number of channels is then expanded back to C, resulting in two activated values. Finally, these two values ​​are summed and passed through a Sigmoid activation function to obtain the channel output. Multiplying the channel output by the initial input restores the feature map size to C×H×W, completing the channel attention module operation.

[0097] The spatial attention module is characterized by maintaining the spatial dimension while compressing the channel dimension, focusing on the target's positional information. Its operation is as follows: The results from the channel attention module are processed through max pooling and average pooling to obtain two tensors of size H×W×1. These two tensors are then stacked together through a concatenation operation to form a tensor of size H×W×2. A convolution operation is then performed to convert channel 2 to channel 1, keeping the feature map's width and height unchanged in this step. The output is then passed through a sigmoid activation function to obtain the spatial attention output. Finally, the spatial attention output is multiplied by the input from the previous step to obtain a feature map of size C×H×W, completing the spatial attention operation.

[0098] Finally, the semantic information of the richly fused image features is processed by 3×3 convolution and then upsampled by 4 times to obtain the segmentation map.

[0099] Step 2.3: Use the images in the training set of Step 1 as input data for the IRN neural network constructed in Step 2, and train the IRN network constructed in Step 2 to obtain the weights for image segmentation;

[0100] This invention standardizes the size of each image to 1024×2048 before inputting it into the IRN network, using it as the final training set.

[0101] The device used in this invention has an Intel(R) Core(TM) i7-10870H CPU@2.20GHz, an NVIDIA GeForce GTX 3080 GPU, and a Windows 10 operating system. It is implemented using Python 3.7 based on the PyTorch network framework and accelerated using the CUDA 11.1 library.

[0102] During training on the dataset, the Stochastic Gradient Descent (SGD) optimization algorithm was used, with momentum set to 0.9, weight decay set to 0.0001, and the base learning rate set to 0.1. The learning rate decay used a "Poly" decay strategy, and end-to-end training was performed via backpropagation. The maximum number of iterations was set to 50,000 during training on the training set.

[0103] This invention utilizes transfer learning to fine-tune the weights of a pre-trained model trained on ImageNet using PyConvResNet-50. Transfer learning effectively prevents gradient vanishing and gradient exploding problems, resulting in faster network training and convergence, saving learning time and improving learning efficiency.

[0104] The training loss function uses the cross-entropy function, which is mainly used to measure the difference between two probability distributions. The formula for the multi-class cross-entropy loss function is as follows: (2)

[0105]

[0106] Where M is the number of categories; y ic It is an indicator variable; it is 1 if the category is the same as that of sample i, and 0 otherwise; p ic It is the predicted probability that observed sample i belongs to category c.

[0107] The evaluation method used in this paper is the average intersection-union ratio (MIoU), and the formula for the average intersection-union ratio is shown in formula (3) below:

[0108]

[0109] Where p ii This represents the total number of pixels whose actual category is i and whose true category is i. ji p represents the total number of pixels whose actual class is j and whose predicted class is i. ij This represents the total number of pixels whose actual category is i and whose predicted category is j, where k represents the number of categories. In this patent, k = 19.

[0110] Step 2.4: Use the IRN network weights trained in Step 3 to re-segment the input image.

[0111] Step 3: Online positioning. Environmental image data is collected through the hardware system of the cross-regional emergency positioning system, mainly through the hardware system installed in the vehicle, which is a binocular camera device.

[0112] Step 4: Process the environmental image data collected in Step 3 using the same semantic segmentation algorithm as in Step 2. Retrieve and match the processed environmental image data in the database established in Step 2, calculate the rotation and translation matrix of the image, obtain the distance and heading angle, further obtain the GNSS information of the most similar image in the offline database, and finally calculate the GNSS information of the current location using the latitude and longitude calculation formula and feed it back to the user.

[0113] For typical urban environments, the matching algorithm used is an image matching algorithm, specifically:

[0114] Step 4.1.1: For the environmental image data retrieved from the database in Step 2, the FAST algorithm is used to detect feature points, specifically detecting the pixel values ​​around the candidate feature points.

[0115]

[0116] Where c(p) represents circle p, I(x) is the gray level of any point on the circumference, I(p) is the gray level of the center of the circle, and ε d The threshold for the difference in gray values: if N is greater than a given threshold, then p is considered a feature point;

[0117] Step 4.1.2: After obtaining feature point p, the BRIEF algorithm is used to calculate a descriptor for feature point p. Specifically, N pairs of points are selected around feature point p, and the comparison results of these N pairs of points are combined as a descriptor. When the similarity between two descriptors is greater than 90%, the two points are determined to be a matching pair.

[0118] Step 4.1.3: After obtaining N sets of matching pairs through steps 4.1.1 and 4.1.2, the RANSAC algorithm is used to optimize these N sets of matching pairs to prevent mismatches.

[0119] In step 4, loop closure detection and correction are applied to address the cumulative error generated during the positioning process. Based on local loop closures, the RANSAC method is optimized to calculate the rotation and translation matrix. The main steps of loop closure detection and correction are as follows:

[0120] 1) Calculate the similarity between the image to be matched and the N-1 similar images retrieved, and take the minimum value among them. When detecting loop closure images, the similarity between the image to be matched and the image to be matched must be greater than this minimum value.

[0121] 2) Obtain candidate images for loop closure and save them in a list. The list will then be purged twice. First, candidate images with a similarity greater than 80% of their maximum value will be retained. Second, the sum of the similarities between the candidate images and their adjacent images in all retrieved images will be calculated, and candidate images with a sum greater than 75% of their maximum value will be retained.

[0122] 3) Further consistency checks are performed on the obtained candidate images. If the current frame passes the consistency check, no loop closure is detected; if it fails the consistency check, a loop closure is detected.

[0123] 4) After detecting a loopback frame, calculate the rotation and translation matrix between the current image and the loopback image;

[0124] 5) The calculated rotation and translation matrix can be used to adjust the pose of the current image. By utilizing the pose relationship between two previously known frames, all poses can be adjusted.

[0125] The cross-regional emergency positioning system includes a hardware system and a software system. The hardware system includes a binocular camera, a combined inertial navigation system, a vehicle-mounted graphics workstation, and a tower-type deep learning graphics workstation. The software system includes an offline system and an online system. The offline system includes an offline data acquisition module, an offline image processing module, and an image storage module. The online system includes an online data acquisition module, an online image processing module, and a real-time positioning module.

[0126] Example 2:

[0127] Unlike Embodiment 1, this invention addresses cross-regional emergency positioning under GNSS unavailability conditions in post-disaster environments. Traditional image feature matching methods are based on two image frames. However, post-disaster images in cities differ significantly from those taken in normal times, due to factors such as damaged buildings and road signs, smoke obscuring images, seasonal changes, and variations in light intensity. Traditional feature matching algorithms are prone to generating false matches between the two images, leading to increased errors in the pose transformation matrix. As the system continues to operate, this error accumulates, eventually causing positioning accuracy to spiral out of control. Furthermore, if the overlap area between IA and IB is too small, resulting in insufficient numbers of similar features, traditional feature matching algorithms will also fail, affecting GNSS positioning accuracy.

[0128] Therefore, this invention proposes a cross-regional emergency positioning method under GNSS unavailability conditions in a post-disaster environment, which differs from the specific embodiments in that:

[0129] This embodiment uses a multi-frame image matching algorithm, which simultaneously associates N images. One of these images is the image to be matched, IA, and the other N-1 images are the top N-1 retrieval results with the highest similarity in the image retrieval process. Specifically:

[0130] Step 4.2.1: When a new image IA to be matched is obtained, the first N-1 images with the highest similarity are obtained through image retrieval;

[0131] Step 4.2.2: Using the same method as steps 4.1.1 and 4.1.2, extract feature points on IA and calculate their descriptors;

[0132] Step 4.2.3: Match IA with the N-1 images obtained from the previous image retrieval;

[0133] a) Use the KNN matching algorithm to extract matching pairs corresponding to feature points in IA. ;

[0134] b) Compare the SIFT matching between the nearest neighbor distance and the second nearest neighbor distance. That is, take a SIFT keypoint in one IA image and find the two keypoints in another image that are closest to it in terms of Euclidean distance. If the ratio obtained by dividing the nearest distance by the second nearest distance between these two keypoints is between 0.4 and 0.6, then accept this pair of matching points. If the ratio is less than 0.4, then there are not enough matching pairs, and return to the beginning of step b to start again.

[0135] c) Solve the matching problem for N images to obtain the final corresponding L N The knnMatch function is used here. A match is considered successful only when the Euclidean distance between two feature points is less than a certain value, and a matching pair is output.

[0136] Step 4.2.3.4: If this is not the last image to be matched, return to step 1; otherwise, stop.

[0137] Meanwhile, to verify the effectiveness of the proposed multi-frame image matching method, this invention designs an image "contamination" method to simulate the damage and occlusion of road signs after a wartime disaster. Due to the uncertainty of war damage, the damage experiment in this invention is based on randomness. For the input original image, to enhance the rationality of the damaged image, this algorithm randomly destroys buildings in the image, and the destroyed area is larger than 100×100. This algorithm is based on semantic segmentation. First, the original image is input. Then, the semantic segmentation algorithm performs semantic segmentation processing on the original image, classifying each pixel in the image and generating a classification matrix. Based on the semantic segmentation results, building parts are randomly searched, and the building areas are randomly destroyed, so that the pixel value of the building area is 0. The processed image matrix is ​​then calculated with the original image to obtain the damaged image. When a multi-frame image matching algorithm is introduced, the "weak" feature points existing in multiple images can provide support for correct matching, realizing comprehensive matching and effectively improving the matching accuracy.

[0138] Example 3:

[0139] Unlike Example 2, this example is a cross-regional emergency positioning method for use in desert environments where GNSS is unavailable.

[0140] Because the geographical features of desert environments are not obvious, and trees and other features vary greatly due to seasonal and environmental influences, ridgelines are used as the features to be extracted. In step 4 of this embodiment, the retrieved image is first processed using the Canny edge detection operator to extract ridgelines, and then a multi-frame image matching algorithm is used for matching. Specifically, the Canny edge detection operator is as follows:

[0141] Step 4.3.1: Calculate the gradient and its direction of the retrieved image;

[0142] Step 4.3.2: After calculating the gradient intensity and direction of each image in step 4.3.1, the non-maximum suppression edge thinning method is used to retain the points with the largest local gradient in order to obtain the thinned edge;

[0143] Step 4.3.3: Apply dual threshold detection to determine the real and potential ridge edges.

[0144] A cross-regional emergency positioning system under GNSS unavailability conditions includes a hardware system and a software system. The hardware system includes a binocular camera, a combined inertial navigation system, a vehicle-mounted graphics workstation, and a tower-type deep learning graphics workstation. The software system includes an offline system and an online system. The offline system includes an offline data acquisition module, an offline image processing module, and an image storage module. The online system includes an online data acquisition module, an online image processing module, and a real-time positioning module.

[0145] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of this invention is defined by the appended claims and their equivalents.

Claims

1. A cross-region emergency positioning method in a GNSS unavailable condition, characterized in that: Comprise the following steps: Step 1: offline state, collect geographic environment information data, that is, collect geographic environment information data during normal work training, wherein the geographic environment includes three kinds of urban ordinary environment, post-disaster environment and desert environment; Step 2: the geographic environment information data collected in step 1 is processed by using an improved semantic segmentation algorithm, and then the segmented image and geographic spatial information are stored and constructed into a database; The semantic segmentation algorithm in step 2 is specifically: Step 2.1: select the geographic environment information data collected in step 1 as the training set; Step 2.2: build an improved IRN neural network through PyCharm IDE; Wherein the structure of the improved IRN neural network is encoder-decoder, and the overall process is: in the backbone network, introduce multi-scale pyramid convolution; Secondly, replace the standard convolution in the empty space convolution pyramid with depth separable convolution to reduce the parameter quantity of the overall model; Finally, use the multi-scale method to capture the global background in the decoding layer, and combine the background features through the attention mechanism, and then combine with the shallow features and the empty space pyramid pooling layer, to enrich the semantic information of the fused shallow features; Step 2.3: the image in the training set of step 2.1 is used as the input data of the IRN neural network constructed in step 2.2, and the IRN network constructed in step 2.2 is trained to obtain the weight of image segmentation; Step 2.4: use the trained IRN network weight in 2.3 to segment the input image; Step 3: online positioning, collect environment image data; Step 4: the environment image data collected in step 3 is processed by using the same semantic segmentation algorithm as step 2, and the processed environment image data is searched and matched in the database established in step 2, the rotation translation matrix of the environment image obtained in step 3 is calculated, the distance and heading angle are obtained, the GNSS information of the most similar image in the offline database is further obtained, and finally the GNSS information of the current position is calculated by using the latitude and longitude calculation formula and fed back to the user; In step 4, for the city ordinary environment, the matching algorithm adopts an image matching algorithm; For post-disaster environment, the matching algorithm adopts a multi-frame graph matching algorithm; For desert environment, the extracted ridge line is first used Canny edge detection operator to extract ridge line, and then multi-frame graph matching algorithm is used for matching; In step 4, for the city ordinary environment, the image matching algorithm for matching is specifically: Step 4.1.1: for the environment image data searched in step 2 database, the FAST algorithm is used to detect feature points, specifically to detect the pixel value around a circle of candidate feature points, ； wherein, with c(p) denotes the circle p, I(x) is the gray value of an arbitrary point on the circumference, I(p) is the gray value of the center, a threshold value for the difference in gray value, if N is greater than a given threshold value, p is considered to be a feature point; Step 4.1.2: after obtaining the feature point p, the BRIEF algorithm is used to calculate the descriptor of a feature point p, specifically, N points are selected around the feature point P, and the comparison results of the N point pairs are combined as the descriptor, when the similarity of two descriptors is greater than 90%, it is determined that the two points are a matching pair; Step 4.1.3: N sets of matching pairs are obtained by step 4.1.1 and step 4.1.2, and the RANSAC algorithm is used to optimize the N sets of matching pairs to prevent false matching; In step 4, in the post-disaster environment, the matching is performed by a multi-frame graph matching algorithm, which simultaneously associates N images, one of which is the image to be matched IA, and the other N-1 images are the top N-1 search results with the highest similarity in the image search process. Specifically, the top N-1 search results with the highest similarity are obtained by image search when a new image to be matched IA is obtained. Step 4.2.1: When a new image to be matched IA is obtained, the top N-1 images with the highest similarity are obtained by image search. Step 4.2.2: The same method as steps 4.1.1 and 4.1.2 is used to extract feature points from IA and calculate their descriptors. Step 4.2.3: Match IA with the N-1 images obtained by image search. a) using KNN matching algorithm to extract the matching pairs corresponding to the feature points in the IA ; b) Compare the nearest neighbor distance with the second nearest neighbor distance SIFT matching, that is, take a SIFT key point in one image, and find the top two key points with the smallest Euclidean distance in another image. Among the two key points, if the ratio of the nearest distance to the second nearest distance is 0.4-0.6, the matching pair is accepted. If the ratio is less than 0.4, there is not enough matching pair, and the process starts again from the beginning of step b. c) solving N image matches to get final correspondences ; Step 4.2.4: If this is not the last image to be matched, go back to step 1, otherwise stop.

2. The cross-region emergency positioning method under GNSS unavailability condition according to claim 1, characterized in that: In step 4, in the three environments, to address the cumulative error generated during the positioning process, loop detection and loop correction are applied. According to the local loop, the RANSAC method is used to calculate the rotation and translation matrix. The main steps of loop detection and loop correction are as follows: 1) Calculate the similarity of the image to be matched with the N-1 similar images obtained by search, and take the minimum value. When detecting the loop image, the similarity with the image to be matched should be greater than the minimum value. 2) Obtain loop candidate images and save them in a list. The list will be pruned twice. First, keep the candidate images with a similarity greater than 80% of the maximum value. Second, calculate the cumulative sum of the similarity of adjacent images with the image to be matched among all search images of the candidate images, and keep the candidate images with a cumulative sum greater than 75% of the maximum value. 3) Further consistency test is performed on the obtained candidate images. If the current frame passes the consistency test, no loop is detected. If it does not pass the consistency test, a loop is detected. 4) After detecting the loop frame, calculate the rotation and translation matrix between the current image and the loop image. 5) The rotation and translation matrix obtained can be used to adjust the pose of the current image. Using the known pose relationship between the two frames, all poses can be adjusted.

3. The cross-region emergency positioning method under GNSS unavailability condition according to claim 2, characterized in that: In step 4, in the desert environment, the specific algorithm for extracting ridge lines using the Canny edge detection operator is as follows: Step 4.3.1: Calculate the gradient and direction of the image after search. Step 4.3.2: After the gradient strength and direction of each image is calculated in step 4.3.1, the non-maximum suppression edge thinning method is used to keep the points with the maximum local gradient to get the thinned edge; Step 4.3.3: The double threshold detection is applied to determine the real and potential ridge edges.

Images