GNSS NLOS data cleaning combination navigation method based on deep learning

Abstract

The invention relates to a GNSS NLOS data cleaning integrated navigation method based on deep learning, and the method comprises the steps: obtaining carrier sky-direction environment image information, carrying out the preprocessing of the carrier sky-direction environment image information, and obtaining a mask image; inputting the mask image and the carrier original sky direction environment image into a DeepLabV3 + model, and obtaining a category label of each pixel in the image; the method comprises the following steps: acquiring a carrier and an elevation angle and an azimuth angle of a GNSS satellite received by the carrier, constructing a satellite projection model according to the elevation angle, the azimuth angle, camera internal reference and carrier azimuth angle information, and acquiring pixel coordinates of the GNSS satellite in a pixel coordinate system of an image labeled by a category label on the basis of the model; and judging the type of an area where the pixel coordinates are located, judging whether the area is a non-line-of-sight signal or not, giving a weight to the non-line-of-sight signal or the line-of-sight signal, carrying out GNSS satellite position calculation according to the weight, and generating a positioning result.

Description

Technical Field

[0001] The present invention relates to the field of navigation technology, and in particular to a GNSS NLOS data cleaning integrated navigation method based on deep learning. Background Art

[0002] In modern navigation systems, the Global Navigation Satellite System (GNSS) is a commonly used positioning technology that can provide users with high-precision location information and plays an increasingly important role in social, economic, and military fields. In a relatively open environment, GNSS can provide more reliable positioning accuracy. However, in complex environments such as urban canyons, forests, and indoors, GNSS signals are easily affected by occlusion, reflection, and multipath effects, resulting in reduced positioning accuracy or even loss of positioning capabilities. Among them, non-line of sight (NLOS) signals are one of the main factors leading to positioning errors.

[0003] At present, many scholars at home and abroad have conducted extensive research on detecting NLOS signals and reducing the errors caused by them. In recent years, thanks to the rapid development of visual technology, the method of identifying NLOS signals based on visual detection has been paid special attention due to its superior performance. Earlier, some scholars proposed to use vision and use binarization to segment and extract the sky and non-sky areas. At present, many scholars also use Fisher clustering algorithm, K-Means clustering algorithm, Canny edge detection and Flood Fill algorithm to segment the sky area. However, in special environments such as overexposure, the color of the building is similar to the color of the sky, the judged sky area will be different from the actual area. In particular, when the center point of the sky environment image is an occluder or the selected center point is an occluder, the sky area extraction using the Flood Fill algorithm will fail, resulting in a completely opposite judgment.

[0004] In terms of weighting NLOS signals, some scholars have taken measures to eliminate them when the number of satellites is small. When the number of satellites is too small, eliminating NLOS signals will affect the geometric precision factor and lead to positioning failure. To address this problem, some scholars have adopted different weighting methods for satellites projected in two areas to replace the elimination operation, thereby compensating for the impact of directly eliminating NLOS signals on the geometric precision factor. When obtaining the carrier's astronomical environmental image information, since the fisheye camera has a wide field of view and panoramic perception of the environment, it can obtain a large range of environmental images with low additional cost investment. By utilizing its advantages, it can better project satellites into images and determine whether they are NLOS signals.

[0005] At present, the research on image segmentation using deep learning is in full swing. Deep learning is used to extract sky areas and non-sky areas, thereby avoiding the misjudgment phenomenon of the above-mentioned visual detection and recognition method, and then improving the recognition accuracy of GNSS NLOS signals, so as to achieve the purpose of improving the GNSS / INS combined navigation and positioning accuracy. Summary of the invention

[0006] In order to solve the problems existing in the above-mentioned prior art, the purpose of the present invention is to propose a GNSSNLOS data cleaning integrated navigation method based on deep learning, which effectively solves the problem of detecting and identifying GNSS NLOS signals with relatively low additional cost.

[0007] To achieve the above object, the present invention provides the following solutions:

[0008] The GNSS NLOS data cleaning integrated navigation method based on deep learning includes:

[0009] Acquire the carrier's anterograde environmental image information, pre-process the carrier's anterograde image information, and acquire a mask image;

[0010] The mask image and the carrier original sky environment image are input into the DeepLabV3+ model to obtain the category label of each pixel in the image; the DeepLabV3+ model enhances the ability to extract multi-scale features in the image by changing the convolution of ASPP to Depth-wise convolution;

[0011] Acquire the altitude angle and azimuth angle of the carrier and the GNSS satellite received by the carrier, construct a satellite projection model according to the altitude angle, the azimuth angle, camera internal parameters and carrier azimuth angle information, and acquire the pixel coordinates of the GNSS satellite in the pixel coordinate system of the image annotated with the category label based on the satellite projection model;

[0012] Determine the type of the area where the pixel coordinates are located and judge it as a non-line-of-sight signal or a line-of-sight signal, assign a weight to the non-line-of-sight signal or the line-of-sight signal, perform GNSS satellite position calculation according to the weight, and generate a positioning result.

[0013] Optionally, acquiring the mask image includes:

[0014] Calibrate the carrier's astronomical environmental image information to obtain a marking file; wherein the calibration content includes: sky area, building area, and tree area;

[0015] Based on the marker file, the mask image is obtained.

[0016] Optionally, the method for obtaining the altitude angle and azimuth angle of the carrier and the GNSS satellite received by the carrier is:

[0017]

[0018] Among them, H and E are azimuth and altitude respectively, L S , B S is the geodetic longitude and latitude of the carrier, ΔX S , ΔY S , ΔZ S are the distances between the carrier and the observation satellite in three directions, r is the distance from the carrier to the observation satellite, and Π is the pi.

[0019] Optionally, a method for obtaining pixel coordinates of the GNSS satellite in a pixel coordinate system of an image annotated with a category label is:

[0020]

[0021] Among them, (x c ,y c ) is the pixel coordinate of the center of the image, A is the satellite azimuth, R l is the pixel distance from the satellite projection point to the image center, and H is the azimuth.

[0022] Optionally, the method for obtaining the pixel distance from the satellite projection point to the image center is:

[0023]

[0024] Among them, F c is the camera pixel focal length, and E is the altitude angle.

[0025] Optionally, the method of assigning weights to the non-line-of-sight signals is:

[0026]

[0027] Among them, SNR n is the signal-to-noise ratio of the nth satellite, a, b, c are the empirical values ​​of the signal-to-noise ratio weighting, γ and β are the empirical values ​​of different obstructions, dist min To calculate the shortest pixel distance to the main sky area based on the pixel coordinates of the satellite projection point, SNR i is the signal-to-noise ratio of the non-line-of-sight signal corresponding to the satellite, R max is the threshold of the distance between the satellite image coordinates and the sky boundary, I is the image type, and I tree The occluding object type is trees, and & is a union.

[0028] Optionally, the method for calculating the shortest pixel distance to the main sky area based on the pixel coordinates of the satellite projection point is:

[0029]

[0030] Among them, n is the number of pixels at the boundary of the sky area, and Respectively represent the horizontal and vertical coordinates of the boundary points, (x c ,y c ) is the pixel coordinate of the center of the image.

[0031] Optionally, the method for solving the GNSS satellite position according to the weight is:

[0032]

[0033] Where G is the geometric matrix determined by the receiver and satellite positions; w is the weight of different GNSS satellite signals, b is the observation value composed of pseudorange vectors, T is the transposed sign, Δx, Δy, Δz, Δt u is the positioning result and the receiver clock difference.

[0034] To achieve the above objectives, the present invention also provides a computer system, comprising: one or more processors, a computer-readable storage medium for storing one or more programs, and processors of multiple computer devices jointly implementing the steps of the method when executing the computer programs.

[0035] To achieve the above object, the present invention further provides a computer-readable storage medium storing computer-executable instructions, wherein the computer program stored in the storage medium implements the steps of the method when executed by a processor.

[0036] The beneficial effects of the present invention are:

[0037] In the context of deep integration of visual positioning technology and GNSS NLOS signal processing technology, the present invention proposes a GNSS NLOS data cleaning combined navigation method based on deep learning to address the problem that GNSS / INS combined navigation is easily affected by GNSS NLOS signals, resulting in poor vehicle navigation positioning accuracy. The method uses a fisheye camera to obtain celestial environmental image information, which is divided into a sky area, a tree occlusion area, and a building occlusion area. The satellite signal is projected onto the processed image to accurately detect and identify the NLOS signal. Furthermore, the NLOS occlusion degree is characterized by the nearest pixel distance from the NLOS signal to the sky area, and the NLOS signal is re-weighted to perform NLOS weight distribution and elimination processing, so as to fully explore and utilize the favorable information of NLOS, thereby better inputting the GNSS positioning solution data into the GNSS / INS combined navigation algorithm. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the embodiments of the present invention or the prior art solutions, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0039] Figure 1 This is a flow chart of DeepLabV3+ model image processing according to an embodiment of the present invention;

[0040] Figure 2 This is a flow chart of a GNSS NLOS data cleaning integrated navigation method based on deep learning according to an embodiment of the present invention;

[0041] Figure 3 This is a sky / building / tree area map of an embodiment of the present invention; red is the sky area, black is the building area, and green is the tree area;

[0042] Figure 4 It is a signal type projection and identification diagram of an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0045] like Figure 2 As shown, this embodiment discloses a GNSS NLOS data cleaning integrated navigation method based on deep learning, including: obtaining carrier celestial environment image information, preprocessing the carrier celestial environment image information, and obtaining a mask image; inputting the mask image and the carrier original celestial environment image into the DeepLabV3+ model to obtain the category label of each pixel in the image; the DeepLabV3+ model enhances the ability to extract multi-scale features in the image by changing the convolution of ASPP to Depth-wise convolution; obtaining the altitude angle and azimuth angle of the carrier and the GNSS satellite received by the carrier, constructing a satellite projection model according to the altitude angle, azimuth angle, camera intrinsic parameters and carrier azimuth angle information, and obtaining the pixel coordinates of the GNSS satellite in the pixel coordinate system of the image marked with the category label based on the satellite projection model; determining the type of the area where the pixel coordinates are located and judging it as an NLOS signal or a LOS signal, assigning weights to the NLOS signal or the LOS signal, solving the GNSS satellite position according to the weights, and generating an integrated navigation positioning result.

[0046] Specifically, this embodiment first captures the carrier's celestial environmental image information through a fisheye camera. Secondly, the sky area and non-sky area of ​​the image are extracted by deep learning, and a satellite projection model is constructed according to the satellite altitude angle, azimuth angle and carrier azimuth angle information. The satellite signal is projected to obtain the pixel coordinates of the projection point of the satellite signal on the image, and the detection and identification of the LOS / NLOS signal is completed. The shortest distance between the projection point and the sky area is used to characterize the degree of GNSS NLOS occlusion. Finally, the processed positioning solution is imported into the data packet for use by the GNSS / INS combined navigation algorithm. Finally, the full exploration and utilization of NLSO signal information is achieved, and the purpose of improving the positioning accuracy of vehicle-mounted combined navigation is achieved.

[0047] This embodiment is based on the theory of image segmentation, uses a fisheye camera to capture the image information of the carrier's astronomical environment, and uses the deeplabv3+ deep learning model to semantically segment the sky area and the non-sky area, and further segment the types of obstructions where the satellite signal is blocked. On this basis, the satellite is projected onto the segmented image according to the altitude angle and azimuth angle of both the satellite and the carrier to achieve GNSS LOS / NLOS discrimination, and NLOS weight distribution or elimination processing is performed to obtain a positioning solution, and then the positioning solution data is combined for combined navigation, thereby achieving the purpose of improving the accuracy of vehicle-mounted navigation positioning. This method can effectively reduce the impact of non-line-of-sight signals on the accuracy of combined navigation positioning with a relatively low additional cost, thereby achieving the goal of improving the accuracy of vehicle-mounted combined navigation positioning in this environment. It includes the following steps:

[0048] Step 1: Use ROS to drive the fisheye camera to obtain the carrier's astronomical environment image information;

[0049] Step 2: Preprocess the image obtained in step 1;

[0050] Step 3: Extract the sky area and different occlusion areas based on the image preprocessed in step 2;

[0051] Step 4: Preliminary calculation of the altitude angle and azimuth angle of the carrier and the GNSS satellite received by the carrier;

[0052] Step 5: According to the altitude angle and azimuth angle obtained in step 4, the fisheye camera internal parameters and the carrier azimuth angle information, a satellite projection model is constructed to obtain the pixel coordinate value of the corresponding satellite in the pixel coordinate system on the image processed in step 3:

[0053]

[0054] Where (x c ,y c ) is the pixel coordinate of the center of the image, A is the satellite azimuth, R l is the pixel distance from the satellite projection point to the center of the image;

[0055] Step 6: According to the weights of LOS and NLOS signals, the GNSS position is solved by weighted least squares method;

[0056]

[0057] Where G represents the set matrix determined by the receiver and satellite position; W is the weight vector of different satellite signals, which is composed of w l and w n b is the observation value composed of pseudo-range vector; w is l is the weight of the LOS signal; w n is the weight of the NLOS signal; Δtu is the receiver clock error.

[0058] Furthermore, obtaining the mask image includes: calibrating the carrier's astronomical environmental image information and obtaining a marking file, such as Figure 3-4 As shown; the calibrated contents include: sky area, building area, tree area; based on the marker file, a mask image is obtained.

[0059] Specifically, a. Use the labelme tool to calibrate the dataset, including the sky area, building area, and tree area. b. Distribute the dataset according to the labeled files, generate the train.txt and val.txt files required in the training process, and generate the mask image according to the labeled json file.

[0060] Specifically, segmentation and occlusion handling, such as Figure 1 As shown:

[0061] a. Image segmentation: Use the DeepLabV3+ model to segment the preprocessed image generated in step 2. The core of this method is to use dilated convolution to capture contextual information of different scales, and to enhance the ability to extract multi-scale features in the image by changing the ASPP convolution to depth-wise convolution. The specific steps include:

[0062] Network architecture: Use the encoder-decoder structure of DeepLabV3+, where the encoder part extracts features, uses dilated convolution, extracts features through DCNN, and the decoder part restores the resolution and precise boundaries, uses upsampling and skip connections to gradually restore the spatial size of the features, and outputs the segmentation results. The encoder uses ResNet as the backbone.

[0063] Segmentation process: The preprocessed image is input into the DeepLabV3+ model, and the sky area and different types of non-sky areas are segmented according to the category label of each pixel in the output image.

[0064] b. Occlusion type extraction: Based on the labels defined during the training process and the original carrier sky-oriented environment image information, the DeepLabV3+ model outputs the type identification of different occlusion areas and divides them into sky, building and tree areas, thus completing image segmentation. The result is the category label of the segmented area and the corresponding pixel mask.

[0065] 3) Detect NLOS signals and assign weights:

[0066] a. Detect NLOS signal:

[0067]

[0068] The satellite elevation angle and azimuth angle are obtained according to formula (1), where H and E are the azimuth and elevation angle respectively, and L S , B S is the geodetic longitude and latitude of the carrier, ΔX S , ΔY S , ΔZ S are the distances between the carrier and the observation satellite in three directions, and r is the distance from the carrier to the observation satellite.

[0069]

[0070] The pixel distance from the satellite projection point to the center point of the image is calculated according to formula (2), and the pixel coordinates are calculated according to formula (3).

[0071]

[0072] Then, it is determined whether it is a NLOS signal based on the type of area where the pixel coordinates are located.

[0073] b. Signal empowerment:

[0074] The shortest pixel distance to the main sky area is calculated based on the pixel coordinates of the satellite projection point as follows:

[0075]

[0076] Where n is the number of pixels at the boundary of the sky area. and Represent the horizontal and vertical coordinates of the boundary points respectively.

[0077] The weight assignment of NLOS signals is mainly based on dist min And the altitude angle judgment is as follows:

[0078]

[0079] Where SNR n is the signal-to-noise ratio of the nth satellite, a, b, c are the empirical values ​​of the signal-to-noise ratio when determining the weights, γ and β are the empirical values ​​of different obstructions, respectively.

[0080] Finally, the weighted least squares method is used to obtain the positioning solution, as follows:

[0081]

[0082] Where G represents the geometric matrix determined by the receiver and satellite positions; w is the weight of different GNSS satellite signals, b is the observation value composed of pseudorange vectors, T is the transposed sign, Δx, Δy, Δz, Δt u is the positioning result and the receiver clock difference.

[0083] The positioning solution data packet is included in the GNSS / INS integrated navigation solution, and finally a better navigation positioning result is achieved.

[0084] This embodiment also discloses a computer system, including: one or more processors, a computer-readable storage medium for storing one or more programs, and processors of multiple computer devices that jointly implement the steps of the method when executing the computer programs.

[0085] This embodiment also discloses a computer-readable storage medium storing computer-executable instructions, and the steps of the method are implemented when the computer program stored in the storage medium is executed by a processor.

[0086] The embodiments described above are only descriptions of the preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary technicians in this field should all fall within the protection scope determined by the claims of the present invention.

Claims

1. A GNSS NLOS data cleaning integrated navigation method based on deep learning, characterized in that: include: Acquire the carrier's anterograde environmental image information, pre-process the carrier's anterograde image information, and acquire a mask image; The mask image and the carrier original sky environment image are input into the DeepLabV3+ model to obtain the category label of each pixel in the image; the DeepLabV3+ model enhances the ability to extract multi-scale features in the image by changing the convolution of ASPP to Depth-wise convolution; Acquire the altitude angle and azimuth angle of the carrier and the GNSS satellite received by the carrier, construct a satellite projection model according to the altitude angle, the azimuth angle, camera internal parameters and carrier azimuth angle information, and acquire the pixel coordinates of the GNSS satellite in the pixel coordinate system of the image annotated with the category label based on the satellite projection model; Determine the type of the area where the pixel coordinates are located and judge it as a non-line-of-sight signal or a line-of-sight signal, assign a weight to the non-line-of-sight signal or the line-of-sight signal, perform GNSS satellite position calculation according to the weight, and generate a positioning result.

2. The GNSS NLOS data cleaning integrated navigation method based on deep learning according to claim 1, characterized in that: Acquiring the mask image includes: Calibrate the carrier's astronomical environmental image information to obtain a marking file; wherein the calibration content includes: sky area, building area, and tree area; Based on the marker file, the mask image is obtained.

3. The GNSS NLOS data cleaning integrated navigation method based on deep learning according to claim 1, characterized in that: The method for obtaining the altitude angle and azimuth angle of a carrier and a GNSS satellite received by the carrier is: Among them, H and E are azimuth and altitude respectively, L S , B S is the geodetic longitude and latitude of the carrier, ΔX S , ΔY S , ΔZ S are the distances between the carrier and the observation satellite in three directions, r is the distance from the carrier to the observation satellite, and Π is the pi.

4. The GNSS NLOS data cleaning integrated navigation method based on deep learning according to claim 1, characterized in that: The method for obtaining the pixel coordinates of the GNSS satellite in the pixel coordinate system of the image annotated with the category label is: Among them, (x c ,y c ) is the pixel coordinate of the image center, A is the satellite azimuth, Rl is the pixel distance from the satellite projection point to the image center, and H is the azimuth.

5. The GNSS NLOS data cleaning integrated navigation method based on deep learning according to claim 4, characterized in that: The method to obtain the pixel distance from the satellite projection point to the center of the image is: Among them, F c is the camera pixel focal length, and E is the altitude angle.

6. The GNSS NLOS data cleaning integrated navigation method based on deep learning according to claim 1, characterized in that: The method of assigning weights to the non-line-of-sight signals is: Among them, SNR n is the signal-to-noise ratio of the nth satellite, a, b, c are the empirical values ​​of the signal-to-noise ratio weighting, γ and β are the empirical values ​​of different obstructions, dist min To calculate the shortest pixel distance to the main sky area based on the pixel coordinates of the satellite projection point, SNR i is the signal-to-noise ratio of the non-line-of-sight signal corresponding to the satellite, R max is the threshold of the distance between the satellite image coordinates and the sky boundary, I is the image type, and I tree The occluding object type is trees, and & is a union.

7. The GNSS NLOS data cleaning integrated navigation method based on deep learning according to claim 6, characterized in that: The method for calculating the shortest pixel distance to the main sky area based on the pixel coordinates of the satellite projection point is: Among them, n is the number of pixels at the boundary of the sky area, and Respectively represent the horizontal and vertical coordinates of the boundary points, (x c ,y c ) is the pixel coordinate of the center of the image.

8. The GNSS NLOS data cleaning integrated navigation method based on deep learning according to claim 1, characterized in that: The method for solving the GNSS satellite position according to the weights is: Where G is the geometric matrix determined by the receiver and satellite positions; w is the weight of different GNSS satellite signals, b is the observation value composed of pseudorange vectors, T is the transposed sign, Δx, Δy, Δz, Δt u is the positioning result and the receiver clock difference.

9. A computer system comprising: One or more processors, a computer-readable storage medium, used to store one or more programs, characterized in that the processors of multiple computer devices jointly implement the steps of the method described in any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium storing computer-executable instructions, characterized in that: When the computer program stored in the storage medium is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

Images