GNSS / vision loosely coupled positioning method integrating environmental semantics

Abstract

The present invention belongs to the field of navigation and positioning technology, and in particular relates to a GNSS / vision loosely coupled positioning method that integrates environmental semantics. This method solves the technical problem of insufficient positioning accuracy in traditional methods when faced with urban occlusion environments and high-dynamic scenes. In this method, a GNSS receiver obtains the position coordinates of a carrier through its double-difference measurement model, and continuous image frames captured by a stereo camera are passed to a VO module for performing carrier pose estimation. A loosely coupled model is used to combine the carrier position coordinates of the GNSS and the carrier pose information of the VO module. The occlusion factor is then calculated based on the environmental semantic information and the local mapping module. An adaptive KF algorithm is introduced in combination with the occlusion factor. Under the constraint of the occlusion factor, the adaptive KF algorithm can calculate the optimal pose estimate. This value is fed back to the GNSS to correct the carrier position coordinates, and the corrected position coordinates are finally output as the positioning result.

Description

Technical Field

[0001] The present invention relates to the technical field of navigation and positioning, and in particular to a GNSS / vision loosely coupled positioning method integrating environmental semantics. Background Art

[0002] With the development of the field of smart transportation, the positioning and navigation technology of autonomous vehicles has become a hot topic of current research. Reliable, continuous and high-precision positioning technology is the key to realizing autonomous driving. The Global Navigation Satellite System (GNSS) can provide reliable positioning services in open sky environments. However, in complex environments such as urban canyons or shaded areas, GNSS signals are limited, and their reliability and continuity will deteriorate significantly. Fusion of GNSS with other positioning sensors to achieve combined positioning is an effective way to improve the positioning performance of GNSS signals in restricted environments. The visual positioning system is a low-cost, highly reliable navigation system that can provide navigation information in environments where GNSS signals are restricted.

[0003] Visual odometry (VO) uses visual sensors to capture environmental information and track camera motion in real time to accurately locate the target. The VO module assumes that objects in the environment are stationary. However, the prevalence of dynamic objects on urban streets can lead to mismatched feature points, affecting pose estimation accuracy. Summary of the Invention

[0004] To address the technical drawbacks of traditional methods, such as insufficient positioning accuracy in urban occlusion environments and highly dynamic scenes, the present invention provides a GNSS / vision loosely coupled positioning method that integrates environmental semantics.

[0005] The present invention provides a GNSS / vision loosely coupled positioning method that integrates environmental semantics. The loosely coupled positioning method is implemented based on a stereo camera, GNSS, and a SLAM system. The stereo camera is fixedly connected to a vehicle for real-time image acquisition during vehicle movement. The SLAM system includes a VO module and a local mapping module. The VO module includes a feature point tracking unit. The steps of the loosely coupled positioning method are as follows:

[0006] Step 1: Improve the VO module by embedding a real-time environment semantic segmentation processing unit in the VO module. The environment semantic segmentation processing unit and the feature point tracking unit work in parallel. The image is input into the VO module in the form of continuous frames. The feature point tracking unit extracts ORB (Oriented FAST and Rotated BRIEF) feature points from the image to form a feature map containing dynamic feature points and static feature points. The environment semantic segmentation processing unit performs environment semantic segmentation on the image and generates a mask image with specific semantic labels. The VO module fuses the feature map and the mask image, marks the dynamic feature points as outliers, and removes the outliers to obtain an image containing only static feature points.

[0007] Step 2: The VO module matches the static feature points of the current image with those of the previous frame one by one, and then obtains the six-degree-of-freedom rotation R and translation T of the stereo camera by minimizing the reprojection error;

[0008] Step 3: independently obtain the preliminary position coordinates of the carrier based on the GNSS double-difference observation model;

[0009] Step 4: The rotation R and translation T of the stereo camera are the rotation R and translation T of the carrier. The rotation R and translation T of the carrier are used to construct a visual positioning error model, where the visual positioning error model is:

[0010]

[0011] In formula (1), l, r n and ψ are the length scale factor of the stereo camera, the position coordinates of the carrier and the attitude of the carrier in the navigation coordinate system, respectively. The position coordinates of the carrier r n The carrier's translation T is integrated, and the carrier's posture ψ is integrated with the carrier's rotation R; δl, δr n and δψ are the errors of length scale factor, position coordinates and attitude in the navigation coordinate system respectively; and are the continuous time derivatives of the errors of length scale factor, position coordinates and attitude in the navigation coordinate system respectively; is the estimated rotation matrix from the vehicle coordinate system to the navigation coordinate system; is the estimated translation in the carrier coordinate system; is an estimate of the length scale factor; and are the translation error and rotation error in the carrier coordinate system respectively; v is the random drift of l;

[0012] Simplify formula (1) into a system model, which can be expressed as:

[0013]

[0014] In formula (2), for F is x is G is u is

[0015] Then, the state model of the KF algorithm is constructed using formula (2):

[0016] X(k)=Φ(k,k-1)X(k-1)+Γ(k-1)W(k-1) (3),

[0017] In formula (3), X(k) is the state vector of epoch k, Φ(k, k-1) is the discrete-time transition matrix from epoch k-1 to epoch k, Γ(k-1) is the noise distribution matrix, and W(k-1) is the system noise sequence;

[0018] Using a loosely coupled model, the difference between the carrier position coordinates obtained by the VO module and the carrier preliminary position coordinates obtained by GNSS in step 3 is used as the observation value input of the KF algorithm. The observation model of the KF algorithm is constructed as follows:

[0019] Z(k)=r VO -r GNSS =H(k)X(k)+V(k) (4),

[0020] In formula (4), r VO and r GNSS are the carrier position coordinates obtained by the VO module and GNSS respectively; H(k) is the observation transfer matrix; V(k) is the observation noise;

[0021] Then the state update equation of the KF algorithm is expressed as:

[0022]

[0023] In formula (5), P(k-1) represents the error covariance at epoch k-1, and Q(k-1) represents the corresponding covariance matrix of W(k-1); represents the prior state vector estimate from epoch k-1 to epoch k; Φ(k,k-1) represents the state transition matrix from epoch k-1 to epoch k; represents the estimated value of the state vector of epoch k-1; P(k, k-1) represents the one-step covariance of the prediction estimation error from epoch k-1 to epoch k; P(k-1) represents the covariance of the prediction estimation error of epoch k-1; Φ T (k, k-1) represents the transpose of the state transition matrix from epoch k-1 to epoch k; Γ T(k-1) represents the transpose of the noise distribution matrix;

[0024] The observation update equation of the KF algorithm is expressed as:

[0025]

[0026] In formula (6), K(k) represents the gain matrix; H T (k) represents the transpose of the observation transfer matrix; R(k) represents the observation noise covariance; represents the estimated value of the state vector at epoch k; P(k) represents the covariance of the prediction estimate error at epoch k;

[0027] The position coordinates of the carrier can be updated through the observation update equation of the KF algorithm, and the updated position coordinates of the carrier are fed back to the GNSS. The GNSS corrects the preliminary position coordinates of the carrier in step 3 based on the updated position coordinates of the carrier, and finally outputs the corrected position coordinates of the carrier as the positioning result.

[0028] Preferably, in step 4, after obtaining the observation update equation of the KF algorithm, the observation noise covariance R(k) is also adaptively adjusted; the mask image with a specific semantic label obtained in step 1 contains the semantic label of the occluder, and the depth information of the occluder is obtained by the local mapping module of the SLAM system. The depth information of the occluder is the distance between the carrier and the occluder, and the occlusion factor is constructed. for:

[0029]

[0030] In formula (7), d is the depth of field, and the range of depth of field d is [1m, 1000m];

[0031] When an occlusion is detected in the image, the occlusion factor Integrated on the original observation noise covariance R(k), the adaptively adjusted observation noise covariance is obtained The adaptive adjustment formula of the observation noise covariance R(k) is:

[0032]

[0033] When an occlusion is detected, the adaptively adjusted observation noise covariance By introducing the observation update equation of the KF algorithm, the optimal estimated value of the carrier's posture can be calculated.

[0034] The size of the gain matrix K(k) is affected by equations (3) to (6). Specifically, when the set observation noise variance R(k) exceeds the actual noise distribution level, the corresponding value of K(k) will decrease, resulting in an inappropriate reduction in the uncertainty range of the true value of the state estimate, which in turn causes a deviation in the estimation result. On the contrary, if the set R(k) is too small, the value of K(k) will increase, which may cause the filter to diverge. If there are abnormal obstructions in the observation data, and the system still uses the original observation noise covariance R(k) without timely adjustment, the influence of the obstruction on the filter will not be suppressed, resulting in a poor filter convergence effect or even filter divergence.

[0035] Therefore, the adaptive adjustment of the original observation noise covariance R(k) is to improve the estimation characteristics and robustness of the KF algorithm. Integrate on the original observation noise covariance R(k), adjust R(k) upward, and obtain the adaptively adjusted observation noise covariance This reduces the gain matrix K(k) and reduces the impact of abnormal measurement data on the KF algorithm.

[0036] Compared with the existing technology, the technical solution provided by the present invention has the following technical effects: the method described in the present invention combines GNSS with visual positioning technology, and uses the absolute position provided by GNSS to correct the cumulative error of visual positioning. At the same time, the local positioning capability of visual positioning can improve the positioning accuracy and stability of GNSS, effectively solving the problem of insufficient positioning accuracy of traditional methods in urban occlusion environments and high-dynamic scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0038] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0039] Figure 1 This is a flowchart of the GNSS / vision loosely coupled positioning method integrating environmental semantics according to an embodiment of the present invention. DETAILED DESCRIPTION

[0040] In order to more clearly understand the above-mentioned objectives, features and advantages of the present invention, the scheme of the present invention will be further described below. It should be noted that, in the absence of conflict, the embodiments of the present invention and the features therein can be combined with each other.

[0041] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein; it is obvious that the embodiments in the specification are only part of the embodiments of the present invention, rather than all the embodiments.

[0042] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0043] In one embodiment, Figure 1 As shown, a GNSS / vision loosely coupled positioning method that integrates environmental semantics is disclosed. The loosely coupled positioning method is based on a stereo camera, GNSS, and a SLAM (Simultaneous Localization and Mapping) system. The stereo camera is fixedly connected to a carrier to collect images of the carrier in real time during its travel, ensuring the continuity and stability of the image data. The SLAM system includes a VO module and a local mapping module. The VO module includes a feature point tracking unit. The steps of the loosely coupled positioning method are as follows:

[0044] Step 1: Improve the VO module and embed a real-time environment semantic segmentation processing unit in the VO module. The environment semantic segmentation processing unit and the feature point tracking unit work in parallel. The image is input into the VO module in the form of continuous frames. The feature point tracking unit extracts ORB feature points from the image to form a feature map containing dynamic feature points and static feature points. The environment semantic segmentation processing unit performs environment semantic segmentation on the image to generate a mask image with specific semantic labels. The VO module fuses the feature map and the mask image, marks the dynamic feature points as outliers, and the dynamic feature points are feature points related to dynamic objects, where dynamic objects include vehicles, pedestrians or animals, etc. The outliers are removed to obtain an image with only static feature points. Identifying and removing feature points related to dynamic objects can ensure that the accuracy of pose estimation is not affected by dynamic objects.

[0045] Step 2: The VO module matches the static feature points of the current image with those of the previous frame one by one, and then obtains the six-degree-of-freedom rotation R and translation T of the stereo camera by minimizing the reprojection error;

[0046] Step 3: independently obtain the preliminary position coordinates of the carrier based on the GNSS double-difference observation model;

[0047] Step 4: The rotation R and translation T of the stereo camera are the rotation R and translation T of the carrier. The rotation R and translation T of the carrier are used to construct a visual positioning error model, where the visual positioning error model is:

[0048]

[0049] In formula (1), I, r n and ψ are the length scale factor of the stereo camera, the position coordinates of the carrier and the attitude of the carrier in the navigation coordinate system, respectively. The position coordinates of the carrier r n The carrier's translation T is integrated, and the carrier's posture ψ is integrated with the carrier's rotation R; δl, δr n and δψ are the errors of length scale factor, position coordinates and attitude in the navigation coordinate system respectively; and are the continuous time derivatives of the errors of length scale factor, position coordinates and attitude in the navigation coordinate system respectively; is the estimated rotation matrix from the vehicle coordinate system to the navigation coordinate system; is the estimated translation in the carrier coordinate system; is an estimate of the length scale factor; and are the translation error and rotation error in the carrier coordinate system respectively; v is the random drift of l;

[0050] Simplify formula (1) into a system model, which can be expressed as:

[0051]

[0052] In formula (2), for F is x is G is u is

[0053] Then, the state model of the Kalman filter (KF) algorithm is constructed using formula (2):

[0054] X(k)=Φ(k,k-1)X(k-1)+T(k-1)W(k-1) (3),

[0055] In formula (3), X(k) is the state vector of epoch k, Φ(k, k-1) is the discrete-time transition matrix from epoch k-1 to epoch k, Γ(k-1) is the noise distribution matrix, and W(k-1) is the system noise sequence;

[0056] Using a loosely coupled model, the difference between the carrier position coordinates obtained by the VO module and the carrier position coordinates obtained by GNSS in step 3 is used as the observation value input of the KF algorithm. The observation model of the KF algorithm is constructed as follows:

[0057] Z(k)=r VO -r GNSS=H(k)X(k)+V(k) (4),

[0058] In formula (4), r VO and r GNSS are the carrier positions obtained by the VO module and GNSS respectively; H(k) is the observation transfer matrix; V(k) is the observation noise;

[0059] Then the state update equation of the KF algorithm is expressed as:

[0060]

[0061] In formula (5), P(k-1) represents the error covariance at epoch k-1, and Q(k-1) represents the corresponding covariance matrix of W(k-1); represents the prior state vector estimate from epoch k-1 to epoch k; Φ(k, k-1) represents the state transition matrix from epoch k-1 to epoch k; represents the estimated value of the state vector of epoch k-1; P(k, k-1) represents the one-step covariance of the prediction estimation error from epoch k-1 to epoch k; P(k-1) represents the covariance of the prediction estimation error of epoch k-1; Φ T (k, k-1) represents the transpose of the state transition matrix from epoch k-1 to epoch k; Γ T (k-1) represents the transpose of the noise distribution matrix;

[0062] The observation update equation of the KF algorithm is expressed as:

[0063]

[0064] In formula (6), K(k) represents the gain matrix; H T (k) represents the transpose of the observation transfer matrix; R(k) represents the observation noise covariance; represents the estimated value of the state vector at epoch k; R(k) represents the covariance of the prediction estimate error at epoch k;

[0065] The position coordinates of the carrier can be updated through the observation update equation of the KF algorithm, and the updated position coordinates of the carrier are fed back to the GNSS. The GNSS corrects the preliminary position coordinates of the carrier in step 3 based on the updated position coordinates of the carrier, and finally outputs the corrected position coordinates of the carrier as the positioning result.

[0066] Based on the above embodiment, in a preferred embodiment, in step 4, after obtaining the observation update equation of the KF algorithm, it also includes adaptive adjustment of the observation noise covariance R(k); the mask image with a specific semantic label obtained in step 1 contains the semantic label of the occluder, and the depth information of the occluder is obtained through the local mapping module of the SLAM system. The depth information of the occluder is the distance between the carrier and the occluder, and the occlusion factor is constructed. for:

[0067]

[0068] In formula (7), d is the depth of field, and the range of depth of field d is [1m, 1000m];

[0069] When an occlusion is detected in the image, the occlusion factor Integrated on the original observation noise covariance R(k), the adaptively adjusted observation noise covariance is obtained The adaptive adjustment formula of the observation noise covariance R(k) is:

[0070]

[0071] When an occlusion is detected, the adaptively adjusted observation noise covariance By introducing the observation update equation of the KF algorithm, the optimal estimated value of the carrier's posture can be calculated.

[0072] The size of the gain matrix K(k) is affected by equations (3) to (6). Specifically, when the set observation noise variance R(k) exceeds the actual noise distribution level, the corresponding value of K(k) will decrease, resulting in an inappropriate reduction in the uncertainty range of the true value of the state estimate, which in turn causes a deviation in the estimation result. On the contrary, if the set R(k) is too small, the value of K(k) will increase, which may cause the filter to diverge. If there are abnormal obstructions in the observation data, and the system still uses the original observation noise covariance R(k) without timely adjustment, the influence of the obstruction on the filter will not be suppressed, resulting in a poor filter convergence effect or even filter divergence.

[0073] Therefore, the adaptive adjustment of the original observation noise covariance R(k) is to improve the estimation characteristics and robustness of the KF algorithm. Integrate on the original observation noise covariance R(k), adjust R(k) upward, and obtain the adaptively adjusted observation noise covariance This reduces the gain matrix K(k) and reduces the impact of abnormal measurement data on the KF algorithm.

[0074] Specifically, in step 3, the basic GNSS observations mainly include pseudorange observations and carrier phase observations. According to the definitions of pseudorange observations and carrier phase observations, their positioning equations can be expressed as:

[0075]

[0076] In formula (9), p and They represent the pseudorange observation value and the carrier phase observation value, respectively, is the real satellite-to-ground distance, c is the speed of light in vacuum, δt u and δt s are the GNSS receiver clock error and satellite clock error, respectively, ron and d trop are the ionospheric refraction error and the tropospheric refraction error, λ is the carrier wavelength, N is the integer ambiguity, and ε ρ and denote the unmodeled residual errors of pseudorange observations and carrier phase observations, respectively.

[0077] Pseudorange observations have relatively low accuracy, while high-precision kinematic positioning using carrier phase observations requires real-time resolution of the integer ambiguity N. For short-baseline positioning, atmospheric errors between stations are highly correlated, so the synchronous observation difference method between stations is used to eliminate or reduce the impact of these errors.

[0078] When two receivers observe multiple satellites simultaneously, the calculation formula of the double-difference observation model is expressed as:

[0079]

[0080] The superscripts m and n represent the satellite numbers, and the subscripts i and j represent the station numbers.

[0081] The double-difference observation model eliminates receiver clock errors by calculating the difference between the observations of two receivers at different satellites. In carrier phase double-difference positioning, eliminating these errors also facilitates the dynamic resolution of integer ambiguities.

[0082] In the method described in the present invention, the GNSS receiver independently obtains the position coordinates of the carrier through its double-difference measurement model. At the same time, the continuous image frames captured by the stereo camera are passed to the VO module to perform the pose estimation of the carrier. Step 4 of the present invention uses a loosely coupled model to combine the carrier position coordinates of the GNSS and the carrier pose information obtained by the VO module to construct a GNSS / vision loosely coupled positioning system. During this period, the GNSS / vision loosely coupled positioning system calculates the occlusion factor based on the environmental semantic information and the local mapping module.

[0083] Subsequently, the GNSS / vision loosely coupled positioning system introduces an adaptively adjusted KF algorithm in combination with the occlusion factor. In this algorithm, the pose output by the VO module is used as the state vector for update, and the difference between GNSS positioning and vision positioning is used as the observation vector for update. Under the constraints of , the adaptive KF algorithm can calculate the optimal estimate, which is then fed directly into the GNSS module to correct its position coordinates. Ultimately, the corrected position coordinates are used as the output of the integrated system.

[0084] Based on the above embodiments, in a preferred embodiment, the image is preprocessed before being input into the VO module. The preprocessing includes dedistorting the acquired image to correct the distortion effect of lens distortion on the image, and then removing random noise in the image through a noise reduction algorithm (such as Gaussian filtering) to improve the clarity and usability of the image.

[0085] Based on the above embodiment, in a preferred embodiment, in step 1, the environment semantic segmentation processing unit uses the YOLOv5 method to perform environment semantic segmentation on the image.

[0086] The above description is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Although detailed descriptions have been made with reference to the aforementioned embodiments, those skilled in the art should understand that they may still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents; and such modifications or replacements do not deviate from the essence of the corresponding technical solutions within the scope of the technical solutions of the embodiments, and they should all be included in the scope of protection of the claims.

Claims

1. A GNSS / vision loosely coupled positioning method integrating environmental semantics, characterized by: The loosely coupled positioning method is implemented based on a stereo camera, GNSS, and SLAM system. The stereo camera is fixedly connected to the carrier to collect images of the carrier in real time during its movement. The SLAM system includes a VO module and a local mapping module. The VO module includes a feature point tracking unit. The steps of the loosely coupled positioning method are as follows: Step 1: Improve the VO module and embed a real-time environment semantic segmentation processing unit in the VO module. The environment semantic segmentation processing unit and the feature point tracking unit work in parallel. The image is input into the VO module in the form of continuous frames. The feature point tracking unit extracts ORB feature points from the image to form a feature map containing dynamic feature points and static feature points. The environment semantic segmentation processing unit performs environment semantic segmentation on the image and generates a mask image with specific semantic labels. The VO module fuses the feature map and the mask image, marks the dynamic feature points as outliers, and removes the outliers to obtain an image with only static feature points. Step 2: The VO module matches the static feature points of the current image with those of the previous frame one by one, and then obtains the six-degree-of-freedom rotation R and translation T of the stereo camera by minimizing the reprojection error; Step 3: independently obtain the preliminary position coordinates of the carrier based on the GNSS double-difference observation model; Step 4: The rotation R and translation T of the stereo camera are the rotation R and translation T of the carrier. The rotation R and translation T of the carrier are used to construct a visual positioning error model, where the visual positioning error model is: （1）， In formula (1), 、 and are the length scale factor of the stereo camera, the position coordinates of the carrier and the attitude of the carrier in the navigation coordinate system, respectively. It is obtained by integrating the translation T of the carrier, and the carrier's posture It is obtained by integrating the rotation R of the carrier; 、 and are the errors of length scale factor, position coordinates and attitude in the navigation coordinate system respectively; 、 and are the continuous time derivatives of the errors of length scale factor, position coordinates and attitude in the navigation coordinate system respectively; is the estimated rotation matrix from the vehicle coordinate system to the navigation coordinate system; is the estimated translation in the carrier coordinate system; is an estimate of the length scale factor; and are the translation error and rotation error in the carrier coordinate system respectively; for Random drift of Simplify formula (1) into a system model, and the system model is expressed as: （2）， In formula (2), for ; for ; for ; for ; for ; Then, the state model of the KF algorithm is constructed using formula (2): （3）， In formula (3), It is the epoch The state vector of From the epoch To epoch The discrete-time transition matrix of is the noise distribution matrix, is the system noise sequence; Using a loosely coupled model, the difference between the carrier position coordinates obtained by the VO module and the carrier preliminary position coordinates obtained by GNSS in step 3 is used as the observation value input of the KF algorithm. The observation model of the KF algorithm is constructed as follows: （4）， In formula (4), and are the carrier position coordinates obtained by the VO module and GNSS respectively; is the observation transfer matrix; is the observation noise; Then the state update equation of the KF algorithm is expressed as: (5), In formula (5), Indicates time The error covariance at the epoch, express The corresponding covariance matrix of ; Represents epoch To epoch The prior state vector estimate of ; Indicates the time from epoch To epoch The state transition matrix of Represents epoch The estimated value of the state vector; Represents epoch To epoch One-step covariance of the forecast estimate error; Represents epoch The covariance of the forecast estimate error; Indicates the time from epoch To epoch The transpose of the state transition matrix; represents the transpose of the noise distribution matrix; The observation update equation of the KF algorithm is expressed as: （6）， In formula (6), represents the gain matrix; represents the transpose of the observation transfer matrix; represents the observation noise covariance; Represents epoch The estimated value of the state vector; Represents epoch The covariance of the forecast estimate error; The position coordinates of the carrier can be updated through the observation update equation of the KF algorithm, and the updated position coordinates of the carrier are fed back to the GNSS. The GNSS corrects the preliminary position coordinates of the carrier in step 3 based on the updated position coordinates of the carrier, and finally outputs the corrected position coordinates of the carrier as the positioning result.

2. The GNSS / vision loosely coupled positioning method integrating environmental semantics according to claim 1, characterized in that: In step 4, after obtaining the observation update equation of the KF algorithm, the observation noise covariance is also included. Adaptive adjustment; The mask image with specific semantic labels obtained in step 1 contains the semantic label of the occluder. The depth information of the occluder is obtained through the local mapping module of the SLAM system. The depth information of the occluder is the distance between the carrier and the occluder, and the occlusion factor is constructed. for: （7）， In formula (7), Depth of field, depth of field The range is ; When an occlusion is detected in the image, the occlusion factor Integrated over the original observation noise covariance The adaptively adjusted observation noise covariance is obtained , observation noise covariance The adaptive adjustment formula is: （8）， When an occlusion is detected, the adaptively adjusted observation noise covariance By introducing the observation update equation of the KF algorithm, the optimal estimated value of the carrier's posture can be calculated.

3. The GNSS / vision loosely coupled positioning method integrating environmental semantics according to claim 1, characterized in that: The image is preprocessed before being input into the VO module. The preprocessing includes dedistorting the acquired image and then removing random noise in the image through a denoising algorithm.

4. The GNSS / vision loosely coupled positioning method integrating environmental semantics according to any one of claims 1 to 3, characterized in that: In step 1, the environment semantic segmentation processing unit uses the YOLOv5 method to perform environment semantic segmentation on the image.

Images