A GNSS / INS / map compact integration navigation and positioning method and system based on road marking matching

Abstract

The application discloses a GNSS / INS / map tightly coupled navigation positioning method and system based on road marking matching, belongs to the technical field of navigation positioning, and comprises the following steps: acquiring prior road map data, forward road image data, GNSS original observation values and INS original observation values; adopting a semantic segmentation network to extract road marking semantic features in the forward road image data, so that road marking feature instances are obtained; performing nearest neighbor search on the road marking feature instances and map elements in the prior road map data, obtaining corresponding map matching observations based on the nearest neighbor search results; and coupling the GNSS original observation values, the INS original observation values and the map matching observations, so that navigation positioning results are obtained. The application proposes a tightly coupled model based on GNSS / INS and map matching original observation values, which not only improves the positioning performance of a navigation system in a complex urban scene, but also provides full-directional pose constraints for a vehicle.

Description

Technical Field

[0001] This invention belongs to the field of navigation and positioning technology, specifically relating to a GNSS / INS / map tightly integrated navigation and positioning method and system based on road marking matching. Background Technology

[0002] High-precision location services are an important foundation for the unification of spatiotemporal references, environmental perception, and planning and control of autonomous vehicles. In order to achieve efficient and reliable environmental perception and decision-making and planning, the development of autonomous driving applications urgently requires accurate and continuous positioning technology solutions.

[0003] Due to the propagation characteristics of radio waves, Global Navigation Satellite System (GNSS) signals are susceptible to signal blockage, reflection, and multipath effects in environments with severe obstruction or interference, thus affecting the accuracy and continuity of positioning results. To enhance the positioning performance of navigation systems in complex scenarios, Inertial Navigation Systems (INS), which are autonomous, passive, and less affected by environmental factors, are often combined with GNSS. This leverages the complementary characteristics of the two systems to mitigate performance degradation in challenging GNSS scenarios. However, this integration also heavily relies on the performance of the Inertial Measurement Unit (IMU). Even with significant GNSS signal degradation, the GNSS / INS combined system can still experience severe accuracy divergence.

[0004] To mitigate the impact of error drift in GNSS / INS systems, autonomous vehicles typically incorporate environmental perception sensors. Among these, visual sensors, due to their mature technology, low cost, and high miniaturization, have led to the development of numerous positioning methods that fuse GNSS / INS with visual sensors. While these methods can improve positioning accuracy to some extent, they still possess certain limitations due to the limited sensing distance and range of cameras, and their susceptibility to factors such as weak textures, lighting variations, and viewpoint occlusion.

[0005] As the architecture of autonomous driving technology continues to improve, in order to overcome this limitation and enhance the safety and reliability of autonomous driving systems, the industry has begun to widely focus on methods that incorporate prior map information to assist in positioning. Prior map data typically includes elements such as traffic signs, road markings, traffic lights, and ramps. By matching and associating these with observation information from environmental perception sensors, redundant and drift-free pose constraint information can be provided to the system. However, existing methods do not fully utilize the rich road map information and raw GNSS observations, making it difficult to maintain the omnidirectional accuracy and robustness of the navigation and positioning system in complex scenarios. Summary of the Invention

[0006] The purpose of this invention is to address existing problems by providing a GNSS / INS / map compact combination navigation and positioning method based on road marking matching. Within the framework of the PPP / INS compact combination system, it achieves the extraction and modeling of road marking features. By effectively associating map data with road marking instances, map matching observations are introduced into the PPP / INS system in a compact combination form, thereby improving the navigation and positioning performance of the navigation system in complex urban scenarios.

[0007] According to one aspect of the present invention, a GNSS / INS / map compact integration navigation and positioning method based on road marking matching is provided, comprising:

[0008] Acquire prior road map data, forward-looking road image data, raw GNSS observations, and raw INS observations;

[0009] A deep learning-based semantic segmentation network is used to extract the semantic features of road markings in the forward-looking road image data. The semantic features of road markings are transformed into spatial geometric features through inverse perspective transformation. The extracted different types of road markings are clustered and modeled to obtain road marking feature instances.

[0010] The road marking feature instances are transformed from a pixel-level coordinate system to a world coordinate system, and a nearest neighbor search is performed on the map elements in the prior road map data based on the approximate coordinates of the road marking feature instances. The corresponding map matching observation is constructed based on the nearest neighbor search results.

[0011] A GNSS / INS / map tightly coupled positioning model is constructed based on raw GNSS observations, raw INS observations, and map matching observations to obtain navigation and positioning results.

[0012] Furthermore, the extracted different types of road markings are clustered and modeled separately, including:

[0013] Based on spatial geometric features, the extracted different types of road markings are clustered to obtain the clustering results;

[0014] Based on the clustering results, different modeling methods are used for modeling.

[0015] Furthermore, based on the clustering results, different modeling methods are used for modeling, including:

[0016] For dashed lane lines and arrows, the minimum bounding rectangle of the dashed lane lines and arrows and the corresponding spatial uncertainty are calculated and modeled.

[0017] For solid lane lines and stop lines, singular value decomposition is used to calculate the principal direction of the solid lane lines and stop lines. The road marking instances are fitted along the principal direction, and resampling is used to transform the solid lane lines and stop lines into multi-vertex polylines to obtain the parametric modeling results.

[0018] Furthermore, based on the nearest neighbor search results, the corresponding map matching observations are obtained, including:

[0019] Calculate the distances from lane lines and parking lines to line segments of similar map features, and find the closest line segment as the matching result;

[0020] The center point is calculated based on the minimum bounding box of the arrow, and the arrow closest to the current center point is found as the matching object;

[0021] Map matching observations are obtained based on the matching results and the matching objects.

[0022] Furthermore, the method also includes:

[0023] Road marking information is extracted from prior road map data, and the geometric coordinates of the road marking information are stored in a KD-Tree structure. Then, the data is clustered and divided into different nodes according to the geometric coordinates.

[0024] Furthermore, based on raw GNSS observations, raw INS observations, and map-matching observations, a GNSS / INS / map tightly coupled positioning model is constructed, including:

[0025] (1) Define the state vector as follows:

[0026] ,

[0027] in, and These represent the parameter vectors to be estimated for INS and PPP, respectively; the superscript " " indicates matrix transpose;

[0028] (2) Define the prediction equation, with the following expression:

[0029] ,

[0030] in, and These represent the state transition matrices for INS and GNSS, respectively. express OK The zero matrix of columns; Represents the process noise vector. It is its corresponding coefficient matrix. This indicates the number of ambiguities in the phase of the carrier wave without ionosphere. and This represents the power spectral density of the gyroscope and accelerometer. Represents the rotation matrix from the carrier coordinate system to the world coordinate system;

[0031] (3) Define the observation equation, with the following expression:

[0032] ,

[0033] in, and Let represent the map matching observation vector and the GNSS observation vector at time k, respectively; and Let K represent the point-to-line and point-to-point map matching observations at time k, respectively. and Represent the GNSS pseudorange and carrier phase observations at time k without ionospheric saturation. and Let represent the model values ​​of the pseudorange and carrier phase observations at time k, respectively. and Let these represent the map matching observation and GNSS observation noise vectors at time k, respectively. This represents the state vector at time k. Let $k$ be the Jacobian matrix of the GNSS observations at time $k$ with respect to the state vector. Let k represent the observation coefficient matrix at time k.

[0034] Furthermore, the deep learning-based semantic segmentation network includes:

[0035] A lightweight semantic segmentation network based on convolutional neural networks is used to extract road markings. The lightweight semantic segmentation network adds a flow alignment module between adjacent layers of the network to transmit semantic information between network layers with different resolutions.

[0036] According to one aspect of the present invention, a GNSS / INS / map tightly integrated navigation and positioning system based on road marking matching is provided, comprising:

[0037] The data acquisition module is used to acquire prior road map data, forward-looking road image data, raw GNSS observations, and raw INS observations.

[0038] The road marking feature acquisition module is used to extract the semantic features of road markings in the forward-looking road image data using a semantic segmentation network based on deep learning, transform the semantic features of road markings into spatial geometric features through inverse perspective transformation, and cluster and model the extracted different types of road markings to obtain road marking feature instances.

[0039] The map matching observation module is used to transform road marking feature instances from a pixel-level coordinate system to a world coordinate system, and perform nearest neighbor search on map elements in prior road map data based on the approximate coordinates of the road marking feature instances, and obtain the corresponding map matching observation based on the nearest neighbor search results.

[0040] The navigation and positioning module is used to construct a GNSS / INS / map tightly coupled positioning model based on raw GNSS observations, raw INS observations, and map matching observations to obtain navigation and positioning results.

[0041] According to one aspect of the present invention, an electronic device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of the GNSS / INS / map compact combination navigation and positioning method based on road marking matching.

[0042] According to one aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the GNSS / INS / map compact combination navigation and positioning method based on road marking matching.

[0043] Compared with the prior art, the beneficial effects of the present invention are:

[0044] 1. This invention effectively associates map data with visual road marking instances, introducing map matching observations into the GNSS / INS system in the form of raw observations, thereby improving the positioning performance of the navigation system in complex urban scenarios;

[0045] 2. This invention proposes a tight combination model based on GNSS / INS and map matching of raw observations. It uses a variety of different road markings during the positioning process, making full use of rich road features and raw GNSS observation information, so that the navigation system can achieve higher positioning accuracy and more robust performance in complex scenarios.

[0046] 3. This invention achieves tight parameterization for various road markings and calculates the uncertainty during cluster modeling, making the constructed map matching observation model more accurate and reliable, and providing omnidirectional pose constraints for vehicles. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0048] Figure 1 This is a flowchart illustrating the GNSS / INS / map tightly integrated navigation and positioning method based on road marking matching provided in an embodiment of the present invention. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] like Figure 1As shown, this embodiment of the invention provides a GNSS / INS / map compact combination navigation and positioning method based on road marking matching, including: acquiring prior road map data, forward-looking road image data, GNSS raw observations, and INS raw observations; extracting road marking semantic features from the forward-looking road image data using a deep learning-based semantic segmentation network, and transforming the road marking semantic features into a bird's-eye view through inverse perspective transformation; clustering and modeling the extracted different types of road markings based on the spatial geometric features under the bird's-eye view to obtain road marking feature instances; transforming the road marking feature instances from a pixel-level coordinate system to a world coordinate system, and performing nearest neighbor search on map elements in the prior road map data based on the approximate coordinates of the road marking feature instances; constructing corresponding map matching observations based on the nearest neighbor search results; and adding the map matching observations to a GNSS / INS compact combination filtering framework based on PPP technology, and obtaining navigation and positioning results by coupling the GNSS raw observations, INS raw observations, and map matching observations. Among them, road markings are solid lane lines, dashed lane lines, stop lines and arrows on the road surface; GNSS is the Global Navigation Satellite System; INS is the Inertial Navigation System; and map matching is the correlation and matching observation between visual images and road markings in prior road maps.

[0051] Specifically, this embodiment of the invention provides data acquisition and preprocessing, preparing a road map covering the experimental site (obtained by purchasing commercial maps or creating custom maps). The prior road map data of this invention is obtained from the road map, and the geometric coordinate information is checked and calibrated. A GNSS receiver, MEMS IMU, and forward-facing camera are installed on the experimental vehicle to collect corresponding raw observations and road images, and various precision orbit products required for GNSS positioning are downloaded from the IGS data center. Preprocessing is performed on the raw GNSS observations, including cycle slip detection and gross error removal.

[0052] Specifically, the embodiments of the present invention provide road marking extraction and modeling. First, the semantic features of road markings in the forward-looking road image are extracted using a semantic segmentation network based on deep learning. Then, these semantic features are transformed into a bird's-eye view through inverse perspective mapping (IPM). Based on their spatial geometric features, different types of road markings are clustered and modeled respectively to obtain instance-level road marking features.

[0053] Specifically, this embodiment of the invention provides map data processing and association matching, extracting road marking information from road maps and storing its geometric coordinates in a KD-Tree structure, clustering and dividing it into different nodes according to spatial coordinates. Subsequently, the instance-level road marking features are transformed from the pixel coordinate system to the world coordinate system. The global coordinates of the current road marking feature instance are calculated using the approximate vehicle location provided by the GNSS / INS system. Then, nearest neighbor search is used to retrieve the closest map elements of the same type to the current road marking feature instance: for lane lines (including both solid and dashed lines) and stop lines, the distance to the line segment of the same type of map element is calculated, and the closest line segment is selected as the matching result; for arrows, the center point is calculated based on its minimum bounding box, and the arrow closest to the current center point is found as the matching object. The pixel coordinate system is represented as a direct coordinate system uv established with the top-left corner of the image as the origin, using pixels as units. The horizontal coordinate u and vertical coordinate v of a pixel are the column number and row number in its image array, respectively.

[0054] Specifically, in this embodiment of the invention, a GNSS / INS / map matching compact combination positioning model based on extended Kalman filtering is constructed. The time prediction part of the filtering is driven by INS mechanical orchestration, and measurement updates are triggered when there are GNSS observations or map matching observations.

[0055] Specifically, this embodiment of the invention uses a lightweight semantic segmentation network SFNet (ResNet-18) based on a convolutional neural network to extract road markings. SFNet (ResNet-18) adds a flow alignment module between adjacent layers of the network to transfer semantic information between network layers of different resolutions, thereby improving the efficiency and performance of semantic extraction and making it suitable for fast and accurate semantic extraction applications. For forward-facing vehicle road images, due to perspective distortion, road markings appear larger in the foreground and smaller in the background. To obtain the true geometric features of the road markings, it is necessary to use IPM transformation to convert them to a bird's-eye view in the world coordinate system. The forward-facing images from the vehicle are all captured by a forward-facing camera. It is collected, assuming there is an optical center position relative to... Overlapping virtual cameras , The XY plane is parallel to the road plane, meaning it's perpendicular to the ground when imaging. Under this assumption, the camera... and There exists a rotational transformation of approximately 90° between them. For a point M in a world coordinate system, the transformation formula is as follows:

[0056] (1)

[0057] in, and They represent point M in Camera coordinate system and Three-dimensional coordinates in the camera coordinate system; express Camera coordinate system to The rotation transformation matrix of the camera coordinate system can usually be approximated through prior calibration. Based on this, according to the pinhole camera model, the rotation transformation matrix of point M on the camera can be obtained. and camera The pinhole camera model below, namely

[0058] (2)

[0059] (3)

[0060] in, and These represent the pixel coordinates of point M in the front view image and the top view image, respectively. and These are their corresponding normalized camera coordinates; and Cameras and The intrinsic parameter matrix; For camera Specific intrinsic parameters, It indicates The height of the camera center above the road surface. w and h represent the width and height of the top-view image, reso represents the true scale represented by each pixel in the top-view image; D represents... The height of the camera center from the road surface, its size is... Equal. Complete the camera. Front view image to camera The transformation of the top-view image completes the IPM transformation, thereby obtaining a road marking image with real geometric features.

[0061] Specifically, after performing IPM transformation on the road marking images, pixels of the same type are clustered to obtain pixel cloud instances of different road markings. Based on their spatial geometric characteristics, different modeling methods are adopted: for road markings such as dashed lane lines and arrows, their minimum bounding rectangle is directly used for simplification. For linear markings with extension, such as solid lane lines and stop lines, resampling is used to transform them into multi-vertex polylines. Before resampling, singular value decomposition is used to calculate the principal directions of solid lane lines and stop lines, with the specific formula as follows:

[0062] (4)

[0063] (5)

[0064] in, Indicates the number of pixels corresponding to road markings, in superscript. Indicates matrix transpose. This represents any pixel point of a road marking. Indicates the center point of its pixel; The covariance matrix representing the coordinates of a pixel; and It is a unitary matrix composed of normalized eigenvectors; It is a diagonal matrix containing singular values; a vector This indicates the main direction of the linear road marking. The road marking instance is parameterized along the main direction, using a cubic polynomial for local description, as shown in the following expression:

[0065] (6)

[0066] in, Represents the pixel coordinates of road markings; polynomial parameters It can be calculated using least squares estimation. Once a well-fitted parametric model is obtained, resampling at equal intervals along the fitted parametric model (line) will yield the parametric modeling result.

[0067] Specifically, this embodiment of the invention provides a process for map data processing and association matching. First, the road marking instances are transformed from the IPM image coordinate system to the e-coordinate system, as shown in the following expression:

[0068] (7)

[0069] in, These are the approximate coordinates of the converted instance. and These represent the approximate attitude and coordinates of the vehicle in the E-series, which can be obtained through one-step prediction by filtering. express Camera coordinate system to Rotation transformation matrix of the camera coordinate system, and These represent overhead camera views. The rotation matrix and translation vector relative to the carrier can be obtained through prior calibration and measurement.

[0070] Specifically, a nearest neighbor search is performed based on the world coordinates of the road marking instances to find the nearest map features and construct the corresponding observation model. For lane lines (including both solid and dashed lines) and stop lines, the distance from the current point to the line segment formed by two points on the map is calculated, and the line segment with the shortest distance is determined as the line segment to be matched. For example, the distance from this point to the line segment The map matching observation expression is as follows:

[0071] (8)

[0072] in, For point to line segment The map matches the observations, i.e., the vertical distance. Representing points respectively , and Coordinate vectors in the e-frame, point For point to line segment The dangling foot, The arrows in the text represent vectors, and the whole text indicates calculating the magnitude of the vector; other parameters are similar. This represents the noise in map matching observations, and for the sake of brevity, it is defined as follows:

[0073] (9)

[0074] in, and Representing line segments respectively and line segments Based on the coordinate vector of the point, and combined with formula (7), it can be seen that the map matching observation is related to the approximate pose of the vehicle. Distance to map features and and Regarding this, we can obtain the following formula:

[0075] (10)

[0076] in, This indicates the position of the inertial navigation system in the e-frame. This indicates the attitude of the inertial navigation system in the e-frame. This represents the rotation matrix from frame b to frame e. for Compared to Jacobian matrix;

[0077] (11)

[0078] (12)

[0079] in, Represents line segment Length, To calculate intermediate quantities, To simplify the intermediate values ​​used in the expression, for arrows, the center point is calculated based on their smallest bounding box, and the arrow closest to the center of the current visual instance is selected as the matching object. (Based on a certain point...) For example, points can be constructed. To map point The distance is shown in the following formula:

[0080] (13)

[0081] in, and Representing points respectively and Coordinate vector in the e-frame, This represents the corresponding map matching observation noise. Solve. about and The partial derivatives of can be used to obtain the following formula:

[0082] (14)

[0083] in, express Compared to Jacobian matrix, symbol This represents the multiplication of vectors.

[0084] For each road marking instance in the current frame, observation equations similar to those in equations (8) and (13) can be established. The theoretical value of these observations is 0, meaning that the map and the visual instance have achieved a correct match. From this, the map matching observation vector can be written, i.e.

[0085] (15)

[0086] in, This represents an observation vector consisting of many point-to-point and point-to-line observations. The vector formed by its corresponding theoretical values ​​is equal to the zero vector.

[0087] Specifically, this embodiment of the invention provides a process for constructing a tightly coupled model of GNSS, INS, and map matching observations, with the following specific steps:

[0088] (1) Construct an inertial navigation system model. The error equation of the inertial navigation system can be expressed as:

[0089] (16)

[0090] in, , and These represent the attitude, velocity, and position of the inertial navigation system in the frame of reference, respectively. Each parameter is marked with a "·" symbol above it, indicating the derivative of that parameter with respect to time. , and The corresponding noise; This represents the vector of Earth's rotational angular velocity. Represents the rotation matrix from frame b to frame e; Indicates the specific force in the b system; and These are the measurement errors of the gyroscope and accelerometer, respectively. Let be the gravity error. In actual positioning processes, it is usually necessary to estimate and compensate for the zero-bias error included in the measurement errors of gyroscopes and accelerometers. This is modeled as a first-order Gaussian-Markov process, as shown in the following equation:

[0091] (17)

[0092] in, and These represent the zero bias of the gyroscope and accelerometer, respectively. and This corresponds to the time of the first-order Gauss-Markov process. and The power spectral density of the gyroscope and accelerometer can be considered to follow a white noise process.

[0093] (2) Constructing the GNSS observation model. The observation equations are constructed using Precise Point Positioning (PPP) technology based on ionosphere-free (IF) combination, as follows:

[0094] (18)

[0095] in, and These represent the GNSS pseudorange and carrier phase observations of the IF combination, respectively. and Represents GNSS signals of different frequencies; This corresponds to the raw GNSS pseudorange and carrier phase observations at different frequencies. Combining this with the non-differential form of the raw GNSS pseudorange and carrier phase observation equations, the IF combined observation model is as follows:

[0096] (19)

[0097] in, This is the geometric distance from the phase center of the satellite antenna to the phase center of the receiver antenna. Represents the speed of light; and These represent receiver clock bias and satellite clock bias, respectively. and These represent the dry delay and wet delay of the tropospheric error, respectively. and The zenith delay and projection function with dry delay can be accurately corrected using a priori models, such as the Saastamoinen model. and The zenith delay and projection function are the wet delay. Since the wet delay is very sensitive to the environment and satellite signal frequency, it is difficult to correct it using a model. Therefore, it is often obtained by parameter estimation together with unknown variables. and This represents the pseudorange and carrier phase observation noise of the IF combination; and These represent the wavelength and integer ambiguity of the carrier phase combination observation, respectively. Furthermore, when using multi-system (GPS, Galileo, GLONASS, and BDS) observations for PPP positioning, the different hardware delays of each system cause inconsistencies in the receiver clock biases. Therefore, it is necessary to introduce the Inter-System Bias (ISB), which is calculated together during parameter estimation. Typically, the GPS receiver clock bias is used as the reference. The expression for ISB is as follows:

[0098] (20)

[0099] in, For the receiver clock bias of the GPS system, This refers to the receiver clock bias of the other three systems.

[0100] (3) Combining the map matching observation model, a three-way compact combination localization model based on extended Kalman filtering is constructed. In the system state vector... The vector contains both the INS and PPP parameter vectors to be estimated. and ,as follows:

[0101] (twenty one)

[0102] To unify the system position errors of GNSS and INS, the position errors between the GNSS antenna phase center and the IMU measurement center need to be converted using a lever arm conversion, as follows:

[0103] (twenty two)

[0104] in, This indicates the position error of the antenna phase center; Let represent the arm from the IMU measurement center to the GNSS antenna phase center in frame b. The expression for the prediction model of the compact combination filter is as follows:

[0105] (twenty three)

[0106] in, The expression is as follows:

[0107] (twenty four)

[0108] (25)

[0109] in, and These represent the state transition matrices for INS and GNSS, respectively. Represents rows and columns The zero matrix; Represents an n-dimensional identity matrix; Represents the process noise vector. It is its corresponding coefficient matrix, and the process noise and its covariance matrix are determined by the parameters of the IMU sensor itself; This represents the number of ambiguities in the ionospherically unbound carrier phase. When a GNSS observation or map matching observation occurs at a certain moment, the algorithm will perform a measurement update. The specific linearized observation equation is shown below:

[0110] (26)

[0111] in, and Let represent the map matching observation vector and the GNSS observation vector at time k, respectively; and Let K represent the point-to-line and point-to-point map matching observations at time k, respectively. and Represent the GNSS pseudorange and carrier phase observations at time k without ionospheric saturation. and Let represent the model values ​​of the pseudorange and carrier phase observations at time k, respectively. and Let these represent the map matching observation and GNSS observation noise vectors at time k, respectively. This represents the state vector at time k. Let $k$ be the Jacobian matrix of the GNSS observations at time $k$ with respect to the state vector. Let k represent the observation coefficient matrix at time k.

[0112] The measurement model combining map matching, observation coefficient matrix The expression is:

[0113] (27)

[0114] Among them, subscript (i takes the values ​​1, 2, ..., n) and (i takes values ​​of 1, 2, ..., n) represent point-to-line and point-to-point map matching observations, respectively; n and m represent the number of these two types of observations, respectively. It is the Jacobian matrix of GNSS observations with respect to the system state vector. This represents the state vector at time k.

[0115] The implementation of the various embodiments of the present invention is based on programmed processing by a device with processor functionality. Therefore, in practical engineering, the technical solutions and functions of the various embodiments of the present invention are encapsulated into various modules. Based on this reality, and building upon the above embodiments, the embodiments of the present invention provide a GNSS / INS / map compact integration navigation and positioning system based on road marking matching. This system is used to execute the GNSS / INS / map compact integration navigation and positioning method based on road marking matching in the above method embodiments.

[0116] The system includes: a data acquisition module for acquiring prior road map data, forward-looking road image data, raw GNSS observations, and raw INS observations; a road marking feature acquisition module for extracting semantic features of road markings from the forward-looking road image data using a deep learning-based semantic segmentation network, transforming the semantic features of road markings into a bird's-eye view through inverse perspective transformation, and clustering and modeling the extracted different types of road markings based on the spatial geometric features under the bird's-eye view to obtain road marking feature instances; a map matching observation module for transforming the road marking feature instances from a pixel-level coordinate system to a world coordinate system, and performing nearest neighbor search on map elements in the prior road map data based on the approximate coordinates of the road marking feature instances, obtaining the corresponding map matching observation based on the nearest neighbor search results; and a navigation and positioning module for adding the map matching observation to a GNSS / / INS tight combination filtering framework based on PPP technology, and obtaining navigation and positioning results by coupling the raw GNSS observations, raw INS observations, and map matching observations.

[0117] The link adaptive system based on event flow-assisted deep learning for vehicle networking provided in this embodiment of the invention addresses the problem in the current navigation and positioning field that the rich road map information and raw GNSS observations are not fully utilized, making it difficult to maintain the accuracy and robustness of the navigation and positioning system in all directions in complex scenarios. It adopts the aforementioned modules and constructs a GNSS / INS / map matching tight combination positioning algorithm framework based on road markings. It associates and matches visual road markings with prior road map data to provide vehicle pose constraints in all directions. At the same time, it uses map matching observations to improve the positioning performance of the GNSS / INS navigation system.

[0118] Based on the same inventive concept as the foregoing embodiments, this embodiment of the invention also provides an electronic device, including a memory and a processor. The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to realize the GNSS / INS / map compact combination navigation and positioning method based on road marking matching as proposed in the above embodiments.

[0119] Based on the same inventive concept as the foregoing embodiments, this invention also provides a computer-readable storage medium storing a computer program thereon. When executed by a processor, this program overcomes the current problem in the field of navigation and positioning that it does not fully utilize rich road map information and raw GNSS observations, making it difficult to maintain the omnidirectional accuracy and robustness of navigation and positioning systems in complex scenarios. By utilizing rich road features and raw GNSS observation information, the navigation system achieves higher positioning accuracy and more robust performance in complex scenarios, making the constructed map matching observation model more accurate and reliable, and providing omnidirectional pose constraints for vehicles.

[0120] The storage medium can be any non-volatile storage device such as a hard disk, solid-state drive, flash drive, or optical disk, used to store computer program code and necessary data files. The stored computer program includes: a data acquisition module, a road marking feature acquisition module, a map matching and observation module, and a navigation and positioning module.

[0121] In summary, the present invention also provides specific working steps of the navigation and positioning method: First, prepare raw observation data from GNSS, INS, and visual sensors, and simultaneously prepare prior road map data. For the road image data, extract and model road markings. Utilize a deep learning-based semantic segmentation network to extract the semantic features of road markings from a series of road images captured by a vehicle-mounted forward-facing camera. Transform these features into a bird's-eye view using IPM technology, and then perform clustering and parametric modeling. Based on this, preprocess the prior road map data, extract the road marking information, and store it in a KD-Tree. Then, associate visual instances with map features in the map based on nearest neighbor matching, constructing map matching observations by building distance residuals between matching features. Subsequently, add the map matching observations as redundant information to the PPP / INS compact combination filtering framework. By achieving coupling of INS, GNSS, and map matching observations at the raw observation level, fully utilize multi-source positioning information, thereby obtaining the globally optimal pose estimation result.

[0122] Finally, it should be noted that the above specific embodiments are merely representative examples of the present invention. Obviously, the present invention is not limited to the above specific embodiments and many variations are possible. Any simple modifications, equivalent changes, and alterations made to the above specific embodiments based on the technical essence of the present invention should be considered within the protection scope of the present invention.

Claims

1. A GNSS / INS / map tightly coupled navigation positioning method based on road marking matching, characterized in that, The method comprises the following steps: acquiring prior road map data, front-view road image data, GNSS raw observation values, and INS raw observation values; extracting road marking semantic features in the front-view road image data by using a deep learning-based semantic segmentation network, converting the road marking semantic features into spatial geometric features by inverse perspective transformation, and respectively clustering and modeling different types of road markings to obtain road marking feature instances; converting the road marking feature instances from a pixel-level coordinate system to a world coordinate system, performing a nearest neighbor search on map elements in the prior road map data based on the approximate coordinates of the road marking feature instances, and obtaining corresponding map matching observations based on the nearest neighbor search results; constructing a GNSS / INS / map tight combination positioning model based on the GNSS raw observation values, the INS raw observation values, and the map matching observations to obtain a navigation positioning result. 2.The GNSS / INS / map tightly coupled navigation positioning method based on road marking matching according to claim 1, wherein, The clustering and modeling of different types of road markings respectively comprise the following steps: respectively clustering different types of road markings based on spatial geometric features to obtain clustering results; respectively clustering different types of road markings based on spatial geometric features to obtain clustering results; 3.The GNSS / INS / map tightly coupled navigation positioning method based on road marking matching according to claim 2, characterized in that, respectively clustering different types of road markings based on spatial geometric features to obtain clustering results. The modeling based on the clustering results by using different modeling methods comprises the following steps: for dashed lane lines and arrows, calculating the minimum circumscribed rectangle of the dashed lane lines and arrows and modeling the spatial uncertainty corresponding to the minimum circumscribed rectangle; 4.The GNSS / INS / map tightly coupled navigation positioning method based on road marking matching of claim 1, wherein, for solid lane lines and stop lines, calculating the principal direction of the solid lane lines and stop lines by singular value decomposition, fitting the road marking instances along the principal direction, and converting the solid lane lines and stop lines into multi-vertex polylines by resampling to obtain parameter modeling results. The corresponding map matching observations based on the nearest neighbor search results comprise the following steps: calculating the distance from the lane lines and stop lines to the line segments of the same type of map elements, and finding the closest line segment as the matching result; calculating the center point based on the minimum circumscribed rectangle of the arrow, and finding the closest arrow to the current center point as the matching object; 5.The GNSS / INS / map tightly coupled navigation positioning method based on road marking matching according to claim 1, wherein, obtaining map matching observations based on the matching results and the matching object. The method further comprises the following steps: 6.The GNSS / INS / map tightly coupled navigation positioning method based on road marking matching according to claim 1, wherein, extracting road marking information from the prior road map data, storing the geometric coordinates of the road marking information in a KD-Tree structure, and clustering and dividing the geometric coordinates into different nodes. The GNSS / INS / map tight combination positioning model based on the GNSS raw observation values, the INS raw observation values, and the map matching observations comprises the following steps: ， where and respectively denote the vectors of the parameters to be estimated for INS and PPP; the superscript denotes the matrix transpose; (1) defining a state vector, the expression of which is: ， wherein, and denote the state transition matrices of INS and GNSS, respectively; denote a row a column zero matrix; denote the process noise vector, is its corresponding coefficient matrix, denote the number of ambiguities of ionosphere-free combined carrier phase, and denote the power spectral density of gyroscope and accelerometer, denote the rotation matrix from body coordinate system to world coordinate system; (2) defining a prediction equation, the expression of which is: ， wherein, and denote the map matching observation vector and the GNSS observation vector at time k, respectively; and denote the point-to-line and point-to-point map matching observations at time k, respectively, and denote the ionosphere-free combined GNSS pseudorange and carrier phase observations at time k, respectively, and denote the model values of the pseudorange and carrier phase observations at time k, respectively, and denote the map matching observation and GNSS observation noise vectors at time k, respectively, denotes the state vector at time k, denotes the Jacobian matrix of the GNSS observations with respect to the state vector at time k, denotes the observation coefficient matrix at time k.

7. The GNSS / INS / map tightly coupled navigation positioning method based on road marking matching according to claim 1, characterized in that, (3) defining an observation equation, the expression of which is: The deep learning-based semantic segmentation network comprises the following steps:

8. GNSS / INS / map tightly coupled navigation positioning system based on road marking matching, characterized in that, extracting road markings by using a lightweight semantic segmentation network based on a convolutional neural network, wherein a flow alignment module is added between adjacent layers of the network to transmit semantic information between network layers at different resolutions. The method comprises the following steps: a data acquisition module for acquiring prior road map data, front-view road image data, GNSS raw observation values, and INS raw observation values; The road marking feature acquisition module is configured to extract road marking semantic features in the front-view road image data by using a deep learning-based semantic segmentation network, convert the road marking semantic features into spatial geometric features by inverse perspective transformation, cluster and model different types of road markings respectively, and obtain road marking feature instances. The map matching observation module is configured to convert the road marking feature instances from a pixel-level coordinate system to a world coordinate system, perform a nearest neighbor search on map elements in prior road map data based on the approximate coordinates of the road marking feature instances, and obtain corresponding map matching observations based on the nearest neighbor search results. The navigation positioning module is configured to construct a GNSS / INS / map tight combination positioning model based on GNSS raw observations, INS raw observations, and map matching observations, and obtain a navigation positioning result. 9.An electronic device comprising a memory and a processor, the memory storing a computer program, wherein, The processor executes the computer program to implement the steps of the GNSS / INS / map tight combination navigation positioning method based on road marking matching according to any one of claims 1-7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the GNSS / INS / map tight combination navigation positioning method based on road marking matching according to any one of claims 1-7.

Images