Positioning method based on monocular inertial navigation visual semantic information

Abstract

The invention provides a positioning method based on monocular inertial navigation visual semantic information, and the method comprises the steps: a vehicle collects a current image through a monocular camera, and extracts image semantic information; obtaining current estimated position information of the vehicle through an inertial measurement unit; locally searching a high-precision map based on the estimated position information to obtain map semantic information; matching the image semantic information with the map semantic information to obtain a semantic feature matching pair; and constructing a loss function, and substituting a calculation result of the loss function into an error state Kalman filtering algorithm to update the pose of the vehicle. Semantic information collected by monocular vision and data of the inertial measurement unit are combined to form a low-cost and efficient sensor fusion system, errors caused by sensor noise and drifting can be effectively suppressed, the problem of numerical instability possibly caused by full-state linearization is avoided by using an error state decomposition method, and the precision of the system is improved. Therefore, rapid convergence and high robustness are realized.

Description

Technical Field

[0001] The present invention relates to the field of unmanned driving technology, and in particular to a positioning method based on monocular inertial navigation visual semantic information. Background Art

[0002] In the autonomous driving technology ecosystem, high-precision positioning is a key prerequisite for autonomous vehicle navigation. Its core goal is to provide centimeter-level spatial position and attitude information for vehicles in dynamic scenarios. With the evolution of autonomous driving, positioning systems must maintain robustness in scenarios such as complex urban roads and areas without satellite signal obstruction, while also meeting the real-time and power consumption constraints of the on-board computing platform. However, existing positioning technologies still face significant challenges in terms of cost, accuracy, and environmental adaptability.

[0003] At present, vehicle positioning technology is mainly divided into the following categories: 1. Positioning methods based on lidar inertial navigation. Although the fusion positioning solution based on lidar and inertial measurement unit can provide high accuracy, the lidar hardware cost is high, and its detection capability is reduced in severe weather such as rain, fog, and dust. It needs to rely on multi-sensor redundant design, which increases the complexity of the system and restricts its application in large-scale commercialization.

[0004] 2. End-to-end positioning method based on deep learning. The end-to-end positioning solution based on deep learning directly regresses the vehicle's posture through a neural network. Although it performs well in some scenarios, it requires massive calibration data and a lot of training time, with high initial investment costs. In addition, the number of model parameters is huge, and the on-board edge computing unit is difficult to meet real-time requirements. At the same time, it lacks explicit modeling of the vehicle's kinematic constraints, and may output unreasonable posture estimates in sudden working conditions.

[0005] As a result, fusion positioning solutions based on monocular cameras, IMUs, and semantic maps have become a research hotspot due to their low hardware costs. These solutions achieve positioning by extracting semantic features from images and matching them with pre-built semantic maps. Compared to these positioning solutions, positioning methods based on monocular inertial visual semantic information offer the advantages of low cost and high accuracy, providing strong support for the commercialization of autonomous driving technology.

[0006] However, currently, monocular cameras can only capture two-dimensional image information and cannot directly obtain the depth (scale) data of the scene. This makes monocular visual positioning inherently disadvantageous in terms of accuracy, processing complexity, and stability, making it difficult to meet the strict real-time and reliability requirements of downstream autonomous driving tasks. In particular, depth information must be inferred through motion parallax or IMU data. In low-speed or short-baseline scenarios, depth estimation errors are significant, resulting in scale uncertainty in the three-dimensional coordinates of semantic features. Traditional semantic features are of a single type and are easily interfered with by factors such as occlusion, wear, and construction, resulting in a high risk of matching failure in scenarios such as rural roads without lane lines. Semantic feature extraction, map matching, and filter updates require a large amount of computing power, while the computing resources of on-board embedded platforms are limited, making it difficult to strike a balance between real-time and accuracy. The existing geometric representation of semantic features lacks a unified paradigm, resulting in inconsistent metrics for feature matching across sensors and maps, limiting the compatibility and scalability of positioning systems.

[0007] Currently, autonomous driving positioning technology is transforming from "single sensor high precision" to "multimodal low cost". The problems that need to be solved urgently include but are not limited to: compensating for the scale ambiguity of monocular vision and the cumulative error of IMU through algorithm innovation without relying on lidar; building a feature library covering multiple types of semantics to improve positioning capabilities on unstructured roads; designing a low-computational complexity semantic feature extraction, matching and filtering framework to adapt to the limitations of on-board computing power; establishing a mathematical description of semantic features compatible with two-dimensional images and three-dimensional maps to solve the problem of spatial alignment of heterogeneous data.

[0008] In summary, how to achieve high-precision and high-robust absolute vehicle positioning in complex dynamic scenes through low-cost hardware configuration through semantic feature system optimization, multi-source data fusion algorithm improvement and computing efficiency improvement is a key technical challenge that needs to be overcome in the current field of autonomous driving. Summary of the Invention

[0009] This application addresses the problems existing in the prior art and provides an effective and fast method for high-precision, multi-scenario and high real-time positioning with low computing resource consumption through improved algorithm combination.

[0010] To achieve the above objectives, the technical solutions adopted in this application are as follows: This application provides a positioning method based on monocular inertial navigation visual semantic information, which includes the following steps: The vehicle collects the current image through a monocular camera and extracts the image semantic information through a semantic segmentation model; Based on the speed information of the vehicle, obtaining the current estimated position information of the vehicle through an inertial measurement unit; Locally searching a high-precision map based on the estimated location information to obtain map semantic information; Matching the image semantic information with the map semantic information to obtain a semantic feature matching pair; A loss function is constructed, and the planar error between each pair of semantic feature matches is used as the input parameter of the loss function; the loss function is embedded in the gain calculation of the error state Kalman filter algorithm, and the vehicle's posture is updated according to the result of the error state Kalman filter algorithm.

[0011] Optionally, extracting image semantic information through a semantic segmentation model includes the following steps: Extracting segmentation semantic information of the current image through the semantic segmentation model; The segmentation semantic information is characterized and represented by line segments to obtain the image semantic information.

[0012] Optionally, the segmentation semantic information includes irregular environmental objects; The line segment characterization comprises the following steps: Segmenting instances in the current image based on a deep learning model and obtaining a pixel point set of the irregular environmental object; Calculating the main direction of the pixel point set by principal component analysis; The endpoint positions of the pixel point set in the main direction are determined based on an orthogonal projection method.

[0013] Optionally, the high-precision map is a three-dimensional point cloud map containing preset semantic information, and the preset semantic information includes one or more combinations of lane lines, tree trunks, poles, and curbs; The point cloud data of the high-precision map all contain global UTM coordinate information.

[0014] Optionally, the map semantic information acquisition includes the following steps: Based on the estimated location information, searching for semantic information within a preset radius of the estimated location in the high-precision map; Eliminate semantic information outside the field of view of the monocular camera from the semantic information within the preset radius to obtain observation range semantic information; The semantic information of the observation range is characterized and represented by three points to obtain the semantic information of the image.

[0015] Optionally, the three-point characterization includes the following steps: Classifying the observation range semantic information and obtaining a classification category; According to the classification category and the point cloud data of the high-precision map, regular sampling, key node extraction or geometric fitting method is selected to extract three spatial points from the semantic information of the observation range.

[0016] Optionally, the matching of the image semantic information and the map semantic information includes the following steps: Based on the matching relationship, establishing a correspondence between the line segments in the image semantic information and the points in the map semantic information to obtain point-line matching pairs; A distance function from a point to a line segment is established in the point-line matching pair, and the solution value of the distance function is used as the plane error between the semantic feature matching pairs.

[0017] Optionally, it also includes vehicle global coordinate initialization: when the vehicle starts positioning, the global coordinates of the vehicle are first obtained through the vehicle-side real-time dynamic positioning system and used as the initialization global coordinates.

[0018] Optionally, the vehicle global coordinate initialization is performed only once when starting positioning.

[0019] Optionally, the inertial measurement unit obtains the current estimated position information of the vehicle, comprising the following steps: The inertial measurement unit collects the velocity information, where the velocity information includes acceleration and angular velocity; The estimated position information is calculated based on the IMU motion model and the speed information of the vehicle.

[0020] Compared with the prior art, this application has the following beneficial effects: This application constructs a loss function through visual semantic information fusion, and combines the semantic information collected by monocular vision and the data of the inertial measurement unit based on efficient data fusion of error state Kalman filtering to form a low-cost and efficient sensor fusion system, which can effectively suppress the errors caused by sensor noise and drift. The error state decomposition method is used to avoid the numerical instability problem that may be caused by full-state linearization, thereby achieving fast convergence and high robustness. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0022] Figure 1 is a flow chart of a method in a specific embodiment of the present application; Figure 2 This is a local sample image of a high-precision map in a specific embodiment of this application; Figure 3 This is an example diagram of a matching pair in a specific embodiment of the present application.

[0023] In the figure: 1, curb, 2, lane line, 3, pole, 4, tree trunk, 5, line segment representation, 6, three-point representation. DETAILED DESCRIPTION

[0024] To make the purpose, technical solutions, and advantages of the embodiments of this application more clear, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0025] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, it does not need to be further defined or explained in subsequent drawings.

[0026] It is worth noting that the methods used in this application are all conventional methods unless otherwise specified; the raw materials and devices used are all conventional commercially available products, and their sources are not specifically limited unless otherwise specified.

[0027] It should also be noted that in order to facilitate understanding, the method steps in the specific embodiments of the present application are described in a certain order, but those skilled in the art can change the order of the steps according to actual needs, so this cannot be used as a limiting condition.

[0028] See also Figure 1 As shown, this embodiment provides a positioning method based on monocular inertial navigation visual semantic information, which mainly includes the following steps: Step 1: Pre-construction of high-precision maps; First, before implementing this embodiment, a high-precision three-dimensional point cloud map containing preset semantic information is established. Unlike traditional three-dimensional point cloud maps, the high-precision map of this embodiment only contains three-dimensional point clouds of pre-defined semantic features of the road environment, such as lane lines, tree trunks, poles, and curbs. Optionally, the high-precision three-dimensional point cloud map of this embodiment also includes irregular environmental objects, such as trees lining the road. Unlike conventional high-precision maps, the addition of irregular environmental objects facilitates subsequent semantic extraction and matching on roads outside of cities. This not only expands the information source but also avoids positioning issues in rural areas, where there are no easily recognizable artificial reference points such as signage. Furthermore, the point cloud data in this embodiment's high-precision map includes global UTM coordinates (Universal Transverse Mercator coordinates), ensuring that the map's coordinate system is consistent with the UTM coordinates of the global positioning system.

[0029] Among them, a local example of the high-precision three-dimensional point cloud map constructed in this embodiment is as follows Figure 2 As shown, the red point cloud is the curb 1, the yellow point cloud is the lane line 2, the blue point cloud is the pole 3, and the green point cloud is the tree trunk 4.

[0030] Step 2: Initialize vehicle global coordinates; After an autonomous vehicle starts, it requires continuous navigation and positioning. To ensure an accurate positioning foundation, this embodiment uses the vehicle's onboard RTK (Real-Time Kinematic Positioning) system to obtain the vehicle's global coordinates during the vehicle positioning algorithm module. These coordinates are used as initialization to complete the vehicle's initial global position. This step is performed only once, upon startup of the positioning module, to ensure that subsequent vehicle positioning is based on an accurate initial state.

[0031] Step 3: Detecting semantic information of vehicle-side images; The vehicle captures the current image using a monocular camera and extracts semantic information from the image using a semantic segmentation model. As the vehicle drives, the onboard monocular camera and pre-trained image segmentation model capture the current road scene in real time and detect various semantic features, including lane markings, tree trunks, poles, and curbs. Consequently, the captured image includes the aforementioned irregular environmental objects, particularly the trees on both sides.

[0032] Thus, the detection results are output in the form of instance segmentation of semantics in the image.

[0033] Step 4: Characterization of real-time image semantic information; Unlike regular man-made objects, natural objects like tree trunks have diverse forms and complex surface textures, making accurate and uniform parameterization a significant challenge. Overly complex representations hinder real-time computation, while oversimplification can lead to loss of critical geometric information. Therefore, this embodiment proposes a mathematically based line segment parameterization method that efficiently and robustly transforms irregular tree trunk outlines into uniform line segment features. The specific steps are as follows: Step 4.1: High confidence instance segmentation; Taking irregular environmental objects as trees as an example, this step is the basis for characterization, and its goal is to accurately identify and separate individual tree trunks from the complex background.

[0034] First, the on-board monocular camera collects RGB images in real time. Then, a pre-trained deep learning instance segmentation network is used, such as a trunk segmentation model optimized based on the Mask R-CNN architecture, to generate a pixel-level mask, and then output one or more pixel point sets. ,in Represents an independent tree trunk, each point is the pixel coordinate in the image coordinate system.

[0035] Step 4.2: Principal axis extraction based on principal component analysis (PCA); First, calculate the center of the point set, which can be specifically the centroid: the method is to calculate the average coordinates of all pixel points ; Thus, the geometric center of the point set is determined respectively: ; ; Where, 、 For the i The horizontal and vertical coordinates of the pixel points, n Point set S The total number of pixels in 、 are the average values ​​of the horizontal and vertical axes, respectively.

[0036] Secondly, construct the covariance matrix; This embodiment constructs a 2×2 covariance matrix C to describe the distribution and correlation of point sets. The matrix is ​​as follows: ; The calculations for each item are as follows: ; ; ; Where, is the horizontal axis u The variance of is the vertical axis v The variance of for u and v The covariance of is the degrees of freedom correction term for the unbiased estimate.

[0037] After that, solve the eigenvalues ​​and eigenvectors; This embodiment solves the eigenvalues ​​of the covariance matrix C λ and eigenvectors e , the formula is: ; This gives us two eigenvalues λ 1≥ λ 2 and the corresponding eigenvector e 1. e 2.

[0038] Where, λ 1. λ 2 is the eigenvalue of the covariance matrix, and λ 1 is the maximum eigenvalue; e 1. e 2 is the corresponding eigenvector, representing the direction of data change.

[0039] Finally, determine the main direction; The maximum eigenvalue of this embodiment λ 1 corresponding eigenvector e 1 is the main direction of the point set, which describes the extension trend of the trunk. Its direction is determined by the direction of the maximum variance of the point set distribution and is robust to local bending and irregular contours of the trunk.

[0040] Step 4.3: Orthogonal projection and line segment endpoint determination; First, projection is performed; The original pixel Subtract the centroid Then, to the main direction vector Orthogonal projection, get the projection scalar ;in, is the projection value of the point in the main direction, represents the vector dot product operation, is the normalized principal direction vector.

[0041] Secondly, find the extreme points; This embodiment finds the minimum value among the projected values and maximum value , the formula is: ; ; In the formula, the minimum value and the maximum value are the minimum and maximum values ​​of the projection value, respectively, corresponding to the two end positions of the trunk in the main axis direction.

[0042] After that, calculate the coordinates of the endpoints of the line segment; Segment endpoints and Move along the main direction from the center of mass and The formula is: ; ; Through the above steps, this embodiment can accurately and quickly parameterize any irregularly shaped tree trunk instance into a two-dimensional line segment defined by its endpoints. This method is mathematically rigorous and computationally efficient, providing standardized input for subsequent semantic feature matching and positioning solutions, and is a key technology for high-precision positioning in complex natural scenes.

[0043] Therefore, the semantic information detected in real time in the above steps is feature-processed, various semantic features are uniformly represented as line segments, and line segment parameters are obtained.

[0044] This processing method extracts line segment parameters by fitting the centerline of the instance segmentation target, providing standardized input data for the subsequent construction of the loss function. Furthermore, this step converts the semantic targets in the image into geometric line segments through the "centerline → line segment" conversion, which is easy to match and calculate with the semantic features in the subsequent map. In addition, the line segment parameters, such as the start point, end point, and angle, are low-dimensional data, which requires less calculation than two-dimensional regions or three-dimensional point clouds and can fully utilize limited computing resources. Furthermore, the instance segmentation area may have edge noise due to lighting and occlusion, and centerline fitting can filter out this noise; line segment fitting further approximates the curve as a straight line, reducing the interference of complex curves on matching. At the same time, the above steps also provide a solution to the problem that existing visual positioning solutions mostly rely on clear lane lines, road signs, poles and other features on structured roads, but their positioning performance will drop sharply in scenarios such as rural roads, parks or parking lots where these features are lacking. This is not just a simple addition of multiple feature types. It also overcomes the difficulty of ensuring the system can operate quickly, stably and reliably in an environment without poles or clear markings through optimized algorithm design. It greatly expands the operational design domain (ODD) of autonomous vehicles, enabling them to adapt to more diverse and challenging real-world scenarios, and is a key step in achieving high-precision positioning in all scenarios.

[0045] Step 5: Use the inertial measurement unit to predict the state; Based on the vehicle's speed information, the inertial measurement unit (IMU) obtains the vehicle's current estimated position. The onboard IMU collects the vehicle's speed information, including acceleration and angular velocity data, in real time. The IMU then predicts the vehicle's position state based on the IMU's motion model to obtain the vehicle's current estimated position. This estimated position serves as the prediction value in the subsequent error state Kalman filter algorithm, providing the initial state for state updates.

[0046] Step 6: Local search of map semantic information; Based on the estimated position information, a local search is performed on the high-precision map to obtain map semantic information. Specifically, based on the estimated vehicle position information obtained in the above steps, the pre-built global high-precision 3D point cloud semantic map is searched for semantic information within a preset radius around the vehicle's estimated position, thereby achieving a local search and reducing system resource consumption. To avoid interference from irrelevant data, semantic information outside the monocular camera's field of view is further eliminated, retaining only feature data that overlaps with the vehicle's field of view to obtain semantic information within the observation range.

[0047] Step 7: Characterization of map semantic information; The semantic information of the observation range filtered in the previous step is characterized and represented by three points to obtain the semantic information of the image, thereby uniformly describing each semantic feature as a feature form composed of three points. Specifically, the three-point representation includes the following steps: Classify the removed semantic information and obtain the classification category; Based on the classification category and the point cloud data of the high-precision map, regular sampling, key node extraction, or geometric fitting methods are selected to extract three spatial points from the semantic information after elimination. In a specific embodiment, classification can be performed based on geometric shapes, such as straight lines, broken lines / curves, and three-dimensional structures; If it is a linear feature, such as a lane line or a pole: the three points can be fitted into a straight line to calculate the slope and direction; If it is a partial broken line / curve, such as a bend along a road: three points can be fitted into a curve segment or an approximate straight line; If it is a three-dimensional structure, such as a traffic sign pole: the three points can cover the bottom, middle and top.

[0048] The three-point representation can describe the "position + form" of semantic features with the lowest complexity, balancing computational efficiency and representation accuracy.

[0049] During the map semantic information processing and matching phase, describing features using three-point structures facilitates more accurate positioning and matching. For example, for specific locations or landmarks on the road, a three-point shape can more accurately determine their position and shape on the map. Three points can also be used to identify specific areas or shapes within a plane, facilitating precise matching with real-time vehicle detection. This representation method accurately captures the spatial distribution and geometry of semantic features, providing a unified metric for calculating observation error when constructing the subsequent loss function.

[0050] Step 8: semantic feature matching; Match image semantic information with map semantic information to obtain semantic feature matching pairs. Match the image semantic features detected in real time by the vehicle with the map semantic features obtained from the high-precision map. Matching establishes a correspondence between the real-time observed features and the map features. Specifically, when matching image semantic information with map semantic information, a better matching method is: First, a coarse match based on semantic labels is performed. Building on the aforementioned local search, label-prioritized matching is employed: only features with consistent semantic categories in the image and the map are matched. Second, based on the aforementioned coarse matching, the three-dimensional spatial representation of the map semantic information is reduced to a three-point representation on the two-dimensional image plane using the intrinsic parameters of the monocular camera and the extrinsic parameters of the vehicle. This is then followed by a fine match based on feature descriptors. Geometric descriptors compare line segment features, calculating geometric properties such as orientation angles and length ratios, and determining semantic matching relationships based on similarity principles.

[0051] Furthermore, in matching the image semantic information and the map semantic information, the following steps are also included: After semantic information matching, the point-line correspondence between image semantic information and map semantic information is established to obtain point-line matching pairs; Figure 3 As shown, the red three-point representation 6 and the corresponding green line segment representation 5 form a matching pair, which together describe the tree trunk or rod-shaped object on the side of the road.

[0052] A planar geometry algorithm is used to establish the planar error for each semantic feature matching pair. Specifically, since the matching relationship involves line segment representations and three-point representations, a distance function is established from the point-to-line matching pair to the corresponding points on the line segment. The solved distance is used as the point-to-line deviation for the point-to-line matching pair. Subsequently, for each semantic feature matching pair, three sets of point-to-line matching pairs are generated. The distance from each point to the line segment is obtained using these three sets of distance functions and recorded as the planar error between the semantic feature matching pairs.

[0053] Although the matching and centering representation forms in the above steps are different, the data of the monocular image mentioned above is two-dimensional plane data. The center line of the semantic area after instance segmentation is fitted into a two-dimensional line segment, which is extremely fast to calculate, meeting the real-time processing needs of vehicle movement and adapting to low-cost computing power support; the three-dimensional point cloud semantic map is high-precision data constructed offline, which needs to accurately describe the spatial distribution and geometric shape of the semantic features: the data has coordinates, so the corresponding point data can be quickly obtained, and the expression is more convenient. For example, for straight line features: three points can define the starting point, direction, and length of the line, which is not only richer than a line segment, but also can locate the spatial position; three points as the simplest three-dimensional representation can not only cover the geometric forms of most semantic features, but also control the amount of calculation.

[0054] At present, positioning methods based on monocular visual semantics usually adopt more complex semantic characterization algorithms. The independent representation of various semantic features leads to a cumbersome matching process, high resource consumption and lack of universality. To solve this problem, this embodiment adopts a differentiated representation method: during the operation of the positioning algorithm, all features on the pre-built semantic map are uniformly described in the form of three points, and the semantic features detected by the monocular camera in real time are uniformly represented as line segments. This classified unified representation reduces the computing resources required to minimize the loss function, and significantly improves the adaptability of the system in the absence or ambiguity of semantic information, ensuring that efficient and accurate positioning can still be achieved in complex environments.

[0055] Step 9: Construct the loss function and perform Error-State Kalman Filter (ESKF) update; Construct a loss function, specifically taking the aforementioned distance function as the main body, and the design goal is to minimize the loss. The loss function is then embedded in the update step of the error state Kalman filter, that is, the loss function is brought into the gain calculation of the error state Kalman filter algorithm. Among them, in the specific filtering process of this embodiment, it is divided into two parts: prediction and update. The estimated position of the aforementioned inertial measurement unit completes the prediction part of the filtering and obtains the covariance matrix; then the above-mentioned loss function is used as a function of the observation equation, thereby completing the embedding of the loss function. Since the loss function of this embodiment is a nonlinear function, it needs to be aligned and linearized, that is, the error state quantity is differentiated to obtain the Jacobian matrix.

[0056] The error state Kalman filter algorithm generates an error state, which serves as the target of the loss function. Minimizing the target loss can be understood as finding the shortest distance from the midpoint of the loss function to the corresponding line segment. When the target is minimized, the resulting error state serves as the vehicle's current update, which corrects the estimated position to update the vehicle's current pose. This step not only completes the measurement update in the Kalman filter algorithm but also effectively mitigates the effects of sensor noise and dynamic errors.

[0057] Finally, it should be noted that the above content is only used to illustrate the technical solution of the present application, and does not limit the scope of protection of the present application. Simple modifications or equivalent replacements of the technical solution of the present application by ordinary technicians in this field do not deviate from the essence and scope of the technical solution of the present application.

Claims

1. A positioning method based on monocular inertial navigation visual semantic information, characterized by: The steps include: The vehicle collects the current image through a monocular camera and extracts the image semantic information through a semantic segmentation model; Based on the speed information of the vehicle, obtaining the current estimated position information of the vehicle through an inertial measurement unit; Locally searching a high-precision map based on the estimated location information to obtain map semantic information; Matching the image semantic information with the map semantic information to obtain a semantic feature matching pair; A loss function is constructed, and the planar error between each pair of semantic feature matches is used as the input parameter of the loss function; the loss function is embedded in the gain calculation of the error state Kalman filter algorithm, and the vehicle's posture is updated according to the result of the error state Kalman filter algorithm.

2. The positioning method based on monocular inertial navigation visual semantic information according to claim 1, characterized in that: Extracting image semantic information through the semantic segmentation model includes the following steps: Extracting segmentation semantic information of the current image through the semantic segmentation model; The segmentation semantic information is characterized and represented by line segments to obtain the image semantic information.

3. The positioning method based on monocular inertial navigation visual semantic information according to claim 2, characterized in that: The segmented semantic information includes irregular environmental objects; The line segment characterization comprises the following steps: Segmenting instances in the current image based on a deep learning model and obtaining a pixel point set of the irregular environmental object; Calculating the main direction of the pixel point set by principal component analysis; The endpoint positions of the pixel point set in the main direction are determined based on an orthogonal projection method.

4. The positioning method based on monocular inertial navigation visual semantic information according to claim 1, 2 or 3, characterized in that: The high-precision map is a three-dimensional point cloud map containing preset semantic information; The point cloud data of the high-precision map all contain global UTM coordinate information.

5. The positioning method based on monocular inertial navigation visual semantic information according to claim 4, characterized in that: The map semantic information acquisition includes the following steps: Based on the estimated location information, searching for semantic information within a preset radius of the estimated location in the high-precision map; Eliminate semantic information outside the field of view of the monocular camera from the semantic information within the preset radius to obtain observation range semantic information; The semantic information of the observation range is characterized and represented by three points to obtain the semantic information of the image.

6. The positioning method based on monocular inertial navigation visual semantic information according to claim 5, characterized in that: The three-point characterization includes the following steps: Classifying the observation range semantic information and obtaining a classification category; According to the classification category and the point cloud data of the high-precision map, regular sampling, key node extraction or geometric fitting method is selected to extract three spatial points from the semantic information of the observation range.

7. The positioning method based on monocular inertial navigation visual semantic information according to claim 6, characterized in that: The matching of the image semantic information and the map semantic information includes the following steps: Based on the matching relationship, establishing a correspondence between the line segments in the image semantic information and the points in the map semantic information to obtain point-line matching pairs; A distance function from a point to a line segment is established in the point-line matching pair, and the solution value of the distance function is used as the plane error between the semantic feature matching pairs.

8. The positioning method based on monocular inertial navigation visual semantic information according to claim 1, characterized in that: It also includes vehicle global coordinate initialization: when the vehicle starts positioning, the global coordinates of the vehicle are first obtained through the vehicle-side real-time dynamic positioning system and used as the initialization global coordinates.

9. The positioning method based on monocular inertial navigation visual semantic information according to claim 8, characterized in that: The vehicle global coordinate initialization is performed only once when starting positioning.

10. The positioning method based on monocular inertial navigation visual semantic information according to claim 1, characterized in that: The inertial measurement unit obtains the current estimated position information of the vehicle, comprising the following steps: The inertial measurement unit collects the velocity information, where the velocity information includes acceleration and angular velocity; The estimated position information is calculated based on the IMU motion model and the speed information of the vehicle.

Images