A method and device for updating vehicle posture data and a vehicle

Abstract

The present application relates to the field of navigation and assisted driving technology, and provides a method, device and vehicle for updating the posture data of a vehicle. The method comprises: obtaining first coordinate data obtained based on an advanced assisted driving map, obtaining second coordinate data perceived by the vehicle, using the first coordinate data as the target point cloud, using the second coordinate data as the source point cloud, performing odometer calculation based on map matching on the target point cloud and the source point cloud, and obtaining first posture data; constructing an optimization model, predicting the motion offset of the vehicle according to the second posture data currently output by the optimization model, and obtaining the target posture data of the vehicle at the next moment, and the target posture data is used to characterize the actual posture of the vehicle at the next moment. The present application can obtain a vehicle posture with sufficient accuracy at the lowest possible cost.

Description

Technical Field

[0001] The present application relates to the field of navigation and assisted driving technology, and more specifically, to a method and device for updating vehicle posture data, and a vehicle. Background Art

[0002] With the continuous evolution of technology and product improvements, advanced driver assistance services are gaining increasing recognition among users. Vehicles equipped with NOA are being released one after another. However, as advanced intelligent driving enters urban roads, the underlying map products are experiencing a new trend. Generally speaking, the use of high-precision maps places higher demands on their freshness, while also increasing costs. Relying solely on navigation maps cannot overcome the limitations of vehicle-side perception. Therefore, high-assisted maps (advanced driver assistance maps) have emerged as an intermediate product between navigation maps and high-precision maps.

[0003] In existing technologies, the fusion positioning technology based on the semantic features of high-precision maps uses a high-precision map engine to broadcast 3D semantic features such as road markings, traffic lights, traffic signs, and telephone poles. The images collected by the camera are then pre-processed and trained to output landmarks through a convolutional neural network. The data obtained by the sensor is associated with the map data to output the map relative position. Finally, a multi-source fusion filtering positioning model is established based on GNSS / INS combined navigation and wheel speedometer. This method is based on a relatively mature filtering theory and is simple and fast to implement, but it is not suitable for high-precision maps. Figure 1 In addition, it places excessively high technical costs and update cycles on perception, regulation, and positioning, which go beyond engineering reality. At the same time, because the filtering model must satisfy the first-order Markov assumption and does not consider historical information, it cannot effectively assist in current state estimation in some challenging scenarios.

[0004] End-to-end "mapless" autonomous driving technology does not rely on pre-produced high-precision maps. Instead, it uses navigation maps to directly utilize vehicle sensor input (such as cameras, lidar, and millimeter-wave radar) for real-time environmental perception and decision-making. The core of this technology is the use of deep learning models to convert sensor data into vehicle driving behaviors, including but not limited to path planning, obstacle avoidance, and speed control. While this approach offers greater generalization and, by not relying on high-precision maps, reduces reliance on map data collection and updates, thereby lowering costs, it also faces critical flaws in data distribution and diversity, and poses safety risks in unknown or complex road conditions.

[0005] Therefore, the present application provides a method, device and vehicle for updating vehicle posture data to solve one of the above technical problems. Summary of the Invention

[0006] The purpose of this application is to provide a method, device, and vehicle for updating vehicle posture data, which can solve at least one of the above-mentioned technical problems. The specific application is as follows:

[0007] According to a specific embodiment of the present application, in a first aspect, the present application provides a method for updating posture data of a vehicle, comprising:

[0008] Acquire first coordinate data obtained based on an advanced driver assistance map, the first coordinate data including first coordinate point data and first semantic label data, and each first coordinate point data is labeled based on the first semantic label data; acquire second coordinate data perceived by the vehicle, the second coordinate data including second coordinate point data and second semantic label data, and each second coordinate point data is labeled based on the second semantic label data; use the first coordinate data as a target point cloud and the second coordinate data as a source point cloud, perform odometer calculation based on map matching on the target point cloud and the source point cloud, and obtain first pose data; construct an optimization model, the optimization model consisting of odometer residuals, inertial measurement unit IMU pre-integration residuals, and global navigation satellite system GNSS data residuals; wherein the odometer residuals are constructed based on the first pose data; predict the motion offset of the vehicle according to the second pose data currently output by the optimization model, and obtain target pose data of the vehicle at the next moment, and the target pose data is used to characterize the pose of the vehicle at the next moment.

[0009] In one embodiment, the odometer calculation method based on map matching includes: ; Wherein, the first posture data is based on and characterization, represents the optimal rotation amount, represents the optimal translation amount, k represents the time mark, S represents the current feature point set, v represents the second coordinate data mark, w represents the first coordinate data mark, Represents the horizontal coordinate of the second coordinate data at time k, Represents the ordinate of the second coordinate data at time k, Represents the horizontal coordinate of the first coordinate data at time k, represents the vertical coordinate of the first coordinate data at time k, r represents the rotation amount of the current frame, t is the translation amount of the current frame, and R represents the rotation matrix of r.

[0010] In one embodiment, the optimization model includes: ;in, represents the IMU pre-integration residual, represents the odometry residual, Represents the GNSS data residual, represents the state vector, , indicating that the state vector takes into account n number of poses, , 、 express The three posture components at the moment, , represents the set of natural numbers, represents the Huber kernel function, represents the information matrix of the IMU factor, represents the IMU pre-integrated observation, represents the information matrix of the matching odometry, represents the matching odometry observation, represents the information matrix of RTK-GNSS, Represents RTK-GNSS observations.

[0011] In one embodiment, before performing map matching-based odometer calculation on the target point cloud and the source point cloud, the method further includes: obtaining position information output by the wheel speed encoder of the vehicle and posture information output by the IMU; fusing the position information with the posture information to obtain third posture data; using the first coordinate data as the target point cloud and the second coordinate data as the source point cloud, and performing map matching-based odometer calculation on the target point cloud and the source point cloud, including: using the first coordinate data as the target point cloud, the second coordinate data as the source point cloud, and the third posture data as the prior posture data, and performing map matching-based odometer calculation on the target point cloud and the source point cloud.

[0012] In one embodiment, before constructing the optimization model, the method further includes: for each coordinate point data group calibrated as the same point position by the first pose data, label matching the first semantic label data and the second semantic label data corresponding to each coordinate point data group; wherein, in each coordinate point data group, each coordinate point data group includes a first coordinate point data and a second coordinate point data, and each coordinate point data group corresponds to a first semantic label data and a second semantic label data; in response to the proportion of the target coordinate point data group in each coordinate point data group not exceeding the preset proportion, using the third pose data as the first pose data to construct the odometer residual; wherein, the target coordinate point data group is a coordinate point data group that matches the first semantic label data and the second semantic label data.

[0013] In one embodiment, obtaining the second coordinate data perceived by the vehicle includes: obtaining visual data based on a fisheye camera configured on the vehicle; performing target detection and semantic feature extraction on the visual data, and projecting the extracted data to the vehicle coordinate system based on an inverse perspective transformation to obtain the second coordinate data.

[0014] In one embodiment, the fisheye camera includes a first camera configured on the front side of the vehicle, a second camera configured on the left side of the vehicle, a third camera configured on the right side of the vehicle, and a fourth camera configured on the rear side of the vehicle; obtaining visual data based on the fisheye camera configured on the vehicle includes: determining an obstacle point with a minimum distance from the vehicle; determining a target camera with a minimum distance from the obstacle point, the target camera being the first camera, the second camera, the third camera, or the fourth camera; in response to the target camera being the first camera or the fourth camera, obtaining visual data based on the second camera and the third camera; in response to the target camera being the second camera or the third camera, obtaining visual data based on the first camera and the fourth camera.

[0015] According to a specific embodiment of the present application, in a second aspect, the present application provides a vehicle posture data updating device, comprising:

[0016] An acquisition unit is used to acquire first coordinate data obtained based on an advanced driver assistance map, wherein the first coordinate data includes first coordinate point data and first semantic label data, and each first coordinate point data is annotated based on the first semantic label data; and a processing unit is used to obtain second coordinate data perceived by the vehicle, wherein the second coordinate data includes second coordinate point data and second semantic label data, and each second coordinate point data is annotated based on the second semantic label data; and a processing unit is used to use the first coordinate data as a target point cloud and the second coordinate data as a source point cloud, perform odometer calculation based on map matching on the target point cloud and the source point cloud, and obtain first pose ... and obtain first pose data based on map matching. Based on the first posture data, an odometer residual is constructed; based on the posture information output by the inertial measurement unit (IMU) of the vehicle, an IMU pre-integration residual is constructed; based on the positioning data output by the real-time differential positioning global navigation satellite system (RTK-GNSS), a GNSS data residual is constructed; based on the odometer residual, the IMU pre-integration residual, and the GNSS data residual, an optimization model is constructed; and according to the second posture data currently output by the optimization model, the motion offset of the vehicle is predicted to obtain the target posture data of the vehicle at the next moment, the target posture data being used to characterize the posture of the vehicle at the next moment.

[0017] According to a specific embodiment of the present application, in a third aspect, the present application provides a vehicle for executing any one of the methods described in the first aspect.

[0018] Compared with the prior art, the above-mentioned embodiments of the present application have at least the following beneficial effects:

[0019] The present application provides a method, device and vehicle for updating the posture data of a vehicle. The method obtains coordinate data and semantic labels in a global coordinate system from an advanced driver-assisted driving map on the one hand, and obtains coordinate data and semantic labels in a vehicle body coordinate system from vehicle perception on the other hand, and then performs coordinate fusion through odometer matching to obtain the first posture data. On this basis, by considering various influences, the odometer residual, the inertial measurement unit (IMU) pre-integration residual, and the global navigation satellite system (GNSS) data residual are designed respectively, and then an optimization model is constructed through multiple residuals to output the second posture data. On this basis, high-precision motion offset prediction is performed using the second posture data to output the posture of the vehicle at the next moment. Among them, the accuracy of the posture prediction is guaranteed by the fusion of vehicle-side data and map-side data, and the posture is enhanced by the optimization model on this basis, so that the posture prediction maintains a high accuracy. At the same time, the data uses an advanced driver-assisted driving map, which has a lower mapping cost than the method of using high-precision maps for data fusion and posture prediction. In summary, the present application provides a low-cost, high-precision posture prediction application. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 A flow chart of a method for updating vehicle posture data according to an embodiment of the present application is shown;

[0021] Figure 2 A flow chart of a method for calculating odometry based on map matching of a target point cloud and a source point cloud is shown;

[0022] Figure 3 A flow chart of a method for verifying the first pose data is shown;

[0023] Figure 4 A flow chart of a method for obtaining second coordinate data perceived by a vehicle is shown;

[0024] Figure 5 A flow chart of a method for acquiring visual data based on a fisheye camera configured on a vehicle is shown;

[0025] Figure 6 A factor graph for characterizing the processing logic of the optimization model of the present application is shown;

[0026] Figure 7 A unit block diagram of a vehicle posture data updating device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0027] To make the objectives, technical applications, and advantages of this application more clear, this application will be further described in detail below with reference to the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of this application.

[0028] The terms used in the examples of this application are for the purpose of describing specific embodiments only and are not intended to limit this application. The singular forms "a," "the," and "the" used in the examples of this application and the appended claims are also intended to include plural forms, and unless the context clearly indicates otherwise, "a plurality" generally includes at least two.

[0029] It should be understood that the term "and / or" as used herein is merely a description of the relationship between associated objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, or B exists alone. Furthermore, the character " / " in this document generally indicates that the associated objects are in an "or" relationship.

[0030] It should be understood that although the terms first, second, third, etc. may be used to describe in the embodiments of the present application, these descriptions should not be limited to these terms. These terms are only used to distinguish the descriptions. For example, without departing from the scope of the embodiments of the present application, the first may also be referred to as the second, and similarly, the second may also be referred to as the first.

[0031] As used herein, the words "if" and "if" may be interpreted as "at the time of" or "when" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrases "if it is determined" or "if (stated condition or event) is detected" may be interpreted as "when it is determined" or "in response to the determination" or "when detecting (stated condition or event)" or "in response to detecting (stated condition or event)," depending on the context.

[0032] It should also be noted that the terms "include," "comprises," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a product or device comprising a series of elements includes not only those elements but also other elements not explicitly listed, or elements inherent to such product or device. In the absence of further limitations, an element defined by the phrase "comprises a..." does not exclude the presence of other identical elements in the product or device comprising the element.

[0033] It should be noted in particular that any symbols and / or numbers in the specification that are not marked in the accompanying drawings are not drawing marks.

[0034] The optional embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0035] The embodiment provided in this application is an embodiment of a method for updating vehicle posture data.

[0036] The following combination Figure 1 The embodiments of the present application are described in detail.

[0037] Figure 1 A flow chart of a method for updating vehicle posture data according to an embodiment of the present application is shown. Figure 1 As shown, the process includes the following steps S101 to S105.

[0038] In step S101 , first coordinate data based on an advanced driver assistance map is acquired.

[0039] The first coordinate data includes first coordinate point data and first semantic label data, and each first coordinate point data is labeled based on the first semantic label data.

[0040] In the above embodiment, the advanced driver assistance Figure 1 Generally refers to maps used in Level 3 and below autonomous driving systems for intelligent connected vehicles. A type of electronic navigation map, these maps complement the road attributes of traditional electronic navigation maps by adding information such as traffic facilities, traffic signs, and road attributes like slope and curvature, essential for advanced driver assistance functions. Compared to high-precision maps, advanced driver assistance maps have lower production costs and can offer comparable or even superior driver assistance capabilities.

[0041] In step S102 , second coordinate data perceived by the vehicle is acquired.

[0042] The second coordinate data includes second coordinate point data and second semantic label data, and each second coordinate point data is labeled based on the second semantic label data.

[0043] In this application, the execution order of step S101 and step S102 is not fixed. For example, step S102 may be executed first and then step S101, or step S101 and step S102 may be executed simultaneously. The above-mentioned execution of step S101 first and then step S102 is only an exemplary embodiment.

[0044] In step S103, the first coordinate data is used as the target point cloud, and the second coordinate data is used as the source point cloud. The target point cloud and the source point cloud are subjected to odometry calculation based on map matching to obtain the first pose data.

[0045] For example, the first pose data can be represented by a rotation amount and a translation amount.

[0046] Furthermore, we consider improving pose accuracy by building an optimization model. The residuals considered in the optimization model may include odometry residuals, inertial measurement unit (IMU) pre-integration residuals, and global navigation satellite system (GNSS) data residuals.

[0047] In step S104, an optimization model is constructed.

[0048] The optimization model consists of odometer residuals, IMU pre-integration residuals, and GNSS data residuals.

[0049] In one embodiment, the odometry residual can be constructed using the first pose data.

[0050] In one embodiment, the IMU pre-integration residual can be obtained through the attitude information output by the IMU.

[0051] In one embodiment, the GNSS data residual may be obtained using navigation data provided by a global navigation satellite system, where the global navigation satellite system may be, for example, a global navigation satellite system based on real-time kinematic (RTK).

[0052] In step S105, the motion offset of the vehicle is predicted according to the second posture data currently output by the optimization model to obtain the target posture data of the vehicle at the next moment. The target posture data is used to represent the posture of the vehicle at the next moment.

[0053] The vehicle posture data updating method provided by the present application obtains the coordinate data and semantic labels in the global coordinate system from the advanced driver-assisted map on the one hand, and obtains the coordinate data and semantic labels in the vehicle body coordinate system from the vehicle perception on the other hand, and then performs coordinate fusion through the odometer matching method to obtain the first posture data. On this basis, by considering various influences, the odometer residual, the inertial measurement unit IMU pre-integration residual, and the global navigation satellite system GNSS data residual are designed respectively, and then an optimization model is constructed through multiple residuals to output the second posture data. On this basis, the second posture data is used to perform high-precision motion offset prediction to output the vehicle's posture at the next moment. Among them, the accuracy of the posture prediction is guaranteed by the fusion of vehicle-side data and map-side data, and the posture is enhanced by the optimization model on this basis, so that the posture prediction maintains a high accuracy. At the same time, the data uses the advanced driver-assisted map, and the mapping cost is lower than the method of using high-precision maps for data fusion and posture prediction. In summary, the present application provides a low-cost, high-precision posture prediction application.

[0054] For example, the odometer calculation method based on map matching includes .

[0055] Among them, the first pose data is based on and characterization, represents the optimal rotation amount, represents the optimal translation amount, k represents the time mark, S represents the current feature point set, v represents the second coordinate data mark, w represents the first coordinate data mark, Represents the horizontal coordinate of the second coordinate data at time k, Represents the ordinate of the second coordinate data at time k, Represents the horizontal coordinate of the first coordinate data at time k, represents the vertical coordinate of the first coordinate data at time k, r represents the rotation amount of the current frame, t is the translation amount of the current frame, and R represents the rotation matrix of r.

[0056] For example, the optimization model is as follows:

[0057] .

[0058] in, represents the IMU pre-integration residual, represents the odometry residual, Represents the GNSS data residual, represents the state vector, , indicating that the state vector takes into account n number of poses, , 、 express The three posture components at the moment, , represents the set of natural numbers, represents the Huber kernel function, represents the information matrix of the IMU factor, represents the IMU pre-integrated observation, represents the information matrix of the matching odometry, represents the matching odometry observation, represents the information matrix of RTK-GNSS, Represents RTK-GNSS observations.

[0059] In this application, the odometry calculation based on map matching can also be accelerated by providing a priori pose.

[0060] Figure 2 A flow chart of a method for calculating odometry based on map matching of a target point cloud and a source point cloud is shown. Figure 2 As shown, the process includes the following steps S201 to S203.

[0061] In step S201, the position information output by the wheel speed encoder of the vehicle and the attitude information output by the IMU are obtained.

[0062] In step S201, the position information and the posture information are fused to obtain third posture data.

[0063] In step S203, the first coordinate data is used as the target point cloud, the second coordinate data is used as the source point cloud, and the third pose data is used as the prior pose data, and an odometer calculation based on map matching is performed on the target point cloud and the source point cloud.

[0064] This application provides a priori posture through third-party posture data, which saves computing costs while accelerating odometer calculations and improves the timeliness of posture prediction.

[0065] In this application, the first pose data is used to characterize the rotation and translation required to move from the first coordinate point data to the second coordinate point data at the same point position. This is equivalent to establishing a matching relationship between the first coordinate point data and the second coordinate point data. On this basis, we further consider semantic matching based on the semantic label data corresponding to the same point under different coordinate point data to further describe the accuracy of the first pose data.

[0066] Figure 3 A flow chart of a method for verifying the first pose data is shown in FIG. Figure 3 As shown, the process includes the following steps S301, S302a, and S302b.

[0067] In step S301, for each coordinate point data group calibrated as the same point position by the first pose data, label matching is performed on the first semantic label data and the second semantic label data corresponding to each coordinate point data group.

[0068] Among them, each coordinate point data group includes a first coordinate point data and a second coordinate point data, and each coordinate point data group corresponds to a first semantic label data and a second semantic label data.

[0069] In step S302a, in response to the fact that the proportion of the target coordinate point data group in each coordinate point data group does not exceed the preset proportion, the third pose data is used as the first pose data to construct the odometer residual.

[0070] The target coordinate point data set is a coordinate point data set that matches the first semantic label data and the second semantic label data.

[0071] In step S302b, in response to the proportion of the target coordinate point data group in each coordinate point data group exceeding the preset proportion, the first pose data is retained and the odometer residual is constructed using the first pose data.

[0072] For example, the preset ratio may be set to, for example, 95%.

[0073] In this application, before building the optimization model, the matching between the semantic label data realizes the verification of the first pose data. When the target coordinate point data group accounts for less than 95% of the coordinate point data groups, it means that the first pose data is not descriptive enough and not accurate enough. In this case, the third pose data is used and RTK-GNSS is called for repositioning.

[0074] In this application, the vehicle is equipped with a fisheye camera. For example, visual data can be acquired by the fisheye camera and second coordinate data can be obtained through a series of processing.

[0075] Figure 4 A flow chart of a method for obtaining the second coordinate data perceived by the vehicle is shown. Figure 4 As shown, the process includes the following steps S401 and S402.

[0076] In step S401 , visual data is acquired based on a fisheye camera configured on a vehicle.

[0077] In step S402, target detection and semantic feature extraction are performed on the visual data, and based on the inverse perspective transformation, the extracted data is projected into the vehicle coordinate system to obtain second coordinate data.

[0078] For example, the fisheye camera may include a first camera configured on the front side of the vehicle, a second camera configured on the left side of the vehicle, a third camera configured on the right side of the vehicle, and a fourth camera configured on the rear side of the vehicle. The present application can obtain visual data that meets the requirements by setting the fisheye camera call rules.

[0079] Figure 5 A flow chart of a method for acquiring visual data based on a fisheye camera configured on a vehicle is shown. Figure 5 As shown, the process includes the following steps S501 to S503b.

[0080] In step S501 , the obstacle point with the shortest distance to the vehicle is determined.

[0081] In step S502 , a target camera having the smallest distance to the obstacle point is determined, where the target camera is the first camera, the second camera, the third camera, or the fourth camera.

[0082] For example, the target camera with the smallest distance to the obstacle point can be determined according to the comparison result between the nearest point output by the drivable area detection (freespace) and the size of the body contour point in the body coordinate system.

[0083] In step S503a, in response to the target camera being the first camera or the fourth camera, visual data is acquired based on the second camera and the third camera.

[0084] In step S503b, in response to the target camera being the second camera or the third camera, visual data is acquired based on the first camera and the fourth camera.

[0085] As some other feasible embodiments, in response to the target camera being the first camera, visual data may be acquired based on the fourth camera, or in response to the target camera being the fourth camera, visual data may be acquired based on the first camera.

[0086] In this application, the camera with the smallest distance to the obstacle point is avoided to ensure that there is a sufficient distance between the camera and the obstacle point to obtain visual data with sufficient field of view, thereby ensuring that vehicle perception and processing are easy to obtain.

[0087] The method provided in this application uses an advanced driver assistance map produced by a vehicle equipped with a panoramic fisheye camera, an IMU and other sensors. The vehicle calls the fisheye camera interface to extract semantic features according to certain rules based on the real-time perception results, so as to prepare more computing resources to call the factor graph optimization model to complete the pose estimation.

[0088] In one implementation, a factor graph optimization model can be constructed using the gtsam library to establish a batch optimization cost function. All historical and current information are then jointly optimized to generate a six-degree-of-freedom pose, thereby achieving a low-cost, low-latency, high-precision fusion positioning for autonomous driving.

[0089] The purpose of this application is to provide a high-precision fusion positioning based on advanced assisted driving maps to overcome the high cost problem brought about by positioning based on high-precision maps, and to solve the data missing diversity problem existing in map-free intelligent driving technology based on navigation maps. At the same time, without adding new sensors to the vehicle, the vehicle-side posture can be obtained by calling the fisheye camera on demand and combining factor graph optimization, thereby maintaining low cost and low computing power and improving positioning accuracy.

[0090] Preferably, an implementation scheme of the present application is shown below, including steps one to seven.

[0091] 1. Map semantic feature reception. The Advanced Driver Assistance Map Engine receives map elements such as lane markings, crosswalks, ground signs, and stop signs within a range of 2 km in front of and 500 m behind the vehicle, and stores their contour point coordinates and semantic label IDs. The broadcast frequency is 50 Hz, and the contour points output in each frame are treated as a frame of 2D point cloud. According to specific clustering rules, each cluster carries semantic label information.

[0092] 2. Fisheye Camera Detection. The vehicle's panoramic fisheye cameras can be controlled and invoked according to specific rules based on the perception results of the drivable area detection. (This means that the fisheye camera interfaces required for detection are selected from multiple fisheye cameras for visual data.) The deep learning model YOLOV8 is used to detect and extract semantic features of the aforementioned target objects in real time based on the visual image information. After inverse perspective transformation, the images are projected onto the ground plane in the vehicle coordinate system to extract contour coordinate points and semantic label IDs. The following rules apply to fisheye camera invocation: 1. If both front and rear fisheye cameras are occluded, only the left and right fisheye cameras are invoked. 2. If the front fisheye camera is occluded, only the rear fisheye camera is invoked. 3. If the rear fisheye camera is occluded, only the front fisheye camera is invoked. The occlusion condition can be determined by comparing the nearest point output from the drivable area detection with the size of the vehicle's contour points in the vehicle coordinate system. The vehicle coordinate system is established according to the right-hand rule, with the X-axis pointing in the vehicle's forward direction, the Y-axis pointing to the left, and the Z-axis pointing toward the sky.

[0093] 3. Inertial solution: Based on the initial absolute positioning data provided by RTK-GNSS, the position information output by the wheel speed encoder and the attitude information output by the IMU are integrated to obtain more accurate and reliable attitude and position information, which can provide a priori pose for the subsequent map matching process.

[0094] 4. Semantic feature association and matching. The semantic feature point contour coordinate point set received by the advanced driver assistance map engine (i.e., the first coordinate data mentioned above in this application) is used as the target point cloud, and the semantic feature contour point set detected in real time by the fisheye camera is used as the source point cloud (i.e., the second coordinate data mentioned above in this application). The predicted prior value is provided by inertial solution, and the generalized-lterative closest point algorithm (GICP) is called to implement the odometry calculation based on map matching. The calculation formula is as follows: Indicates. Among them, represents the optimal rotation amount, represents the optimal translation amount, k represents the time mark, S represents the current feature point set, v represents the second coordinate data mark, w represents the first coordinate data mark, Represents the horizontal coordinate of the second coordinate data at time k, Represents the ordinate of the second coordinate data at time k, Represents the horizontal coordinate of the first coordinate data at time k, represents the vertical coordinate of the first coordinate data at time k, r represents the rotation of the current frame, t is the translation of the current frame, and R represents the rotation matrix of r. is the position of the current feature point in the vehicle body coordinate system, is the closest point matching the feature in the global coordinate system.

[0095] 5. Matching Accuracy Assessment: After S4 completes the matching process and determines the relative position between the vehicle and the map, the semantic tag ID can be used to further verify the matching accuracy. If the accuracy is less than 95%, the prior position is used and RTK-GNSS is used for repositioning.

[0096] 6. Constructing an optimization model. Generally speaking, a window optimization model with a basic structure is constructed by Indicates. Among them, is the residual, is the Jacobian of the residual with respect to the state, is the information matrix, for The transposed matrix of is the optimization variable.

[0097] On this basis, this application considers the residuals of matching the relative pose of the odometer and the optimization variables (i.e., the odometer residuals mentioned in this application), the residuals of the IMU pre-integration and the optimization variables (i.e., the IMU pre-integration residuals mentioned in this application), and the residuals corresponding to the prior factors formed by RTK-GNSS (i.e., the GNSS data residuals mentioned in this application). Specifically, it involves the residual construction of each factor and the calculation of the Jacobian matrix. Furthermore, as a feasible embodiment, it is considered to perform frame-by-frame optimization through a sliding window method, for example, setting the optimization model to ;in, represents the IMU pre-integration residual, represents the odometry residual, Represents the GNSS data residual, represents the state vector, , indicating that the state vector takes into account n number of poses, , 、 express The three posture components at the moment, , represents the set of natural numbers, represents the Huber kernel function, represents the information matrix of the IMU factor, represents the IMU pre-integrated observation, represents the information matrix of the matching odometry, represents the matching odometry observation, represents the information matrix of RTK-GNSS, Represents RTK-GNSS observations.

[0098] 7. Posture Release. The optimization model outputs the pose of the current frame in real time. Based on the vehicle's kinematic model, the vehicle's motion offset from the current time to the latest pose is calculated, and the predicted vehicle pose data for the next moment is released.

[0099] In this application, the processing logic of the optimization model can be represented by a factor graph.

[0100] Figure 6 A factor graph for characterizing the processing logic of the optimization model of the present application is shown.

[0101] For example, Figure 6As shown, the present application considers the residual of the matching odometer relative pose and optimization variables (i.e., the odometer residual mentioned above in this application), the residual of the IMU pre-integration and optimization variables (i.e., the IMU pre-integration residual mentioned above in this application), and the residual corresponding to the prior factor formed by RTK-GNSS (i.e., the GNSS data residual mentioned above in this application), and sets the matching odometer factor, RTK-GNSS factor, and IMU and integration factor respectively. The optimization model performs graph optimization in a window sliding manner, updating the pose frame by frame from left to right. In addition, the sliding window contains multiple poses arranged in a time-delayed sequence, specifically including optimized poses and poses to be optimized. The optimized poses adjacent to the pose to be optimized can be used as reference poses for pose optimization. Among them, each IMU pre-integration factor is used to intervene in two poses to be optimized, each matching odometer factor is used to intervene in two poses to be optimized, and each TTK-GNSS factor is used to intervene in one pose to be optimized. In this application, in order to maintain real-time performance during graph optimization, the dimension of the optimization variable is controlled to a certain extent, which corresponds to the processing of the sliding window and the output of the sliding window.

[0102] The present application also provides an apparatus embodiment that is consistent with the above embodiment, which is used to implement the method steps of the above embodiment. The explanation based on the same name meaning is the same as the above embodiment, and has the same technical effect as the above embodiment, and will not be repeated here.

[0103] Figure 7 A unit block diagram of a vehicle posture data updating device according to an embodiment of the present application is shown.

[0104] like Figure 7 As shown, the vehicle posture data updating device includes an acquisition unit 701 and a processing unit 702.

[0105] An acquisition unit 701 acquires first coordinate data based on an advanced driver assistance map, the first coordinate data including first coordinate point data and first semantic label data, each first coordinate point data being annotated based on the first semantic label data. The acquisition unit 701 is configured to acquire second coordinate data perceived by the vehicle, the second coordinate data including second coordinate point data and second semantic label data, each second coordinate point data being annotated based on the second semantic label data.

[0106] Processing unit 702 is used to use the first coordinate data as the target point cloud and the second coordinate data as the source point cloud, and perform odometer calculation based on map matching on the target point cloud and the source point cloud to obtain the first pose data. It is used to construct the odometer residual based on the first pose data. It is used to construct the IMU pre-integration residual based on the posture information output by the vehicle's inertial measurement unit IMU. It is used to construct the GNSS data residual based on the positioning data output by the real-time differential positioning global navigation satellite system RTK-GNSS. It is used to construct the optimization model based on the odometer residual, the IMU pre-integration residual, and the GNSS data residual. And it is used to predict the motion offset of the vehicle according to the second pose data currently output by the optimization model to obtain the target pose data of the vehicle at the next moment, and the target pose data is used to characterize the pose of the vehicle at the next moment.

[0107] In one embodiment, the first pose data is obtained by calculating the odometry based on map matching using the following formula:

[0108] .

[0109] Among them, the first pose data is based on and characterization, represents the optimal rotation amount, represents the optimal translation amount, k represents the time mark, S represents the current feature point set, v represents the second coordinate data mark, w represents the first coordinate data mark, Represents the horizontal coordinate of the second coordinate data at time k, Represents the ordinate of the second coordinate data at time k, Represents the horizontal coordinate of the first coordinate data at time k, represents the vertical coordinate of the first coordinate data at time k, r represents the rotation amount of the current frame, t is the translation amount of the current frame, and R represents the rotation matrix of r.

[0110] In one embodiment, the optimization model includes: .in, represents the IMU pre-integration residual, represents the odometry residual, Represents the GNSS data residual, represents the state vector, , indicating that the state vector takes into account n number of poses, , 、 express The three posture components at the moment, , represents the set of natural numbers, represents the Huber kernel function, represents the information matrix of the IMU factor, represents the IMU pre-integrated observation, represents the information matrix of the matching odometry, represents the matching odometry observation, represents the information matrix of RTK-GNSS, Represents RTK-GNSS observations.

[0111] In one embodiment, the acquisition unit 701 is further used to obtain the position information output by the vehicle's wheel speed encoder and the posture information output by the IMU. The processing unit 702 is further used to fuse the position information with the posture information to obtain third posture data. The processing unit 702 uses the first coordinate data as the target point cloud and the second coordinate data as the source point cloud to perform an odometer calculation based on map matching on the target point cloud and the source point cloud in the following manner: using the first coordinate data as the target point cloud, the second coordinate data as the source point cloud, and the third posture data as the prior posture data to perform an odometer calculation based on map matching on the target point cloud and the source point cloud.

[0112] In one embodiment, the processing unit 702 is also used to: for each coordinate point data group that is calibrated as the same point position by the first pose data, respectively, perform label matching on the first semantic label data corresponding to each coordinate point data group and the second semantic label data. Wherein, in each coordinate point data group, each coordinate point data group includes a first coordinate point data and a second coordinate point data, and each coordinate point data group corresponds to a first semantic label data and a second semantic label data. In response to the fact that the proportion of the target coordinate point data group in each coordinate point data group does not exceed the preset proportion, the third pose data is used as the first pose data to construct the odometer residual. Wherein, the target coordinate point data group is a coordinate point data group that matches the first semantic label data and the second semantic label data.

[0113] In one embodiment, the processing unit 702 obtains the second coordinate data perceived by the vehicle, including: obtaining visual data using a fisheye camera disposed on the vehicle, performing object detection and semantic feature extraction on the visual data, and projecting the extracted data into the vehicle coordinate system based on an inverse perspective transformation to obtain the second coordinate data.

[0114] In one embodiment, the fisheye cameras include a first camera located at the front of the vehicle, a second camera located at the left side of the vehicle, a third camera located at the right side of the vehicle, and a fourth camera located at the rear of the vehicle. The processing unit 702 acquires visual data based on the fisheye cameras located on the vehicle in the following manner: The obstacle point with the shortest distance from the vehicle is determined. A target camera with the shortest distance from the obstacle point is determined, where the target camera can be the first camera, the second camera, the third camera, or the fourth camera. In response to the target camera being the first camera or the fourth camera, visual data is acquired based on the second and third cameras. In response to the target camera being the second camera or the third camera, visual data is acquired based on the first and fourth cameras.

[0115] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0116] The present application also provides a system embodiment that is consistent with the above embodiment, which is used to implement the method steps of the above embodiment. The explanation based on the same name meaning is the same as the above embodiment, and has the same technical effect as the above embodiment, which will not be repeated here.

[0117] The system provided in this application can be divided into three parts: a sensor data preprocessing module, a semantic feature association and matching module, and a factor graph optimization model construction module. The sensor data preprocessing module primarily processes data at the data level to obtain target-level positioning results. It receives contour point data such as lane lines, crosswalks, ground signs, and stop lines within a certain range from a high-level assistance map engine. Furthermore, it uses a neural network model to detect these semantic features in real time using a fisheye camera interface after inverse perspective transformation according to specific rules. Simultaneously, it calculates vehicle inertial force based on the initial absolute pose provided by GPS, wheel speed encoders, and IMU. This serves as prior information to help narrow the search range and improve data association accuracy. The semantic feature association and matching module first extracts contour points from the semantic features detected in real time by the fisheye camera. It then associates them with the semantic features of the advanced driver assistance map in terms of both semantic labels and coordinate values. It then uses the GICP matching algorithm to determine the relative pose of the vehicle in the current frame and the local advanced driver assistance map. The inertial force estimation results provide a relatively accurate prior estimate to assist in the data association and matching process. The factor graph optimization construction module is mainly a method of tightly coupling visual-inertial odometry, which mainly obtains the globally consistent vehicle posture by jointly optimizing the visual-map matching odometry factor, IMU pre-integration factor, and RTK-GNSS factor.

[0118] The core technologies of this application are primarily focused on the following areas. 1) Low-cost sensor integration: Utilizing low-cost sensors such as panoramic fisheye cameras and IMUs installed in urban taxis, high-precision positioning is achieved, reducing reliance on expensive equipment. 2) Cloud-based crowdsourcing for lightweight map production: Through cloud-based crowdsourcing, lightweight maps are produced using data collected by a large number of low-cost vehicles, improving map data coverage and update frequency. 3) Real-time semantic feature extraction: Based on real-time perception results, the vehicle calls the fisheye camera interface to extract semantic features, providing key information for subsequent pose estimation. 4) Factor graph optimization model: A factor graph optimization model is constructed that integrates map matching factors, RTK-GNSS factors, and IMU pre-integration factors to improve pose estimation accuracy. 5) Joint optimization of historical and current information: By batch-optimizing the cost function, historical and current information are combined for joint optimization to generate accurate six-degree-of-freedom poses. 6) Modular system design: The system is divided into a sensor data preprocessing module, a semantic feature association and matching module, and a factor graph optimization model construction module, achieving clear functional division and efficient collaboration. 7) Prior information assistance: The DR inference results based on GPS, wheel speed encoder and IMU are used as prior information to improve the accuracy of data association and the efficiency of the matching process. 8) GICP matching algorithm: The GICP algorithm is used to achieve relative pose matching between the current frame of the vehicle and the local high-assisted map, which improves the accuracy and robustness of the matching. 9) Fusion structure design: The visual-map matching odometer factor, IMU pre-integration factor and RTK-GNSS factor are comprehensively considered to achieve a tightly coupled visual-inertial odometer. 10) Sliding window optimization: Through the sliding window technology, the real-time update of the optimization model and the optimization of the key frame pose are achieved, ensuring the real-time and stability of the system. These core technical points together constitute the high-precision fusion positioning of this application, realizing low-cost, low-latency, high-precision autonomous driving positioning technology, which has broad application prospects and market potential.

[0119] This application has the following beneficial effects. 1) Cost-Effectiveness: By utilizing low-cost panoramic fisheye cameras, IMUs, and other sensors installed in urban taxis, combined with lightweight maps generated through cloud crowdsourcing, the hardware cost of the high-precision positioning system is significantly reduced. 2) Real-Time Performance: The vehicle promptly accesses the fisheye camera interface to extract semantic features based on real-time perception, ensuring the system's responsiveness and real-time performance. 3) Optimization Algorithm: A batch optimization cost function based on a factor graph improves positioning accuracy and reliability by jointly optimizing historical and current information. 4) Modular Design: The system is designed with a sensor data preprocessing module, a semantic feature association and matching module, and a factor graph optimization model construction module. Each module has a clear division of labor and is easy to maintain and upgrade. 5) Data Association Accuracy: The prior information provided by DR inference narrows the search scope and improves data association accuracy. 6) Robustness: The system leverages real-time semantic feature detection and map matching to maintain stable positioning performance even in urban canyons or indoor environments where GPS signals are limited. 7) Flexibility: By accessing the fisheye camera on demand, the system can flexibly adapt to different driving scenarios and conditions. 8) Scalability: Lightweight maps produced through cloud crowdsourcing can be continuously updated and expanded to support a wider range of road and environmental information. 9) Autonomy: The system can autonomously calculate and optimize vehicle-side position and posture without adding new sensors, reducing dependence on external systems. 10) Low Latency: Through sliding window optimization and real-time updates, the system can quickly publish position and posture data, meeting the high latency requirements of autonomous driving. 11) Balance between high precision and low computing power: While maintaining high-precision positioning, the optimized algorithm reduces the demand for computing resources, making it suitable for vehicle platforms with limited computing power.

[0120] The present application also provides a vehicle that is consistent with the above embodiment, which is used to implement the method steps of the above embodiment. The explanation based on the same name meaning is the same as the above embodiment, and has the same technical effect as the above embodiment, which will not be repeated here.

[0121] Although operations are described in a particular order in the drawings, this should not be understood as requiring that the operations be performed in the particular order shown or in serial order, or that all shown operations be performed to obtain the desired results. In certain circumstances, multitasking and parallel processing may be advantageous.

[0122] The methods and apparatus of the present application can be implemented using standard programming techniques, using rule-based logic or other logic to implement the various method steps. It should also be noted that the terms "means" and "module" as used herein and in the claims are intended to include implementations using one or more lines of software code and / or hardware implementations and / or devices for receiving input.

[0123] The foregoing description of the implementation of the present application has been provided for purposes of illustration and description. The foregoing description is not intended to be exhaustive or to limit the present application to the precise form disclosed, and various variations and modifications are possible in accordance with the above teachings or may result from the practice of the present application. These embodiments have been selected and described in order to illustrate the principles of the present application and its practical application, so as to enable those skilled in the art to utilize the present application in various embodiments and modifications as appropriate for the particular use contemplated.

[0124] Regarding the apparatus in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0125] It is further understood that, unless otherwise specified, “connection” includes a direct connection where there are no other components between the two elements, and also includes an indirect connection where there are other elements between the two elements.

[0126] It should be further understood that although operations are described in a particular order in the drawings in the embodiments of the present application, this should not be construed as requiring that these operations be performed in the particular order shown or in a serial order, or that all of the illustrated operations be performed to obtain the desired result. In certain circumstances, multitasking and parallel processing may be advantageous.

[0127] Those skilled in the art will readily appreciate other implementations of the present application after considering the specification and practicing the application disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present application that follow the general principles of this application and include common knowledge or customary techniques in the field of this application that are not disclosed herein. The description and examples are to be considered merely exemplary, and the true scope and spirit of this application are indicated by the following claims.

[0128] It should be understood that the present application is not limited to the precise structures described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the scope of the appended claims.

[0129] The above embodiments are only used to illustrate the technical application of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, ordinary technicians in this field should understand that they can still modify the technical applications recorded in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical application to deviate from the spirit and scope of the technical application of each embodiment of the present application.

Claims

1. A method for updating vehicle posture data, characterized in that: include: Acquire first coordinate data obtained based on an advanced driver assistance map, where the first coordinate data includes first coordinate point data and first semantic label data, and each of the first coordinate point data is labeled based on the first semantic label data; Acquire second coordinate data perceived by the vehicle, the second coordinate data including second coordinate point data and second semantic label data, each of the second coordinate point data being annotated based on the second semantic label data; The first coordinate data is used as the target point cloud, the second coordinate data is used as the source point cloud, and the odometer calculation based on map matching is performed on the target point cloud and the source point cloud to obtain the first pose data; the first coordinate data is used as the target point cloud, the second coordinate data is used as the source point cloud, and the odometer calculation based on map matching is performed on the target point cloud and the source point cloud, including: the first coordinate data is used as the target point cloud, the second coordinate data is used as the source point cloud, and the third pose data is used as the prior pose data, and the odometer calculation based on map matching is performed on the target point cloud and the source point cloud; the odometer calculation method based on map matching includes: ； Wherein, the first posture data is based on and Characterization, represents the optimal rotation amount, represents the optimal translation amount, k represents the time mark, S represents the current feature point set, v represents the second coordinate data mark, w represents the first coordinate data mark, Represents the horizontal coordinate of the second coordinate data at time k, Represents the ordinate of the second coordinate data at time k, Represents the horizontal coordinate of the first coordinate data at time k, represents the ordinate of the first coordinate data at time k, r represents the rotation of the current frame, t is the translation of the current frame, and R represents the rotation matrix of r; Constructing an optimization model, wherein the optimization model is composed of an odometer residual, an inertial measurement unit IMU pre-integration residual, and a global navigation satellite system GNSS data residual; wherein the odometer residual is constructed based on the first posture data; and the optimization model includes: ； in, represents the IMU pre-integration residual, represents the odometer residual, Represents the GNSS data residual, represents the state vector, , indicating that the state vector takes into account n number of postures, , , express The three posture components at the moment, , represents the set of natural numbers, represents the Huber kernel function, represents the information matrix of the IMU factor, represents the IMU pre-integrated observation, represents the information matrix of the matching odometer, represents the matching odometry observation, Represents the information matrix of RTK-GNSS, Represents the observation quantity of RTK-GNSS; According to the second posture data currently output by the optimization model, the motion deviation of the vehicle is predicted to obtain the target posture data of the vehicle at the next moment, and the target posture data is used to characterize the posture of the vehicle at the next moment.

2. The method according to claim 1, characterized in that Before performing odometer calculation based on map matching on the target point cloud and the source point cloud, the method further includes: Obtaining position information output by a wheel speed encoder of the vehicle and attitude information output by the IMU; The position information is fused with the posture information to obtain third posture data.

3. The method according to claim 2, characterized in that Before constructing the optimization model, the method further includes: For each coordinate point data group calibrated as the same point position by the first posture data, the first semantic label data and the second semantic label data corresponding to each coordinate point data group are respectively matched; wherein, in each coordinate point data group, each coordinate point data group includes a first coordinate point data and a second coordinate point data, and each coordinate point data group corresponds to a first semantic label data and a second semantic label data; In response to the proportion of the target coordinate point data group in the coordinate point data groups not exceeding a preset proportion, using the third posture data as the first posture data to construct an odometer residual; The target coordinate point data set is a coordinate point data set that matches the first semantic label data and the second semantic label data.

4. The method according to claim 1, characterized in that: The obtaining of the second coordinate data perceived by the vehicle includes: Acquiring visual data based on a fisheye camera configured on the vehicle; Target detection and semantic feature extraction are performed on the visual data, and based on an inverse perspective transformation, the extracted data is projected to a vehicle coordinate system to obtain the second coordinate data.

5. The method according to claim 4, characterized in that The fisheye camera includes a first camera disposed on the front side of the vehicle, a second camera disposed on the left side of the vehicle, a third camera disposed on the right side of the vehicle, and a fourth camera disposed on the rear side of the vehicle; The obtaining of visual data based on a fisheye camera configured on the vehicle includes: determining an obstacle point having a minimum distance from the vehicle; Determine a target camera having the shortest distance from the obstacle point, wherein the target camera is the first camera, the second camera, the third camera, or the fourth camera; In response to the target camera being the first camera or the fourth camera, acquiring visual data based on the second camera and the third camera; In response to the target camera being the second camera or the third camera, visual data is acquired based on the first camera and the fourth camera.

6. A vehicle posture data updating device, characterized in that: The method according to any one of claims 1 to 5, wherein the device comprises: an acquisition unit, for acquiring first coordinate data obtained based on an advanced driver assistance map, the first coordinate data including first coordinate point data and first semantic label data, each of which is labeled based on the first semantic label data; and for acquiring second coordinate data perceived by the vehicle, the second coordinate data including second coordinate point data and second semantic label data, each of which is labeled based on the second semantic label data; A processing unit, configured to use the first coordinate data as a target point cloud and the second coordinate data as a source point cloud, perform odometer calculation based on map matching on the target point cloud and the source point cloud, and obtain first pose data; to construct odometer residuals based on the first pose data; to construct IMU pre-integration residuals based on the pose information output by the inertial measurement unit IMU of the vehicle; to construct GNSS data residuals based on the positioning data output by a real-time differential positioning global navigation satellite system RTK-GNSS; to construct an optimization model based on the odometer residuals, the IMU pre-integration residuals, and the GNSS data residuals; and to predict the motion offset of the vehicle according to the second pose data currently output by the optimization model, and obtain target pose data of the vehicle at the next moment, wherein the target pose data is used to characterize the pose of the vehicle at the next moment; The first coordinate data is used as the target point cloud, the second coordinate data is used as the source point cloud, and the odometer calculation based on map matching is performed on the target point cloud and the source point cloud, including: the first coordinate data is used as the target point cloud, the second coordinate data is used as the source point cloud, and the third posture data is used as the prior posture data, and the odometer calculation based on map matching is performed on the target point cloud and the source point cloud; the odometer calculation method based on map matching includes: ； Wherein, the first posture data is based on and Characterization, represents the optimal rotation amount, represents the optimal translation amount, k represents the time mark, S represents the current feature point set, v represents the second coordinate data mark, w represents the first coordinate data mark, Represents the horizontal coordinate of the second coordinate data at time k, Represents the ordinate of the second coordinate data at time k, Represents the horizontal coordinate of the first coordinate data at time k, represents the ordinate of the first coordinate data at time k, r represents the rotation of the current frame, t is the translation of the current frame, and R represents the rotation matrix of r; The optimization model includes: ； in, represents the IMU pre-integration residual, represents the odometer residual, Represents the GNSS data residual, represents the state vector, , indicating that the state vector takes into account n number of postures, , , express The three posture components at the moment, , represents the set of natural numbers, represents the Huber kernel function, represents the information matrix of the IMU factor, represents the IMU pre-integrated observation, represents the information matrix of the matching odometer, represents the matching odometry observation, Represents the information matrix of RTK-GNSS, Represents the RTK-GNSS observation.

7. A vehicle comprising a memory, a processor and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the method according to any one of claims 1 to 5.

Images