Map updating method and device, vehicle and storage medium

Abstract

The invention relates to the field of intelligent driving, and discloses a map updating method and device, a vehicle and a storage medium, and the method comprises the steps: obtaining inertial measurement data, vehicle-mounted wheel speed and camera data on the vehicle; calculating according to the inertial measurement data and the vehicle-mounted wheel speed, and determining the vehicle pose; semantic segmentation is carried out on the camera data, a semantic image is determined, and the semantic image comprises ground semantic information; projecting ground semantic information on a road to a three-dimensional space on a world coordinate system based on a vehicle pose, and determining point cloud information of a target lane line; and the point cloud information of the target lane line is matched with the point cloud data on the high-precision map, the map is updated according to a matching result, and the point cloud data on the high-precision map is determined by the priori map. According to the method, the limitation of traditional high-precision map static updating is broken through, real-time dynamic correction of map data is realized through multi-sensor space-time synchronization and three-dimensional reconstruction, the cost is reduced, and the coverage use scene is expanded.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent driving, and in particular to a map updating method and device, a vehicle, and a storage medium. BACKGROUND

[0002] A high-definition map (HD Map) is a map with high positioning accuracy and real-time data updating capability. The high-definition map mainly serves intelligent (automatic) driving vehicles and provides lane-level planning and self-positioning assistance for intelligent driving vehicles.

[0003] In related technologies, the HD Map is updated based on data collected by professional map collection vehicles. However, the professional map collection vehicles are equipped with laser radars and high-precision GNSS devices, which on the one hand, the collection vehicles have high use costs and are in small numbers, and the collection efficiency is low; on the other hand, the professional map collection vehicles face frequent changes in urban roads (such as construction and marking adjustment), have safety hazards, and are difficult to meet the needs of automatic driving vehicles for hourly or even minute-level data updating of the map, and cannot cover remote areas and underground parking lots and other scenes. SUMMARY

[0004] Embodiments of the present application provide a map updating method and device, a vehicle, and a storage medium to solve the technical problems of high cost, low coverage, and long cycle in updating a high-definition map in related technologies.

[0005] The embodiments of the present application provide a map updating method, which includes: obtaining inertial measurement data, vehicle-mounted wheel speed, and camera data on a vehicle; determining a vehicle pose by calculating the inertial measurement data and the vehicle-mounted wheel speed; performing semantic segmentation on the camera data to determine a semantic image, the semantic image including ground semantic information; projecting the ground semantic information on a road to a three-dimensional space on a world coordinate system based on the vehicle pose to determine target lane line point cloud information; matching the target lane line point cloud information with point cloud data on a high-definition map, and updating the map according to a matching result, wherein the point cloud data on the high-definition map is determined by a prior map.

[0006] In an embodiment of the present application, the target lane line point cloud information is matched with the point cloud data on the high-definition map, and the map is updated according to a matching result, which includes: if the target lane line point cloud information does not exist in the high-definition map, it is determined that the current road section does not have a prior map, and the target lane line point cloud information is generated into the prior map; if the target lane line point cloud information matches the point cloud data in the high-definition map, the current target lane line point cloud information is discarded; and if the target lane line point cloud information does not match the point cloud data in the high-definition map, the current matching point cloud data is updated according to the target lane line point cloud information to complete the map updating.

[0007] In an embodiment of the present application, the calculating according to the inertial measurement data and the vehicle wheel speed further comprises: The pre-integration calculation is performed on the inertial measurement data to determine a pre-integration residual and a speed residual, and the pre-integration residual and the speed residual are taken as pre-integration objects; a factor graph is constructed with the pre-integration objects and pose observation values, and the factor graph is iteratively optimized for multiple times to output vehicle pose and motion data, the vehicle pose including coordinates and an attitude of the vehicle, and the motion data including speed, acceleration bias and angular velocity bias of the vehicle; wherein the pre-integration residual is used to constrain consistency of the inertial measurement data and a kinematic model, the speed residual is used to constrain relevance of the wheel speed data and the inertial measurement data, and the pose observation values provide positioning reference information.

[0008] In an embodiment of the present application, the pre-integration calculation on the inertial measurement data to determine a pre-integration residual comprises: determining a state of the vehicle based on the vehicle wheel speed, if the vehicle is in a stationary state, performing static initialization on the inertial measurement data to determine bias vectors and noise vectors of the acceleration and the angular velocity in the stationary state; determining a gravity direction according to the bias vector of the acceleration, and determining initial attitude information in combination with the gravity direction and the bias vector; calculating a first angular velocity value and a first acceleration value at a current time, and a median integral of the angular velocity and a median integral of the acceleration from the current time to a next time based on the bias vectors and the noise vectors of the acceleration and the angular velocity; calculating position information and speed information at the current time according to the first angular velocity value and the first acceleration value; and determining pose four elements at the next time based on the position information and the speed information at the current time; performing incremental calculation on the pose four elements, the position information and the speed information corresponding to the current time and the next time to determine a rotational pose increment, a displacement change and a median integral increment of the speed; determining a rotational attitude pre-integration amount, a displacement pre-integration amount and a speed pre-integration amount based on the rotational pose increment, the displacement change and the median integral increment of the speed respectively; and determining the pre-integration residual according to the residual operation on the rotational attitude pre-integration amount, the displacement pre-integration amount and the speed pre-integration amount between the current time and the next time.

[0009] In an embodiment of the present application, before determining the state of the vehicle based on the vehicle wheel speed, further comprising: calibrating the coordinate system relationship of the camera and the inertial measurement unit to determine the rotation matrix and the camera intrinsic parameter of each of the camera and the inertial measurement unit; and determining the spatial rotation matrix and the translation vector between the wheel and the inertial measurement unit.

[0010] In an embodiment of the present application, the manner of determining the speed residual includes: obtaining an inverse matrix of the pose four elements, an inverse rotation matrix of the vehicle wheel speed to the world coordinate system, the speed at the current time, the skew-symmetric matrix of the angular velocity at the current time, the translation parameter of the vehicle wheel speed to the world coordinate system, and the vehicle speed determined by the wheel speed model; determining a first product according to the multiplication operation of the speed at the current time and the inverse rotation matrix; determining a second product according to the multiplication operation of the translation parameter and the skew-symmetric matrix; determining a third product according to the multiplication of the sum of the second product and the first product and the inverse matrix; and determining the speed residual according to the difference operation of the third product and the vehicle speed.

[0011] In an embodiment of the present application, the ground semantic information on the road is projected into a three-dimensional space in the world coordinate system based on the vehicle pose to determine the target lane line point cloud information, which includes: converting the two-dimensional point cloud data in each frame of the ground semantic information into initial three-dimensional point cloud data in the camera coordinate according to the preset camera parameter; determining the lane line point cloud from the feature extraction of each frame of the initial three-dimensional point cloud data according to the ground semantic information; obtaining the vehicle pose data corresponding to each frame of the lane line point cloud through the vehicle pose data in the world coordinate system and the linear interpolation time synchronization mechanism; and converting the lane line point cloud from the vehicle body coordinate system to the world coordinate system through the rigid transformation principle to obtain the target lane line point cloud information in the world coordinate system.

[0012] The embodiment of the present application also provides a map updating device, which includes: a data acquisition module configured to acquire the inertial measurement data, the vehicle wheel speed, and the camera data on the vehicle; a pose determination module configured to determine the vehicle pose by calculating the inertial measurement data and the vehicle wheel speed; a semantic segmentation module configured to perform semantic segmentation on the camera data to determine a semantic image, the semantic image including the ground semantic information; a lane line determination module configured to project the ground semantic information on the road into a three-dimensional space in the world coordinate system based on the vehicle pose to determine the target lane line point cloud information; and a map updating module configured to match the target lane line point cloud information with the point cloud data on the high-definition map, and update the map according to the matching result, wherein the point cloud data on the high-definition map is determined by the prior map.

[0013] The embodiment of the present application also provides a vehicle adopting the method of any one of the above-mentioned embodiments.

[0014] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the method of any one of the above-mentioned embodiments.

[0015] The scheme implemented by the map updating method, device, vehicle and storage medium provided above effectively compensates for single sensor error by combining the short-time high-frequency characteristics of the IMU and the odometry information of the wheel speed data, and enhances the positioning robustness through multi-modal data fusion; the camera data is subjected to pixel-level semantic segmentation, which not only accurately extracts key ground semantic information such as lane lines and road boundaries, but also filters dynamic obstacle interference; the two-dimensional semantic image based on the vehicle pose is projected to a three-dimensional world coordinate system, a lane line point cloud model is constructed, the space-time synchronization of visual features and vehicle kinematics is realized, the lane line point cloud generated in real time is matched with the prior point cloud of the high-definition map, and the spatial transformation matrix of the overlapping area is calculated to realize incremental updating of the map data. The present application breaks through the limitations of traditional high-definition map static updating, constructs a three-dimensional lane line point cloud by fusing vehicle inertial measurement data, wheel speed data and camera semantic information, and dynamically matches the high-definition map, realizes real-time updating of map data by using ordinary vehicle sensors, and improves the timeliness of map updating; compared with the pure visual SLAM scheme, the anti-interference ability is improved; compared with the laser radar mapping scheme, the lane line detection accuracy is maintained, and the cost is reduced. BRIEF DESCRIPTION OF DRAWINGS

[0016] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the description of the embodiments of the present application will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0017] Figure 1 An exemplary system architecture diagram of the map updating device provided by the embodiments of the present application is provided. Figure 2 A flowchart of the map updating method provided by the embodiments of the present application is provided. Figure 3 A structural schematic diagram of the map updating device provided by the embodiments of the present application is provided. Figure 4 An incremental flowchart of the map updating device provided by the embodiments of the present application is provided. Figure 5 A structural schematic diagram of the electronic device in an embodiment of the present application is provided. Figure 6 Another structural schematic diagram of the electronic device in an embodiment of the present application is provided. DETAILED DESCRIPTION

[0018] With reference to the drawings of the embodiments of the present application, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are some of the embodiments of the present application, but not all the embodiments of the present application. Based on the embodiments in the present application, all the other embodiments obtained by those of ordinary skill in the art without creative work are within the scope of protection of the present application.

[0019] In order to enable those skilled in the art to better understand the improvement of the technical solutions provided by the present disclosure, the present disclosure briefly introduces the implementation scenarios and related information of the map updating method in the related art.

[0020] Figure 1 is a functional block diagram of a vehicle 100 provided by an embodiment of the present application. The vehicle 100 includes but is not limited to a new energy vehicle and a fuel vehicle, wherein a map updating device 101 is configured in the vehicle, and the vehicle 100 can include a perception system, a display device and a computing platform, wherein the perception system can include several kinds of sensors for sensing information about the environment around the vehicle 100. For example, the perception system can include a positioning system, which can be a global positioning system (GPS), a Beidou system or other positioning systems, an inertial measurement unit (IMU), a laser radar, a millimeter wave radar, an ultrasonic radar and a camera device.

[0021] Some or all functions of the vehicle 100 can be controlled by a computing platform. The computing platform can include multiple processors, which are circuits with processing capabilities of signals, in one implementation, the processors can be circuits with instruction reading and running capabilities, such as central processing units (CPUs), microprocessors, graphics processing units (GPUs) (which can be understood as a kind of microprocessor), digital signal processors (DSPs), and the like; in another implementation, the processors can implement certain functions through the logical relationship of hardware circuits, which is fixed or can be reconfigured, such as application-specific integrated circuits (ASICs) or programmable logic devices (PLDs) implemented hardware circuits, such as FPGAs. In the reconfigurable hardware circuit, the processor loads the configuration document to implement the hardware circuit configuration process, which can be understood as the process of the processor loading instructions to implement the functions of the above part or all units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as a kind of ASIC, such as neural network processing unit (NPU), tensor processing unit (TPU), deep learning processing unit (DPU), and the like. In addition, the computing platform can also include a memory for storing instructions, and some or all of the multiple processors can call the instructions in the memory to implement corresponding functions.

[0022] The vehicle 100 can include an advanced driving assistant system (ADAS) that utilizes various sensors (including but not limited to laser radar, millimeter wave radar, camera, ultrasonic sensor, global positioning system, inertial measurement unit) on the vehicle to obtain information from the surroundings of the vehicle, and analyzes and processes the obtained information to implement functions such as obstacle perception, target recognition, vehicle positioning, path planning, driver monitoring / reminding, etc., thereby improving the safety, automation level and comfort of vehicle driving.

[0023] The sensors can include a laser radar, a millimeter wave radar, a camera, an ultrasonic sensor, wherein the millimeter wave radar can be divided into a long-range radar and a medium / short-range radar. At present, the sensing range of the laser radar is about 80-150 meters, the sensing range of the long-range millimeter wave radar is about 1-250 meters, the sensing range of the medium / short-range millimeter wave radar is about 30-120 meters, the sensing range of the camera is about 50-200 meters, and the sensing range of the ultrasonic radar is about 0-5 meters.

[0024] From the logical function, the ADAS system generally includes three main functional modules: a perception module, a decision module and an execution module. The perception module perceives the environment around the vehicle body through sensors, inputs corresponding real-time data to the decision layer processing center, and the perception module mainly includes a vehicle-mounted camera, an ultrasonic radar, a millimeter wave radar, a laser radar and the like. The decision module uses a computing device and an algorithm to make a corresponding decision according to the information obtained by the perception module. The execution module takes corresponding actions such as driving, lane changing, turning, braking and warning after receiving the decision signal from the decision module.

[0025] As described above, the current high-precision map acquisition mode of the vehicle acquisition mode has the following significant disadvantages: 1) high-beam laser radar (single cost of tens of thousands to hundreds of thousands of yuan), high-precision GNSS and other equipment are required, the cost of centimeter-level map surveying is 1000 yuan per kilometer, and about 20 billion yuan is required for national road full coverage; 2) the acquisition efficiency of a single vehicle per day is only 100 kilometers (centimeter level) or 500 kilometers (decimeter level), and the comprehensive cost is difficult to reduce due to the superposition of manpower and equipment maintenance costs; 3) relying on professional surveying vehicle team, the update cycle is more than 3 months, which cannot match the frequent changes of urban roads (such as construction, marking adjustment), and there is a safety hazard; 4) weak intelligence acquisition capability, traditional "street sweeping type" acquisition is only 50%-60% effective, and resources are wasted seriously; 5) the surveying vehicle team is difficult to cover remote areas, underground parking lots and other scenes, and the mobility is poor and the adjustment plan lags behind; 6) relying on fixed route acquisition, sudden road changes (such as temporary traffic control) cannot be responded in real time; 7) massive raw data need manual annotation and verification, the automation degree is low, and the compilation period is long; 8) storage and transmission pressure is large, the high-precision map data volume is large, and the vehicle end update bandwidth demand is high.

[0026] To solve the above problems, the contradiction between map updating efficiency, cost and coverage range in the prior art is increasingly prominent. Through analysis, it is found that ordinary vehicles are equipped with inertial measurement units, wheel speed sensors and cameras and other devices which have basic data collection capabilities. If multi-source sensor data can be effectively fused to construct a low-cost lane line recognition and map updating mechanism, high-frequency and wide-coverage map maintenance can be theoretically realized. In view of this, real-time road data is collected by ordinary vehicles, lane line information is extracted by multi-sensor fusion positioning and semantic segmentation technology, dynamic updating is realized in combination with prior maps, and a distributed map maintenance network is formed.

[0027] Please refer to Figure 2 , Figure 2 A flowchart of a map updating method provided by the embodiment of the present application is shown in the figure. The method comprises the following steps: Step S201, inertial measurement data, vehicle-mounted wheel speed and camera data on the vehicle are acquired. Step S202, the vehicle pose is determined by calculating according to the inertial measurement data and the vehicle-mounted wheel speed. Step S203, semantic segmentation is performed on the camera data to determine a semantic image, and the semantic image comprises ground semantic information. Step S204, the ground semantic information on the road is projected onto a three-dimensional space in a world coordinate system based on the vehicle pose to determine target lane line point cloud information. Step S205, the target lane line point cloud information is matched with point cloud data on a high-precision map, and the map is updated according to the matching result, wherein the point cloud data on the high-precision map is determined by a prior map.

[0028] The inertial measurement data refers to vehicle motion state parameters acquired by an accelerometer and a gyroscope, for example, a six-axis sensor can be used to calculate the vehicle attitude and displacement change. The vehicle-mounted wheel speed refers to the vehicle driving speed acquired by a wheel speed sensor, for example, a Hall effect sensor can be used to assist in correcting the cumulative error of the inertial measurement unit. Semantic segmentation refers to extracting ground semantic information such as lane lines and road shoulders from camera data by image recognition algorithms, for example, a convolutional neural network model can be used to generate structured road features. World coordinate system projection refers to mapping semantic information in the vehicle body coordinate system to the global coordinate system through geometric transformation, for example, a rigid body transformation matrix can be used to unify local data collected at different times. Point cloud matching refers to spatially aligning the lane line point cloud generated in real time with the prior map, for example, an iterative closest point algorithm can be used to detect map differences and trigger updates.

[0029] Exemplarily, by eliminating the timing misalignment caused by the difference in sensor sampling frequency, the spatio-temporal consistency of pose calculation and image semantic segmentation is ensured; the conversion relationship (such as rotation matrix and translation vector) between the coordinate systems of various sensors is established, for example, the timestamps of different sensors are aligned by using hardware triggering (such as PPS signal) or software interpolation (such as linear interpolation); for another example, the spatial transformation matrix of IMU, wheel speed sensor and camera is determined by joint calibration (such as using a checkerboard calibration method).

[0030] Exemplarily, according to the camera intrinsic parameters (focal length f, optical center) and camera extrinsic parameters (R, t), the image pixel coordinates (u, v) are converted into three-dimensional points (Xc, Yc, Zc) in the camera coordinate system, and the three-dimensional points are converted to the world coordinate system (Xw, Yw, Zw) combined with the vehicle pose (T_vehicle); the lane line pixels in the semantic image are three-dimensionally reconstructed to generate a point cloud with semantic labels.

[0031] Specifically, during the driving of the vehicle, the inertial measurement unit continuously acquires angular velocity and acceleration data, and the vehicle-mounted wheel speed sensor synchronously records the vehicle speed information; the sensor noise is eliminated by a pre-integration algorithm, and the factor graph is constructed combined with the wheel speed data for pose optimization, and the accurate position and attitude of the vehicle in the world coordinate system are output; at the same time, the road image captured by the vehicle-mounted camera is processed by the semantic segmentation network to separate the lane line pixel area; based on the camera intrinsic and extrinsic parameters and the optimized vehicle pose, the two-dimensional pixel coordinates are converted into three-dimensional point cloud, and the point cloud is projected to the world coordinate system through rigid body transformation, and the point cloud data is registered with the corresponding area in the high-precision map: if new lane lines are found, a new prior map is generated; if it is consistent with the existing data, redundant information is filtered; if there is a difference, the map database is updated.

[0032] Specifically, the traditional method relies on professional acquisition vehicles with laser radar and high-precision positioning equipment, while the present application realizes data acquisition by using existing sensors of ordinary vehicles, which greatly reduces the hardware cost. The prior art adopts a centralized map updating mechanism, while the present application realizes real-time data acquisition and local updating by a distributed vehicle group, which significantly improves the map updating frequency. In addition, traditional semantic segmentation is mainly used for environmental perception rather than map construction, and the present application creatively fuses semantic information and multi-source positioning data to form a dynamically maintained high-precision road model.

[0033] By the above manner, by combining the short-time high-frequency characteristics of the IMU and the odometry information of the wheel speed data, the single sensor error is effectively compensated, the positioning robustness is enhanced through multi-modal data fusion; the camera data is subjected to pixel-level semantic segmentation, not only the key ground semantic information such as lane lines and road boundaries is accurately extracted, but also the dynamic obstacle interference is filtered; the two-dimensional semantic image based on the vehicle pose is projected to a three-dimensional world coordinate system, a lane line point cloud model is constructed, the space-time synchronization of the visual features and the vehicle kinematics is realized, the lane line point cloud generated in real time is matched with the prior point cloud of the high-definition map, the spatial transformation matrix of the overlapping area is calculated, and the incremental updating of the map data is realized. The application breaks through the limitation of traditional high-definition map static updating, constructs a three-dimensional lane line point cloud by fusing vehicle inertial measurement data, wheel speed data and camera semantic information, and dynamically matches the high-definition map, realizes real-time updating of map data by using ordinary vehicle sensors, and improves the timeliness of map updating; compared with the pure visual SLAM scheme, the anti-interference ability is improved; compared with the laser radar mapping scheme, the lane line detection accuracy is maintained, and the cost is reduced.

[0034] The application aims to solve the problem that the traditional high-definition map updating relies on professional acquisition vehicles for regular inspection, and has the problems of long updating cycle and limited coverage.

[0035] In an embodiment, the target lane line point cloud information is matched with the point cloud data on the high-definition map, and the map is updated according to the matching result, including: If the target lane line point cloud information does not exist in the high-definition map, it is determined that there is no prior map in the current road section, and the target lane line point cloud information is generated as a prior map; If the target lane line point cloud information matches the same point cloud data in the high-definition map, the current target lane line point cloud information is discarded; If the target lane line point cloud information matches different point cloud data in the high-definition map, the current matched point cloud data is updated according to the target lane line point cloud information to complete the map updating.

[0036] Illustratively, by querying the high-definition map database, it is confirmed whether there is pre-stored point cloud data in the target area; if there is no matching data, a new map generation process is triggered; the matching degree (such as Hausdorff space distance, overlap rate) of the real-time point cloud and the map point cloud is calculated; if the matching degree is greater than 100% (the threshold value can be adjusted), it is determined that the data is the same, and the redundant point cloud is directly discarded; otherwise, it is determined that the data is different, and the current matched point cloud data is updated according to the target lane line point cloud information to complete the map updating.

[0037] Specifically, when a vehicle travels to an area where no prior map is stored, onboard sensors collect lane line data in real time to generate a 3D point cloud, which is then stored as new map data in a high-precision map database. For road sections with existing map data, if the real-time point cloud perfectly matches the map data, it indicates that the road features have not changed, and in this case, there is no need to repeatedly store the data to save computing resources. When a difference is detected between the real-time point cloud and the map data, coordinate transformation and data fusion are performed on the differing parts to achieve dynamic updates of the map data. For example, after road markings are redrawn, the differences between the old and new lane lines are automatically identified and the corresponding point cloud coordinates are updated to ensure that the map data is consistent with the physical environment in real time.

[0038] Through the above methods, this application enables automated incremental updates of high-precision maps, quickly correcting map data errors when road environments change, and preventing positioning deviations in autonomous vehicles due to outdated map information. For newly constructed roads or temporary detours, prior maps can be automatically generated by the vehicles, significantly improving the coverage and real-time performance of map data.

[0039] This application aims to solve the problem that traditional methods, which typically rely solely on inertial measurement data or wheel speed data for independent calculations, are prone to pose drift due to sensor noise or errors from a single data source.

[0040] Optionally, the calculation based on inertial measurement data and vehicle wheel speed also includes: Pre-integration calculations are performed on the inertial measurement data to determine the pre-integration residual and velocity residual, and the pre-integration residual and velocity residual are used as the pre-integration objects; A factor graph is constructed using the pre-integrated object and pose observations. The factor graph is then iteratively optimized multiple times to output vehicle pose and motion data. Vehicle pose includes the vehicle's coordinates and attitude; motion data includes the vehicle's velocity, acceleration offset, and angular velocity offset. The pre-integrated residual is used to constrain the consistency between the inertial measurement data and the kinematic model, the velocity residual is used to constrain the correlation between the wheel speed data and the inertial measurement data, and the pose observations provide positioning reference information.

[0041] Specifically, by integrating inertial measurement data in the local coordinate system, repeated integration in the global coordinate system is avoided, thus reducing the amount of computation; by using the residual definition principle, the difference between the predicted value (based on the current state estimate) and the measured value (IMU integration result) is taken as the optimization objective to suppress drift error.

[0042] Specifically, when the vehicle is in a static state, the accelerometer measurement value should theoretically only reflect the gravity component, the bias vectors of the acceleration and angular velocity are calculated through static initialization, the rotation attitude increment, displacement increment and velocity increment between adjacent time instants can be calculated by inputting the inertial measurement data and wheel speed data into the pre-integration model; subsequently, the pre-integration residual, velocity residual and vehicle pose observation value are jointly constructed into a factor graph, and the optimal vehicle pose and sensor bias parameters are solved through multiple rounds of iterative optimization. For example, in each iteration, the pre-integration residual is used to constrain the consistency of the inertial measurement data and the kinematic model, the velocity residual is used to constrain the correlation of the wheel speed data and the inertial measurement data, and the pose observation value provides reference information from an external positioning system. In this way, the coordinates, attitude and motion parameters of the vehicle pose are gradually optimized in the iteration process.

[0043] Specifically, the present application jointly models the inertial data and wheel speed data in the time domain through the pre-integration model, and introduces a factor graph optimization framework, which can effectively fuse the spatio-temporal correlation between multiple sources of data; at the same time, the wheel speed model and the inertial measurement data are mathematically associated through the velocity residual term, which enhances the consistency judgment of the vehicle motion state.

[0044] In the above manner, the problem of decline in positioning accuracy caused by accumulation of sensor noise and errors in traditional pose estimation methods is solved, and through the synergistic effect of the pre-integration model and the factor graph optimization, the zero bias error and random noise of the inertial measurement unit can be effectively suppressed, and the estimation accuracy of the vehicle pose is improved; at the same time, the acceleration bias and angular velocity bias parameters can be dynamically adjusted in the multiple rounds of iterative optimization process, so as to adapt to the changes of sensor characteristics with the environment or time, thereby ensuring the stability and reliability of vehicle positioning in the map updating process.

[0045] The present application aims to solve the problem that the existing method does not independently initialize the vehicle in a static state, resulting in the direct introduction of bias noise in the inertial measurement data into the subsequent calculation process.

[0046] In an embodiment, the inertial measurement data is pre-integrated to determine a pre-integration residual, comprising: Based on the vehicle wheel speed, the state of the vehicle is determined, and if the vehicle is in a static state, the inertial measurement data is statically initialized to determine the bias vectors and noise vectors of the acceleration and angular velocity in the static state; The gravity direction is determined according to the bias vector of the acceleration, and the initial attitude information is determined in combination with the gravity direction and the bias vector; Based on the bias vectors and noise vectors of the acceleration and angular velocity, a preset integration model is used to calculate a first angular velocity value and a first acceleration value at the current time instant, and a median integral of the angular velocity and a median integral of the acceleration from the current time instant to the next time instant; Based on the first angular velocity value and the first acceleration value, calculate the position and velocity information at the current moment; and based on the position and velocity information at the current moment, determine the pose four elements at the next moment. Based on the pose four elements, position information and velocity information corresponding to the current moment and the next moment, incremental calculations are performed to determine the rotational pose increment, displacement change and velocity median integral increment. The pre-integral quantities of rotational attitude, displacement, and velocity are determined by calculating the rotational attitude increment, displacement change, and velocity median integral increment respectively. The pre-integral residual is determined by performing residual calculations based on the pre-integral quantities of rotational attitude, displacement, and velocity between the current and next time steps.

[0047] Specifically, when the vehicle is stationary, a static initialization process is triggered by the onboard wheel speed signal. First, multiple sets of inertial measurement data are collected. The gravity direction vector is determined by calculating the average acceleration, and the initial attitude angle of the vehicle is solved by combining the gravity vector and the acceleration bias vector. Next, the inertial measurement data is discretized using a preset integral model. The changes in angular velocity and acceleration at adjacent time points are calculated using the median integration method to generate a four-element description of the rotational state of the pose. By accumulating the rotation increment, displacement increment, and velocity increment within each time interval, a pre-integral expression is constructed. Finally, by comparing the theoretical value of the pre-integral with the actual observed value at adjacent time points, the residual is calculated to evaluate the pose solution error.

[0048] By employing the above method, dynamic and static error sources are separated through static initialization. Wheel speed signals are used as trigger conditions to complete sensor bias calibration in a static state, effectively suppressing noise interference in pose calculation. Furthermore, the use of median integration instead of traditional Euler integration reduces the accumulation of integration errors caused by high-frequency vibrations. This application improves the attitude initialization accuracy of a stationary vehicle and reduces pose calculation errors caused by sensor bias. Simultaneously, the pre-integration model achieves temporal alignment between inertial data and wheel speed data, avoiding matching deviations caused by asynchronous multi-source data.

[0049] This application aims to address the problem that traditional solutions often overlook the impact of sensor installation errors on data fusion, resulting in systematic deviations between the data from the inertial measurement unit and the wheel speed meter.

[0050] In one embodiment, before determining the vehicle's state based on the onboard wheel speed, the method further includes: The coordinate system relationship between the camera and the inertial measurement unit is calibrated, and the rotation matrices and intrinsic parameters of the camera and the inertial measurement unit are determined respectively; as well as the spatial rotation matrix and translation vector between the rotating wheel and the inertial measurement unit are determined.

[0051] Specifically, during the operation of the vehicle, the image data of the camera and the motion data of the inertial measurement unit need to be fused in a unified coordinate system; to achieve high-precision alignment of multi-sensor data, the installation positions of the camera and the inertial measurement unit need to be calibrated in advance to obtain the rotation matrix between the two; at the same time, the internal parameter calibration of the camera can eliminate the influence of lens distortion on semantic segmentation. For the installation deviation of the wheel speed sensor and the inertial measurement unit, by constructing a rigid transformation model from the wheel coordinate system to the inertial measurement unit coordinate system, the rotation matrix and the translation vector are calibrated, so as to compensate for the mechanical installation error in subsequent pose calculation. For example, by using a hand-eye calibration method, the wheel speed sensor readings and inertial measurement unit data of the vehicle at multiple poses are collected, and a least squares optimization equation is established to solve the spatial transformation parameters.

[0052] In the above manner, the coordinate system alignment problem in multi-sensor data fusion is effectively solved, ensuring that the measurement data of the inertial measurement unit, the wheel speed sensor and the camera have a unified spatial reference, not only improving the accuracy of vehicle pose estimation, but also ensuring the quality of lane line point cloud generation, and ultimately improving the reliability and adaptability of the map updating system.

[0053] The present application aims to solve the problem that the traditional method usually directly calculates the vehicle speed using the wheel speed pulse number, without considering the influence of sensor installation position difference and vehicle attitude change on speed observation. For example, under sharp turns or bumpy road conditions, the spatial position difference between the wheel speed sensor and the inertial measurement unit will cause speed measurement error, and the rotation of the coordinate system caused by angular velocity is not effectively compensated.

[0054] In an embodiment, the manner of determining the speed residual comprises: obtaining the inverse matrix of the pose four elements, the inverse rotation matrix of the vehicle-mounted wheel speed to the world coordinate system, the speed at the current time, the skew-symmetric matrix of the angular velocity at the current time, the translation extrinsic parameter of the vehicle-mounted wheel speed to the world coordinate system, and the vehicle speed determined by the wheel speed model; determining a first product according to the multiplication operation of the speed at the current time and the inverse rotation matrix; determining a second product according to the multiplication operation of the translation extrinsic parameter and the skew-symmetric matrix; determining a third product by multiplying the sum of the second product and the first product with the inverse matrix; determining the speed residual by performing difference operation on the third product and the vehicle speed.

[0055] Specifically, the calculation process of the speed residual can be decomposed into multiple matrix operation steps. First, the speed vector at the current time is multiplied by the inverse rotation matrix to obtain the speed component in the inertial measurement unit coordinate system. Then, the translational extrinsic parameter and the angular velocity skew-symmetric matrix are multiplied to obtain the speed offset caused by angular motion. After adding the above two results, the inverse matrix of the pose four elements is multiplied to finally obtain the theoretical speed value predicted by the wheel speed model. The difference between the theoretical speed and the actual vehicle speed output by the wheel speed model is the speed residual. This residual is used in subsequent factor graph optimization to adjust the vehicle pose estimation parameters by minimizing the residual, such as optimizing the bias error of the inertial measurement unit or correcting the calibration parameters of the wheel speed sensor, thereby improving the accuracy of vehicle motion state calculation.

[0056] In the above manner, by introducing the inverse rotation matrix, the translational extrinsic parameter, and the skew-symmetric matrix, a more accurate speed residual model is constructed, which eliminates the influence of sensor installation errors and dynamic attitude changes on speed observation at the mathematical level; effectively improves the accuracy of vehicle pose estimation, especially in complex motion scenarios, it can more reliably fuse multi-sensor data. For example, when the vehicle continuously changes lanes or there is a pitch and yaw angular velocity, by accurately compensating the coupling effect of angular motion on speed observation, cumulative errors in the pose calculation process can be avoided, thereby ensuring the accuracy of projecting ground semantic information to the world coordinate system, making the matching result of lane line point cloud and high-precision map more reliable, and finally supporting real-time updating and dynamic maintenance of map data.

[0057] In an embodiment of the present application, based on the vehicle pose, the ground semantic information on the road is projected onto the three-dimensional space of the world coordinate system to determine the target lane line point cloud information, including: According to the preset camera extrinsic parameter, the two-dimensional point cloud data in each frame of ground semantic information is converted into initial three-dimensional point cloud data in the camera coordinate system; according to the feature extraction from the initial three-dimensional point cloud data in each frame of point cloud, the lane line point cloud is determined; through the vehicle pose data in the world coordinate system and the linear interpolation time synchronization mechanism, the vehicle pose data corresponding to each frame of lane line point cloud is obtained; through the rigid transformation principle, the lane line point cloud is converted from the vehicle body coordinate system to the world coordinate system to obtain the target lane line point cloud information in the world coordinate system.

[0058] Specifically, the two-dimensional point cloud is converted into a three-dimensional point cloud in the camera coordinate system by presetting the camera external parameters, and the three-dimensional point cloud corresponding to the lane line is further screened in combination with the semantic segmentation result. For each frame of lane line point cloud, the linear interpolation is performed on the time stamp of the vehicle pose data and the point cloud collection time to obtain the accurate pose parameters corresponding to the frame of point cloud. Subsequently, the point cloud is converted from the vehicle body coordinate system to the world coordinate system based on the rigid transformation matrix composed of the rotation matrix and the translation vector, and finally the target lane line point cloud with global position information is generated. This process eliminates the time difference of sensor data through the time synchronization mechanism, and at the same time, the geometric accuracy of coordinate conversion is ensured by using the rigid transformation.

[0059] In the above manner, by means of semantic segmentation and multi-sensor data fusion, low-cost and high-robust three-dimensional point cloud generation is realized by using vehicle pose and time synchronization mechanism while reducing hardware dependence, and further by rigid transformation, the global consistency of the point cloud in the world coordinate system is ensured, thereby providing a reliable data source for map updating.

[0060] Even without high-cost laser radar equipment, lane line point cloud information in the world coordinate system can be accurately generated; by combining the time synchronization mechanism and the rigid transformation, the asynchronous problem of sensor data is effectively solved and the coordinate conversion accuracy is improved, so that the generated point cloud data can be directly used for matching and updating of high-precision maps, especially suitable for scenes with frequent changes of urban roads or lack of satellite signals.

[0061] Please refer to Figure 4 An incremental flowchart of a map updating device provided by the embodiment of the present application is described as follows: (1) Calibration By calibrating the coordinate system relationship of the inertial unit (IMU) and the camera (Camera), the rotation matrix of the Camera coordinate system to the IMU coordinate system is obtained and and the camera intrinsic parameter K is obtained; at the same time, the spatial rotation matrix and the translation vector between the IMU and the wheel speed are measured. A) Synchronous trigger, hardware synchronization (such as trigger signal) ensures the time synchronization of the camera and the IMU.

[0062] B) Motion incentive, sufficient motion (rotation and translation) is needed in the calibration process to excite all degrees of freedom, but avoid motion blur caused by violent motion.

[0063] (2) Static initialization of odometry According to the wheel speed detection vehicle state whether to be in static, if in static, use the inertial navigation static initialization, collect multiple frames of IMU data, average the acceleration and angular velocity in each frame of data to obtain the mean value, and then calculate the variance of the acceleration and angular velocity of multiple frames of data by using the mean value. According to the error model of IMU, the mean value and variance of acceleration and angular velocity in the static state are equal to the bias of acceleration and angular velocity , and noise and ; at the same time, since the IMU is in a static state, it is only affected by gravity, so the formula 1 can be used to calculate the gravity direction of the IMU, and then the formula 2, formula 3, formula 4 are used to calculate the initialization attitude R of the IMU (which can be converted into four elements q by using Eigen (i.e. C++ linear algebra library)).

[0064] g Formula 1 Where, G=9.81,| | is the modulus of vector Formula 2 Formula 3 Formula 4 Where, g is the gravity of the IMU, I is the unit matrix, k is the rotation axis of the vector g and b, is the rotation angle between the vector g and b, b is the vector (0, 0, 1), and R is the rotation matrix.

[0065] The IMU integral model of formula 5, formula 6, formula 7, formula 8, formula 9, formula 10, formula 11 is used to calculate the vehicle pose and speed, and the wheel speed model in formula 12 is used to calculate the vehicle speed.

[0066] Formula 5 Formula 6 Formula 7 Formula 8 Formula 9 Formula 10 Formula 11 Formula 12 Where, in formula 5, formula 6, formula 7, formula 8, is the measured value of the angular velocity of the IMU at time k, is the measured value of the acceleration of the IMU at time k;​ is the median value integration result of the angular velocity of the IMU from time k to time k+1, is the median value integration result of the acceleration of the IMU from time k to time k+1.

[0067] wherein in formula 9, formula 10, formula 11 and formula 12, is the position at time k, is the velocity at time k+1, is the pose four-element at time k+1, and a is the median value integration of the acceleration from time k to time k+1, 、 all represent the time interval between two consecutive IMU measurements, is the vehicle speed calculated by the wheel speed, is the right wheel speed of the vehicle, is the left wheel speed of the vehicle.

[0068] (3) Mileage factor graph optimization calculation The pre-integration residual and velocity residual of formula 13, formula 14, formula 15, formula 16, formula 17, formula 18, formula 19 and formula 20 are constructed, and the optimal pose and velocity are calculated by using the Levenberg Marquard (i.e. LM) optimization algorithm, wherein the LM algorithm is an optimization algorithm for solving nonlinear least squares problems, and is widely used in parameter estimation, curve fitting, machine learning and other fields.

[0069] Formula 13 Formula 14 Formula 15 Formula 16 Formula 17 Formula 18 Formula 19 Formula 20 wherein, is a three-dimensional vector of [x, y, z] composed of the imaginary part of the four-element, and in formula 13, formula 14 and formula 15 、 、 is the median value integration of the rotation attitude increment, displacement increment and velocity, so as to obtain the pre-integration quantity from time i to time j 、 、 .

[0070] The displacement, attitude, velocity change, acceleration bias and angular velocity bias at i to j moments in formula 16, formula 17, formula 18, formula 19 and formula 20 are respectively pre-integrated residuals. 、 、 、 、

[0071] Velocity residual Formula 21 In formula 21, is the rotation matrix of wheel speed to IMU coordinate system The inverse of the corresponding four elements, is the inverse of the IMU attitude four elements at i moment, is the IMU velocity at i moment, is the skew-symmetric matrix of IMU angular velocity at i moment, is the translation parameter of wheel speed to IMU coordinate system, is the velocity residual at i moment.

[0072] Finally, the results calculated after the odometer factor graph optimization include the coordinates P, attitude Q, velocity V and imu accelerometer bias ba and gyroscope bias bg of the vehicle.

[0073] (4) Semantic segmentation Model selection is used to select the open source yolov8 semantic segmentation model Load model is used to load the trained YoLov8-seg model.

[0074] Input image is used to input the image into the model, and multi-scale features are extracted through Backbone (main network, CSPDarknet) and Neck (PAN-FPN).

[0075] Dual-branch output is used to detect the target box and category predicted by the detection head, and the segmentation head generates prototype maps and mask coefficients.

[0076] Mask calculation is used to multiply the prototype map with the mask coefficient, and the sigmoid activation is used to obtain the pixel-wise binary mask.

[0077] Post-processing is used to merge the binary mask to obtain a semantic image (5) Lane line three-dimensional reconstruction The relative pose relationship between the pose, camera and IMU calculated in the third step is used to project the road semantic information calculated in the fourth step to three-dimensional space to obtain lane line semantic point cloud data, and the specific method is as follows. ​

[0078] Projection formula: Formula 22 Formula 23 In formulas 22 and 23, denotes the inverse matrix of the camera intrinsic, denotes the pixel coordinates of the ground elements in the camera, and h denotes the camera installation height, denotes the depth of the pixel in the camera coordinate system, denotes the pose (rotation matrix) of the IMU at time i, denotes the position of the IMU at time i, denotes the position of the pixel projected into the local coordinate system, thereby obtaining the 3D semantic information of the road.

[0079] (6) Semantic ICP The reconstructed 3D road semantic information and the road information of the prior map are matched using semantic ICP. Whether the matching is successful is determined according to a threshold. If the matching fails, it is considered that the map information has changed and needs to be updated. If the matching is successful, the reconstructed 3D lane line information is discarded.

[0080] Objective function of semantic ICP: Minimize the semantic-geometric joint error: Formula 24 is the rotation matrix and translation vector to be solved; is a point in the source point cloud, and the semantic category is ; is the nearest neighbor point in the target point cloud corresponding to , and the semantic category is ; is a geometric weight (such as a distance weight or a covariance weight); is a weight coefficient of the semantic error term.

[0081] Semantic nearest neighbor search: When matching corresponding points, the same semantic points are preferentially selected; Formula 25 is a candidate point set of the target point cloud.

[0082] is an indicator function (penalizing when semantic matching fails).

[0083] Penalty coefficient for semantic mismatch (usually set to a large value, such as 10^6).

[0084] Iterative optimization, solve optimal R, t by SVD or Gauss-Newton iteration.

[0085] (7) Incremental update If the similarity of semantic ICP matching is lower than the threshold, it is considered that the regional map region has changed and needs to be updated, and then the incremental update of the map is realized according to the odometer using the reconstructed 3D road semantic information, and the invalid data of the original map is deleted.

[0086] In the above manner, the application only updates the changed part in the environment (such as a moving object and a newly added obstacle), avoids the calculation overhead of global reconstruction, can quickly realize map updating, and is relatively robust and stable as a whole without the influence of surrounding buildings.

[0087] In an embodiment, a map updating apparatus is provided for performing the map updating method provided by any of the above embodiments. Please refer to Figure 3 , Figure 3 A structural schematic diagram of the map updating apparatus provided by the embodiment of the application is shown in Figure 3 The map updating apparatus comprises a data acquisition module 301, a pose determination module 302, a semantic segmentation module 303, a lane line determination module 304, and a map updating module 305, wherein: The data acquisition module 301 is configured to acquire inertial measurement data, vehicle-mounted wheel speed, and camera data on the vehicle. The pose determination module 302 is configured to determine the vehicle pose by calculating based on the inertial measurement data and the vehicle-mounted wheel speed. The semantic segmentation module 303 is configured to perform semantic segmentation on the camera data to determine a semantic image, wherein the semantic image comprises ground semantic information. The lane line determination module 304 is configured to project the ground semantic information on the road to a three-dimensional space on a world coordinate system based on the vehicle pose to determine target lane line point cloud information. The map updating module 305 is configured to match the target lane line point cloud information with point cloud data on a high-precision map, and update the map according to the matching result, wherein the point cloud data on the high-precision map is determined by a priori map.

[0088] Specific limitations regarding the map update device can be found in the limitations of the map update method described above, and will not be repeated here. Each module in the aforementioned map update device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in the electronic device, or stored in software in the memory of the electronic device, so that the processor can call and execute the corresponding operations of each module.

[0089] In this embodiment, the map update device is actually set up with multiple modules to execute the map update method in any of the above embodiments. The specific functions and technical effects can be referred to the above embodiments, and will not be repeated here.

[0090] In one embodiment, a vehicle is provided that includes the map updating device provided in any of the above embodiments.

[0091] Specific limitations regarding vehicles can be found in the map update method section above, and will not be repeated here. The various modules within the vehicle can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in hardware or independently of the processor in the electronic device, or stored in software within the memory of the electronic device, so that the processor can call and execute the corresponding operations of each module.

[0092] In one embodiment, an electronic device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 5 As shown, the electronic device includes a processor, memory, network interface, and database connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile and / or volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and database. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface is used to communicate with external clients via a network connection. The computer program is executed by the processor to implement the functions or steps of the server-side method described above.

[0093] In one embodiment, an electronic device is provided, which may be a client, and its internal structure diagram may be as follows: Figure 6As shown. The electronic device includes a processor, a memory, a network interface, a display screen and an input device connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium, an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The network interface of the electronic device is used to communicate with the external server through the network connection. The computer program is executed by the processor to realize the functions or steps of the client side of the above method.

[0094] In one embodiment, an electronic device is provided, comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor executing the computer program to implement the following steps: Obtaining inertial measurement data, vehicle-mounted wheel speed and camera data on the vehicle; determining the vehicle pose by calculating the inertial measurement data and the vehicle-mounted wheel speed; performing semantic segmentation on the camera data to determine a semantic image, the semantic image comprising ground semantic information; projecting the ground semantic information on the road to a three-dimensional space on a world coordinate system based on the vehicle pose to determine target lane line point cloud information; and matching the target lane line point cloud information with point cloud data on a high-definition map, and updating the map according to the matching result, wherein the point cloud data on the high-definition map is determined from a prior map.

[0095] In one embodiment, a computer readable storage medium is provided, having a computer program stored thereon, the computer program being executed by a processor to implement the following steps: Obtaining inertial measurement data, vehicle-mounted wheel speed and camera data on the vehicle; determining the vehicle pose by calculating the inertial measurement data and the vehicle-mounted wheel speed; performing semantic segmentation on the camera data to determine a semantic image, the semantic image comprising ground semantic information; projecting the ground semantic information on the road to a three-dimensional space on a world coordinate system based on the vehicle pose to determine target lane line point cloud information; and matching the target lane line point cloud information with point cloud data on a high-definition map, and updating the map according to the matching result, wherein the point cloud data on the high-definition map is determined from a prior map.

[0096] It should be noted that the functions or steps that the above computer readable storage medium or electronic device can implement can be referred to the related descriptions of the server side and the client side in the foregoing method embodiments. To avoid repetition, they will not be described one by one here.

[0097] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program. The above-mentioned computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, the processes of the above-mentioned embodiments of each method can be included. Any reference to memory, storage, database or other medium used in each embodiment provided by the present application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration but not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0098] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the above-mentioned division of each functional unit and module is exemplified. In actual application, the above-mentioned functions can be completed by different functional units and modules according to needs, that is, the internal structure of the above-mentioned device and system is divided into different functional units or modules to complete all or part of the above-mentioned functions.

[0099] The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, but not to limit them. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part of the technical features. The modification or replacement does not make the essence of the corresponding technical solution deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application.

Claims

1. A map updating method, characterized in that, The method includes: Acquire inertial measurement data, wheel speed, and camera data from the vehicle; The vehicle's position and orientation are determined by calculating the inertial measurement data and the vehicle's wheel speed. The camera data is semantically segmented to determine a semantic image, which includes ground semantic information; Based on the vehicle pose, the ground semantic information on the road is projected onto a three-dimensional space in the world coordinate system to determine the target lane line point cloud information. The target lane line point cloud information is matched with the point cloud data on the high-precision map, and the map is updated according to the matching result. The point cloud data on the high-precision map is determined by the prior map.

2. The map updating method as described in claim 1, characterized in that, Matching the target lane line point cloud information with point cloud data on a high-precision map, and updating the map based on the matching results, includes: If the target lane line point cloud information does not exist in the high-precision map, it is determined that there is no prior map for the current road segment, and the target lane line point cloud information is used to generate a prior map. If the target lane line point cloud information matches the point cloud data in the high-precision map, then discard the current target lane line point cloud information. If the target lane line point cloud information does not match the point cloud data in the high-precision map, the currently matched point cloud data is updated based on the target lane line point cloud information to complete the map update.

3. The map updating method as described in claim 1, characterized in that, The calculation based on the inertial measurement data and the vehicle wheel speed also includes: The inertial measurement data is pre-integrated to determine the pre-integration residual and the velocity residual, and the pre-integration residual and the velocity residual are used as the pre-integration objects; A factor graph is constructed using the pre-integrated object and pose observations. The factor graph is iteratively optimized multiple times to output the vehicle pose and motion data. The vehicle pose includes the vehicle's coordinates and attitude. The motion data includes the vehicle's velocity, acceleration bias, and angular velocity bias. Among them, the pre-integral residual is used to constrain the consistency between inertial measurement data and kinematic model, the velocity residual is used to constrain the correlation between wheel speed data and inertial measurement data, and the pose observation value provides positioning reference information.

4. The map updating method as described in claim 3, characterized in that, Pre-integration calculation is performed on the inertial measurement data to determine the pre-integration residual, including: The state of the vehicle is determined based on the on-board wheel speed. If the vehicle is stationary, the inertial measurement data is statically initialized to determine the bias vector and noise vector of the acceleration and angular velocity in the stationary state. The direction of gravity is determined based on the offset vector of the acceleration, and the initial attitude information is determined by combining the direction of gravity and the offset vector. Based on the bias vectors of the acceleration and the angular velocity and the noise vector, the first angular velocity value and the first acceleration value at the current moment are calculated using a preset integral model, as well as the median integral of the angular velocity and the median integral of the acceleration from the current moment to the next moment; Based on the first angular velocity value and the first acceleration value, calculate the position information and velocity information at the current moment; and based on the position information and velocity information at the current moment, determine the pose four elements at the next moment. Based on the pose four elements, position information and velocity information corresponding to the current time and the next time, incremental calculations are performed to determine the rotation pose increment, displacement change and velocity median integral increment; Based on the rotational pose increment, the displacement change, and the velocity median integral increment, the rotational attitude pre-integral, displacement pre-integral, and velocity pre-integral are calculated and determined respectively. The pre-integral residual is determined by performing residual calculations based on the pre-integral quantities of rotational attitude, displacement, and velocity between the current and next time steps.

5. The map updating method as described in claim 4, characterized in that, Before determining the vehicle's state based on the onboard wheel speed, the method further includes: The coordinate system relationship between the camera and the inertial measurement unit is calibrated, and the rotation matrices and intrinsic parameters of the camera and the inertial measurement unit are determined respectively; and the spatial rotation matrix and translation vector between the rotating wheel and the inertial measurement unit are determined.

6. The map updating method as described in claim 3, characterized in that, The methods for determining the velocity residual include: Obtain the inverse matrix of the pose four elements, the inverse rotation matrix from the vehicle wheel speed to the world coordinate system, the current speed, the antisymmetric matrix of the current angular velocity, the translation extrinsic parameters from the vehicle wheel speed to the world coordinate system, and the vehicle speed determined by the wheel speed model. The first product is determined by multiplying the current velocity with the inverse rotation matrix. The second product is determined based on the multiplication operation of the antisymmetric matrix involving the translation; The third product is determined by summing the second product and the first product, and then multiplying it by the inverse matrix. The speed residual is determined by performing a difference calculation between the third product and the vehicle speed.

7. The map updating method as described in any one of claims 1 to 6, characterized in that, Based on the vehicle pose, the ground semantic information on the road is projected onto a three-dimensional space in the world coordinate system to determine the target lane line point cloud information, including: Based on preset camera extrinsic parameters, the two-dimensional point cloud data in the ground semantic information of each frame is converted into initial three-dimensional point cloud data in camera coordinates; Based on the ground semantic information, features are extracted from each frame of the initial 3D point cloud data to determine the lane line point cloud; The vehicle pose data corresponding to each frame of lane line point cloud is obtained by using vehicle pose data in the world coordinate system and a linear interpolation time synchronization mechanism. By using the rigid body transformation principle, the lane line point cloud is transformed from the vehicle coordinate system to the world coordinate system, thus obtaining the target lane line point cloud information in the world coordinate system.

8. A map updating device, characterized in that, include: The data acquisition module is used to acquire inertial measurement data, onboard wheel speed, and camera data from the vehicle. The pose determination module is used to calculate and determine the vehicle pose based on the inertial measurement data and the vehicle wheel speed. A semantic segmentation module is used to perform semantic segmentation on the camera data to determine a semantic image, the semantic image including ground semantic information; The lane line determination module projects the ground semantic information on the road onto a three-dimensional space in the world coordinate system based on the vehicle pose to determine the target lane line point cloud information. The map update module is used to match the target lane line point cloud information with the point cloud data on the high-precision map, and update the map according to the matching result. The point cloud data on the high-precision map is determined by the prior map.

9. A vehicle, characterized in that, The vehicle is described using the method described in any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 7.

Images