Construction method and device of high-precision map, storage medium and electronic equipment

Abstract

The invention discloses a high-precision map construction method and device, a storage medium and electronic equipment. The method comprises the following steps: determining a plurality of driving data sets acquired in a vehicle driving process; determining a single sparse point cloud model corresponding to each driving data set and an aggregation sequence of each driving data set; aggregating the single sparse point cloud models corresponding to the driving data sets based on the aggregation sequence to obtain a global sparse point cloud model; and generating a high-precision map based on the global sparse point cloud model. According to the method, the single sparse point cloud models corresponding to the driving data sets are determined, and the single sparse point cloud models are aggregated according to the aggregation sequence corresponding to the driving data sets, so that the map construction precision can be improved, three-dimensional information of all pixels of the image does not need to be calculated, the calculation data volume can be reduced, the time consumption is shortened, and the map construction efficiency is improved. Therefore, the high-precision map can be quickly constructed.

Description

Technical Field

[0001] The present disclosure relates to the field of three-dimensional reconstruction technology, and in particular to a method, device, storage medium and electronic device for constructing a high-precision map. Background Art

[0002] With the development of science and technology, 3D maps are increasingly used in various technical fields (such as navigation, urban planning, game design, intelligent driving, etc.). At present, the construction of 3D maps mainly has the defects of high cost, low efficiency, large amount of calculation data, and long time consumption, which makes it difficult to be widely popularized. Summary of the invention

[0003] The existing technology for building high-precision maps has the disadvantages of high cost, low efficiency, large amount of calculation data, and long processing time.

[0004] In order to solve the above technical problems, the present disclosure provides a method, device, storage medium and electronic device for constructing a high-precision map, which can achieve the effect of quickly constructing a high-precision map.

[0005] The first embodiment of the present disclosure provides a method for constructing a high-precision map, including:

[0006] Determine a plurality of driving data sets collected during the driving process of the vehicle;

[0007] Determining a single sparse point cloud model corresponding to each of the driving data sets and an aggregation order of each of the driving data sets;

[0008] Aggregating the single sparse point cloud models corresponding to each of the driving data sets based on the aggregation order to obtain a global sparse point cloud model;

[0009] The high-precision map is generated based on the global sparse point cloud model.

[0010] A second aspect of the present disclosure provides a device for constructing a high-precision map, including:

[0011] A first determination module is used to determine a plurality of driving data sets collected during the driving process of the vehicle;

[0012] A second determination module is used to determine a single sparse point cloud model corresponding to each of the driving data sets and an aggregation order of each of the driving data sets;

[0013] An aggregation module, used for aggregating the single sparse point cloud models corresponding to each of the driving data sets based on the aggregation order to obtain a global sparse point cloud model;

[0014] A generation module is used to generate the high-precision map based on the global sparse point cloud model.

[0015] An embodiment of the third aspect of the present disclosure provides a computer-readable storage medium, wherein the storage medium stores a computer program, and the computer program is used to execute the method for constructing a high-precision map proposed in the embodiment of the first aspect of the present disclosure.

[0016] In a fourth aspect of the present disclosure, an electronic device is provided, the electronic device comprising:

[0017] processor;

[0018] a memory for storing instructions executable by the processor;

[0019] The processor is used to read the instructions from the memory and execute the instructions to implement the method for constructing a high-precision map proposed in the embodiment of the first aspect of the present disclosure.

[0020] The fifth aspect embodiment of the present disclosure provides a computer program product, which, when an instruction processor in the computer program product executes, executes the method for constructing a high-precision map proposed in the first aspect embodiment of the present disclosure.

[0021] The method for constructing a high-precision map provided by the embodiment of the present disclosure first determines multiple driving data sets collected during the driving process of the vehicle; determines the single sparse point cloud model corresponding to each driving data set and the aggregation order of each driving data set; aggregates the single sparse point cloud model corresponding to each driving data set based on the aggregation order to obtain a global sparse point cloud model; and generates a high-precision map based on the global sparse point cloud model. By determining the single sparse point cloud model corresponding to each of the multiple driving data sets, and aggregating the single sparse point cloud models according to the aggregation order corresponding to each driving data set, the accuracy of map construction can be improved, and there is no need to calculate the three-dimensional information of all pixels of the image, which can reduce the amount of data calculation and shorten the time consumption, thereby realizing the rapid construction of a high-precision map. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a schematic diagram of the network architecture of a high-precision map construction system provided by an exemplary embodiment of the present disclosure;

[0023] Figure 2 It is a flowchart of a method for constructing a high-precision map provided by an exemplary embodiment of the present disclosure;

[0024] Figure 3 is a flowchart of a method for constructing a high-precision map provided by another exemplary embodiment of the present disclosure;

[0025] Figure 4 is a flowchart of a method for constructing a high-precision map provided by another exemplary embodiment of the present disclosure;

[0026] Figure 5is a flowchart of a method for constructing a high-precision map provided by another exemplary embodiment of the present disclosure;

[0027] Figure 6 is a flowchart of a method for constructing a high-precision map provided by another exemplary embodiment of the present disclosure;

[0028] Figure 7 is a flowchart of a method for constructing a high-precision map provided by another exemplary embodiment of the present disclosure;

[0029] Figure 8 is a flowchart of a method for constructing a high-precision map provided by another exemplary embodiment of the present disclosure;

[0030] Fig. 9 is a flowchart of a method for constructing a high-precision map provided by another exemplary embodiment of the present disclosure;

[0031] Fig.10 is a flowchart of a method for constructing a high-precision map provided by another exemplary embodiment of the present disclosure;

[0032] Fig.11 It is a flowchart of a step of determining a sparse point cloud of each driving data set in a method for constructing a high-precision map provided by an exemplary embodiment of the present disclosure;

[0033] Fig.12 It is a schematic diagram of the composition structure of a device for constructing a high-precision map provided by an exemplary embodiment of the present disclosure;

[0034] Fig.13 is a schematic diagram of the composition structure of a device for constructing a high-precision map provided by another exemplary embodiment of the present disclosure;

[0035] Fig.14 It is a schematic diagram of the composition structure of an electronic device provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0036] To explain the present disclosure, example embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. It should be understood that the present disclosure is not limited to the example embodiments.

[0037] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

[0038] Application Overview

[0039] 3D Reconstruction refers to the establishment of a mathematical model suitable for computer representation and processing of three-dimensional objects. It is the basis for processing, operating and analyzing its properties in a computer environment. It is also a key technology for establishing virtual reality in computers to express the objective world.

[0040] In the field of intelligent driving, scene perception is the prerequisite for the safe and stable driving of intelligent driving vehicles (referred to as vehicles). Through 3D reconstruction technology, the environment around the vehicle is digitally modeled to provide more accurate scene information to the intelligent driving system, so as to better perform safe driving. Compared with traditional sensor technology, 3D reconstruction technology can provide more accurate environmental information, help improve the decision-making ability of the intelligent driving system, and provide support for the safe and stable driving of intelligent driving vehicles.

[0041] In related technologies, 3D reconstruction is mainly achieved through feature point matching and Structure from Motion (SFM) algorithm. The algorithm detects and matches feature points by inputting multi-view images, and then uses the SFM algorithm to restore the camera parameters and the relative position relationship between cameras, thereby constructing a sparse point cloud of the scene. Since the points used by SFM are provided by feature matching, the matching points are sparse. The map constructed based on this has the defect of low precision, which affects the safe and stable driving of the vehicle.

[0042] The dense reconstruction method is to calculate the 3D information of most or all pixels of the image through algorithms such as stereo matching, and perform 3D reconstruction. Compared with sparse reconstruction, dense reconstruction obtains more comprehensive 3D information of pixels, and building a map based on this can improve the accuracy of map construction. However, since it requires calculating massive amounts of data and takes a long time, it is impossible to build a high-precision map in real time.

[0043] In order to solve the defects of low accuracy, large amount of calculation and long time consumption in the construction of maps in the related technologies, a method for constructing a high-precision map is provided in the embodiment of the present disclosure. During the driving process, the vehicle collects the driving data of the vehicle and the driving environment data of the surrounding environment of the driving section through the sensors on the vehicle, and determines multiple driving data sets. The single sparse point cloud model and aggregation order corresponding to each driving data set are determined according to the driving data set collected at each moment, and then the single sparse point cloud model corresponding to each driving data set is aggregated based on the aggregation order to obtain a global sparse point cloud model, and finally a high-precision map is generated based on the global sparse point cloud model. This method can improve the accuracy of map construction by aggregating multiple single sparse point cloud models, without calculating the three-dimensional information of all pixels of the image. The sparse point cloud model can reduce the amount of calculated data and shorten the time consumption, thereby achieving the purpose of quickly constructing a high-precision map.

[0044] Exemplary Systems

[0045] Figure 1 It is a schematic diagram of the network architecture of a system for constructing a high-precision map provided by an exemplary embodiment of the present disclosure.

[0046] In some embodiments, Figure 1 As shown in (a), the high-precision map construction system may include a terminal device 100, a network 200, and a server device 300. The terminal device 100 is connected to the server device 300 via the network 200. The network 200 may be a wide area network or a local area network, or a combination of the two, and a wired or wireless link is used to achieve data transmission. The terminal device 100 and the server device 300 may be directly or indirectly connected via wired or wireless communication, which is not limited in the embodiments of the present disclosure.

[0047] Exemplarily, the terminal device 100 in the high-precision map construction system includes a collection device and a transceiver. Specifically, the terminal device is a device set on a vehicle that can interact with the server device 300, or the terminal device 100 can be a vehicle including an intelligent driving function. Here, the terminal device 100 is taken as an example of a vehicle. The collection device on the vehicle can include multiple on-board cameras, radar sensors, laser sensors, wheel speed sensors, inertial measurement unit (Inertial Measurement Unit, IMU) sensors and other sensors, and the transceiver can include a receiving module and a sending module. Among them, the receiving module is used to receive global positioning system (Global Positioning System, GPS) data, and the sending module is used to send driving data sets.

[0048] During the driving process of the vehicle, the vehicle driving data and driving environment data are collected by the collection device. Among them, the vehicle driving data may include the wheel speed meter data collected by the wheel speed meter sensor, the inertial unit data collected by the IMU sensor and the global positioning system data received by the receiving module. The driving environment data may include the driving environment image data collected by multiple vehicle-mounted cameras during driving. The terminal device 100 can package the collected vehicle driving data and driving environment data into multiple driving data sets according to preset rules (such as according to the collection time) and transmit them to the server device 300.

[0049] After the server device 300 receives multiple driving data sets sent by the terminal device 100 (such as a vehicle), it determines the single sparse point cloud model and aggregation order corresponding to each driving data set; then, based on the aggregation order, the single sparse point cloud models corresponding to each driving data set are aggregated to obtain a global sparse point cloud model; finally, a high-precision map is generated based on the global sparse point cloud model.

[0050] For example, after the server device 300 generates a high-precision map, it can transmit the generated high-precision map to the terminal device 100 (such as a vehicle), so that the terminal device 100 controls the vehicle to perform intelligent driving based on the high-precision map. The server device 300 can also transmit the high-precision map to other vehicles ( Figure 1 (a) (not shown) to enable other vehicles to perform intelligent driving based on high-precision map control.

[0051] In other embodiments, Figure 1 As shown in (b) of FIG. 1 , the high-precision map construction system may include a terminal device 100, the above Figure 1 The step of constructing a high-precision map as shown in (a) of FIG. 1 and executed by the server device 300 may also be performed by Figure 1 The terminal device 100 shown in (b) in FIG. 1 may also be executed by other terminal devices on other vehicles ( Figure 1 (b) is not shown in the figure and is not limited in the embodiments of the present disclosure.

[0052] For example, Figure 1 As shown in (b), the terminal device 100 may include a collection device 101, a transceiver 102 and a high-precision map construction device 103, and the terminal device 100 may be a device set on a vehicle. In another exemplary embodiment, the terminal device 100 may be a vehicle including an intelligent driving function. Here, the terminal device 100 is taken as a vehicle as an example for explanation. The collection device 101 in the vehicle may include multiple on-board cameras, radar sensors, laser sensors, wheel speed sensors, IMU sensors and other sensors. The transceiver 102 in the terminal device 100 may include a receiving module, which is used to receive global positioning system data. The high-precision map construction device 103 in the terminal device 100 can obtain the data collected by the collection device 101 and the data received by the transceiver 102, determine multiple driving data sets based on the acquired data, and then determine the single sparse point cloud model and aggregation order corresponding to each driving data set; then aggregate the single sparse point cloud models corresponding to each driving data set based on the aggregation order to obtain a global sparse point cloud model; finally, generate a high-precision map based on the global sparse point cloud model.

[0053] For example, after the terminal device 100 (such as a vehicle) generates a high-precision map, the generated high-precision map can be transmitted to other vehicles, so that other vehicles can perform intelligent driving based on the high-precision map. Alternatively, after the terminal device 100 (such as a vehicle) generates a high-precision map, the generated high-precision map can be transmitted to a server, so that the server sends the high-precision map to other vehicles, thereby enabling other vehicles to perform intelligent driving based on the high-precision map.

[0054] In some other embodiments, the high-precision map construction system may include a high-precision map construction device, which may be a device set in a terminal device that can be used to construct a high-precision map, and specifically may be hardware such as a chip (such as a system on a chip SoC) or an on-board computing device. The high-precision map construction device obtains multiple driving data sets, each driving data set includes data collected during the driving process of the vehicle; then determines the single sparse point cloud model and aggregation order corresponding to each driving data set; then aggregates the single sparse point cloud models corresponding to each driving data set based on the aggregation order to obtain a global sparse point cloud model; and finally generates a high-precision map based on the global sparse point cloud model.

[0055] In the disclosed embodiment, by determining a single sparse point cloud model corresponding to each of the multiple driving data sets and aggregating each single sparse point cloud model according to the aggregation order corresponding to each driving data set, the accuracy of map construction can be improved, and there is no need to calculate the three-dimensional information of all pixels of the image, which can reduce the amount of calculated data and shorten the time, thereby realizing the rapid construction of a high-precision map.

[0056] Exemplary Methods

[0057] Figure 2 This is a flowchart of a method for constructing a high-precision map provided by an exemplary embodiment of the present disclosure. Figure 1 The server device shown in (a) can also be applied to Figure 1 The terminal device shown in (b) in FIG. 1 can also be applied to a device for constructing a high-precision map. Here, the present disclosure is described by taking the application to a device for constructing a high-precision map as an example. Figure 2 As shown, the method for constructing the high-precision map includes the following steps:

[0058] Step S201, determining a plurality of driving data sets collected during the driving process of the vehicle.

[0059] The present disclosure can be implemented by Figure 1 The server device shown in (a) can also be executed by Figure 1 The method can be executed by the terminal device shown in (b) in the figure, or by a high-precision map construction device. The method provided by the embodiment of the present disclosure is exemplarily described below by taking a high-precision map construction device as an example.

[0060] The high-precision map construction device determines multiple driving data sets collected during the driving process of the vehicle, and the driving data sets may include multiple driving data sets collected at different times during the driving process of the vehicle. Among them, one driving data set may include the vehicle driving data and driving environment data collected by the vehicle at the same time.

[0061] The vehicle driving data in the driving data set acquired by the high-precision map construction device may include wheel speed meter data (or wheel data) collected by the wheel speed meter sensor, inertial unit data (or IMU data) collected by the inertial measurement unit IMU sensor, and global positioning system data (or GPS data) received by the receiving module. The IMU sensor may be composed of an acceleration sensor and a gyroscope sensor. The acceleration sensor may be used to detect the acceleration signal of the vehicle, and the gyroscope sensor may be used to detect the angular velocity signal of the vehicle. The vehicle posture is determined based on the acceleration signal and the angular velocity signal. Based on the vehicle posture and the relative position of each on-board camera on the vehicle, the position and posture of each on-board camera can be determined.

[0062] The driving environment data in the driving data set acquired by the high-precision map construction device may include driving environment images captured by multiple on-board cameras on the vehicle, denoted as vi, vi represents the image captured by the i-th on-board camera, i=1, 2, ..., m, where m is a positive integer representing the number of on-board cameras. Different on-board cameras capture images from different perspectives. In some embodiments, images from different perspectives captured by different on-board cameras can be spliced ​​into a panoramic image, which can construct a high-definition map with a 360° (degree) perspective.

[0063] A driving data set can contain multiple types of data collected at the same time. For example, the number of driving data sets can be n, that is, n driving data sets are collected at n times, recorded as {clip1, clip2, clip3, ..., clipn}, where each driving data set includes {v1, v2, ..., vm, GPS data, wheel data, IMU data}. For example, the vehicle can collect data every 0.1s (seconds), and the data collected at the same time are packaged into a driving data set.

[0064] In some embodiments, the high-precision map construction device can determine the objects in the vehicle's surrounding environment based on the driving environment image data collected by the m vehicle-mounted cameras; determine the vehicle's position based on the acquired GPS data, determine the vehicle's speed based on the acquired wheel data, and determine the vehicle's posture based on the acquired IMU data. Furthermore, the high-precision map construction device can determine the posture of each vehicle-mounted camera based on the vehicle's posture.

[0065] Step S202: determining a single sparse point cloud model corresponding to each driving data set and an aggregation order of each driving data set.

[0066] The aggregation order is the order in which the single sparse point cloud models corresponding to the multiple driving data sets are aggregated. The high-precision map construction device can determine the single sparse point cloud model corresponding to the driving data set collected at the jth moment based on the driving environment image data, GPS data, wheel data and IMU data included in the driving data set clipj at the jth moment in the n driving data sets. Repeat this process to obtain the single sparse point cloud models corresponding to the n driving data sets.

[0067] When the high-precision map construction device aggregates n single sparse point cloud models, it needs to aggregate them in a certain order. In the disclosed embodiment, the high-precision map construction device can determine the aggregation order of each single sparse point cloud model in subsequent aggregation according to the size of the overlapping area between each single sparse point cloud model. Where j = 1, 2, ..., n, n is a positive integer, indicating the number of driving data sets.

[0068] Step S203 , aggregating the single sparse point cloud models corresponding to each driving data set based on the aggregation order to obtain a global sparse point cloud model.

[0069] In some embodiments, a high-precision map construction device can determine the aggregation order of each single sparse point cloud model according to the chronological order of the collection time of each driving data set, starting from the single sparse point cloud model corresponding to the first driving data set with the earliest collection time, and according to the aggregation order of each single sparse point cloud model, the single sparse point cloud models corresponding to the remaining n-1 driving data sets are aggregated in turn, and a global sparse point cloud model is obtained after the aggregation is completed.

[0070] In the embodiment of the present disclosure, a global sparse point cloud model can be directly determined by aggregating single sparse point cloud models corresponding to each driving data set. Compared with the implementation method in the related art of splicing images of different perspectives in each driving data set, extracting all features of all pixels of the spliced ​​image, and then calculating the three-dimensional information of all pixels of the image, the embodiment of the present disclosure can reduce the amount of calculation data and shorten the time.

[0071] Step S204: Generate a high-precision map based on the global sparse point cloud model.

[0072] In some embodiments, the high-precision map construction device can extract and identify features based on the aggregated global sparse point cloud model to generate a high-precision map. The extracted features can be object features included in the high-precision map, such as lane lines, road signs, road boundaries and other object features. After the high-precision map is generated, the high-precision map construction device can further perform decision control based on the high-precision map to enable the vehicle to perform intelligent driving.

[0073] The method for constructing a high-precision map provided by the embodiment of the present disclosure first determines multiple driving data sets collected during the driving process of the vehicle; determines the single sparse point cloud model corresponding to each driving data set and the aggregation order of each driving data set; aggregates the single sparse point cloud model corresponding to each driving data set based on the aggregation order to obtain a global sparse point cloud model; and generates a high-precision map based on the global sparse point cloud model. By determining the single sparse point cloud model corresponding to each of the multiple driving data sets, and aggregating the single sparse point cloud models according to the aggregation order corresponding to each driving data set, the accuracy of map construction can be improved, and there is no need to calculate the three-dimensional information of all pixels of the image, which can reduce the amount of data calculation and shorten the time consumption, thereby realizing the rapid construction of a high-precision map.

[0074] In some embodiments, based on the above embodiments, the above step S202 "determining the single sparse point cloud model corresponding to each driving data set and the aggregation order of each driving data set" can be performed by Figure 3 The steps S2021 to S2023 shown are implemented as follows:

[0075] Step S2021, determining, according to each driving data set, the global positioning system data, the wheel speed meter data, the inertial unit data and the image data respectively collected by a plurality of vehicle-mounted cameras provided on the vehicle in each driving data set.

[0076] After obtaining each driving data set, the high-precision map construction device determines the GPS data used to determine the vehicle position, the wheel data used to determine the vehicle speed, the IMU data used to determine the vehicle posture, and the driving environment image data {v1, v2, ..., vm} collected by the m on-board cameras on the vehicle in the driving data set.

[0077] Step S2022: Generate a single sparse point cloud model corresponding to each driving data set according to the global positioning system data, wheel speed meter data, inertial unit data and each image data in each driving data set.

[0078] Taking the jth driving data set among n driving data sets as an example, after obtaining the GPS data, wheel data, IMU data, and m driving environment image data {v1, v2, ..., vm} included in the jth driving data set, the GPS data, wheel data, IMU data, and m driving environment image data {v1, v2, ..., vm} in the jth driving data set are processed to generate a single sparse point cloud model corresponding to the jth driving data set. For the specific implementation process of generating the point cloud model, see the following Figure 4 The embodiment shown.

[0079] Step S2023: Determine the aggregation order of each driving data set according to the single sparse point cloud model corresponding to each driving data set.

[0080] In some embodiments, the aggregation order of each driving data set can be determined according to the size of the overlapping area between the single sparse point cloud models corresponding to the driving data set. In other embodiments, the aggregation order of each driving data set can also be determined according to the time sequence of the timestamps of each driving data set, for example, the earlier the driving data set is collected, the earlier the driving data set is aggregated, wherein the timestamp of the driving data set can be the collection time of the driving data, or the time when the high-precision map construction device obtains the driving data set.

[0081] In the disclosed embodiment, the sparse point cloud model and aggregation order corresponding to each driving data set are determined to provide data support for the subsequent aggregation of the global sparse point cloud model. The sparse point cloud model of each driving data set determined in this method can reduce the amount of data calculation and shorten the time consumption compared to the dense point cloud model, thereby achieving the purpose of quickly building a high-precision map.

[0082] In some embodiments, based on the above embodiments, the above step S2022 "generating a single sparse point cloud model corresponding to each driving data set according to the global positioning system data, wheel speed meter data, inertial unit data and each image data in each driving data set", taking any one of the driving data sets as an example, the single sparse point cloud model corresponding to the driving data set can be obtained by Figure 4 Steps S20221 to S20224 shown are used to implement:

[0083] Step S20221: determining feature points of each image data according to each image data in the driving data set.

[0084] Still taking the j-th driving data set as an example, feature extraction is performed on the m driving environment image data {v1, v2, ..., vm} in the j-th driving data set to obtain feature points of each driving environment image.

[0085] In some embodiments, before extracting features from the driving environment image, semantic segmentation can be performed on it first, and the segmentation graphs that are not needed to build the map can be removed by cropping in combination with the scene. For example, in a scene that focuses more on ground objects, the segmentation graph of the sky can be removed. For example, in building a static map, the segmentation graph of dynamic objects (such as vehicles, pedestrians, etc.) can be removed. By removing the segmentation graphs that are not involved in building the map, only feature points are extracted for the relevant objects involved in building the map during feature extraction, which can reduce the amount of data processed and improve the processing speed, thereby further accelerating the construction of high-precision maps.

[0086] Step S20222, determining the image pose of each image data according to the global positioning system data, the wheel speed meter data and the inertial unit data in the driving data set.

[0087] The high-precision map construction device can determine the image pose of each image data, that is, the shooting position and shooting posture of each image based on GPS data, wheel data and IMU data.

[0088] In some embodiments, the image pose of each image data may be based on Figure 5 Steps S2221 to S2223 are shown to determine:

[0089] Step S2221, determining the optimized posture of the vehicle according to the global positioning system data, wheel speed meter data and inertial unit data in the driving data set.

[0090] The device for constructing a high-precision map can determine the vehicle posture independently based on any one of the GPS data, wheel data, and IMU data. In the disclosed embodiment, in order to improve the accuracy, the vehicle posture can be determined based on the GPS data, wheel data, and IMU data to obtain the optimized posture of the vehicle. The optimized posture of the vehicle corresponding to the driving data set clipj refers to the posture of the vehicle when the vehicle collects the driving data set clipj. The optimized posture of the vehicle may include the optimized position and optimized posture of the vehicle.

[0091] In some embodiments, the optimized posture of the vehicle is determined based on the GPS data, wheel data, and IMU data in the driving data set clipj, which can be determined specifically according to the following steps:

[0092] Step 1: Get measurements based on GPS data, wheel data, and IMU data.

[0093] Taking the jth driving data set as an example, the measured value of the jth driving data set (denoted as z j(j-1) ) can be calculated based on the data included in the j-th driving data set and the driving data set obtained at the previous moment (i.e., the j-1-th driving data set). Specifically, the measurement value z j(j-1) It can be calculated according to the following formula (1):

[0094] z j(j-1) =z gps +z wheel +z imu (1);

[0095] In the formula, z gps is the GPS position change measurement data. For example, z gps It can be calculated based on the two GPS data received at adjacent time points. The GPS data in the (j-1)th driving data set is recorded as GPS j-1 , expressed as The GPS data in the jth driving data set can be recorded as GPS j , expressed as Then the GPS position change measurement data z gps It can be calculated according to the following formula (2):

[0096]

[0097] In the formula, z wheel is the wheel posture change measurement data. For example, z wheel It can be calculated based on the two wheel data received at adjacent time points. The wheel data in the j-1th driving data set is denoted as wheel j-1 , expressed as The wheel data in the jth driving data set can be recorded as wheel j , expressed as Then the wheel posture change measurement data z wheel It can be calculated according to the following formula (3):

[0098]

[0099] In the formula, z imu is the IMU pose change measurement data. For example, z imu It can be calculated based on the two IMU data received at adjacent time points. The IMU data in the j-1th driving data set is denoted as IMU j-1 , expressed as The IMU data in the jth driving data set can be recorded as IMU j , expressed as Then the IMU posture change measurement data z imu It can be calculated according to the following formula (4):

[0100]

[0101] Step 2: Construct the error term.

[0102] In the pose graph, the error term is used to measure the difference between the measured value and the estimated value. There will be a certain error in the GPS pose change measurement data, wheel pose change measurement data, and IMU pose change measurement data. The error term corresponding to the jth driving data set is denoted as e j(j-1) . Exemplarily, the error term constructed is shown in the following formula (5):

[0103] e j(j-1) =z j(j-1) -h j(j-1) (5);

[0104] Among them, z j(j-1) is the measurement value, which represents the measured data of the vehicle's posture change from the position when the j-1th driving data set was collected to the position when the jth driving data set was collected; h j(j-1) is the estimated value calculated according to the estimation model, which represents the estimated data of the vehicle's posture change from the position when the j-1th driving data set was collected to the position when the jth driving data set was collected. The estimation model can be a model pre-trained based on pre-collected sample data and user-labeled data.

[0105] Step 3: Determine the optimal pose of the vehicle.

[0106] First, the vehicle's posture is initialized according to the sensor data; then the sum of the error terms of the GPS posture change measurement data, the wheel posture change measurement data, and the IMU posture change measurement data is calculated to form the total error function E, as shown in the following equation (6):

[0107] E=Σe j(j-1) T Ω j(j-1) e j(j-1) (6);

[0108] In the formula, Ω j(j-1) is the information matrix, which indicates the confidence of the measured value. For example, if the IMU data is relatively accurate in angular velocity measurement but has a large error in acceleration measurement, the corresponding information matrix can be defined as Among them, ω xx ,ω yy and ω θθ Represent the confidence of the angular velocity ω in the x, y and θ directions respectively.

[0109] After constructing the total error function E, a nonlinear optimization algorithm (such as the Gauss-Newton method or the LM method) can be used to minimize the error function E. The optimal posture corresponding to the minimum E is the optimal posture of the vehicle.

[0110] Step S2222, determining the motion trajectory of each vehicle-mounted camera based on the optimized posture of the vehicle and the external parameter data of each vehicle-mounted camera.

[0111] The motion trajectory of the i-th on-board camera is determined by using the optimized posture of the vehicle and the external parameter data of the i-th on-board camera among the m on-board cameras. The external parameter data of the on-board camera includes the posture data of the on-board camera, and further includes the position and posture of the on-board camera in the world coordinate system. The posture of the on-board camera describes its rotation angle in the world coordinate system.

[0112] According to the optimized posture of the vehicle and the external parameter data of the m on-board cameras, the motion trajectories of the m on-board cameras are determined. Generally, the relative position of the on-board camera on the vehicle is fixed. When the rotation angles of the m on-board cameras are also fixed, the motion trajectories of the m on-board cameras are consistent; when the rotation angles of the on-board cameras are not fixed, if rotation occurs, the motion trajectories are inconsistent.

[0113] Step S2223, determining the image pose of each image data according to the motion trajectory of each vehicle-mounted camera.

[0114] After the motion trajectory of the vehicle-mounted camera is determined, the position and posture of each frame of image collected by the vehicle-mounted camera is calculated according to the motion trajectory of the vehicle-mounted camera and the exposure time of the vehicle-mounted camera, and the image position and posture of each image data is obtained.

[0115] In some embodiments, the image pose of each frame can be obtained by interpolation based on the exposure time of the vehicle-mounted camera. For example, based on the motion trajectory of the vehicle-mounted camera, the image poses of 0.1s (seconds) and 0.2s can be determined, and the image poses between 0.1s and 0.2s can be determined. Based on the exposure time of the vehicle-mounted camera, it can be determined that a frame of image was taken at 0.13s, and the image poses of 0.1s and 0.2s can be interpolated to determine the pose of the image at 0.13s.

[0116] According to the motion trajectory of the vehicle-mounted camera, the position and posture of one frame of image can be determined per second, and the position and posture change can be determined based on the position and posture of two frames of image within two seconds obtained from the motion trajectory. According to the position change and the frame rate, the interpolation time interval for each frame can be 1s (second).

[0117] In the disclosed embodiment, the vehicle posture is first optimized using GPS data, wheel data, and IMU data to determine the optimized posture of the vehicle, and the motion trajectory of the vehicle is determined based on the optimized postures at multiple moments. Then, the motion trajectory of each on-board camera is determined by combining the motion trajectory of the vehicle and the external parameters of each on-board camera (including the position and posture of each on-board camera on the vehicle). Finally, interpolation is performed based on the motion trajectory of each on-board camera and the exposure time of the on-board camera to determine the image posture of each frame. Determining the image posture in this way can improve the calculation speed and accuracy.

[0118] Step S20223, determining the three-dimensional information of the feature points of each image data according to the feature points and image pose of each image data.

[0119] The feature points of each image data determined in the above step S20221 are two-dimensional feature points. After the image pose of each image data is determined, the three-dimensional information of the feature points of each image data is determined based on the image pose of each image data.

[0120] In some embodiments, the three-dimensional information of the feature points of each image data may be specifically obtained according to Figure 6 Steps S2231 to S2235 are shown to determine:

[0121] Step S2231, determining matching image data for each image data according to the image pose of each image data.

[0122] If the acquisition angles of the vehicle-mounted cameras are the same or overlap, there will be a common view area in the multiple frames of images collected at similar positions. In the disclosed embodiment, all image data can be associated according to the common view area that meets the preset conditions. First, the corresponding matching image data is determined according to the image pose of each image data.

[0123] In some embodiments, determining the corresponding matching image data according to the image pose of each image data can be achieved according to the following steps: first, determining a plurality of candidate image data corresponding to the target image data from each image data according to the image pose of each image data; then determining the common viewing area between the target image data and its corresponding candidate image data; finally, determining the matching image data of the target image data according to the candidate image data that meets preset conditions in the common viewing area.

[0124] The construction device of the high-precision map obtains n driving data sets, each of which includes m image data. Taking the jth driving data set as an example, when determining the matching image data of the m image data in the jth driving data set, first, according to the image pose of the i-th image data among the m image data in the jth driving data set, select the image data within a certain threshold range from the i-th image data as the candidate image data of the i-th image data (i.e., the current target image data); calculate the common view area between the i-th image data and each of its candidate image data. When the common view area between a candidate image data and the i-th image data among the m image data in the jth driving data set meets the preset conditions, the candidate image data is used as a matching image data of the i-th image data.

[0125] In one implementation, candidate image data for each target image data may be selected according to a K-Nearest Neighbor (KNN) algorithm.

[0126] In the disclosed embodiment, the common viewing area refers to the area observed by the current image and the candidate image. In one implementation, it is possible to determine whether there is a common viewing relationship between the two frames of images based on key feature points, and determine the common viewing area based on the key feature points of the two frames of images when there is a common viewing relationship. In another implementation, it is also possible to determine whether there is a common viewing area between the two frames of images based on the viewing angle range and posture of the vehicle-mounted camera, and determine the common viewing area based on the viewing angle range and posture of the two vehicle-mounted cameras when there is a common viewing area.

[0127] In one implementation, whether the common view area satisfies the preset condition can be determined based on the size of the common view area. If the size of the common view area exceeds the preset range threshold, it is determined that the common view area between the candidate image data and the i-th image data satisfies the preset condition, and the candidate image data is used as a matching image data of the i-th image data. If the size of the common view area does not exceed the preset range threshold, it is determined that the common view area between the candidate image data and the i-th image data does not satisfy the preset condition, and the judgment step of the next candidate image data is continued until all candidate image data in the m image data in the j-th driving data set are judged, and all matching image data of the i-th image data in the j-th driving data set are obtained.

[0128] The same as the above steps, continue to determine the matching image data of the next image data in the m image data until the matching image data of the m image data are obtained. In this way, the matching image data of each image data in the j-th driving data set are obtained.

[0129] Step S2232, performing feature point matching on each image data and its corresponding matching image data, and obtaining a matching relationship between some feature points in the feature points of each image data.

[0130] For m image data in the jth driving data set, if the similarity between the kth feature point in the ith image data and the pth feature point of the matching image data corresponding to the ith image data is greater than the preset similarity threshold, it is determined that the kth feature point of the ith image data and the pth feature point of the matching image data corresponding to it are the same feature point, that is, the kth feature point of the ith image data and the pth feature point of the matching image data corresponding to it match. Compare all feature points of the ith image data and all feature points of the corresponding matching image data, select some feature points with matching feature points from all feature points of the ith image data, and obtain some feature points with matching relationships from the feature points of the ith image data, where the matching relationship of the kth feature point can be expressed as (k, p).

[0131] Among them, k and p are positive integers, the kth feature point represents the kth feature point in the ith image data, and the pth feature point represents the matching feature point in the matching image data corresponding to the ith image data that represents the same feature point as the kth feature point.

[0132] Based on the above steps, the matching relationship of some feature points in the feature points of the i-th image data is obtained, and the next image data is determined until all m image data are determined. At this point, the matching relationship of some feature points in the feature points of each image data in the j-th driving data set is obtained.

[0133] Step S2233: Determine the three-dimensional information of the partial feature points of each image data according to the matching relationship of the partial feature points of each image data.

[0134] For example, according to the position of the same feature point of the same object in two of the m image data, the distance between the feature point and the vehicle camera can be calculated based on the binocular vision principle as the depth of the feature point. Similarly, the depths of all feature points with matching relationships are calculated.

[0135] According to the depth of some feature points among the feature points of the i-th image data among the m image data, combined with the two-dimensional information of each feature point determined based on the i-th image data, the three-dimensional information of some feature points of the i-th image data is obtained. This step is performed on the m image data respectively to obtain the three-dimensional information of some feature points of each image data.

[0136] Step S2234: Determine the three-dimensional information of the remaining feature points of each image data according to the three-dimensional information of the partial feature points of each image data.

[0137] After determining the three-dimensional information of the feature points of the common view area based on two of the m image data, the three-dimensional information of the remaining feature points of the non-common view area in the two images can be determined based on the three-dimensional information of the feature points of the common view area. Specifically, the depth of the remaining feature points can be determined based on the feature points of known depth located on the same object, or the depth of the remaining feature points can be determined based on the imaging size and the feature points of known depth. After obtaining the depth of the remaining feature points except for some feature points among the feature points of the i-th image data, the three-dimensional information of the remaining feature points of the i-th image data is determined in combination with the two-dimensional information of each feature point. This step is performed on the m image data respectively to obtain the three-dimensional information of the remaining feature points of each image data.

[0138] Step S2235 , determining the three-dimensional information of the feature points of each image data according to the three-dimensional information of some feature points of each image data and the three-dimensional information of the remaining feature points among the feature points of each image data except some feature points.

[0139] After obtaining the 3D information of some feature points in the common view area and the 3D information of the remaining feature points in the non-common view area in the i-th image data, the 3D information of some feature points in the i-th image data and the 3D information of the remaining feature points are determined as the 3D information of the feature points of the i-th image data. This step is performed on m image data respectively to obtain the 3D information of the feature points of each image data.

[0140] Step S20224: Generate a single sparse point cloud model corresponding to the driving data set based on the three-dimensional information of the feature points of each image data.

[0141] For the m image data in the jth driving data set, the three-dimensional information of the feature points of the i-th image data can be the three-dimensional information in the world coordinate system, that is, the spatial position. According to the three-dimensional positions of all the feature points of each image data in the m image data in the world coordinate system, the point cloud model corresponding to the m image data is formed, that is, the point cloud model corresponding to the jth driving data set.

[0142] The above steps are performed on n driving data sets respectively to obtain a single sparse point cloud model corresponding to each driving data set.

[0143] Since in the embodiment of the present disclosure, when extracting and processing features of each image data of each driving data set, only the features of key points are extracted for calculation and processing, rather than all feature points, the generated point cloud model is a sparse point cloud model. Compared with a dense point cloud model, a sparse point cloud model can reduce the amount of calculated data and shorten the time to generate a point cloud model, thereby shortening the time required to build a high-definition map and achieving the purpose of quickly building a high-precision map.

[0144] In some embodiments, based on the above embodiments, the above step S2023 "determine the aggregation order of each driving data set according to the single sparse point cloud model corresponding to each driving data set" can be performed by Figure 7 The steps S20231 and S20232 shown are implemented as follows:

[0145] Step S20231: determining a first overlapping area between different driving data sets according to a single sparse point cloud model corresponding to each driving data set.

[0146] In the driving data set acquired by the high-precision map construction device, there are overlapping areas between the data of the driving data sets collected at similar times, such as driving data sets collected at adjacent collection times all include data of the common overlapping area. Based on this, after determining the single sparse point cloud model corresponding to each driving data set, whether the corresponding driving data sets overlap is determined based on whether the point cloud models overlap. Specifically, it can be determined whether there are the same feature points between the two point cloud models. If there are the same feature points, it is determined that the two point cloud models overlap. Furthermore, the overlapping area of ​​the two point cloud models and the size of the overlapping area can be determined based on the number and position of the feature points.

[0147] Step S20232: Determine the aggregation order of each driving data set according to the first overlapping area between different driving data sets.

[0148] After determining the first overlapping area between different driving data sets in each driving data set, the aggregation order of each driving data set can be determined according to the size of the overlapping area. The larger the overlapping area, the larger the common view area corresponding to the two driving data sets, and the higher the priority of aggregation during aggregation.

[0149] In another implementation, the aggregation order of the driving data sets can also be determined according to the interval between the collection times of the driving data sets. The shorter the interval, the closer the collection time of the two driving data sets is, the larger the overlap area between them is, and the higher the priority of aggregation is during aggregation.

[0150] In the embodiment of the present disclosure, the aggregation order of each driving data set is determined, and the larger the overlapping area, the higher the priority of aggregation, which can reduce the aggregation error.

[0151] In some embodiments, based on the above embodiments, the above step S203 "aggregating the single sparse point cloud models corresponding to each driving data set based on the aggregation order to obtain a global sparse point cloud model" can be performed by Figure 8 The steps S2031 to S2035 shown are implemented as follows:

[0152] Step S2031, determining the aggregated sparse point cloud model.

[0153] When aggregating the sparse point cloud models corresponding to n driving data sets, first determine the aggregated sparse point cloud model. When it is the first aggregation, the aggregated sparse point cloud model is the single sparse point cloud model corresponding to the driving data set ranked first in the aggregation order; when it is not the first aggregation, the aggregated sparse point cloud model is the sparse point cloud model obtained by the last aggregation.

[0154] Determine whether there are any unaggregated single sparse point cloud models. If there are any unaggregated single sparse point cloud models, go to step S2032; if there are no unaggregated single sparse point cloud models, that is, all single sparse point cloud models have been aggregated, go to step S2035.

[0155] Step S2032: in response to the existence of unaggregated single sparse point cloud models in the single sparse point cloud models corresponding to each driving data set, a single sparse point cloud model to be aggregated is determined from the unaggregated single sparse point cloud models based on the aggregation order.

[0156] When there is an unaggregated single sparse point cloud model, based on the aggregation order, the single sparse point cloud model that needs to be aggregated this time is determined from the unaggregated single sparse point cloud models.

[0157] Step S2033: determine a second overlapping area between the aggregated sparse point cloud model and the single sparse point cloud model to be aggregated.

[0158] According to the sparse point cloud model obtained by the last aggregation and the single sparse point cloud model to be aggregated this time, the overlapping area of ​​the two point cloud models is determined to obtain a second overlapping area.

[0159] Step S2034: based on the second overlapping area, aggregate the aggregated sparse point cloud model and the single sparse point cloud model to be aggregated, and update the aggregated sparse point cloud model.

[0160] Since the second overlapping area has been aggregated into the sparse point cloud model, there is no need to repeatedly aggregate the already aggregated second overlapping area when aggregating the single sparse point cloud model that needs to be aggregated this time. Based on the position of the second overlapping area, the non-overlapping area except the second overlapping area is aggregated into the result obtained from the last aggregation to complete this aggregation and obtain a new aggregated sparse point cloud model.

[0161] Then determine again whether there are any unaggregated single sparse point cloud models. If there are any unaggregated single sparse point cloud models, return to execute steps S2032 to S2034 again; if there are no unaggregated single sparse point cloud models, that is, all single sparse point cloud models have been aggregated, then enter step S2035.

[0162] Step S2035: In response to the fact that there is no unaggregated single sparse point cloud model in the single sparse point cloud models corresponding to each driving data set, the aggregated sparse point cloud model is determined as the global sparse point cloud model.

[0163] When there is no unaggregated single sparse point cloud model, that is, all single sparse point cloud models have been aggregated, the current aggregated sparse point cloud model is determined as the global sparse point cloud model, so that all aggregation results obtained by aggregating n single sparse point cloud models corresponding to n driving data sets are obtained, that is, the global sparse point cloud model.

[0164] In the disclosed embodiment, starting from the sparse point cloud model corresponding to the first ranked driving data set, the remaining n-1 single sparse point cloud models are aggregated in sequence according to the aggregation order to obtain a global sparse point cloud model. When aggregating, the models are aggregated in sequence according to the size of the overlapping area, which can reduce the error of the global sparse point cloud model.

[0165] Based on the above embodiments, the present disclosure further provides a method for constructing a high-precision map. Fig. 9 is a flowchart of a method for constructing a high-precision map provided by another exemplary embodiment of the present disclosure. Fig. 9 As shown, the method comprises the following steps:

[0166] Step S901, determining a plurality of driving data sets collected during the driving process of the vehicle.

[0167] The disclosed embodiments can be executed by a high-precision map construction device. The high-precision map construction device obtains multiple driving data sets collected during the driving process of the vehicle. Among them, a driving data set may contain multiple types of data collected at the same time. Exemplarily, the number of driving data sets may be n, that is, n driving data sets are collected at n moments, recorded as {clip1, clip2, ..., clipj, ..., clipn}. Exemplarily, the vehicle can collect once every 0.1s (seconds), and the data collected at the same time is packaged into a driving data set. Among them, j = 1, 2, ..., n, n is a positive integer, indicating the number of driving data sets.

[0168] Fig.10 is a flowchart of a method for constructing a high-precision map provided by another exemplary embodiment of the present disclosure. Fig.10 As shown, the high-precision map construction device obtains n driving data sets {clip1, clip2, ..., clipj, ..., clipn} from the vehicle.

[0169] Step S902: determining, according to each driving data set, the global positioning system data, the wheel speed meter data, the inertial unit data and the image data respectively collected by a plurality of vehicle-mounted cameras arranged on the vehicle in each driving data set.

[0170] Fig.11: is a flowchart of the step of determining the sparse point cloud of each driving data set in the method for constructing a high-precision map provided by an exemplary embodiment of the present disclosure. Take the jth driving data set among n driving data sets as an example. Fig.11 As shown, the j-th driving data set includes {v1, v2, ..., vi, ...vm, GPS data, wheel data, IMU data}, where i = 1, 2, ..., m, where m is a positive integer representing the number of vehicle-mounted cameras.

[0171] Step S903: determining feature points of each image data according to each image data in the driving data set.

[0172] See also Fig.11 , for a total of m image data from v1 to vm, segmentation and key point extraction are performed respectively to obtain the feature points of each image data.

[0173] Step S904: determining the image pose of each image data according to the global positioning system data, the wheel speed meter data and the inertial unit data in the driving data set.

[0174] like Fig.11 As shown in the figure, firstly, the posture optimization is performed according to the wheel data, GPS data and IMU data to obtain the optimized posture of the vehicle, and then the optimized posture of the vehicle and the external parameter data of each vehicle-mounted camera are used for transformation processing to determine the motion trajectory of each vehicle-mounted camera, and finally the posture of each image data is obtained by interpolation according to the exposure time of the vehicle-mounted camera, that is, the image posture of each image data.

[0175] Step S905: Determine the three-dimensional information of the feature points of each image data according to the feature points and image pose of each image data.

[0176] After the image pose of each image data is determined, the depth of the feature point of each image data can be determined according to the image pose of each image data. The depth of the feature point is combined with the two-dimensional information of the feature point determined based on the image data to obtain the three-dimensional information of the feature point of each image data.

[0177] Specifically, when determining the three-dimensional information of the feature points of each image data, first determine the matching image data of each image data according to the image posture of each image; perform feature point matching on each image data and its corresponding matching image data to obtain the matching relationship of some feature points among the feature points of each image data; then determine the three-dimensional information of some feature points of each image data according to the matching relationship of some feature points of each image data; then determine the three-dimensional information of the remaining feature points of each image data except some feature points according to the three-dimensional information of some feature points of each image data; finally, determine the three-dimensional information of the feature points of each image data according to the three-dimensional information of some feature points of each image data and the three-dimensional information of the remaining feature points among the feature points of each image data except some feature points.

[0178] Step S906, optimizing the three-dimensional information of the feature points of each image data based on the bundle adjustment method to obtain optimized three-dimensional information of the feature points of each image data.

[0179] Bundle adjustment is a technique used in photogrammetry and computer vision to accurately estimate the position and attitude of a camera and to restore the three-dimensional structure of a scene using multiple photos. The main principle is that for any three-dimensional point Q in the scene, the light rays emitted from the optical center of the on-board camera corresponding to each image and passing through the pixel corresponding to Q in the image all intersect at point Q. Based on this principle, after determining the three-dimensional information of the feature points of each image data, the depth of each feature point can be optimized based on the bundle adjustment method, which can reduce the cumulative error and improve the accuracy of the point cloud model.

[0180] In some embodiments, the external parameters of each camera and the image pose of each image data can also be optimized based on the bundle adjustment method. When processing the driving data set at the next moment (e.g., the jth driving data set is processed at the current moment, and the j+1 driving data set can be processed at the next moment), the optimized camera external parameters and image pose are used for calculation, which can further reduce the cumulative error and improve the accuracy of the point cloud model.

[0181] Step S907 , generating a single sparse point cloud model corresponding to the driving data set according to the optimized three-dimensional information of the feature points of each image data.

[0182] The bundle adjustment method is used to perform incremental mapping based on the 3D information of the feature points of each optimized image data. The 3D structure is restored from the multi-view images through the motion recovery SFM method to obtain a single sparse point cloud model corresponding to the j-th driving data set.

[0183] Repeat the above steps to determine a single sparse point cloud model corresponding to the n driving data sets.

[0184] Step S908: determining the aggregation order of each driving data set according to the single sparse point cloud model corresponding to each driving data set.

[0185] In some embodiments, the aggregation order of each driving data set can be determined according to the size of the overlapping area between the single sparse point cloud models corresponding to the driving data set. In other embodiments, the aggregation order of each driving data set can also be determined according to the order of the timestamps of each driving data set, for example, the earlier the driving data set is collected, the earlier it is aggregated, where the timestamp of the driving data set can be the collection time of the data, or the time when the high-precision map construction device obtains the driving data set.

[0186] In the disclosed embodiment, the sparse point cloud model and aggregation order corresponding to each driving data set are determined to provide data support for the subsequent aggregation of the global sparse point cloud model. The sparse point cloud model of each driving data set determined in this method can reduce the amount of data calculation and shorten the time consumption compared to the dense point cloud model, thereby achieving the purpose of quickly building a high-precision map.

[0187] Step S909: Aggregate the single sparse point cloud models corresponding to each driving data set based on the aggregation order to obtain a global sparse point cloud model.

[0188] In some embodiments, the first driving data set collected earliest can be determined according to the collection time of each driving data set. Starting from the first driving data set, the single sparse point cloud models corresponding to the n-1 remaining driving data sets are aggregated in sequence according to the aggregation order of each single sparse point cloud model. After the aggregation is completed, a global sparse point cloud model is obtained.

[0189] In the disclosed embodiment, a global sparse point cloud model is determined by aggregating single sparse point cloud models corresponding to each driving data set. There is no need to stitch images of different perspectives in each driving data set, nor is there any need to extract all features of all pixels in the stitched image. Furthermore, there is no need to calculate the three-dimensional information of all pixels in the image, which can reduce the amount of data calculation and shorten the time.

[0190] Step S910, generating a high-precision map based on the global sparse point cloud model.

[0191] In some embodiments, feature extraction and recognition may be performed based on the aggregated global sparse point cloud model to generate a high-precision map, wherein the extracted features may be object features included in the high-precision map, such as features of lane lines, road signs, road boundaries, and other objects.

[0192] The method for constructing a high-precision map provided by the embodiment of the present disclosure can improve the accuracy of map construction by determining a single sparse point cloud model corresponding to each of a plurality of driving data sets and aggregating each single sparse point cloud model according to the aggregation order corresponding to each driving data set. There is no need to calculate the three-dimensional information of all pixels of the image, which can reduce the amount of calculated data and shorten the time, thereby realizing the rapid construction of a high-precision map.

[0193] Exemplary Devices

[0194] Fig.12 FIG. 1 is a schematic diagram of the composition structure of a device for constructing a high-precision map provided by an exemplary embodiment of the present disclosure. Fig.12 As shown, the high-precision map construction device 1200 may include:

[0195] The first determination module 1201 is used to determine a plurality of driving data sets collected during the driving process of the vehicle;

[0196] A second determination module 1202 is used to determine a single sparse point cloud model corresponding to each of the driving data sets and an aggregation order of each of the driving data sets;

[0197] Aggregation module 1203, used for aggregating the single sparse point cloud models corresponding to the driving data sets based on the aggregation order to obtain a global sparse point cloud model;

[0198] The generation module 1204 is used to generate the high-precision map based on the global sparse point cloud model.

[0199] See also Fig.13 In some embodiments, the second determining module 1202 may include:

[0200] A first determining submodule 12021 is used to determine, according to each of the driving data sets, the global positioning system data, the wheel speed meter data, the inertial unit data and the image data respectively collected by the multiple vehicle-mounted cameras provided on the vehicle in each of the driving data sets;

[0201] A generating submodule 12022, configured to generate a single sparse point cloud model corresponding to each driving data set according to the global positioning system data, the wheel speed meter data, the inertial unit data and each image data in each driving data set;

[0202] The second determination submodule 12023 is used to determine the aggregation order of each driving data set according to the single sparse point cloud model corresponding to each driving data set.

[0203] In some embodiments, the generating submodule 12022 may include:

[0204] A first determining unit, configured to determine, for each of the driving data sets, a feature point of each of the image data according to each of the image data in the driving data set;

[0205] A second determination unit, configured to determine an image pose of each of the image data according to the global positioning system data, the wheel speed meter data and the inertial unit data in the driving data set;

[0206] A third determining unit, configured to determine three-dimensional information of the feature points of each of the image data according to the feature points and the image pose of each of the image data;

[0207] A generating unit is used to generate the single sparse point cloud model corresponding to the driving data set according to the three-dimensional information of the feature points of each of the image data.

[0208] In some embodiments, the second determining unit may include:

[0209] an optimization subunit, configured to determine an optimized posture of the vehicle according to the global positioning system data, the wheel speed meter data and the inertial unit data in the driving data set;

[0210] A first determination subunit, configured to determine a motion trajectory of each of the vehicle-mounted cameras according to the optimized posture of the vehicle and external parameter data of each of the vehicle-mounted cameras;

[0211] The second determining subunit is used to determine the image pose of each image data according to the motion trajectory of each vehicle-mounted camera.

[0212] In some embodiments, the third determining unit may include:

[0213] A third determining subunit, configured to determine matching image data for each of the image data according to the image pose of each of the image data;

[0214] A matching subunit, used for performing feature point matching on each of the image data and the corresponding matching image data, to obtain a matching relationship between some of the feature points in the feature points of each of the image data;

[0215] A fourth determining subunit, configured to determine three-dimensional information of the partial feature points of each of the image data according to a matching relationship between the partial feature points of each of the image data;

[0216] a fifth determining subunit, configured to determine, based on the three-dimensional information of the partial feature points of each of the image data, the three-dimensional information of the remaining feature points among the feature points of each of the image data except for the partial feature points;

[0217] The sixth determining subunit is used to determine the three-dimensional information of the feature points of each image data based on the three-dimensional information of the part of the feature points of each image data and the three-dimensional information of the remaining feature points among the feature points of each image data except the part of the feature points.

[0218] In some embodiments, the third determination subunit is specifically used to: determine a plurality of candidate image data corresponding to the target image data from each of the image data according to the image pose of each of the image data; determine a common viewing area between the target image data and each of the candidate image data corresponding to the target image data; and determine the matching image data of the target image data according to the candidate image data that meets preset conditions in the common viewing area.

[0219] In some embodiments, the second determining submodule 12023 may include:

[0220] a fourth determining unit, configured to determine a first overlapping area between different driving data sets according to a single sparse point cloud model corresponding to each driving data set;

[0221] A fifth determining unit is used to determine an aggregation order of each of the driving data sets according to the first overlapping areas between different driving data sets.

[0222] See also Fig.13 In some embodiments, the aggregation module 1203 may include:

[0223] The third determination submodule 12031 is used to determine the aggregated sparse point cloud model;

[0224] The fourth determining submodule 12032 is configured to, in response to the existence of unaggregated single sparse point cloud models in the single sparse point cloud models corresponding to each of the driving data sets, determine the single sparse point cloud model to be currently aggregated from the unaggregated single sparse point cloud models based on the aggregation order;

[0225] A fifth determining submodule 12033 is used to determine a second overlapping area between the aggregated sparse point cloud model and the single sparse point cloud model to be aggregated;

[0226] Aggregation submodule 12034, configured to aggregate the aggregated sparse point cloud model and the single sparse point cloud model to be aggregated based on the second overlapping area, and update the aggregated sparse point cloud model;

[0227] The sixth determination submodule 12035 is configured to determine the aggregated sparse point cloud model as the global sparse point cloud model in response to the absence of an unaggregated single sparse point cloud model in the single sparse point cloud models corresponding to each of the driving data sets.

[0228] In some embodiments, the generating submodule 12022 may further include:

[0229] An optimization unit, used for optimizing the three-dimensional information of the feature points of each of the image data based on a bundle adjustment method to obtain optimized three-dimensional information of the feature points of each of the image data;

[0230] The generating unit is further configured to generate a single sparse point cloud model corresponding to the driving data set according to the optimized three-dimensional information of the feature points of each of the image data.

[0231] The beneficial technical effects corresponding to the exemplary embodiment of the high-precision map construction device provided in the present disclosure can be found in the corresponding beneficial technical effects of the above-mentioned exemplary method part, which will not be repeated here.

[0232] Exemplary Electronic Devices

[0233] Fig.14 is a schematic diagram of the composition structure of an electronic device provided by an exemplary embodiment of the present disclosure, such as Fig.14 As shown, the electronic device 1400 includes at least one processor 1401 and a memory 1402 .

[0234] The processor 1401 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 1400 to perform desired functions.

[0235] The memory 1402 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on a computer-readable storage medium, and the processor 1401 may execute one or more computer program instructions to implement the high-precision map construction method and / or other desired functions of the various embodiments of the present disclosure described above.

[0236] In one example, the electronic device 1400 may further include: an input device 1403 and an output device 1404 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0237] The input device 1403 may also include, for example, a keyboard, a mouse, etc.

[0238] The output device 1404 can output various information to the outside, and may include, for example, a display, a speaker, a printer, a communication network and its connected remote output device, etc.

[0239] Of course, to simplify, Fig.14 Only some of the components related to the present disclosure in the electronic device 1400 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 1400 may also include any other appropriate components.

[0240] Exemplary computer program products and computer-readable storage media

[0241] In addition to the above-mentioned methods and devices, embodiments of the present disclosure may also provide a computer program product, including computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the method for constructing a high-precision map of various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section.

[0242] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0243] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by a processor, the processor executes the steps in the method for constructing a high-precision map of various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section.

[0244] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium is, for example, but not limited to, a system, device or device including electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0245] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and cannot be considered as necessary for each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, rather than limitation, and the above details do not limit the present disclosure to being implemented by adopting the above specific details.

[0246] Those skilled in the art may make various changes and modifications to the present disclosure without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present disclosure claims and their equivalents, the present disclosure is also intended to include these modifications and variations.

Claims

1. A method for constructing a high-precision map. include: Determine a plurality of driving data sets collected during the driving process of the vehicle; Determining a single sparse point cloud model corresponding to each of the driving data sets and an aggregation order of each of the driving data sets; Aggregating the single sparse point cloud models corresponding to each of the driving data sets based on the aggregation order to obtain a global sparse point cloud model; The high-precision map is generated based on the global sparse point cloud model.

2. The method according to claim 1, in, The determining of the single sparse point cloud model corresponding to each of the driving data sets and the aggregation order of each of the driving data sets includes: According to each of the driving data sets, determining the global positioning system data, the wheel speed meter data, the inertial unit data and the image data respectively collected by the multiple vehicle-mounted cameras provided on the vehicle in each of the driving data sets; Generate a single sparse point cloud model corresponding to each driving data set according to the global positioning system data, the wheel speed meter data, the inertial unit data and each image data in each driving data set; The aggregation order of each driving data set is determined according to the single sparse point cloud model corresponding to each driving data set.

3. The method according to claim 2, in, The step of generating a single sparse point cloud model corresponding to each driving data set according to the global positioning system data, the wheel speed meter data, the inertial unit data and each image data in each driving data set comprises: For each of the driving data sets, determining a feature point of each of the image data according to each of the image data in the driving data set; Determining the image pose of each of the image data according to the global positioning system data, the wheel speed meter data and the inertial unit data in the driving data set; Determining three-dimensional information of the feature points of each of the image data according to the feature points and the image pose of each of the image data; The single sparse point cloud model corresponding to the driving data set is generated according to the three-dimensional information of the feature points of each image data.

4. The method according to claim 3, in, The step of determining the image pose of each image data according to the global positioning system data, the wheel speed meter data and the inertial unit data in the driving data set comprises: Determining an optimized posture of the vehicle according to the global positioning system data, the wheel speed meter data and the inertial unit data in the driving data set; Determining the motion trajectory of each of the on-board cameras according to the optimized posture of the vehicle and the external parameter data of each of the on-board cameras; The image pose of each of the image data is determined according to the motion trajectory of each of the vehicle-mounted cameras.

5. The method according to claim 3, in, Determining the three-dimensional information of the feature points of each image data according to the feature points and the image pose of each image data includes: Determining matching image data for each of the image data according to the image pose of each of the image data; Performing feature point matching on each of the image data and the corresponding matching image data to obtain a matching relationship between some of the feature points in the feature points of each of the image data; Determining three-dimensional information of the partial feature points of each of the image data according to a matching relationship between the partial feature points of each of the image data; Determine, according to the three-dimensional information of the part of feature points of each of the image data, the three-dimensional information of the remaining feature points among the feature points of each of the image data except the part of feature points; The three-dimensional information of the feature points of each image data is determined according to the three-dimensional information of the part of feature points of each image data and the three-dimensional information of the remaining feature points among the feature points of each image data except the part of feature points.

6. The method according to claim 5, in, Determining matching image data for each of the image data according to the image pose of each of the image data comprises: Determining a plurality of candidate image data corresponding to the target image data from each of the image data according to the image pose of each of the image data; determining a common viewing area between the target image data and each of the candidate image data corresponding to the target image data; The matching image data of the target image data is determined according to the candidate image data that meets the preset condition in the common viewing area.

7. The method according to claim 2, in, The step of determining the aggregation order of each driving data set according to the single sparse point cloud model corresponding to each driving data set includes: Determining a first overlapping area between different driving data sets according to a single sparse point cloud model corresponding to each driving data set; An aggregation order of the driving data sets is determined according to the first overlapping areas between different driving data sets.

8. The method according to claim 1, in, The step of aggregating the single sparse point cloud models corresponding to each of the driving data sets based on the aggregation order to obtain a global sparse point cloud model includes: determining an aggregated sparse point cloud model; In response to the existence of unaggregated single sparse point cloud models in the single sparse point cloud models corresponding to each of the driving data sets, based on the aggregation order, determining the single sparse point cloud model to be currently aggregated from the unaggregated single sparse point cloud models; Determine a second overlapping area between the aggregated sparse point cloud model and the single sparse point cloud model to be aggregated; Based on the second overlapping area, aggregate the aggregated sparse point cloud model and the single sparse point cloud model to be aggregated, and update the aggregated sparse point cloud model; In response to the fact that there is no unaggregated single sparse point cloud model in the single sparse point cloud models corresponding to the driving data sets, the aggregated sparse point cloud model is determined as the global sparse point cloud model.

9. The method according to claim 3, in, The method further comprises: Optimizing the three-dimensional information of the feature points of each of the image data based on the bundle adjustment method to obtain optimized three-dimensional information of the feature points of each of the image data; Generating a single sparse point cloud model corresponding to the driving data set according to the three-dimensional information of the feature points of each of the image data includes: A single sparse point cloud model corresponding to the driving data set is generated according to the optimized three-dimensional information of the feature points of each of the image data.

10. A device for constructing a high-precision map, include: A first determination module is used to determine a plurality of driving data sets collected during the driving process of the vehicle; A second determination module is used to determine a single sparse point cloud model corresponding to each of the driving data sets and an aggregation order of each of the driving data sets; An aggregation module, used for aggregating the single sparse point cloud models corresponding to each of the driving data sets based on the aggregation order to obtain a global sparse point cloud model; A generation module is used to generate the high-precision map based on the global sparse point cloud model.

11. A computer-readable storage medium, wherein the storage medium stores a computer program, wherein the computer program is used to execute the method for constructing a high-precision map as described in any one of claims 1 to 9.

12. An electronic device, the electronic device include: processor; a memory for storing instructions executable by the processor; The processor is used to read the instructions from the memory and execute the instructions to implement the method for constructing a high-precision map as described in any one of claims 1-9 above.

Images