Map construction method and device, equipment and storage medium

Abstract

The application relates to a map construction method and device, equipment and a storage medium, and relates to the technical field of map construction. The method comprises the following steps: acquiring a plurality of initial positions of a vehicle, a plurality of collection images corresponding to the plurality of initial positions, a plurality of semantic images corresponding to the plurality of collection images, and a plurality of vehicle poses corresponding to the plurality of collection images, wherein one semantic image comprises a map element category corresponding to each pixel in the collection image corresponding to the semantic image. One semantic image is determined according to the collection image corresponding to the semantic image and a preset semantic model. Then, a plurality of target positions of the vehicle are determined according to the plurality of initial positions, the plurality of collection images, the plurality of semantic images, the plurality of vehicle poses and a preset map, and a target map is constructed according to the plurality of target positions, the plurality of collection images, the plurality of semantic images and the plurality of vehicle poses. The application can efficiently, accurately and at low cost construct the target map.

Description

Technical Field

[0001] This application relates to the field of map building technology, and more particularly to the field of autonomous driving technology, specifically to a map building method, apparatus, device, and storage medium. Background Technology

[0002] High-precision maps can provide centimeter-level road network information, intersection information, and traffic information such as urban arterial roads for autonomous driving, helping to improve the safety of autonomous driving. Currently, high-precision maps are typically constructed using two methods: 1. Building high-precision maps based on LiDAR: Using LiDAR to scan, analyze, and reconstruct the real-world scene to obtain a 3D point cloud map. Then, the 3D point cloud map is manually labeled to obtain a high-precision map containing semantic information. However, this method is labor-intensive and cannot efficiently and accurately construct maps. Furthermore, the cost of using LiDAR is high.

[0003] II. Building High-Precision Maps Based on Visual Sensors: Visual sensors are used to extract road features such as lane lines and traffic signs as semantic information, and triangulation techniques are used to obtain the positions of these road features relative to vehicles. Then, the vehicle's position and attitude are obtained using an inertial navigation system and real-time kinematic (RTK) carrier phase differential technology. Combining this semantic information with the positions of road features relative to vehicles, a high-precision map is constructed. However, RTK is costly, making low-cost map construction impossible. Summary of the Invention

[0004] This application provides a map building method, apparatus, device, and storage medium to at least solve the technical problems of existing technologies that cannot build maps with high accuracy, high efficiency, and low cost. The technical solution of this application is as follows:

[0005] According to a first aspect of this application, a map construction method is provided, comprising: acquiring multiple initial positions of a vehicle, multiple acquired images corresponding one-to-one with the multiple initial positions, multiple semantic images corresponding one-to-one with the multiple acquired images, and multiple vehicle poses corresponding one-to-one with the multiple acquired images. A semantic image includes a map feature category corresponding to each pixel in the acquired image corresponding to the semantic image. A semantic image is determined based on the acquired image corresponding to the semantic image and a preset semantic model. Multiple target positions of the vehicle are determined based on the multiple initial positions, multiple acquired images, multiple semantic images, multiple vehicle poses, and the preset map. A target map is constructed based on the multiple target positions, multiple acquired images, multiple semantic images, and multiple vehicle poses.

[0006] Based on the aforementioned technical means, this application can efficiently obtain the map feature category corresponding to each pixel in the acquired image, thereby improving the efficiency of constructing the target map. Then, based on multiple initial positions, multiple acquired images, multiple semantic images, multiple vehicle poses, and a preset map, multiple target positions of the vehicle are determined. Since the initial position of the vehicle obtained through the GPS receiver has errors, the preset map can correct the initial position, thereby obtaining a high-precision vehicle position. Furthermore, a target map is constructed based on multiple target positions, multiple acquired images, multiple semantic images, and multiple vehicle poses. In this way, the high-precision vehicle position allows for more accurate determination of the map feature positions, thus enabling the construction of a high-precision map. Simultaneously, the cost of using a GPS receiver to obtain the initial position is relatively low.

[0007] In one possible implementation, determining multiple target locations of a vehicle based on multiple initial locations, multiple acquired images, multiple semantic images, multiple vehicle poses, and a preset map includes: for each initial location, determining the position of the initial location in the preset map as a first location corresponding to the initial location; if the first location is located in a preset deviation region, acquiring a first acquired image. The first acquired image is an image of the region to which the first location belongs in the preset map. Obtaining the matching number between the first acquired image and the second acquired image. The second acquired image is the image corresponding to the initial location. If the matching number is greater than a preset number, determining the target location based on the initial location, the second acquired image, the acquisition position of the first acquired image, and the vehicle pose corresponding to the second acquired image. If the matching number is less than or equal to the preset number, updating the first acquired image until the matching number is greater than the preset number. If the first location is not located in the preset deviation region, determining the first location as the target location.

[0008] Based on the aforementioned technical means, this application can correct the initial position of a vehicle acquired by a GPS receiver using a preset map, thereby achieving accurate and low-cost acquisition of high-precision vehicle positions. In this way, a high-precision map can be constructed based on the vehicle's target position.

[0009] In one possible implementation, obtaining the matching count of the first and second acquired images includes: obtaining multiple first feature points and multiple second feature points. The multiple first feature points are feature points in the first acquired image whose grayscale values ​​satisfy a preset grayscale value condition, and the multiple second feature points are feature points in the second acquired image whose grayscale values ​​satisfy the preset grayscale value condition. For each first feature point, if the map feature category of the first feature point is the same as the map feature category of the third feature point, and the positions of the first feature point and the third feature point are within a preset range, then the first feature point and the third feature point are determined to be matched. The third feature point is any one of the multiple second feature points. The above steps are repeated until multiple first feature points are traversed to obtain the matching count of the first and second acquired images.

[0010] Based on the aforementioned technical means, this application can use map feature categories as matching filtering conditions, which can improve the accuracy and efficiency of the association matching of feature points in the first and second acquired images. Simultaneously, the number of matches between the first and second acquired images can be accurately determined based on the location of the feature points. This improves the efficiency of subsequently determining the target location of the vehicle based on the number of matches, further enhancing the efficiency of constructing the target map.

[0011] In one possible implementation, when the target map includes a road layer, the above-described construction of the target map based on multiple target locations, multiple acquired images, multiple semantic images, and multiple vehicle poses includes: for each acquired image, determining the position of each map element in the world coordinate system based on a second location, camera pose, the vehicle pose corresponding to the acquired image, and the transformation relationship between the camera coordinate system and the world coordinate system. The second location is the target location corresponding to the acquired image. Based on the position of each map element in the world coordinate system, performing a distance transformation on each map element to obtain a first distance map. Based on the position of each map element in a preset map, performing a distance transformation on each map element to obtain a second distance map. Aligning the first distance map and the second distance map at their edges, and constructing the road layer of the target map based on the position of each map element in the world coordinate system and the position of each map element in the preset map.

[0012] Based on the aforementioned technical means, this application can supplement potentially missing map elements by using map elements in a preset map. This allows for a more accurate construction of the road layer in the target map.

[0013] In one possible implementation, when the target map includes a lane line layer and a traffic sign layer, the above-described construction of the target map based on multiple target locations, multiple acquired images, multiple semantic images, and multiple vehicle poses includes: for each acquired image, determining the position of each map feature in the acquired image in the camera coordinate system based on a second position, camera pose, and the acquired image. The second position is the target location corresponding to the acquired image. The position of each map feature in the acquired image in the world coordinate system is determined based on the vehicle pose corresponding to the acquired image and the transformation relationship between the camera coordinate system and the world coordinate system. The lane line layer of the target map is constructed based on the position of each map feature in the acquired image in the world coordinate system and the semantic image corresponding to the acquired image. The traffic sign layer of the target map is constructed based on the position of each map feature in the acquired image in the world coordinate system and the semantic image corresponding to the acquired image.

[0014] Based on the aforementioned technical means, this application can transform map features from a pixel coordinate system to a camera coordinate system using triangulation technology. Then, through the transformation relationship between the camera coordinate system and the world coordinate system, map features can be transformed from the camera coordinate system to the world coordinate system. This allows for the efficient construction of lane line layers and traffic sign layers in the target map.

[0015] In one possible implementation, before constructing the lane line layer of the target map based on the position of each map feature in the world coordinate system and the semantic image corresponding to the acquired image, the method further includes: determining a segmented image corresponding to each map feature in the acquired image based on the semantic image corresponding to the acquired image; determining a map feature marker corresponding to each map feature based on the segmented image corresponding to each map feature and multiple preset marker images; and determining the geometric structure information of each map feature based on the geometric structure information of the map feature marker corresponding to each map feature. Constructing the lane line layer of the target map based on the position of each map feature in the world coordinate system and the semantic image corresponding to the acquired image includes: constructing the lane line layer of the target map based on the position of each map feature in the world coordinate system, the semantic image corresponding to the acquired image, and the geometric structure information of each map feature.

[0016] Based on the aforementioned technical means, this application can obtain the geometric structure information of map elements. Since geometric structure information can more accurately represent map elements, the lane line layer constructed based on the geometric structure information is more accurate.

[0017] In one possible implementation, when the target map includes a dynamic object layer, the above-described construction of the target map based on multiple target locations, multiple acquired images, multiple semantic images, and multiple vehicle poses includes: inputting multiple semantic images into a target tracking model to obtain the predicted position of each of the multiple first map features in the multiple semantic images. The target tracking model is used to identify the same map feature in the multiple semantic images and predict the position of the same map feature in the next semantic image. The first map feature is the same map feature in the multiple semantic images. Based on the multiple semantic images and the predicted position of each of the multiple first map features in the multiple semantic images, the position of each first map feature in the multiple semantic images is determined to be multiple third positions corresponding to each first map feature, thereby obtaining multiple third positions corresponding to each of the multiple first map features. For each acquired image, based on the multiple third positions corresponding to each of the multiple first map features, the second position, the camera pose, the vehicle pose corresponding to the acquired image, and the transformation relationship between the camera coordinate system and the world coordinate system, the position of each map feature in the acquired image in the world coordinate system is determined. The second position is the target location corresponding to the acquired image. Based on the position of each map feature in the world coordinate system in the acquired image and the semantic image corresponding to the acquired image, a dynamic object layer in the target map is constructed.

[0018] Based on the above technical means, this application combines the predicted location of map elements with the location of map elements in the acquired image, which can more accurately determine the location of map elements and further improve the accuracy of constructing the dynamic object layer in the target map.

[0019] According to a second aspect of this application, a map building apparatus is provided, comprising an acquisition unit, a determination unit, and a building unit. The acquisition unit is configured to acquire multiple initial positions of a vehicle, multiple acquired images corresponding one-to-one with the multiple initial positions, multiple semantic images corresponding one-to-one with the multiple acquired images, and multiple vehicle poses corresponding one-to-one with the multiple acquired images. Each semantic image includes a map feature category corresponding to each pixel in the acquired image corresponding to the semantic image. A semantic image is determined based on the acquired image corresponding to the semantic image and a preset semantic model. The determination unit is configured to determine multiple target positions of the vehicle based on the multiple initial positions, multiple acquired images, multiple semantic images, multiple vehicle poses, and a preset map. The building unit is configured to build a target map based on the multiple target positions, multiple acquired images, multiple semantic images, and multiple vehicle poses.

[0020] In one possible implementation, the determining unit is specifically configured to: for each initial position, determine the position of the initial position in a preset map as the first position corresponding to the initial position; if the first position is located in a preset deviation area, acquire a first acquired image. The first acquired image is an acquired image of the area to which the first position belongs in the preset map; acquire the matching number between the first acquired image and the second acquired image. The second acquired image is the acquired image corresponding to the initial position; if the matching number is greater than a preset number, determine a target position based on the initial position, the second acquired image, the acquisition position of the first acquired image, and the vehicle pose corresponding to the second acquired image; if the matching number is less than or equal to the preset number, update the first acquired image until the matching number is greater than the preset number; if the first position is not located in the preset deviation area, determine the first position as the target position.

[0021] In one possible implementation, the determining unit is specifically used to: acquire a plurality of first feature points and a plurality of second feature points. The plurality of first feature points are feature points in the first acquired image whose grayscale values ​​satisfy a preset grayscale value condition, and the plurality of second feature points are feature points in the second acquired image whose grayscale values ​​satisfy the preset grayscale value condition. For each first feature point, if the map feature category of the first feature point is the same as the map feature category of the third feature point, and the positions of the first feature point and the third feature point are within a preset range, then the first feature point and the third feature point are determined to match. The third feature point is any one of the plurality of second feature points. The above steps are repeated until the plurality of first feature points are traversed, obtaining the number of matches between the first acquired image and the second acquired image.

[0022] In one possible implementation, when the target map includes a road layer, the aforementioned construction unit is specifically used to: for each acquired image, determine the position of each map element in the world coordinate system based on a second position, camera pose, vehicle pose corresponding to the acquired image, and the transformation relationship between the camera coordinate system and the world coordinate system. The second position is the target position corresponding to the acquired image. Based on the position of each map element in the world coordinate system, perform a distance transformation on each map element to obtain a first distance map. Based on the position of each map element in a preset map, perform a distance transformation on each map element to obtain a second distance map. Align the first distance map and the second distance map at their edges, and construct the road layer of the target map based on the position of each map element in the world coordinate system and the position of each map element in the preset map.

[0023] In one possible implementation, when the target map includes a lane line layer and a traffic sign layer, the aforementioned construction unit is specifically used for: for each acquired image, determining the position of each map feature in the acquired image in the camera coordinate system based on a second position, camera pose, and the acquired image. The second position is the target position corresponding to the acquired image. Based on the vehicle pose corresponding to the acquired image and the transformation relationship between the camera coordinate system and the world coordinate system, determining the position of each map feature in the acquired image in the world coordinate system. Based on the position of each map feature in the acquired image in the world coordinate system and the semantic image corresponding to the acquired image, constructing the lane line layer of the target map. Based on the position of each map feature in the acquired image in the world coordinate system and the semantic image corresponding to the acquired image, constructing the traffic sign layer of the target map.

[0024] In one possible implementation, the determining unit is further configured to determine a segmented image corresponding to each map feature in the acquired image based on the semantic image corresponding to the acquired image. The determining unit is further configured to determine a map feature marker corresponding to each map feature based on the segmented image corresponding to each map feature and multiple preset marker images. The determining unit is further configured to determine the geometric structure information of each map feature based on the geometric structure information of the map feature marker corresponding to each map feature. Specifically, the construction unit is configured to construct a lane line layer of the target map based on the position of each map feature in the world coordinate system, the semantic image corresponding to the acquired image, and the geometric structure information of each map feature.

[0025] In one possible implementation, when the target map includes a dynamic object layer, the aforementioned construction unit is specifically used to: input multiple semantic images into a target tracking model to obtain the predicted position of each of the multiple first map features in the multiple semantic images. The target tracking model is used to identify the same map feature in the multiple semantic images and predict the position of the same map feature in the next semantic image. The first map feature is the same map feature in the multiple semantic images. Based on the multiple semantic images and the predicted position of each of the multiple first map features in the multiple semantic images, the position of each first map feature in the multiple semantic images is determined to be multiple third positions corresponding to each first map feature, so as to obtain multiple third positions corresponding to each of the multiple first map features. For each acquired image, based on the multiple third positions corresponding to each of the multiple first map features, the second position, the camera pose, the vehicle pose corresponding to the acquired image, and the transformation relationship between the camera coordinate system and the world coordinate system, the position of each map feature in the acquired image in the world coordinate system is determined. The second position is the target position corresponding to the acquired image. Based on the position of each map feature in the acquired image in the world coordinate system and the semantic image corresponding to the acquired image, a dynamic object layer in the target map is constructed.

[0026] According to a third aspect provided in this application, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions. The processor is configured to execute instructions to implement the methods described in the first aspect and any possible embodiments thereof.

[0027] According to a fourth aspect provided in this application, a computer-readable storage medium is provided that, when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to perform the method described in the first aspect and any possible implementation thereof.

[0028] According to the fifth aspect provided in this application, a computer program product is provided, the computer program product including computer instructions, which, when executed on an electronic device, cause the electronic device to perform the method described in the first aspect and any possible implementation thereof.

[0029] Therefore, the above-mentioned technical features of this application have the following beneficial effects:

[0030] (1) By acquiring images and using a pre-set semantic model, the map feature category corresponding to each pixel in the acquired images can be efficiently obtained, thereby improving the efficiency of constructing the target map. Then, based on multiple initial locations, multiple acquired images, multiple semantic images, multiple vehicle poses, and the pre-set map, multiple target locations of the vehicle are determined. Since the initial location of the vehicle obtained through the GPS receiver has errors, the pre-set map can be used to correct the initial location, thus obtaining a high-precision vehicle location. Further, based on multiple target locations, multiple acquired images, multiple semantic images, and multiple vehicle poses, a target map is constructed. In this way, the high-precision vehicle location allows for more accurate determination of the map feature locations, thereby enabling the construction of a high-precision map. Simultaneously, the cost of acquiring the initial location using a GPS receiver is relatively low.

[0031] (2) Since GPS receivers have positioning errors, the initial position of the vehicle obtained by the GPS receiver can be corrected by using a preset map, thereby achieving accurate and low-cost acquisition of high-precision vehicle positions. In this way, a high-precision map can be constructed based on the target position of the vehicle.

[0032] (3) Using map feature categories as matching filtering conditions can improve the accuracy and efficiency of the association matching of feature points in the first and second acquired images. Simultaneously, the number of matches between the first and second acquired images can be accurately determined based on the location of the feature points. This improves the efficiency of subsequently determining the target location of the vehicle based on the number of matches, further enhancing the efficiency of constructing the target map.

[0033] (4) Since map features in the road layer determined by the acquired images may be missing, map features in the preset map can be used to supplement any missing map features. In this way, the road layer in the target map can be constructed more accurately.

[0034] (5) Triangulation technology can be used to transform map features from pixel coordinates to camera coordinates. Then, by using the transformation relationship between camera coordinates and world coordinates, map features can be transformed from camera coordinates to world coordinates. In this way, lane line layers and traffic sign layers in the target map can be constructed efficiently.

[0035] (6) It can obtain the geometric structure information of map elements. Since the geometric structure information can represent map elements more accurately, the lane line layer constructed based on the geometric structure information is more accurate.

[0036] (7) The target tracking model can track the same map feature in multiple semantic images and predict the location of the map feature in multiple semantic images. In this way, by combining the predicted location of the map feature with its location in the acquired image, the location of the map feature can be determined more accurately, further improving the accuracy of constructing the dynamic object layer in the target map.

[0037] It should be noted that the technical effects of any of the implementation methods in aspects two through five can be found in the technical effects of the corresponding implementation methods in aspect one, and will not be repeated here.

[0038] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0039] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application, and do not constitute an undue limitation of this application.

[0040] Figure 1 This is a schematic diagram illustrating the structure of a map building system according to an exemplary embodiment;

[0041] Figure 2 This is a flowchart illustrating a map construction method according to an exemplary embodiment;

[0042] Figure 3 This is a flowchart illustrating yet another map construction method according to an exemplary embodiment;

[0043] Figure 4 This is a schematic diagram illustrating the classification of a target map according to an exemplary embodiment;

[0044] Figure 5 This is a block diagram illustrating a map building apparatus according to an exemplary embodiment;

[0045] Figure 6 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0046] To enable those skilled in the art to better understand the technical solutions of this application, the technical solutions in the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings.

[0047] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0048] Before providing a detailed description of the map construction method provided in this application, let's briefly introduce the existing technologies related to this application.

[0049] Autonomous driving technology is a collaborative system that relies on the integration of perception, decision-making, planning, and control. Its implementation requires obtaining more accurate vehicle positioning information and richer static and dynamic road environment information. Ordinary electronic navigation maps can no longer meet the needs of autonomous driving.

[0050] High-precision maps serve as a crucial infrastructure for realizing autonomous driving, playing a vital role in vehicle positioning, planning, and control. They provide centimeter-level road network information, accurately depicting scenarios with dense traffic elements such as complex intersections and urban arterial roads, thus providing a fundamental guarantee for vehicle beyond-line-of-sight perception and precise positioning.

[0051] Before providing a detailed introduction to the map construction method provided in this application, let's first briefly introduce the implementation environment (implementation architecture) involved in this application.

[0052] Figure 1 A schematic diagram of a map building system is shown. Figure 1As shown, the map building system 10 includes a map building device 11 and an electronic device 12. The map building device 11 is connected to the electronic device 12. The map building device 11 and the electronic device 12 can be connected by wired or wireless means, and this embodiment does not limit the connection.

[0053] The map building device 11 can be used to interact with the electronic device 12, for example, to obtain multiple initial positions of the vehicle, multiple acquired images corresponding to the multiple initial positions, multiple semantic images corresponding to the multiple acquired images, and multiple vehicle poses corresponding to the multiple acquired images from the electronic device 12, and send the target map to the electronic device 12.

[0054] The map building device 11 can also be used to process multiple initial positions of the vehicle, multiple acquired images corresponding to the multiple initial positions, multiple semantic images corresponding to the multiple acquired images, and multiple vehicle poses corresponding to the multiple acquired images. For example, based on the multiple initial positions, multiple acquired images, multiple semantic images, multiple vehicle poses, and a preset map, multiple target positions of the vehicle are determined. A target map is then constructed based on the multiple target positions, multiple acquired images, multiple semantic images, and multiple vehicle poses.

[0055] The electronic device 12 can be used to interact with the map building device 11, for example, to send a target map to the map building device 11 and to receive multiple initial positions of vehicles, multiple acquired images corresponding to the multiple initial positions, multiple semantic images corresponding to the multiple acquired images, and multiple vehicle poses corresponding to the multiple acquired images sent by the map building device 11.

[0056] Optionally, the electronic device can be a physical machine, such as a desktop computer, mobile phone, tablet computer, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), and other terminal devices. The electronic device can also be a server or a server cluster composed of multiple servers.

[0057] Optionally, the map building device 11 described above can also achieve the functions to be performed by a virtual machine (VM) deployed on a physical machine.

[0058] It should be noted that the map building device 11 and the electronic device 12 can be independent devices or integrated into the same device; this application does not make any specific limitation in this regard.

[0059] When the map building device 11 and the electronic device 12 are integrated into the same device, the communication method between the map building device 11 and the electronic device 12 is the same as the communication method between the modules within the device. In this case, the communication process between the two is the same as the communication process between the map building device 11 and the electronic device 12 when they are independent of each other.

[0060] In the following embodiments provided in this application, the map building apparatus 11 and the electronic device 12 are described as being configured independently of each other.

[0061] For ease of understanding, the map construction method provided in this application will be described in detail below with reference to the accompanying drawings.

[0062] Figure 2 This is a flowchart illustrating a map construction method according to an exemplary embodiment. For example... Figure 2 As shown, the map construction method includes the following steps:

[0063] S201, The electronic device acquires multiple initial positions of the vehicle, multiple acquired images corresponding to the multiple initial positions, multiple semantic images corresponding to the multiple acquired images, and multiple vehicle poses corresponding to the multiple acquired images.

[0064] A semantic image includes the map feature category corresponding to each pixel in the acquired image. A semantic image is determined based on the acquired image corresponding to the semantic image and a preset semantic model.

[0065] As one possible implementation, the electronic device sets the acquisition frequency of the vehicle-mounted camera to a preset frequency and the resolution of the acquired images to a preset resolution. Then, the electronic device calibrates the vehicle-mounted camera using the Zhang Zhengyou calibration method, obtaining the intrinsic parameter matrix, extrinsic parameter matrix, and distortion coefficients of the vehicle-mounted camera. Next, based on the intrinsic parameter matrix, the electronic device obtains the projection relationship of the vehicle-mounted camera from the camera coordinate system to the pixel coordinate system, and based on the extrinsic parameter matrix, obtains the transformation relationship between the camera coordinate system and the world coordinate system.

[0066] Then, the electronic device acquires multiple initial positions collected by the Global Positioning System (GPS) receiver and captures images from the vehicle-mounted camera at each initial position, resulting in multiple captured images corresponding to each initial position. Next, the electronic device performs distortion correction on the multiple captured images using distortion coefficients and a pinhole camera model, resulting in distortion-corrected captured images. Furthermore, the electronic device inputs the distortion-corrected captured images into a semantic segmentation model to obtain multiple semantic images corresponding to each captured image.

[0067] Furthermore, the electronic device uses the inertial measurement unit (IMU) toolkit to calibrate the axisymmetric error, scale factor, temperature drift, noise, and zero bias. Then, the electronic device acquires the vehicle pose measured by the IMU at the initial position, obtaining multiple vehicle poses corresponding one-to-one with the acquired images. Next, the electronic device performs multi-sensor calibration on the vehicle-mounted camera and the IMU using the visual-inertial navigation calibration toolkit, obtaining the relative positional relationship between the vehicle-mounted camera and the IMU.

[0068] For example, the preset frequency can be 25 frames per second, and the preset resolution can be 1280*720. The pinhole camera model can be a pinhole camera model from an open-source computer vision library (OpenCV).

[0069] For example, the distortion coefficients and pinhole camera model satisfy the following expression:

[0070]

[0071] Where x and y are pixel coordinates in the pixel coordinate system, the origin of the pixel coordinate system is the upper left corner of the acquired image, radial distortion coefficients k1, k2, and k3 are the first three coefficients of the Taylor series expansion around the principal point, tangential distortion coefficients p1 and p2, r is the radial distance, and x... distorted and y distorted These are the pixel coordinates after distortion correction.

[0072] For example, when the coordinates of the vehicle-mounted camera are rotated by α degrees around the X-axis, β degrees around the Y-axis, and γ degrees around the Z-axis, the transformation relationship between the camera coordinate system and the world coordinate system satisfies the following expression:

[0073]

[0074] Where x, y, and z are coordinates in the camera coordinate system, and x′, y′, and z′ are coordinates in the world coordinate system.

[0075] For example, axis deflection angle error, scale factor, and temperature drift are deterministic errors, while noise and zero bias are random errors.

[0076] For example, vehicle pose includes: acceleration, angular velocity, and orientation. The inertial measurement unit (IMU) can be a six-axis IMU, which includes a three-axis accelerometer and a three-axis gyroscope. The accelerometer is used to acquire acceleration signals, and the gyroscope is used to acquire angular velocity signals.

[0077] For example, the electronic device uses a visual inertial navigation calibration toolkit to perform multi-sensor calibration on the vehicle-mounted camera and the inertial measurement unit to obtain the relative positional relationship between the vehicle-mounted camera and the inertial measurement unit.

[0078] For example, the images captured by the vehicle-mounted camera include road surface features, sky features, and vehicle features.

[0079] For example, a semantic segmentation model can stitch together and cluster similar pixels in a captured image to obtain the outline of each map feature in the captured image.

[0080] For example, a GPS receiver can obtain the initial position of a vehicle using differential positioning technology.

[0081] S202. The electronic device determines multiple target positions of the vehicle based on multiple initial positions, multiple acquired images, multiple semantic images, multiple vehicle poses, and a preset map.

[0082] As one possible implementation, the electronic device determines the position of each initial position on a preset map as the first position corresponding to the initial position. Then, if the first position is within a preset deviation area, the electronic device acquires a first image. Next, the electronic device obtains the number of matches between the first and second acquired images.

[0083] Then, if the number of matches exceeds a preset number, the electronic device determines the target position based on the initial position, the second acquired image, the acquisition position of the first acquired image, and the vehicle pose corresponding to the second acquired image. Further, if the number of matches is less than or equal to a preset number, the electronic device updates the first acquired image until the number of matches exceeds the preset number. If the first position is not located within a preset deviation area, the electronic device determines the first position as the target position.

[0084] The specific implementation method of this step can be referred to in the subsequent description of the embodiments of this application, and will not be repeated here.

[0085] S203. The electronic device constructs a target map based on multiple target locations, multiple acquired images, multiple semantic images, and multiple vehicle poses.

[0086] As one possible implementation, for each acquired image, the electronic device determines the position of each map feature in the world coordinate system based on the second position, camera pose, vehicle pose corresponding to the acquired image, and the transformation relationship between the camera coordinate system and the world coordinate system. Then, based on the position of each map feature in the world coordinate system, the electronic device performs a distance transformation on each map feature to obtain a first distance map. Next, based on the position of each map feature in a preset map, the electronic device performs a distance transformation on each map feature to obtain a second distance map. Finally, the electronic device aligns the edges of the first and second distance maps and constructs the road layer of the target map based on the positions of each map feature in the world coordinate system and the positions of each map feature in the preset map.

[0087] Furthermore, for each acquired image, the electronic device determines the position of each map feature in the camera coordinate system based on the second position, camera pose, and the acquired image itself. Then, based on the vehicle pose corresponding to the acquired image and the transformation relationship between the camera coordinate system and the world coordinate system, the electronic device determines the position of each map feature in the world coordinate system. Next, based on the position of each map feature in the world coordinate system and the corresponding semantic image, the electronic device constructs the lane line layer in the target map. Finally, based on the position of each map feature in the world coordinate system and the corresponding semantic image, the electronic device constructs the traffic sign layer in the target map.

[0088] Furthermore, the electronic device inputs multiple semantic images into the target tracking model to obtain the predicted position of each first map feature in the multiple semantic images. Then, based on the multiple semantic images and the predicted positions of each first map feature in the multiple semantic images, the electronic device determines the position of each first map feature in the multiple semantic images as multiple third positions. Next, for each acquired image, the electronic device determines the position of each map feature in the world coordinate system based on the multiple third positions, second positions, camera pose, the vehicle pose corresponding to the acquired image, and the transformation relationship between the camera coordinate system and the world coordinate system.

[0089] It should be noted that the second location is the target location corresponding to the acquired image. The target tracking model is used to identify the same map feature in multiple semantic images and predict the location of the same map feature in the next semantic image. The first map feature is the same map feature in multiple semantic images.

[0090] The specific implementation method of this step can be referred to in the subsequent description of the embodiments of this application, and will not be repeated here.

[0091] Understandably, existing technologies for building high-precision maps using LiDAR require manual annotation of semantic information, resulting in low efficiency and poor accuracy. Furthermore, existing technologies using visual sensors and RTK to build high-precision maps suffer from high costs. This application acquires multiple initial positions of the vehicle, multiple acquired images corresponding to each initial position, multiple semantic images corresponding to each acquired image, and multiple vehicle poses corresponding to each acquired image. Each semantic image includes the map feature category corresponding to each pixel in the acquired image. A semantic image is determined based on the acquired image corresponding to it and a preset semantic model. Thus, by using the acquired image and the preset semantic model, the map feature category corresponding to each pixel in the acquired image can be efficiently obtained, thereby improving the efficiency of constructing the target map.

[0092] Subsequently, multiple target locations of the vehicle are determined based on multiple initial positions, multiple acquired images, multiple semantic images, multiple vehicle poses, and a pre-set map. Since the initial vehicle position obtained through the GPS receiver has errors, the pre-set map can correct for these errors, resulting in a high-precision vehicle position. Furthermore, a target map is constructed based on the multiple target locations, multiple acquired images, multiple semantic images, and multiple vehicle poses. In this way, the high-precision vehicle position allows for more accurate positioning of map features determined from the vehicle's location, enabling the construction of a high-precision map. Simultaneously, using a GPS receiver to obtain the initial position is relatively inexpensive.

[0093] In some embodiments, in order to accurately determine the target location of the vehicle, the above-mentioned S202 can be implemented in the following manner:

[0094] S301. For each initial position, the electronic device determines the position of the initial position in the preset map as the first position corresponding to the initial position.

[0095] For example, the first location includes: longitude, latitude, and altitude.

[0096] S302. When the first position is located in the preset deviation area, the electronic device acquires the first image.

[0097] The first acquired image is the acquired image of the area to which the first location belongs in the preset map.

[0098] For example, the preset deviation area can be the area where the double yellow lines, yellow lines, and white solid lines are located in the preset map.

[0099] For example, the first captured image may be a street view.

[0100] For example, when the first location is located within the double yellow lines on a preset map, the electronic device determines the image of the area to which the first location belongs in the preset map as the first captured image.

[0101] S303, The electronic device acquires the number of matches between the first acquired image and the second acquired image.

[0102] The second acquired image is the acquired image corresponding to the initial position.

[0103] As one possible implementation, the electronic device acquires multiple first feature points and multiple second feature points. Then, for each first feature point, the electronic device determines a match between the first feature point and the third feature point if the map feature category of the first feature point is the same as that of the third feature point, and the positions of the first and third feature points are within a preset range. The electronic device then repeats the above steps until multiple first feature points are traversed, obtaining the number of matches between the first and second acquired images.

[0104] The specific implementation method of this step can be referred to in the subsequent description of the embodiments of this application, and will not be repeated here.

[0105] S304. When the number of matched objects is greater than the preset number, the electronic device determines the target position based on the initial position, the second acquired image, the acquisition position of the first acquired image, and the vehicle pose corresponding to the second acquired image.

[0106] As one possible implementation, when the number of matches exceeds a preset number, the electronic device determines the target position based on the initial position, the second acquired image, the first acquired image, the camera pose, the acquisition position of the first acquired image, the vehicle pose corresponding to the second acquired image, and the transformation relationship between the camera coordinate system and the world coordinate system.

[0107] For example, the camera pose includes the height and angle at which the vehicle-mounted camera captures the image.

[0108] S305. When the number of matches is less than or equal to the preset number, the electronic device updates the first acquired image until the number of matches is greater than the preset number.

[0109] S306. If the electronic device is not located in the preset deviation area at the first position, it determines the first position as the target position.

[0110] Understandably, due to positioning errors in GPS receivers, a pre-set map can be used to correct the initial vehicle position acquired by the GPS receiver, thereby achieving accurate and low-cost high-precision vehicle location acquisition. This allows for the construction of a high-precision map based on the vehicle's target location.

[0111] In some embodiments, to accurately obtain the number of matches, the above-mentioned S303 can be implemented in the following manner:

[0112] S401, The electronic device acquires multiple first feature points and multiple second feature points.

[0113] Among them, multiple first feature points are feature points in the first acquired image whose gray values ​​meet preset gray value conditions, and multiple second feature points are feature points in the second acquired image whose gray values ​​meet preset gray value conditions.

[0114] As one possible implementation, the electronic device inputs a first acquired image into a corner detection algorithm to obtain multiple first feature points. Then, the electronic device inputs a second acquired image into the corner detection algorithm to obtain multiple second feature points.

[0115] For example, the preset grayscale conditions include: the grayscale value of the feature point is the maximum grayscale value, or the grayscale value of the feature point is the minimum grayscale value, or the grayscale value of the feature point is the minimum grayscale value among seven or eight consecutive pixels.

[0116] For example, multiple first feature points can be visual features of a first acquired image, and multiple second feature points can be visual features of a second acquired image.

[0117] S402. For each first feature point, the electronic device determines that the first feature point and the third feature point are matched if the map feature category of the first feature point and the map feature category of the third feature point are the same and the positions of the first feature point and the third feature point are within a preset range.

[0118] The third feature point is any one of the multiple second feature points.

[0119] As one possible implementation, the electronic device inputs a first acquired image into a semantic segmentation model to obtain a semantic image corresponding to the first acquired image. Then, based on the semantic image corresponding to the first acquired image, the electronic device determines the map feature category of each of a plurality of first feature points. Next, based on the semantic image corresponding to a second acquired image, the electronic device determines the map feature category of each of a plurality of second feature points.

[0120] Next, for each first feature point, the electronic device determines any one of the multiple second feature points as the third feature point. Then, the electronic device checks if the road pixel categories of the first and third feature points are the same. If they are the same, it determines the position of each pixel in the second acquired image in the camera coordinate system based on the second acquired image and the camera pose when the vehicle-mounted camera acquired the second image. Finally, the electronic device determines the position of the third feature point based on the position of each pixel in the second acquired image in the camera coordinate system.

[0121] Furthermore, the electronic device determines whether the positions of the first feature point and the third feature point are within a preset range. If the positions of the first feature point and the third feature point are within the preset range, the electronic device determines that the first feature point and the third feature point match. If the road pixel categories of the first feature point and the third feature point are not the same, or if the positions of the first feature point and the third feature point are not within the preset range, the electronic device determines the third feature point from the remaining multiple second feature points, until multiple second feature points are traversed.

[0122] For example, map element categories include: traffic signs and billboards.

[0123] For example, the position of the first feature point is the position of the pixel of the first feature point in the camera coordinate system in the first acquired image.

[0124] For example, the position in the camera coordinate system can constitute the vector data of map features.

[0125] S403. The electronic device repeats the above steps until it traverses multiple first feature points to obtain the number of matches between the first acquired image and the second acquired image.

[0126] As one possible implementation, the electronic device determines the number of matches between the first feature point and the third feature point as the number of matches between the first acquired image and the second acquired image.

[0127] As another possible implementation, the electronic device determines the matching quantity weight as a first weight when the map feature category of the first feature point is a first map feature category. Then, the electronic device determines the matching quantity weight as a second weight when the map feature category of the third feature point is a second map feature category.

[0128] Next, the electronic device determines a first value by multiplying the first weight by the number of matching map feature categories. It then determines a second value by multiplying the second weight by the number of matching map feature categories. Further, the electronic device sums the first and second values ​​to determine the number of matching pairs between the first and second acquired images.

[0129] For example, when the first map feature category is fixed landmarks and the second map feature category is moving objects, the first weight is greater than the second weight.

[0130] Understandably, using map feature categories as matching filters can improve the accuracy and efficiency of associating feature points in the first and second acquired images. Simultaneously, the location of feature points allows for the accurate determination of the number of matches between the first and second acquired images. This improves the efficiency of subsequently determining the vehicle's target location based on the number of matches, further enhancing the efficiency of constructing the target map.

[0131] In some embodiments, in order to construct the road layer in the target map, S203 above can be implemented in the following manner:

[0132] S501. For each acquired image, the electronic device determines the position of each map element in the world coordinate system based on the second position, camera pose, vehicle pose corresponding to the acquired image, and the transformation relationship between the camera coordinate system and the world coordinate system.

[0133] The second position is the target position corresponding to the acquired image.

[0134] As one possible implementation, for each acquired image, the electronic device determines the position of each map feature in the camera coordinate system based on the second position, camera pose, and the acquired image itself. Then, based on the vehicle pose corresponding to the acquired image, the transformation relationship between the camera coordinate system and the world coordinate system, and the position of each map feature in the camera coordinate system, the electronic device determines the position of each map feature in the world coordinate system.

[0135] S502. The electronic device performs distance transformation on each map element according to its position in the world coordinate system to obtain the first distance map.

[0136] S503. The electronic device performs distance transformation on each map element according to its position in the preset map to obtain a second distance map.

[0137] As one possible implementation, the electronic device acquires raw road structure data provided by a map provider. Then, based on the location of each map feature in a preset map and a specific data interface, the electronic device performs distance transformation on each map feature to obtain a second distance map.

[0138] S504. The electronic device aligns the edges of the first distance map and the second distance map, and constructs the road layer of the target map based on the position of each map element in the world coordinate system and the position of each map element in the preset map.

[0139] As one possible implementation, the electronic device aligns the edges of the first distance map with the second distance map and constructs the road layer of the target map based on the position of each map feature in the world coordinate system, the position of each map feature in the preset map, and the orientation of each map feature in the preset map.

[0140] For example, the road layer includes road boundary topology information and intersection topology information.

[0141] For example, the map feature categories for the road layer include: road edge lines and road center lines.

[0142] Understandably, since map features in the road layer determined by image acquisition may be missing, pre-setting map features can supplement any missing elements. This allows for a more accurate construction of the road layer in the target map.

[0143] In some embodiments, to construct the lane line layer and traffic sign layer in the target map, the above-mentioned S203 can be implemented in the following manner:

[0144] S601. For each acquired image, the electronic device determines the position of each map feature in the camera coordinate system based on the second position, camera pose, and the acquired image.

[0145] The second position is the target position corresponding to the acquired image.

[0146] As one possible implementation, the electronic device determines the position of each map feature in the camera coordinate system in each acquired image based on the second position, camera pose, acquired image, and triangulation technique.

[0147] For example, the triangulation technique satisfies the following expression:

[0148] s2x2=s1Rx1+t

[0149]

[0150] Where (x1, x2) are the coordinates of the pixels in the acquired image in the pixel coordinate system, s1 is the first scale, s2 is the second scale, R is the rotation matrix, t is the translation matrix, and ∧ is the left multiplication sign.

[0151] For example, the electronic device uses the least squares method to solve for s1 and s2, and determines the depth information of the pixel. Then, the electronic device determines the distance between the pixel and the vehicle based on the pixel's depth information.

[0152] S602. The electronic device determines the position of each map element in the acquired image in the world coordinate system based on the vehicle pose corresponding to the acquired image and the transformation relationship between the camera coordinate system and the world coordinate system.

[0153] For example, the position of a map feature in the world coordinate system includes the three-dimensional information of the map feature in the world coordinate system, which includes: road planar information and aerial feature information.

[0154] S603. The electronic device constructs the lane line layer of the target map based on the position of each map element in the world coordinate system in the acquired image and the semantic image corresponding to the acquired image.

[0155] For example, the lane line layer includes: lane edge lines, guide lines, and other actual lane data.

[0156] S604. The electronic device constructs the traffic sign layer of the target map based on the position of each map element in the world coordinate system and the semantic image corresponding to the acquired image.

[0157] For example, traffic sign layers include: ground-based text arrows, stop lines, traffic signs, guardrails, pedestrian crossings, and other ground and air traffic signs.

[0158] Understandably, triangulation can transform map features from a pixel coordinate system to a camera coordinate system. Then, using the transformation relationship between the camera and world coordinate systems, map features can be converted from the camera coordinate system to the world coordinate system. This allows for the efficient construction of lane line and traffic sign layers in the target map.

[0159] In some embodiments, in order to construct the lane line layer in the target map, the method provided in this application further includes:

[0160] S701. The electronic device determines the segmented image corresponding to each map element in the acquired image based on the semantic image corresponding to the acquired image.

[0161] As one possible implementation, the electronic device uses dividing lines to segment the semantic image corresponding to the acquired image, thereby obtaining a segmented image corresponding to each map feature in the acquired image.

[0162] S702. The electronic device determines the map feature marker corresponding to each map feature based on the segmented image corresponding to each map feature and multiple preset marker images.

[0163] As one possible implementation, the electronic device adjusts the segmented image corresponding to each map feature to a preset size. Then, the electronic device acquires multiple preset sign images from a road sign database and determines the similarity between the segmented image corresponding to the map feature and each of the multiple preset sign images using a template matching algorithm. Finally, the electronic device identifies the preset sign images with a similarity greater than a preset similarity as the map feature icons corresponding to the map features.

[0164] The template matching algorithm satisfies the following expression:

[0165]

[0166] Among them, S i Let X be the similarity between the segmented image corresponding to the map feature and the i-th preset label image among multiple preset label images, and let T be the similarity between X and the segmented image corresponding to the map feature. i Let M be the i-th preset logo image, and M*N be the preset size.

[0167] For example, the preset size can be M*N. The preset similarity can be S. y The default size of the logo image is M*N.

[0168] S703. The electronic device determines the geometric structure information of each map element by taking the geometric structure information of the map element marker corresponding to each map element.

[0169] As one possible implementation, the electronic device obtains the geometric structure information of the map feature markers corresponding to each map feature based on traffic sign databases and road marking databases. Then, the electronic device uses the geometric structure information of the map feature markers corresponding to each map feature to determine the geometric structure information of each map feature.

[0170] For example, geometric information includes: length, width, and height.

[0171] S704. The electronic device constructs the lane line layer of the target map based on the position of each map feature in the world coordinate system, the semantic image corresponding to the acquired image, and the geometric structure information of each map feature.

[0172] As one possible implementation, electronic devices construct lane line layers of the target map based on the position of each map feature in the world coordinate system, the semantic image corresponding to the acquired image, and the geometric structure information of each map feature.

[0173] For example, the vector data of map features includes: the position of the map feature in the camera coordinate system and the geometric structure information of the map feature. The vector data of map features includes spatial vector structure.

[0174] Understandably, geometric structure information can more accurately represent map features, thus making the lane line layer constructed based on geometric structure information more accurate.

[0175] In some embodiments, in order to construct a dynamic object layer in the target map, the above-mentioned S203 can be implemented in the following manner:

[0176] S801, The electronic device inputs multiple semantic images into the target tracking model to obtain the predicted position of each first map element in the multiple semantic images.

[0177] The target tracking model is used to identify the same map feature in multiple semantic images and predict the location of the same map feature in the next semantic image. The first map feature is the same map feature in multiple semantic images.

[0178] As one possible implementation, the electronic device sorts multiple semantic images according to the acquisition time, inputs the sorted semantic images into the target tracking model, and obtains the predicted position of each first map element in the multiple semantic images and the motion trajectory of each first map element in the multiple first map elements.

[0179] It should be noted that since overlapping areas exist between adjacent captured images, the target tracking model can represent map features in these overlapping areas using the same identity (ID). Furthermore, because the target tracking model has cross-frame shared query capabilities, it can track map features in semantic images, ensuring that the IDs of map features remain consistent across multiple semantic images. Moreover, since the target tracking model incorporates a Kalman filter algorithm, which can predict the location of the same map feature across multiple semantic images, inputting multiple semantic images into the target image can yield predicted locations of the same map feature across all semantic images.

[0180] For example, cross-frame shared query functionality can be implemented using a data association algorithm. This algorithm matches map features in semantic images of two adjacent frames.

[0181] S802. The electronic device determines the position of each first map element in the multiple semantic images and the predicted position of each first map element in the multiple semantic images as multiple third positions corresponding to each first map element, so as to obtain multiple third positions corresponding to each first map element in the multiple first map elements.

[0182] S803. For each acquired image, the electronic device determines the position of each map element in the world coordinate system based on the multiple third positions, second positions, camera poses, vehicle poses corresponding to the acquired image, and the transformation relationship between the camera coordinate system and the world coordinate system.

[0183] The second position is the target position corresponding to the acquired image.

[0184] As one possible implementation, for each acquired image, the electronic device determines the position of each map element in the camera coordinate system based on multiple third positions, second positions, camera poses, and the acquired image corresponding to each of the multiple first map elements. Then, the electronic device determines the position of each map element in the world coordinate system based on the vehicle pose corresponding to the acquired image, the transformation relationship between the camera coordinate system and the world coordinate system, and the position of each map element in the camera coordinate system.

[0185] S804. The electronic device constructs a dynamic object layer in the target map based on the position of each map element in the world coordinate system and the semantic image corresponding to the acquired image.

[0186] Understandably, object tracking models can track the same map feature across multiple semantic images and predict the location of the map feature within those images. By combining the predicted location of the map feature with its actual position in the acquired images, the location of the map feature can be determined more accurately, further improving the accuracy of constructing dynamic object layers within the target map.

[0187] In some embodiments, such as Figure 3 As shown, another map building method is provided, which includes the following steps:

[0188] S901, the electronic device divides the target map into a road layer, a lane network layer, a lane line layer, a traffic sign layer, and a dynamic object layer.

[0189] As one possible implementation, electronic devices divide the target map into a road layer, a lane network layer, a lane line layer, a traffic sign layer, and a dynamic object layer based on the road scene element hierarchy, physical relationships, and prior human knowledge.

[0190] For example, Figure 4 This is a schematic diagram illustrating the classification of a target map provided in an embodiment of this application. For example... Figure 4 As shown, the target map includes: a road layer, a lane network layer, a lane line layer, a traffic sign layer, and a dynamic object layer. The road layer includes: road edge lines and road center lines. The lane network layer includes: lane connectors, lane center lines, and lane nodes. The lane line layer includes: lane edge lines and guide lines. The traffic sign layer includes: stop lines, ground traffic signs, traffic signs, and road facilities. The dynamic object layer includes: vehicles, pedestrians, and other moving objects.

[0191] For example, other moving objects are moving objects other than vehicles and pedestrians.

[0192] For example, ground traffic signs include: ground text signs, pedestrian crossings, traffic signs include: traffic signs, and road facilities include: guardrails.

[0193] Understandably, dividing the target map into layers can more accurately represent the various features of the road scene.

[0194] S902: Electronic equipment calibrates on-board sensors and collects data on the traffic environment around the vehicle.

[0195] For example, the vehicle-mounted sensors include: a vehicle-mounted camera, an inertial measurement unit, and a GPS receiver.

[0196] For example, the inertial measurement unit can be an inertial navigation system.

[0197] For example, the vehicle determines key nodes based on distance and time thresholds. Then, the electronic equipment configures the initial position, acquired image, visual features, and vehicle pose for each key node based on the acquired images captured by the onboard camera, the initial position acquired by the GPS receiver, the vehicle pose obtained by the inertial measurement unit, and the visual features obtained from the acquired images.

[0198] S903. The electronic device performs semantic segmentation on the acquired images to obtain vector data of map elements.

[0199] As one possible implementation, the electronic device inputs the acquired image into a semantic segmentation model to obtain the map feature category and pixel coordinates of each map feature in the acquired image. Then, based on the camera pose and the acquired image, the electronic device determines the position of each map feature in the camera coordinate system. Next, the electronic device defines the position of the map feature in the camera coordinate system as vector data of the map feature.

[0200] For example, electronic devices can classify map elements into three categories based on their semantics and attributes: 1. Ground-based markers, such as lane lines, pedestrian crossings, ground text signs, stop lines, etc. 2. Dynamic objects on the road surface, such as vehicles, pedestrians, bicycles, animals, etc. 3. Aerial markers, such as traffic signs, traffic lights, streetlights, billboards, etc.

[0201] S904, Electronic equipment constructs the road layer.

[0202] S905, electronic devices construct the lane network layer.

[0203] As one possible implementation, the electronic device obtains geographic data of the intersection and roads from traffic management authorities. Then, the electronic device combines the intersection and road geographic data into road nodes. Furthermore, the electronic device configures each road node with a vehicle's target location, acquired images, visual features, and vehicle pose.

[0204] For example, geographic data of intersections and road topology can also be obtained from map data providers.

[0205] For example, the geographic data of intersection information includes: location, traffic lights, number of lanes and other related data, while the geographic data of the road includes: road shape, lane markings, speed limits and other information.

[0206] For example, a road node is a key point in the road topology, such as an intersection or a road bend. Each road node has a unique identifier.

[0207] For example, the target location includes: longitude, latitude, and altitude.

[0208] For example, the electronic device generates variable and fixed nodes in a progressive manner, with variable distances between nodes. Fixed nodes are determined by lane attributes and are used to describe the connection nodes between intersections and roads (lanes).

[0209] For example, the intersection information is shown in the following expression:

[0210] W m =(C m R m )

[0211] Among them, C m R is a set of lane-level intersections. m The representative set of roads.

[0212] The set of lane-level intersections is shown in the following expression:

[0213] C m=(P m E m T m )

[0214] Among them, P m It is the set of lane-level nodes entering the intersection, E m It is the set of lane-level nodes leaving the intersection, T m It is a lane-level traffic matrix, indicating whether there is a topological connection between two lane-level nodes.

[0215] T m The definition is shown in the following expression:

[0216]

[0217]

[0218] in, Indicates whether and how a vehicle enters node P. m，i Drive to E m，j .

[0219] The road information is shown in the following expression:

[0220] r m =(L, c in c out Q r )

[0221] Where L is the set of lanes, c in and c out It is based on the entrance and exit intersection indices of the road traffic direction, Q r It is a road attribute.

[0222] The definition of a lane set is:

[0223] L=(S l Q l )

[0224] Among them, S l It is the control node on the center line of the lane, Q l It is a set of lane attributes, including lane width, speed limit, and lane length.

[0225] S906, Electronic equipment constructs lane line layer and traffic sign layer.

[0226] S907, Electronic devices construct dynamic object layers.

[0227] It should be noted that, in the embodiments of this application, the execution order of S904, S905, S906, and S907 is not limited. For example, the electronic device can execute S904, S905, S906, and S907 simultaneously, or it can execute S904, S905, S906, and S907 sequentially, or it can execute S904 and S905 simultaneously first, and then execute S906 and S907 sequentially.

[0228] S908, electronic devices perform multi-level map fusion.

[0229] As one possible implementation, the electronic device extends road intersections based on traffic sign guidance information and fits the feature data in the road vector data to form road centerlines and lane centerlines. Then, the electronic device groups the road feature data, dividing the data according to a "layer-feature-attribute" data logic organization model. Further, the electronic device progressively divides the road surface vector data into road layer, lane network layer, and lane line layer, classifying the distribution of aerial vector data according to its type, location, and attribute information, integrating it into traffic sign layer and dynamic object layer. Next, the electronic device sequentially associates the road layer, lane network layer, traffic sign layer, lane line layer, and dynamic object layer.

[0230] Understandably, the above method uses the vehicle's location information to fuse the perception results of the onboard sensors with the prior navigation map, realizes data association between map layers, establishes spatiotemporal association constraints for sequential data, and constructs a high-precision map.

[0231] The foregoing mainly describes the solutions provided by the embodiments of this application from a methodological perspective. To achieve the above functions, the map building apparatus or electronic device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0232] This application embodiment can, based on the above method, exemplarily divide a map building device or electronic device into functional modules. For example, the map building device or electronic device may include functional modules corresponding to each functional division, or two or more functions may be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be noted that the module division in this application embodiment is illustrative and only represents one logical functional division; in actual implementation, there may be other division methods.

[0233] Figure 5 This is a block diagram illustrating a map building apparatus according to an exemplary embodiment. (Refer to...) Figure 5 The map building device 1000 includes an acquisition unit 1001, a determination unit 1002, and a building unit 1003.

[0234] The acquisition unit 1001 is used to acquire multiple initial positions of the vehicle, multiple acquired images corresponding one-to-one with the multiple initial positions, multiple semantic images corresponding one-to-one with the multiple acquired images, and multiple vehicle poses corresponding one-to-one with the multiple acquired images. A semantic image includes the map feature category corresponding to each pixel in the acquired image corresponding to the semantic image. A semantic image is determined based on the acquired image corresponding to the semantic image and a preset semantic model.

[0235] The determining unit 1002 is used to determine multiple target positions of the vehicle based on multiple initial positions, multiple acquired images, multiple semantic images, multiple vehicle poses, and a preset map.

[0236] Construction unit 1003 is used to construct a target map based on multiple target locations, multiple acquired images, multiple semantic images, and multiple vehicle poses.

[0237] Optionally, in order to accurately determine the target location of the vehicle, such as Figure 5 As shown, the aforementioned determining unit 1002 is specifically used for:

[0238] For each initial position, the location of the initial position on the preset map is determined as the first position corresponding to the initial position.

[0239] If the first location is within a preset deviation area, a first image is acquired. The first image is an image of the area to which the first location belongs in a preset map.

[0240] Obtain the number of matches between the first and second acquired images. The second acquired image is the image corresponding to the initial position.

[0241] If the number of matches exceeds the preset number, the target position is determined based on the initial position, the second acquired image, the acquisition position of the first acquired image, and the vehicle pose corresponding to the second acquired image.

[0242] If the number of matches is less than or equal to the preset number, update the first captured image until the number of matches is greater than the preset number.

[0243] If the first position is not located within the preset deviation area, the first position is determined as the target position.

[0244] Optionally, in order to accurately obtain the number of matches, the aforementioned determining unit 1002 is specifically used for:

[0245] Multiple first feature points and multiple second feature points are acquired. The multiple first feature points are feature points in the first acquired image whose grayscale values ​​meet preset grayscale value conditions, and the multiple second feature points are feature points in the second acquired image whose grayscale values ​​meet preset grayscale value conditions.

[0246] For each first feature point, if the map feature category of the first feature point is the same as that of the third feature point, and the positions of the first feature point and the third feature point are within a preset range, then the first feature point and the third feature point are determined to be matched. The third feature point is any one of multiple second feature points.

[0247] Repeat the above steps until multiple first feature points are traversed to obtain the number of matches between the first and second acquired images.

[0248] Optionally, to construct the road layer in the target map, such as Figure 5 As shown, the aforementioned building unit 1003 is specifically used for:

[0249] For each acquired image, the position of each map feature in the acquired image in the world coordinate system is determined based on the second position, camera pose, vehicle pose corresponding to the acquired image, and the transformation relationship between the camera coordinate system and the world coordinate system. The second position is the target position corresponding to the acquired image.

[0250] Based on the position of each map feature in the world coordinate system, a distance transformation is performed on each map feature to obtain the first distance map.

[0251] Based on the position of each map element in the preset map, a distance transformation is performed on each map element to obtain a second distance map.

[0252] Align the first distance map with the second distance map at the edges, and construct the road layer of the target map based on the position of each map feature in the world coordinate system and the position of each map feature in the preset map.

[0253] Optionally, to construct the lane line layer and traffic sign layer in the target map, such as Figure 5 As shown, the aforementioned building unit 1003 is specifically used for:

[0254] For each acquired image, the position of each map feature in the acquired image within the camera coordinate system is determined based on the second position, camera pose, and the acquired image itself. The second position is the target position corresponding to the acquired image.

[0255] Based on the vehicle pose corresponding to the acquired image and the transformation relationship between the camera coordinate system and the world coordinate system, the position of each map element in the acquired image in the world coordinate system is determined.

[0256] Based on the position of each map feature in the acquired image in the world coordinate system and the semantic image corresponding to the acquired image, the lane line layer of the target map is constructed.

[0257] Based on the position of each map feature in the acquired image in the world coordinate system and the corresponding semantic image, a traffic sign layer of the target map is constructed.

[0258] Optionally, to construct the lane line layer in the target map, such as Figure 5 As shown, the determining unit 1002 is also used for,

[0259] Based on the semantic image corresponding to the acquired image, determine the segmented image corresponding to each map feature in the acquired image.

[0260] The determining unit 1002 is also used to determine the map feature label corresponding to each map feature based on the segmented image corresponding to each map feature and multiple preset label images.

[0261] The determining unit 1002 is also used to determine the geometric structure information of the map feature marker corresponding to each map feature as the geometric structure information of each map feature.

[0262] Construction unit 1003 is specifically used to construct the lane line layer of the target map based on the position of each map feature in the world coordinate system, the semantic image corresponding to the acquired image, and the geometric structure information of each map feature.

[0263] Optionally, to construct a layer of dynamic objects in the target map, such as Figure 5 As shown, the aforementioned building unit 1003 is specifically used for:

[0264] Multiple semantic images are input into a target tracking model to obtain the predicted position of each first map feature in the multiple semantic images. The target tracking model is used to identify the same map feature in multiple semantic images and predict the position of the same map feature in the next semantic image. The first map feature is the same map feature in multiple semantic images.

[0265] Based on the predicted position of each first map element in the multiple semantic images and multiple first map elements, the position of each first map element in the multiple semantic images is determined as multiple third positions corresponding to each first map element, so as to obtain multiple third positions corresponding to each first map element in the multiple first map elements.

[0266] For each acquired image, the position of each map element in the acquired image in the world coordinate system is determined based on multiple third positions, second positions, camera pose, vehicle pose corresponding to the acquired image, and the transformation relationship between the camera coordinate system and the world coordinate system. The second position is the target position corresponding to the acquired image.

[0267] Based on the position of each map feature in the world coordinate system in the acquired image and the semantic image corresponding to the acquired image, a dynamic object layer in the target map is constructed.

[0268] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0269] Figure 6 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Figure 6 As shown, the electronic device 1100 includes, but is not limited to, a processor 1101 and a memory 1102.

[0270] The memory 1102 described above is used to store the executable instructions of the processor 1101. It is understood that the processor 1101 is configured to execute instructions to implement the map construction method in the above embodiments.

[0271] It should be noted that those skilled in the art will understand that Figure 6 The electronic device structure shown does not constitute a limitation on the electronic device; the electronic device may include, but is not limited to, other electronic devices. Figure 6 This may indicate more or fewer components, or combinations of certain components, or different component arrangements.

[0272] Processor 1101 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in memory 1102, and by calling data stored in memory 1102, it performs various functions and processes data, thereby providing overall monitoring of the electronic device. Processor 1101 may include one or more processing units. Optionally, processor 1101 may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into processor 1101.

[0273] The memory 1102 can be used to store software programs and various data. The memory 1102 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and application programs required by at least one functional module (such as acquisition units, determination units, and construction units). Furthermore, the memory 1102 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0274] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory 1102 including instructions, which can be executed by a processor 1101 of an electronic device 1100 to implement the map building method in the above embodiments.

[0275] In actual implementation, Figure 5 The functions of the acquisition unit 1001, the determination unit 1002, and the construction unit 1003 can all be provided by... Figure 6 The processor 1101 calls the computer program stored in the memory 1102 to implement the process. The specific execution process can be found in the description of the map construction method section of the previous embodiment, and will not be repeated here.

[0276] Optionally, the computer-readable storage medium may be a non-transitory computer-readable storage medium, such as a read-only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device.

[0277] In an exemplary embodiment, this application also provides a vehicle including an electronic control unit, a relay, a battery, and an electronic water pump for performing the methods described in the first aspect and any possible implementation thereof.

[0278] In an exemplary embodiment, this application also provides a computer program product including one or more instructions, which can be executed by a processor of an electronic device to complete the map building method in the above embodiments.

[0279] It should be noted that when one or more instructions in the computer-readable storage medium or computer program product are executed by the processor of an electronic device, they implement the various processes of the above-described map construction method embodiments and achieve the same technical effect as the above-described map construction method. To avoid repetition, these will not be repeated here.

[0280] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0281] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another apparatus, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0282] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0283] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0284] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially, or the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, ROM, RAM, magnetic disks, or optical disks.

[0285] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A map construction method, characterized in that, The method includes: The system acquires multiple initial positions of the vehicle, multiple acquired images corresponding to the multiple initial positions, multiple semantic images corresponding to the multiple acquired images, and multiple vehicle poses corresponding to the multiple acquired images. Each semantic image includes the map feature category corresponding to each pixel in the acquired image corresponding to the semantic image. The semantic image is determined based on the acquired image corresponding to the semantic image and a preset semantic model. Based on the multiple initial positions, the multiple acquired images, the multiple semantic images, the multiple vehicle poses, and the preset map, multiple target positions of the vehicle are determined; A target map is constructed based on the multiple target locations, the multiple acquired images, the multiple semantic images, and the multiple vehicle poses; The step of determining multiple target locations of the vehicle based on the multiple initial locations, the multiple acquired images, the multiple semantic images, the multiple vehicle poses, and the preset map includes: For each initial position, the location of the initial position in the preset map is determined as the first position corresponding to the initial position; When the first location is located in a preset deviation area, a first image is acquired; the first image is an image of the area to which the first location belongs in the preset map; Obtain the number of matches between the first and second acquired images; the second acquired image is the acquired image corresponding to the initial position; If the number of matches is greater than a preset number, the target position is determined based on the initial position, the second acquired image, the acquisition position of the first acquired image, and the vehicle pose corresponding to the second acquired image. If the number of matches is less than or equal to the preset number, update the first acquired image until the number of matches is greater than the preset number; If the first position is not located within the preset deviation area, the first position is determined as the target position.

2. The method according to claim 1, characterized in that, The step of obtaining the number of matches between the first acquired image and the second acquired image includes: Multiple first feature points and multiple second feature points are acquired; the multiple first feature points are feature points in the first acquired image whose gray values ​​satisfy a preset gray value condition, and the multiple second feature points are feature points in the second acquired image whose gray values ​​satisfy the preset gray value condition; For each first feature point, if the map feature category of the first feature point is the same as that of the third feature point, and the positions of the first feature point and the third feature point are within a preset range, then the first feature point and the third feature point are determined to be matched; the third feature point is any one of the plurality of second feature points. Repeat the above steps until all the first feature points are traversed to obtain the number of matches between the first acquired image and the second acquired image.

3. The method according to claim 1, characterized in that, When the target map includes a road layer, constructing the target map based on the plurality of target locations, the plurality of acquired images, the plurality of semantic images, and the plurality of vehicle poses includes: For each acquired image, the position of each map element in the acquired image in the world coordinate system is determined based on the second position, camera pose, vehicle pose corresponding to the acquired image, and the transformation relationship between the camera coordinate system and the world coordinate system; the second position is the target position corresponding to the acquired image. Based on the position of each map element in the world coordinate system, a distance transformation is performed on each map element to obtain a first distance map; Based on the position of each map element in the preset map, a distance transformation is performed on each map element to obtain a second distance map; The first distance map and the second distance map are edge-aligned, and the road layer of the target map is constructed based on the position of each map element in the world coordinate system and the position of each map element in the preset map.

4. The method according to claim 1, characterized in that, When the target map includes a lane line layer and a traffic sign layer, constructing the target map based on the multiple target locations, the multiple acquired images, the multiple semantic images, and the multiple vehicle poses includes: For each acquired image, the position of each map feature in the acquired image in the camera coordinate system is determined based on the second position, the camera pose, and the acquired image; the second position is the target position corresponding to the acquired image. Based on the vehicle pose corresponding to the acquired image and the transformation relationship between the camera coordinate system and the world coordinate system, the position of each map element in the acquired image in the world coordinate system is determined; Based on the position of each map feature in the acquired image in the world coordinate system and the semantic image corresponding to the acquired image, construct the lane line layer of the target map; Based on the position of each map element in the acquired image in the world coordinate system and the semantic image corresponding to the acquired image, a traffic sign layer of the target map is constructed.

5. The method according to claim 4, characterized in that, Before constructing the lane line layer of the target map based on the position of each map feature in the acquired image in the world coordinate system and the semantic image corresponding to the acquired image, the method further includes: Based on the semantic image corresponding to the acquired image, determine the segmented image corresponding to each map feature in the acquired image; Based on the segmented image and multiple preset marker images corresponding to each map element, determine the map element marker corresponding to each map element; The geometric structure information of the map feature marker corresponding to each map feature is determined as the geometric structure information of each map feature; The step of constructing the lane line layer of the target map based on the position of each map feature in the acquired image in the world coordinate system and the semantic image corresponding to the acquired image includes: Based on the position of each map feature in the world coordinate system, the semantic image corresponding to the acquired image, and the geometric structure information of each map feature, the lane line layer of the target map is constructed.

6. The method according to claim 1, characterized in that, When the target map includes a dynamic object layer, constructing the target map based on the multiple target locations, the multiple acquired images, the multiple semantic images, and the multiple vehicle poses includes: The multiple semantic images are input into the target tracking model to obtain the predicted position of each first map element in the multiple semantic images; the target tracking model is used to identify the same map element in the multiple semantic images and predict the position of the same map element in the next semantic image; the first map element is the same map element in the multiple semantic images; Based on the multiple semantic images and the predicted position of each first map element in the multiple semantic images, the position of each first map element in the multiple semantic images is determined as multiple third positions corresponding to each first map element, so as to obtain multiple third positions corresponding to each first map element in the multiple semantic images. For each acquired image, based on the multiple third positions, second positions, camera poses, vehicle poses corresponding to the acquired image, and the transformation relationship between the camera coordinate system and the world coordinate system, the position of each map element in the acquired image in the world coordinate system is determined; the second position is the target position corresponding to the acquired image. Based on the position of each map element in the acquired image in the world coordinate system and the semantic image corresponding to the acquired image, a dynamic object layer in the target map is constructed.

7. A map building device, characterized in that, The device includes: an acquisition unit, a determination unit, and a construction unit; The acquisition unit is used to acquire multiple initial positions of the vehicle, multiple acquired images corresponding to the multiple initial positions, multiple semantic images corresponding to the multiple acquired images, and multiple vehicle poses corresponding to the multiple acquired images; a semantic image includes the map feature category corresponding to each pixel in the acquired image corresponding to the semantic image; the semantic image is determined based on the acquired image corresponding to the semantic image and a preset semantic model; The determining unit is used to determine multiple target locations of the vehicle based on the multiple initial locations, the multiple acquired images, the multiple semantic images, the multiple vehicle poses, and a preset map. The construction unit is used to construct a target map based on the multiple target locations, the multiple acquired images, the multiple semantic images, and the multiple vehicle poses; The determining unit is specifically used to determine the position of each initial position in the preset map as the first position corresponding to the initial position for each initial position. When the first location is located in a preset deviation area, a first image is acquired; the first image is an image of the area to which the first location belongs in the preset map; Obtain the number of matches between the first and second acquired images; the second acquired image is the acquired image corresponding to the initial position; If the number of matches is greater than a preset number, the target position is determined based on the initial position, the second acquired image, the acquisition position of the first acquired image, and the vehicle pose corresponding to the second acquired image. If the number of matches is less than or equal to the preset number, update the first acquired image until the number of matches is greater than the preset number; If the first position is not located within the preset deviation area, the first position is determined as the target position.

8. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, When the computer-executable instructions stored in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is capable of performing the method as described in any one of claims 1 to 6.

Images