A method and device for constructing a semantic map

Abstract

The embodiments of the present disclosure disclose a method and device for constructing a semantic map. In the method, a point cloud dataset and an image containing roads for constructing a semantic map are first determined, and then the correspondence between the points in the point cloud dataset and the pixels in the image is determined. Then, based on the semantics of the pixels in the image and the correspondence, the semantics of the points in the point cloud dataset are determined, and then a semantic map is constructed based on the semantics of the points in the point cloud dataset. When constructing a semantic map through this solution, the point cloud dataset is not semantically segmented, but the image is semantically segmented, and then the semantics of the points in the point cloud dataset are determined through the semantics of the pixels in the image and the correspondence between the pixels and the points in the point cloud dataset. Compared with semantic segmentation of point cloud datasets, semantic segmentation of images is more efficient. Therefore, the solution of the embodiments of the present disclosure can improve the efficiency of constructing semantic maps.

Description

Technical Field

[0001] The present disclosure relates to the technical field of electronic maps, and in particular to a method and device for constructing a semantic map. Background Art

[0002] In application scenarios such as assisted driving or autonomous driving, it is sometimes necessary to build a semantic map for the vehicle. When building a semantic map, the point cloud dataset of the lidar is usually obtained first, and then the point cloud dataset is semantically segmented, and a semantic map is created based on the results of the semantic segmentation.

[0003] However, the efficiency of semantic segmentation of point cloud datasets is low, which leads to low efficiency in constructing semantic maps. How to improve the efficiency of constructing semantic maps has become a problem that needs to be solved urgently. Summary of the invention

[0004] In order to solve the above technical problems, the present disclosure is proposed. The embodiments of the present disclosure provide a method and device for constructing a semantic map.

[0005] According to one aspect of the present disclosure, a method for constructing a semantic map is provided, comprising:

[0006] Identify point cloud datasets and images containing roads for building semantic maps;

[0007] Determining a correspondence between points in the point cloud data set and pixels in the image;

[0008] Determining the semantics of points in the point cloud dataset based on the semantics of pixels in the image and the corresponding relationship;

[0009] A semantic map is constructed based on the semantics of the points in the point cloud dataset.

[0010] According to another aspect of an embodiment of the present disclosure, a device for constructing a semantic map is provided, including:

[0011] A point cloud and image determination module, for determining a point cloud dataset and an image containing roads for building a semantic map;

[0012] A correspondence determination module, used to determine the correspondence between the points in the point cloud data set determined by the point cloud and image determination module and the pixels in the image;

[0013] A semantic determination module, configured to determine the semantics of points in the point cloud data set based on the semantics of pixels in the image and the correspondence determined by the correspondence determination module;

[0014] A map construction module is used to construct a semantic map based on the semantics of the point determined by the semantic determination module.

[0015] According to another aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute the method for constructing a semantic map described in any of the above embodiments of the present disclosure.

[0016] According to another aspect of the embodiments of the present disclosure, an electronic device is provided, the electronic device comprising:

[0017] processor;

[0018] a memory for storing instructions executable by the processor;

[0019] The processor is used to read the executable instructions from the memory and execute the instructions to implement the method for constructing a semantic map described in any of the above embodiments of the present disclosure.

[0020] When constructing a semantic map through the solution of the embodiment of the present disclosure, it is not necessary to perform semantic segmentation on the point cloud dataset, but to perform semantic segmentation on the image, and then determine the semantics of the points in the point cloud dataset through the semantics of the pixels in the image and the correspondence between the pixels and the points in the point cloud dataset. Images are two-dimensional data and are ordered. Compared with semantic segmentation of point cloud datasets, semantic segmentation of images is more efficient. Therefore, the solution of the embodiment of the present disclosure can improve the efficiency of constructing semantic maps. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and other purposes, features and advantages of the present disclosure will become more apparent by describing the embodiments of the present disclosure in more detail in conjunction with the accompanying drawings. The accompanying drawings are used to provide a further understanding of the embodiments of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. In the drawings, the same reference numerals generally represent the same components or steps.

[0022] Figure 1 It is a schematic diagram of a scenario to which the present disclosure is applicable.

[0023] Figure 2 It is a flowchart of a method for constructing a semantic map provided by an exemplary embodiment of the present disclosure.

[0024] Figure 3 It is a flowchart of a method for constructing a semantic map provided by another exemplary embodiment of the present disclosure.

[0025] Figure 4 It is a schematic diagram of a scanning scene of a laser radar applicable to the present disclosure.

[0026] FIG5( a ) is a schematic diagram of another scenario to which the present disclosure is applicable.

[0027] FIG5( b ) is a schematic diagram of semantics determined by a point cloud dataset to which the present disclosure is applicable.

[0028] Figure 6 It is a flowchart of a method for constructing a semantic map provided by another exemplary embodiment of the present disclosure.

[0029] Figure 7 It is a flowchart of a method for constructing a semantic map provided by another exemplary embodiment of the present disclosure.

[0030] Figure 8 It is a flowchart of a method for constructing a semantic map provided by another exemplary embodiment of the present disclosure.

[0031] FIG9( a ) is an example diagram of a semantic map of the present disclosure;

[0032] FIG9( b ) is an example diagram of another semantic map of the present disclosure;

[0033] Fig.10 It is a flowchart of a method for constructing a semantic map provided by another exemplary embodiment of the present disclosure.

[0034] Fig.11 It is a structural diagram of an apparatus for constructing a semantic map provided by an exemplary embodiment of the present disclosure.

[0035] Fig.12 It is a structural diagram of an apparatus for constructing a semantic map provided by another exemplary embodiment of the present disclosure.

[0036] Fig.13 is a structural diagram of an electronic device provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0037] Below, the exemplary embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present disclosure, rather than all the embodiments of the present disclosure, and it should be understood that the present disclosure is not limited to the exemplary embodiments described here.

[0038] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

[0039] Those skilled in the art can understand that the terms "first" and "second" in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor indicate the necessary logical order between them.

[0040] It should also be understood that in the embodiments of the present disclosure, “plurality” may refer to two or more than two, and “at least one” may refer to one, two, or more than two.

[0041] It should also be understood that any component, data or structure mentioned in the embodiments of the present disclosure can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0042] In addition, the term "and / or" in the present disclosure is only a description of the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present disclosure generally indicates that the associated objects before and after are in an "or" relationship.

[0043] It should also be understood that the description of the various embodiments in the present disclosure focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.

[0044] At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship.

[0045] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present disclosure, its application, or uses.

[0046] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered as part of the specification.

[0047] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0048] The disclosed embodiments can be applied to electronic devices such as terminal devices, computer systems, servers, etc., which can operate with many other general or special computing system environments or configurations. Examples of well-known terminal devices, computing systems, environments and / or configurations suitable for use with electronic devices such as terminal devices, computer systems, servers, etc. include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, small computer systems, large computer systems, and distributed cloud computing technology environments including any of the above systems, etc.

[0049] Electronic devices such as terminal devices, computer systems, servers, etc. can be described in the general context of computer system executable instructions (such as program modules) executed by computer systems. Generally, program modules can include routines, programs, object programs, components, logic, data structures, etc., which perform specific tasks or implement specific abstract data types. Computer systems / servers can be implemented in a distributed cloud computing environment, where tasks are performed by remote processing devices linked through a communication network. In a distributed cloud computing environment, program modules can be located on local or remote computing system storage media including storage devices.

[0050] Application Overview

[0051] In application scenarios such as assisted driving or autonomous driving, it is sometimes necessary to build a semantic map for the vehicle, based on which the vehicle can adjust its driving route.

[0052] Currently, when constructing a semantic map, the lidar first emits laser to the road and its surroundings, and obtains a point cloud dataset corresponding to the laser; the server can perform semantic segmentation on the point cloud dataset, and through semantic segmentation, determine the objects contained in the road and its surroundings (such as road edges, road signs, and buildings around the road), and construct a semantic map based on the vehicle's location.

[0053] However, each point in the point cloud dataset is three-dimensional and disordered, which leads to low efficiency in semantic segmentation of the point cloud dataset, and further leads to low efficiency in constructing semantic maps.

[0054] In view of this, an embodiment of the present disclosure provides a method and device for constructing a semantic map. When constructing a semantic map through the scheme of the present disclosure, first obtain a point cloud dataset and an image containing roads for constructing a semantic map, perform semantic segmentation on the image, and obtain the semantics of each pixel in the image; then, based on the correspondence between each point in the point cloud dataset and each pixel in the image, as well as the semantics of each pixel in the image, determine the semantics of the points in the point cloud dataset, and construct a semantic map accordingly. Among them, the point cloud dataset includes multiple points, and each point refers to corresponding point cloud data. In one example, the point cloud dataset is determined by laser radar measurement, and the laser emitted by the laser radar is reflected after encountering a certain position of an object. Then, each point in the point cloud dataset (i.e., point cloud data) generally includes the three-dimensional coordinates of a certain position of the object and the laser reflection intensity.

[0055] When constructing a semantic map through the solution of the embodiment of the present disclosure, there is no need to perform semantic segmentation on the LiDAR point cloud data. Instead, the semantics of the point cloud are determined by the semantics of the image and the correspondence between the pixels of the image and the points of the point cloud data set. The image is two-dimensional data and is ordered. Compared with the semantic segmentation of the point cloud data set, the efficiency of semantic segmentation of the image is higher. Therefore, constructing a semantic map through the solution of the embodiment of the present disclosure can improve the efficiency of constructing the semantic map.

[0056] Exemplary Systems

[0057] The embodiments of the present disclosure may be applied to application scenarios where a semantic map needs to be constructed, and the application scenarios may include application scenarios such as assisted driving or autonomous driving.

[0058] For example, in application scenarios of assisted driving or autonomous driving, the vehicle's on-board camera can capture images of the road on which the vehicle is located to obtain images containing the road, and on-board sensors can be set on the vehicle to determine a point cloud data set for constructing a semantic map through the on-board sensors, and then the semantic map is constructed through the solution of the embodiment of the present disclosure.

[0059] Figure 1 is a scene graph to which the present disclosure is applicable. Figure 1 As shown, the embodiment of the present disclosure is applied to an application scenario of assisted driving or autonomous driving, wherein a device 100 for constructing a semantic map can be connected to a vehicle-mounted sensor 200 and a vehicle-mounted camera 300.

[0060] The device 100 for constructing a semantic map may be electrically connected to the vehicle-mounted sensor 200 and the vehicle-mounted camera 300, respectively. For example, the device 100 for constructing a semantic map may be an on-board computer. In this case, the device 100 for constructing a semantic map may be electrically connected to the vehicle-mounted sensor 200 and the vehicle-mounted camera 300, respectively. In addition, the connection between the device 100 for constructing a semantic map and the vehicle-mounted sensor 200 and the vehicle-mounted camera 300 may also be a network connection.

[0061] The vehicle-mounted sensor 200 may be used to generate a point cloud data set, and transmit the point cloud data set to the device 100 for constructing a semantic map. In addition, the vehicle-mounted camera 300 may capture an image containing a road, and transmit the image to the device 100 for constructing a semantic map.

[0062] After acquiring an image containing a road, the device 100 for constructing a semantic map may perform semantic segmentation on the image to determine the semantics of pixels in the image. In addition, the device 100 for constructing a semantic map may also determine the correspondence between points in a point cloud data set and pixels in the image, and determine the semantics of points in the point cloud data set based on the correspondence and the semantics of pixels in the image, and then construct a semantic map based on the semantics of points in the point cloud data set.

[0063] Exemplarily, the device 100 for constructing a semantic map may be a server in a vehicle (such as an in-vehicle smart terminal). In this case, the server may utilize the semantic map to implement assisted driving or autonomous driving functions.

[0064] Exemplary Methods

[0065] Figure 2 FIG. 1 is a flow chart of a method for constructing a semantic map provided by an exemplary embodiment of the present disclosure. This embodiment can be applied to electronic devices, such as Figure 2 As shown, the following steps are included:

[0066] Step 201: Determine a point cloud dataset and an image containing roads for constructing a semantic map.

[0067] The device for executing the method for building a semantic map can be connected to a vehicle-mounted sensor and a vehicle-mounted camera, and exchange information. The connection between the device and the vehicle-mounted sensor and the vehicle-mounted camera is not limited to a physical connection. In one example, the connection between the device and the vehicle-mounted sensor and the vehicle-mounted camera can be an electrical connection. In another example, the connection between the device and the vehicle-mounted sensor and the vehicle-mounted camera can be a network connection, that is, the device can exchange information with the vehicle-mounted sensor and the vehicle-mounted camera respectively through the network.

[0068] The vehicle-mounted sensor is a device capable of generating a point cloud data set. For example, the vehicle-mounted sensor may include a laser radar, in which case the point cloud data set may include a point cloud data set generated by the laser radar according to lasers; or the vehicle-mounted sensor may include an imaging processing device that generates a point cloud data set according to an image, in which case the point cloud data set may include a point cloud data set generated by the imaging processing device according to an image.

[0069] In addition, after generating the point cloud dataset, the on-board sensor can transmit the point cloud dataset generated by itself to the device used to execute the method for constructing the semantic map through information interaction with the on-board sensor. If the point cloud dataset generated by the on-board sensor is called the original point cloud dataset, in a feasible design, the point cloud dataset used to construct the semantic map includes the original point cloud dataset. Alternatively, in another feasible design, in order to improve the accuracy of the semantic map, the original point cloud dataset can be processed, and the point cloud dataset used to construct the semantic map includes the point cloud dataset obtained after processing the original point cloud dataset.

[0070] The vehicle-mounted camera can capture images containing roads and transmit the images to a device for executing the method for constructing a semantic map, so that the device obtains images containing roads.

[0071] Step S202: Determine the correspondence between the points in the point cloud data set and the pixels in the image.

[0072] A point cloud dataset consists of multiple points. If a point in the point cloud dataset corresponds to a pixel in the image, it means that the semantics contained in the point is the same as that contained in the pixel.

[0073] Step S203: Determine the semantics of the points in the point cloud data set based on the semantics of the pixels in the image and the corresponding relationship.

[0074] By semantically segmenting an image, the semantics of each pixel in the image can be determined. In this case, the semantics of each point in the point cloud dataset can be determined based on the correspondence between the points in the point cloud dataset and the pixels in the image.

[0075] Step S204: construct a semantic map based on the semantics of the points in the point cloud data set.

[0076] Through the semantics of the points in the point cloud dataset, we can determine the various objects contained in the surrounding environment in the current scene and build a semantic map based on this.

[0077] In the method for constructing a semantic map disclosed in the present invention, after determining a point cloud dataset and an image containing roads for constructing a semantic map, the correspondence between points in the point cloud dataset and pixels in the image is determined; then, based on the semantics of the pixels in the image and the correspondence, the semantics of the points in the point cloud dataset are determined, and a semantic map is constructed through the semantics of the points in the point cloud dataset.

[0078] In the existing technology for building semantic maps, the semantics of each point in the point cloud dataset is determined by semantic segmentation of the point cloud dataset. However, each point in the point cloud dataset is three-dimensional and disordered, so the efficiency of semantic segmentation of the point cloud dataset is low, which further leads to the low efficiency of building semantic maps in the existing technology.

[0079] When constructing a semantic map through the solution of the embodiment of the present disclosure, semantic segmentation is not performed on the point cloud dataset, but on the image. Then, the semantics of the points in the point cloud dataset are determined through the semantics of the pixels in the image and the correspondence between the pixels and the points in the point cloud dataset. Images are two-dimensional data and are ordered. Compared with semantic segmentation of point cloud datasets, semantic segmentation of images is more efficient. Therefore, the solution of the embodiment of the present disclosure can improve the efficiency of constructing semantic maps.

[0080] Furthermore, since each point in the point cloud data set is three-dimensional and disordered, while the image is two-dimensional data and ordered, the accuracy of semantic segmentation of the image is often higher than the accuracy of semantic segmentation of the point cloud data set. Therefore, the solution of the embodiment of the present disclosure can also improve the accuracy of constructing a semantic map, thereby being able to construct a high-precision semantic map.

[0081] In addition, since the existing technology performs semantic segmentation on point cloud datasets, and the accuracy of semantic segmentation on point cloud datasets is low, it is often necessary to manually re-label the results of the semantic segmentation.

[0082] In the solution provided by the embodiment of the present disclosure, semantic segmentation is performed on images, and the accuracy of semantic segmentation of images is high, thereby reducing the need for manual rework and annotation, reducing the waste of human resources, and further improving the efficiency of semantic map construction.

[0083] In the solution provided by the embodiment of the present disclosure, the point cloud dataset used to construct the semantic map may include the original point cloud dataset generated by the vehicle-mounted sensor, or include the point cloud dataset after processing the original point cloud dataset.

[0084] See also Figure 3 In another exemplary embodiment of the present disclosure, in the above Figure 2 Based on the embodiment shown, the point cloud dataset used to construct the semantic map is determined through the following steps:

[0085] Step S2011: Obtain the original point cloud data set through the vehicle-mounted sensor.

[0086] The original point cloud dataset generally refers to a point cloud dataset generated by a vehicle-mounted sensor.

[0087] Step S2012: performing motion distortion correction on the points in the original point cloud data set. The point cloud data set formed by the points after the motion distortion correction is the point cloud data set used to construct the semantic map.

[0088] When the vehicle-mounted sensors generate point cloud datasets, the movement of the vehicle may cause motion distortion in the point cloud datasets.

[0089] If the vehicle's onboard sensors include lidar, see Figure 4 As shown in the scene diagram, during the scanning process, the laser radar rotates once every time period and scans again after the rotation to obtain the points that constitute the point cloud data set. Figure 4 In the example, the laser radar rotates 2° every time period. Figure 4The dotted line in the figure is the laser emitted by the lidar. However, during this process, the vehicle may move, resulting in motion distortion in the point cloud dataset.

[0090] For example, if the scanning frequency of the lidar is 5 Hz, the time difference between the beginning and the end of a frame of data is 200 ms. If the vehicle moves in the x direction at a speed of 0.5 m / s, and the lidar scans the wall in front of the vehicle while the vehicle is moving, then after 200 ms, the measured distance of the tail and the measured distance of the front will differ by 10 cm in the x direction.

[0091] In this step, motion distortion correction can be performed through an inertial measurement unit (IMU). The IMU can be installed on the vehicle. Through the pre-integration of the IMU, the position and posture of the on-board sensor at each point in the point cloud data set can be estimated, and each point can be converted accordingly so that the position and posture of the on-board sensor corresponding to each point are the same, thereby achieving motion distortion correction of the point cloud data set.

[0092] Directly determining the semantics of each point in a point cloud dataset with motion distortion will result in errors in the determined semantics. In this exemplary embodiment, motion distortion correction is performed on the points in the point cloud dataset, and the point cloud dataset composed of the points after motion distortion correction is used as the point cloud dataset for constructing a semantic map, thereby reducing the impact of motion distortion and improving the accuracy of the semantic map.

[0093] For example, as shown in Figure 5(a), there is an actual scene with an object in front of the vehicle, and the arrow in the figure indicates the direction of the vehicle. In this case, the semantics determined based on the point cloud dataset with motion distortion is shown by the dotted line in Figure 5(b), and the semantics determined based on the point cloud dataset after motion distortion correction is shown by the solid line in Figure 5(b).

[0094] Figure 6 FIG. 1 is a flow chart of a method for constructing a semantic map provided by another exemplary embodiment of the present disclosure. Figure 6 As shown in the above Figure 2 Based on the illustrated embodiment, step S202 includes the following steps S2021 and S2024.

[0095] Step S2021: Based on the time when the vehicle-mounted camera captures the image and the time when the vehicle-mounted laser radar emits laser light, determine the first coefficient by the following formula:

[0096] k = (tc-tl0)\(tl1-tl0) Formula (1);

[0097] Among them, k is the first coefficient, tc is the time when the vehicle-mounted camera takes the image, tl0 is the time when the vehicle-mounted laser radar emits the first laser, tl1 is the time when the vehicle-mounted laser radar emits the second laser, the first laser is the laser corresponding to the last frame of point cloud before tc, and the second laser is the laser corresponding to the first frame of point cloud after tc. The vehicle-mounted sensor includes a vehicle-mounted laser radar.

[0098] Since the vehicle-mounted sensor includes a vehicle-mounted laser radar, the point cloud data set includes a point cloud data set determined by the laser emitted by the laser radar. tc is the time when the vehicle-mounted camera captures the image, tl0 is the time when the vehicle-mounted laser radar emits the first laser, and tl1 is the time when the vehicle-mounted laser radar emits the second laser, that is, the first laser and the second laser are the lasers emitted before and after the image is captured.

[0099] Step S2022: determine the position and posture of the vehicle-mounted laser radar.

[0100] Step S2023: Based on the position and posture of the vehicle-mounted laser radar and the first coefficient, determine the second coefficient by the following formula:

[0101]

[0102] Among them, TC is the second coefficient, T1 is the position of the vehicle-mounted laser radar at tl0, T2 is the position of the vehicle-mounted laser radar at tl1, The linear difference between the quaternion interpolation and translation representing the rotation of the pose, Multiplication of the quaternion representing the pose with a constant value for the translation vector.

[0103] Step S2024: Based on the second coefficient, determine the correspondence between the points in the point cloud data set and the pixels in the image by the following formula:

[0104] Px=K×T cl ×TC×P L Formula (3);

[0105] Wherein, Px is the coordinate of the pixel in the image in the image coordinate system, K is the intrinsic calibration matrix of the vehicle-mounted camera, T cl is the extrinsic calibration matrix for converting the map coordinate system of the point cloud dataset to the image coordinate system, P L are the coordinates of the point cloud of the second laser in the map coordinate system.

[0106] Based on this embodiment, the correspondence between points in the point cloud dataset and pixels in the image may be determined, so as to determine the semantics of each point in the point cloud dataset according to the correspondence and the semantics of the pixels in the image.

[0107] Furthermore, in the above Figure 2 Based on the illustrated embodiment, another exemplary embodiment of the present disclosure further includes the following steps:

[0108] Before determining the semantics of the points in the point cloud data set, a first region in the image is deleted to determine the semantics of the points in the point cloud data set based on the semantics of the pixels in the image after deleting the first region and the corresponding relationship, wherein the depth of the pixels in the first region is greater than the depth of the pixels in other regions of the image.

[0109] Points in a point cloud dataset usually include three dimensions: x, y, and z. The depth of a certain pixel can be represented by the z value of the point in the point cloud dataset corresponding to the pixel. In this case, the z values ​​of the points in the point cloud dataset can be sorted, and the sorting can reflect the sorting of the depths of the pixels corresponding to the points. Through the sorting results, the pixels with greater depth can be determined, and then the first area can be determined.

[0110] The installation positions of vehicle-mounted sensors and vehicle-mounted cameras are usually different. Therefore, there may be a phenomenon that the vehicle-mounted sensors are not blocked but the vehicle-mounted cameras are blocked, which will cause areas with large pixel depths to appear in the image, and the semantics determined by the pixels in this area are often inaccurate. In addition, when the vehicle-mounted camera shoots distant scenes, the resolution is small, so the same pixel in the image may be associated with multiple points in the point cloud dataset, that is, the same pixel has a corresponding relationship with multiple points in the point cloud dataset, and the depth of the area where the pixel is located is large. In this case, the semantics determined by the pixels in this area are often inaccurate. In other words, if the first area is semantically segmented, the accuracy of the obtained semantics is low, and accordingly, the accuracy of the semantics of each point in the point cloud dataset determined by the semantics of the pixels in the first area is also low.

[0111] The present exemplary embodiment deletes the first area in the image through the above steps, and determines the semantics of the points in the point cloud dataset based on the semantics and correspondence of the pixels in the image after deleting the first area, thereby improving the accuracy of determining the semantics of the points in the point cloud dataset and further improving the accuracy of the semantic map.

[0112] Figure 7 FIG. 1 is a flow chart of a method for constructing a semantic map provided by another exemplary embodiment of the present disclosure. Figure 7 As shown in the above Figure 2 Based on the illustrated embodiment, this exemplary embodiment further includes the following steps:

[0113] Step S205 : before determining the semantics of the points in the point cloud data set, determining a first object in the image based on the semantics of the pixels in the image and the preset semantics of the first object.

[0114] An image often includes a road and a variety of objects around the road, wherein the first object usually includes an object that does not need to be presented in a semantic map.

[0115] In a feasible design, the first object may include a movable object, such as a vehicle parked on the roadside, etc. Since the first object is movable, the first object often leaves the road or the periphery of the road at other times, and therefore, does not need to be presented in the semantic map.

[0116] In this step, the semantics of the first object may be preset. In this case, after the semantics of each pixel in the image is determined, the first object in the image may be determined based on the preset semantics of the first object.

[0117] Step S206: Delete the first object in the image to determine the semantics of the points in the point cloud data set based on the semantics of the pixels in the image after deleting the first object and the corresponding relationship.

[0118] Since the first object does not need to be presented in the semantic map, there is no need to determine the semantics of the point cloud corresponding to the first object.

[0119] Through the operation of the above exemplary embodiments, the operation of determining the semantics of points in the point cloud data set through the semantics and correspondence of the first object in the image can be reduced, thereby improving the efficiency of constructing the semantic map.

[0120] Furthermore, since the first object is not presented in the semantic map constructed by the above exemplary embodiment, the semantic map is relatively concise.

[0121] In addition, Figure 7 In the flowchart shown, step S206 is located after step S202, but in the actual semantic map construction scenario, there is no strict time limit for these two steps. For example, the correspondence between the points in the point cloud dataset and the pixels in the image can be determined after deleting the first object in the image.

[0122] Figure 8 FIG. 1 is a flow chart of a method for constructing a semantic map provided by another exemplary embodiment of the present disclosure. Figure 8 As shown in the above Figure 2 Based on the illustrated embodiment, this exemplary embodiment further includes the following steps:

[0123] Step S207: before determining the semantics of the points in the point cloud data set, determine a second object in the image based on the semantics of the pixels in the image and a target tracking algorithm, where the second object includes a moving object.

[0124] Based on the semantics of the pixels in the image, the types of the objects contained in the image can be determined, and thus whether there are movable objects in the image. Based on the target tracking algorithm, it can be determined whether the object is moving, and then the second object in the image can be determined.

[0125] Step S208: Delete the second object in the image, so as to determine the semantics of the points in the point cloud data set based on the semantics of the pixels in the image after deleting the second object and the corresponding relationship.

[0126] When constructing a semantic map through the solution provided by the embodiment of the present disclosure, since the area captured by each image is limited, it is often necessary to capture multiple images, determine the semantics of each point in the point cloud dataset through the semantics of the pixels in the multiple images, and the correspondence between the pixels and each point in the point cloud dataset, and then construct a semantic map based on the semantics of each point in the point cloud dataset. In other words, the solution provided by the embodiment of the present disclosure usually requires multiple images to construct a semantic map.

[0127] In this case, if the image contains a moving object (such as a moving vehicle), when a semantic map is constructed based on multiple images, there may be ghosting in the constructed semantic map, which reduces the quality of the semantic map.

[0128] With the solution provided by the embodiment of the present disclosure, the semantics of each point in the point cloud data set can be determined based on the semantics of the pixels in the image after deleting the second object, thereby reducing the ghosting caused by moving objects and improving the quality of the semantic map.

[0129] In addition, Figure 8 In the flowchart shown, step S208 is located after step S202, but in the actual semantic map construction scenario, there is no strict time limit for these two steps. For example, the correspondence between the points in the point cloud data set and the pixels in the image can be determined after deleting the second object in the image.

[0130] In order to clarify the advantages of this exemplary embodiment, FIG9(a) and FIG9(b) are disclosed below, wherein FIG9(a) is a semantic map constructed by using an image containing a second object, and the application scenario for the semantic map is an intersection, in which a vehicle travels from left to right, resulting in smear in the semantic map. In FIG9(a), the dotted line is the smear in the semantic map.

[0131] The semantic map constructed by deleting the image after the second object is shown in FIG9( b ). After deleting the second object, no smear appears in the constructed semantic map.

[0132] The comparison between FIG. 9( a ) and FIG. 9( b ) shows that constructing a semantic map by applying the semantics of pixels in the image after deleting the second object can effectively improve the quality of the semantic map.

[0133] In step S022 of the above exemplary embodiment, an operation of determining the position and posture of the vehicle-mounted laser radar is disclosed. In another exemplary embodiment, the operation can be implemented by the following steps:

[0134] First, based on the point cloud dataset used to build the semantic map, the white noise and zero bias of the vehicle's inertial measurement unit (IMU) are adjusted.

[0135] Then, the position and posture of the vehicle-mounted laser radar is determined based on the global positioning system (GPS) and the inertial measurement unit after adjusting the white noise and the zero bias.

[0136] IMU is usually equipped with a three-axis gyroscope and three-directional accelerometers, which can be used to measure the angular velocity and acceleration of an object in three-dimensional space. However, as the measurement time increases, the white noise and zero bias of the IMU gradually increase, which will affect the measurement accuracy of the IMU.

[0137] In view of this situation, in an exemplary embodiment of the present disclosure, the white noise and zero bias of the IMU are adjusted based on the point cloud data set used to construct the semantic map. The point cloud data set used to construct the semantic map has been corrected for motion distortion, and the white noise and zero bias of the IMU can be reduced through adjustment.

[0138] After adjusting the white noise and zero bias of the IMU, the position and posture of the vehicle-mounted laser radar can be determined through GPS and IMU. Among them, the receiver in GPS can receive the ephemeris parameters and time information sent by the satellite, and calculate the position of the receiver based on the ephemeris parameters and time information. In addition, the vehicle-mounted IMU can measure the angular velocity and acceleration of the vehicle in three-dimensional space. In this case, the combination of GPS and IMU can determine the position and posture of the vehicle-mounted laser radar.

[0139] In the semantic map construction method provided by the present disclosure, it is necessary to determine the correspondence between the points in the point cloud data set and the pixels in the image based on the position and posture of the vehicle-mounted laser radar. In addition, when constructing a semantic map based on the semantics of the points in the point cloud data set, it is often necessary to determine the position and posture of the vehicle based on the position and posture of the vehicle-mounted laser radar to achieve the positioning of the vehicle, and based on the positioning of the vehicle and the point cloud data set, determine the positions of various objects contained in the road and the surrounding area of ​​the road.

[0140] In the existing technology for building semantic maps, the position and posture of the lidar is usually determined based on IMU and real-time kinematic (RTK) positioning system. However, the cost of RTK is relatively high, which leads to the high cost of the existing technology for building semantic maps, which also hinders the promotion of semantic map construction technology.

[0141] However, in the solution provided by the exemplary embodiment of the present disclosure, the position and posture of the laser radar can be determined by IMU and GPS. The cost of GPS is usually less than that of RTK, so it can reduce the cost required for building semantic maps, which is conducive to the promotion of semantic map construction technology.

[0142] Furthermore, in the solution provided by the exemplary embodiment of the present disclosure, when determining the posture of the vehicle-mounted LiDAR, the white noise and zero bias of the IMU are adjusted through the point cloud data set after motion distortion correction, thereby improving the accuracy of the IMU measurement, and correspondingly improving the accuracy of determining the posture of the vehicle-mounted LiDAR, and further improving the accuracy of the semantic map, so that a high-precision semantic map can be constructed.

[0143] Fig.10 It is a flowchart of a method for constructing a semantic map provided by another exemplary embodiment of the present disclosure, and the flowchart is an example of the overall process of constructing a semantic map through the solution of the exemplary embodiment of the present disclosure.

[0144] See also Fig.10 In the method for constructing a semantic map provided in the exemplary embodiment of the present disclosure, a vehicle-mounted camera, a vehicle-mounted sensor, an IMU, a GPS, and an electronic device for executing the method are applied.

[0145] In the process of building a semantic map, the on-board camera can capture images containing roads and transmit the images to electronic devices. The electronic devices perform semantic segmentation on the images to obtain the semantics of the pixels in the images.

[0146] The on-board sensor can obtain a point cloud data set and transmit the point cloud data set to an electronic device, which performs motion distortion correction on the point cloud data set to determine a point cloud data set for building a semantic map. In addition, the electronic device can adjust the white noise and zero bias of the IMU through the point cloud data set with motion distortion correction, and use GPS and the adjusted IMU to determine the pose of the on-board sensor. If the on-board sensor includes an on-board laser radar, the pose of the on-board sensor includes the pose of the on-board laser radar.

[0147] In addition, the electronic device not only needs to determine the point cloud dataset used to construct the semantic map, but also needs to determine the image containing the road, that is, to establish an association between the point cloud dataset and the image, and to determine the correspondence between the points in the point cloud dataset and the pixels in the image.

[0148] In order to improve the quality of the semantic map, the electronic device may further delete the first object and the second object in the image. The semantics of the first object may be preset, and the second object generally includes a moving object. The first object and the second object do not need to be presented in the semantic map, and deleting the first object and the second object in the image helps to improve the accuracy of the semantic map.

[0149] Furthermore, the electronic device may also delete a first area in the image, wherein the depth of pixels in the first area is greater than the depth of pixels in other areas of the image.

[0150] After deleting the first object, the second object and the first area in the image, the electronic device can determine the semantics of the points in the point cloud dataset based on the semantics of the pixels in the image and the correspondence between the points in the point cloud dataset and the pixels in the image, and construct a semantic map based on the semantics of the points in the point cloud dataset.

[0151] Exemplary Devices

[0152] Fig.11 is a structural diagram of a device for constructing a semantic map provided by an exemplary embodiment of the present disclosure. The device for constructing a semantic map can be set in an electronic device such as a terminal device, a server, or an object such as a vehicle to execute the method for constructing a semantic map in any of the above embodiments of the present disclosure. Fig.11 As shown, the apparatus for constructing a semantic map in this embodiment includes: a point cloud and image determination module 201 , a correspondence determination module 202 , a semantic determination module 203 and a map construction module 204 .

[0153] The point cloud and image determination module 201 is used to determine a point cloud data set and an image containing roads for constructing a semantic map.

[0154] The correspondence determination module 202 is used to determine the correspondence between the points in the point cloud data set and the pixels in the image.

[0155] The semantic determination module 203 is used to determine the semantics of points in the point cloud data set based on the semantics of the pixels in the image and the corresponding relationship.

[0156] The map construction module 204 is used to construct a semantic map based on the semantics of the points in the point cloud dataset.

[0157] When constructing a semantic map through the device of the embodiment of the present disclosure, it is not necessary to perform semantic segmentation on the point cloud dataset, but to perform semantic segmentation on the image, and then determine the semantics of the points in the point cloud dataset through the semantics of the pixels in the image and the correspondence between the pixels and the points in the point cloud dataset. The image is two-dimensional data and is ordered. Compared with the semantic segmentation of the point cloud dataset, the efficiency of semantic segmentation of the image is higher. Therefore, the device of the embodiment of the present disclosure can improve the efficiency of constructing the semantic map.

[0158] For further information, see Fig.12 As shown in the structural schematic diagram, in a feasible example, the point cloud and image determination module 201 includes:

[0159] The original data set acquisition unit 2011 is used to acquire the original point cloud data set through the vehicle-mounted sensor.

[0160] The point cloud data set acquisition unit 2012 is used to perform motion distortion correction on the points in the original point cloud data set acquired from the original data set acquisition unit 2011. The point cloud data set composed of the points after motion distortion correction is the point cloud data set used to construct the semantic map.

[0161] In another feasible example, the correspondence determination module 202 includes:

[0162] The first coefficient determination unit 2021 is used to determine the first coefficient by the following formula based on the time when the vehicle-mounted camera captures the image and the time when the vehicle-mounted laser radar emits laser light when the vehicle-mounted laser radar includes a vehicle-mounted laser radar:

[0163] k = (tc-tl0)\(tl1-tl0);

[0164] Among them, k is the first coefficient, tc is the time when the vehicle-mounted camera takes the image, tl0 is the time when the vehicle-mounted laser radar emits the first laser, tl1 is the time when the vehicle-mounted laser radar emits the second laser, the first laser is the laser corresponding to the last frame of point cloud before tc, and the second laser is the laser corresponding to the first frame of point cloud after tc.

[0165] The posture determination unit 2022 is used to determine the posture of the vehicle-mounted laser radar.

[0166] The second coefficient determination unit 2023 is used to determine the second coefficient by the following formula based on the posture of the vehicle-mounted laser radar determined by the posture determination unit 2022 and the first coefficient determined by the first coefficient determination unit 2021:

[0167]

[0168] Among them, TC is the second coefficient, T1 is the position of the vehicle-mounted laser radar at tl0, T2 is the position of the vehicle-mounted laser radar at tl1, The linear difference between the quaternion interpolation and translation representing the rotation of the pose, Multiplication of the quaternion representing the position and the constant value of the translation vector;

[0169] The correspondence determination unit 2024 is configured to determine the correspondence between the points in the point cloud data set and the pixels in the image based on the second coefficient determined by the second coefficient determination unit 2023 by using the following formula:

[0170] Px=K×T cl ×TC×P L ;

[0171] Wherein, Px is the coordinate of the pixel in the image in the image coordinate system, K is the intrinsic calibration matrix of the vehicle-mounted camera, T cl is the extrinsic calibration matrix for converting the map coordinate system of the point cloud dataset to the image coordinate system, P L are the coordinates of the point cloud of the second laser in the map coordinate system.

[0172] In the above Fig.11 or Fig.12 Based on the illustrated embodiment, in another exemplary embodiment of the present disclosure, a device for constructing a semantic map further includes:

[0173] The first region deletion module is used to delete a first region in the image before determining the semantics of the points in the point cloud dataset, so as to determine the semantics of the points in the point cloud dataset based on the semantics of the pixels in the image after deleting the first region and the corresponding relationship, wherein the depth of the pixels in the first region is greater than the depth of the pixels in other regions of the image.

[0174] In the above Fig.11 or Fig.12 Based on the illustrated embodiment, in another exemplary embodiment of the present disclosure, a device for constructing a semantic map further includes:

[0175] A first object determination module, configured to determine a first object in the image based on the semantics of pixels in the image and the semantics of a preset first object before determining the semantics of points in the point cloud data set;

[0176] The first object deletion module is used to delete the first object in the image determined by the first object determination module, so as to determine the semantics of the points in the point cloud data set based on the semantics of the pixels in the image after deleting the first object and the corresponding relationship.

[0177] In the above Fig.11 or Fig.12 Based on the illustrated embodiment, in another exemplary embodiment of the present disclosure, a device for constructing a semantic map further includes:

[0178] A second object determination module is used to determine a second object in the image based on the semantics of the pixels in the image and a target tracking algorithm before determining the semantics of the points in the point cloud data set, wherein the second object includes a moving object;

[0179] The first object deletion module is used to delete the second object in the image determined by the second object determination module, so as to determine the semantics of the points in the point cloud data set based on the semantics of the pixels in the image after deleting the second object and the corresponding relationship.

[0180] In the above Fig.12 Based on the illustrated embodiment, in an apparatus for constructing a semantic map provided by another exemplary embodiment of the present disclosure, the posture determination unit 2022 includes:

[0181] An adjustment subunit, used for adjusting white noise and zero bias of an onboard inertial measurement unit based on the point cloud data set used to construct a semantic map;

[0182] The posture determination subunit is used to determine the posture of the vehicle-mounted laser radar based on the global positioning system and the inertial measurement unit after the white noise and zero bias are adjusted by the adjustment subunit.

[0183] Exemplary Electronic Devices

[0184] Below, reference Fig.13 To describe the electronic device according to the embodiment of the present disclosure. Fig.13 As shown, the electronic device 10 includes one or more processors 11 and a memory 12 .

[0185] The processor 11 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.

[0186] The memory 12 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory, etc. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 11 may run the program instructions to implement the method for constructing a semantic map and / or other desired functions of the various embodiments of the present disclosure described above. Various contents such as images containing roads, point cloud data sets, etc. may also be stored in the computer-readable storage medium.

[0187] In one example, the electronic device 10 may further include: an input device 13 and an output device 14, and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0188] In addition, the input device 13 may also include, for example, a keyboard, a mouse, and the like.

[0189] The output device 14 can output various information to the outside, including the constructed semantic map, etc. The output device 14 can include, for example, a display, a speaker, a printer, a communication network and a remote output device connected thereto, and the like.

[0190] Of course, to simplify, Fig.13 Only some of the components related to the present disclosure in the electronic device 10 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 10 may also include any other appropriate components.

[0191] Exemplary computer program products and computer-readable storage media

[0192] In addition to the above-mentioned methods and devices, an embodiment of the present disclosure may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps in the method of constructing a semantic map according to various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section of this specification.

[0193] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0194] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon, which, when executed by a processor, enable the processor to execute the steps of the method for constructing a semantic map according to various embodiments of the present disclosure described in the above “Exemplary Method” section of this specification.

[0195] The computer readable storage medium may adopt any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may include, for example, but is not limited to, a system, device or device of electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0196] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, it should be noted that the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, and are not limitations. The above details do not limit the present disclosure to the necessity of adopting the above specific details to be implemented.

[0197] The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and / or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

[0198] It should also be noted that in the apparatus, device and method of the present disclosure, each component or each step can be decomposed and / or recombined. Such decomposition and / or recombination should be regarded as equivalent solutions of the present disclosure.

[0199] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present disclosure. Therefore, the present disclosure is not intended to be limited to the aspects shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

[0200] The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present disclosure to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims

1. A method for constructing a semantic map, comprising: Determine a point cloud data set and an image containing a road for constructing a semantic map; wherein the point cloud data set is determined by laser emitted by a vehicle-mounted laser radar; Determining a correspondence between points in the point cloud data set and pixels in the image; Determining the semantics of points in the point cloud dataset based on the semantics of pixels in the image and the corresponding relationship; Building a semantic map based on the semantics of points in the point cloud dataset; Wherein, determining the correspondence between the points in the point cloud data set and the pixels in the image includes: Based on the time when the vehicle-mounted camera captures the image and the time when the vehicle-mounted laser radar emits laser, the first coefficient is determined by the following formula: k = (tc-tl0)\(tl1-tl0); Wherein, k is the first coefficient, tc is the time when the vehicle-mounted camera captures the image, tl0 is the time when the vehicle-mounted laser radar emits the first laser, tl1 is the time when the vehicle-mounted laser radar emits the second laser, the first laser is the laser corresponding to the last frame of point cloud before tc, and the second laser is the laser corresponding to the first frame of point cloud after tc; Determining the position and pose of the laser radar; Based on the position and posture of the laser radar and the first coefficient, the second coefficient is determined by the following formula: Among them, TC is the second coefficient, T1 is the position of the vehicle-mounted laser radar at tl0, T2 is the position of the vehicle-mounted laser radar at tl1, The linear difference between the quaternion interpolation and translation representing the rotation of the pose, Multiplication of the quaternion representing the position and the constant value of the translation vector; Based on the second coefficient, the correspondence between the points in the point cloud data set and the pixels in the image is determined by the following formula: Px=K×T cl ×TC×P L ; Wherein, Px is the coordinate of the pixel in the image in the image coordinate system, K is the intrinsic calibration matrix of the vehicle-mounted camera, T cl is the extrinsic calibration matrix for converting the map coordinate system of the point cloud dataset to the image coordinate system, P L are the coordinates of the point cloud of the second laser in the map coordinate system.

2. The method according to claim 1, wherein: The step of determining a point cloud data set for constructing a semantic map includes: Obtain the original point cloud data set through vehicle-mounted sensors; Motion distortion correction is performed on the points in the original point cloud data set, and the point cloud data set formed by the points after motion distortion correction is the point cloud data set used to construct the semantic map.

3. The method according to claim 1 or 2, wherein: Also includes: Before determining the semantics of the points in the point cloud dataset, a first area in the image is deleted to determine the semantics of the points in the point cloud dataset based on the semantics of the pixels in the image after deleting the first area and the correspondence, wherein the depth of the pixels in the first area is greater than the depth of the pixels in other areas of the image.

4. The method according to claim 1 or 2, wherein: Also includes: Before determining the semantics of the points in the point cloud data set, determining a first object in the image based on the semantics of the pixels in the image and the predetermined semantics of the first object; A first object in the image is deleted to determine semantics of points in the point cloud data set based on semantics of pixels in the image after the first object is deleted and the corresponding relationship.

5. The method according to claim 1 or 2, wherein: Also includes: Before determining the semantics of the points in the point cloud data set, determining a second object in the image based on the semantics of the pixels in the image and a target tracking algorithm, the second object comprising a moving object; The second object in the image is deleted so as to determine the semantics of the points in the point cloud data set based on the semantics of the pixels in the image after the second object is deleted and the corresponding relationship.

6. The method according to claim 1 or 2, characterized in that: The determining the position and posture of the vehicle-mounted laser radar includes: Based on the point cloud data set for building the semantic map, adjusting the white noise and zero bias of the vehicle-mounted inertial measurement unit; The position and posture of the vehicle-mounted laser radar are determined based on the global positioning system and the inertial measurement unit after adjusting the white noise and the zero bias.

7. A device for constructing a semantic map, comprising: A point cloud and image determination module, used to determine a point cloud data set and an image containing a road for constructing a semantic map; wherein the point cloud data set is determined by laser emitted by a vehicle-mounted laser radar; A correspondence determination module, used to determine the correspondence between the points in the point cloud data set determined by the point cloud and image determination module and the pixels in the image; A semantic determination module, configured to determine the semantics of points in the point cloud data set based on the semantics of pixels in the image and the correspondence determined by the correspondence determination module; A map construction module, configured to construct a semantic map based on the semantics of the point determined by the semantic determination module; Wherein, the corresponding relationship determination module is specifically used to determine the first coefficient based on the time when the vehicle-mounted camera shoots the image and the time when the vehicle-mounted laser radar emits the laser, by the following formula: k=(tc-tl0)\(tl1-tl0); wherein k is the first coefficient, tc is the time when the vehicle-mounted camera shoots the image, tl0 is the time when the vehicle-mounted laser radar emits the first laser, tl1 is the time when the vehicle-mounted laser radar emits the second laser, the first laser is the laser corresponding to the last frame of point cloud before tc, and the second laser is the laser corresponding to the first frame of point cloud after tc; Determine the position and posture of the laser radar; determine the second coefficient by a coefficient formula based on the position and posture of the laser radar and the first coefficient; Among them, TC is the second coefficient, T1 is the position of the vehicle-mounted laser radar at tl0, T2 is the position of the vehicle-mounted laser radar at tl1, The linear difference between the quaternion interpolation and translation representing the rotation of the pose, Multiplication of the quaternion representing the position and the constant value of the translation vector; Based on the second coefficient, the correspondence between the points in the point cloud data set and the pixels in the image is determined by the following formula: Px=K×T cl ×TC×P L ; Wherein, Px is the coordinate of the pixel in the image in the image coordinate system, K is the intrinsic calibration matrix of the vehicle-mounted camera, T cl is the extrinsic calibration matrix for converting the map coordinate system of the point cloud dataset to the image coordinate system, P L are the coordinates of the point cloud of the second laser in the map coordinate system.

8. A computer-readable storage medium storing a computer program for executing the method for constructing a semantic map as described in any one of claims 1 to 6.

9. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the method for constructing a semantic map as described in any one of claims 1-6 above.

Images