Three-dimensional map element generation method and device, equipment and storage medium

Abstract

The present disclosure provides a three-dimensional map element generation method and device, electronic equipment and storage medium, relates to the technical field of computer vision and high-definition map, and particularly relates to the field of three-dimensional map element generation. The specific implementation scheme is: obtaining three-dimensional map element data of each frame of to-be-detected image in multiple frames of to-be-detected images; obtaining an element association relationship between a first map element and a second map element based on multiple three-dimensional map element data, wherein the first map element and the second map element are the same map element belonging to different frames of to-be-detected images; and generating a three-dimensional map element in a three-dimensional map based on the element association relationship, wherein the three-dimensional map element is used at least to represent a signboard that needs to be marked in the three-dimensional map.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer vision and high-definition map, and in particular to a three-dimensional map element generation method, device, equipment and storage medium. BACKGROUND

[0002] With the continuous progress of science and technology, high-precision maps (also known as high-definition maps) are being used more and more widely. High-definition maps are electronic maps with high precision and wide data dimensions, and are usually used to serve autonomous driving technology, providing structured road data for autonomous driving for reference in the autonomous driving process; and play a crucial role in autonomous driving positioning, decision-making control and other tasks.

[0003] However, the traditional high-precision map technology acquisition device includes the following several core components: laser radar, inertial measurement unit (IMU), global navigation satellite system (GNSS), high-precision wheel speedometer, and camera, etc., resulting in the high cost and long acquisition update cycle of the high-definition map production method based on laser radar.

[0004] At present, no effective solution has been proposed for the above problems.

[0005] SUMMARY

[0006] The present disclosure provides a method, device, equipment and storage medium for generating a three-dimensional map element.

[0007] According to an aspect of the present disclosure, a three-dimensional map element generation method is provided, comprising: obtaining three-dimensional map element data of each frame of a plurality of frames of to-be-detected images; based on a plurality of three-dimensional map element data, obtaining an element association relationship between a first map element and a second map element, wherein the first map element and the second map element are the same map element belonging to different frames of to-be-detected images; and generating a three-dimensional map element in a three-dimensional map based on the element association relationship, wherein the three-dimensional map element is used to at least represent a signboard that needs to be marked in the three-dimensional map.

[0008] Optionally, the three-dimensional map element data of each frame of the plurality of frames of to-be-detected images is acquired by: acquiring image data of each frame of the to-be-detected images, wherein the image data at least includes: an image timestamp, two-dimensional coordinate information of a map element, and a map element orientation; determining a two-dimensional detection frame corresponding to each frame of the to-be-detected images; and performing detection processing on the image data by using the two-dimensional detection frame to obtain the three-dimensional map element data, wherein the three-dimensional map element data at least includes the following degrees of freedom: three-dimensional coordinate information of a map element center point, a map element orientation, and a map element length.

[0009] Optionally, the element association relationship between the first map element and the second map element is acquired based on the plurality of three-dimensional map element data by: determining, based on the plurality of three-dimensional map element data, a current frame image containing the first map element and an associated frame image containing the second map element in the plurality of frames of to-be-detected images; and establishing the element association relationship between the first map element and the second map element according to the current frame image and the associated frame image.

[0010] Optionally, the element association relationship between the first map element and the second map element is established according to the current frame image and the associated frame image by: acquiring a first pose of the current frame image and a second pose of the associated frame image; determining an epipolar constraint relationship between the first pose and the second pose; determining a first position of the first map element in the current frame image and a second position of the second map element in the associated frame image; and determining the element association relationship between the first map element and the second map element based on the epipolar constraint relationship, the first position, and the second position.

[0011] Optionally, the element association relationship between the first map element and the second map element is established according to the current frame image and the associated frame image by: acquiring a first matching matrix of the current frame image and the associated frame image, wherein the first matching matrix is used to represent a similarity degree between the same map element at different positions in the current frame image and the associated frame image; calculating an epipolar line of a map element center point in the current frame image on the associated frame image and a first relative distance of each map element in the associated frame image to the epipolar line; and establishing the element association relationship between the first map element and the second map element according to the first matching matrix and the first relative distance.

[0012] Optionally, the element correlation between the first map element and the second map element is established based on the current frame image and the two-dimensional map, including: obtaining a second matching matrix of the current frame image and the two-dimensional map, wherein the second matching matrix is used to represent the similarity between the same map element at different positions in the current frame image and the two-dimensional map; projecting the first map element onto the two-dimensional map, and calculating a second relative distance between the first map element and the map element center point of the second map element; and establishing the element correlation between the first map element and the second map element based on the second matching matrix and the second relative distance.

[0013] Optionally, the element correlation between the first map element and the second map element is established based on the current frame image and the two-dimensional map, including: obtaining a second matching matrix of the current frame image and the two-dimensional map, wherein the second matching matrix is used to represent the similarity between the same map element at different positions in the current frame image and the two-dimensional map; projecting the first map element onto the two-dimensional map, and calculating a second relative distance between the first map element and the map element center point of the second map element; and establishing the element correlation between the first map element and the second map element based on the second matching matrix and the second relative distance.

[0014] Optionally, the three-dimensional map element is generated in the three-dimensional map based on the element correlation, including: calculating three-dimensional coordinate information of the map element center point of the three-dimensional map element by using a multi-frame observation estimation algorithm; obtaining three-dimensional element information based on the three-dimensional coordinate information and two-dimensional element information of the three-dimensional map element in the plurality of frames of the to-be-detected image; determining an initial orientation of the three-dimensional map element according to an inverse number of an average yaw angle of the plurality of frames of the to-be-detected image; and generating the three-dimensional map element in the three-dimensional map based on the three-dimensional element information, the initial orientation, and the element correlation.

[0015] Optionally, the method further includes: performing element optimization processing on the plurality of three-dimensional map element data to obtain optimized three-dimensional map element data, wherein the element optimization processing at least includes: initial orientation optimization based on map element orientation constraints, and projection optimization based on three-dimensional map element re-projection errors.

[0016] According to another aspect of the present disclosure, a device for generating a three-dimensional map element is provided, comprising: a first obtaining module configured to obtain three-dimensional map element data of each frame of a plurality of frames of to-be-detected images; a second obtaining module configured to obtain an element association relationship between a first map element and a second map element based on a plurality of the three-dimensional map element data, wherein the first map element and the second map element are the same map element belonging to different frames of to-be-detected images; and a generating module configured to generate a three-dimensional map element in a three-dimensional map based on the element association relationship, wherein the three-dimensional map element is used to at least represent a signboard that needs to be marked in the three-dimensional map.

[0017] Optionally, the first obtaining module comprises: an obtaining unit configured to obtain image data of each frame of the to-be-detected images, wherein the image data at least includes: an image timestamp, two-dimensional coordinate information of a map element, and a map element orientation; a first determining unit configured to determine a two-dimensional detection frame corresponding to each frame of the to-be-detected images; and a detection unit configured to perform detection processing on the image data using the two-dimensional detection frame to obtain the three-dimensional map element data, wherein the three-dimensional map element data at least includes the following degrees of freedom: three-dimensional coordinate information of a map element center point, a map element orientation, and a map element length.

[0018] Optionally, the second obtaining module comprises: a second determining unit configured to determine, based on a plurality of the three-dimensional map element data, a current frame image containing the first map element and an associated frame image containing the second map element in the plurality of frames of to-be-detected images; and a processing unit configured to establish an element association relationship between the first map element and the second map element according to the current frame image and the associated frame image.

[0019] Optionally, the processing unit comprises: a first obtaining subunit configured to obtain a first pose of the current frame image and a second pose of the associated frame image; a first determining subunit configured to determine a epipolar constraint relationship between the first pose and the second pose; a second determining subunit configured to determine a first position of the first map element in the current frame image and a second position of the second map element in the associated frame image; and a third determining subunit configured to determine the element association relationship between the first map element and the second map element based on the epipolar constraint relationship, the first position, and the second position.

[0020] Optionally, the processing unit comprises: a second acquisition subunit, configured to acquire a first matching matrix of the current frame image and the associated frame image, wherein the first matching matrix is used to represent the similarity between the same map elements in different positions in the current frame image and the associated frame image; a calculation subunit, configured to calculate the epipolar line of the map element center point in the current frame image on the associated frame image, and the first relative distance of each map element in the associated frame image to the epipolar line; and a processing subunit, configured to establish the element association relationship between the first map element and the second map element according to the first matching matrix and the first relative distance.

[0021] Optionally, the second acquisition module comprises: a third determination unit, configured to determine, based on the plurality of three-dimensional map element data, a current frame image containing the first map element and a two-dimensional map containing the second map element in the plurality of to-be-detected images; and a second processing unit, configured to establish the element association relationship between the first map element and the second map element according to the current frame image and the two-dimensional map.

[0022] Optionally, the second processing unit comprises: a third acquisition subunit, configured to acquire a second matching matrix of the current frame image and the two-dimensional map, wherein the second matching matrix is used to represent the similarity between the same map elements in different positions in the current frame image and the two-dimensional map; a second calculation subunit, configured to project the first map element onto the two-dimensional map and calculate the second relative distance between the map element center points of the first map element and the second map element; and a second processing subunit, configured to establish the element association relationship between the first map element and the second map element according to the second matching matrix and the second relative distance.

[0023] Optionally, the generation module comprises: a calculation unit, configured to calculate the three-dimensional coordinate information of the map element center point of the three-dimensional map element by using a multi-frame observation estimation algorithm; a third processing unit, configured to obtain three-dimensional element information according to the three-dimensional coordinate information and the two-dimensional element information of the three-dimensional map element in the plurality of to-be-detected images; a third determination unit, configured to determine the initial orientation of the three-dimensional map element according to the inverse of the average yaw angle of the plurality of to-be-detected images; and a generation unit, configured to generate the three-dimensional map element in the three-dimensional map based on the three-dimensional element information, the initial orientation, and the element association relationship.

[0024] Optionally, the device is further configured to perform element optimization processing on the plurality of three-dimensional map element data to obtain optimized three-dimensional map element data, wherein the element optimization processing comprises at least initial orientation optimization based on a map element orientation constraint and projection optimization based on a three-dimensional map element reprojection error.

[0025] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform any one of the three-dimensional map element generation methods described above.

[0026] According to another aspect of the present disclosure, a non-transitory computer readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to perform any one of the three-dimensional map element generation methods described above.

[0027] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein the computer program, when executed by a processor, implements any one of the three-dimensional map element generation methods described above.

[0028] It should be understood that the content described in this section is not intended to identify key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0029] The accompanying drawings are used to better understand the present scheme, and do not constitute a limitation on the present disclosure. Among them:

[0030] FIG. 1 is a step flowchart of a three-dimensional map element generation method according to the first embodiment of the present disclosure;

[0031] FIG. 2 is a sign reconstruction flowchart according to the first embodiment of the present disclosure;

[0032] FIG. 3 is a sign association relationship diagram between images according to the first embodiment of the present disclosure;

[0033] FIG. 4 is a sign association relationship diagram between a map and an image according to the first embodiment of the present disclosure;

[0034] FIG. 5 is a three-dimensional sign projection diagram according to the first embodiment of the present disclosure;

[0035] FIG. 6is a structural schematic diagram of a three-dimensional map element generation apparatus according to a second embodiment of the present disclosure;

[0036] FIG. 7 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0037] Exemplary embodiments of the present disclosure are described herein with reference to the accompanying drawings, which are included to provide a thorough understanding of embodiments of the present disclosure by a person of ordinary skill in the art, and should not be construed as a literal limitation to the scope of the present disclosure. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, in the following description, descriptions of well-known functions and constructions are omitted for clarity and conciseness.

[0038] It should be noted that the terms "first", "second", and the like in the description and claims of the present disclosure and the above drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described herein can be implemented in other than the order illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product, or device that includes a list of steps or units does not necessarily limit to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to such processes, methods, products, or devices.

[0039] According to embodiments of the present disclosure, an embodiment of a three-dimensional map element generation method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0040] Embodiment 1

[0041] FIG. 1 is a step flowchart of a three-dimensional map element generation method according to a first embodiment of the present disclosure, as shown in FIG. 1 the method comprises the following steps:

[0042] In step S102, three-dimensional map element data of each frame of the plurality of frames of to-be-detected images is obtained.

[0043] In step S104, an element association relationship between a first map element and a second map element is obtained based on a plurality of the three-dimensional map element data, wherein the first map element and the second map element are the same map element belonging to different frames of the to-be-detected images.

[0044] In step S106, a three-dimensional map element is generated in the three-dimensional map based on the element association relationship, wherein the three-dimensional map element is used at least to represent a signboard that needs to be marked in the three-dimensional map.

[0045] In the embodiments of the present disclosure, a three-dimensional reconstruction system is used to obtain a plurality of frames of to-be-detected images obtained by a rolling shutter camera, and three-dimensional map element data in each frame of the to-be-detected images is obtained. An element association relationship of the same map element in different frames of the to-be-detected images is obtained based on a plurality of the three-dimensional map element data, and finally a three-dimensional map element is generated in the three-dimensional map based on the element association relationship.

[0046] It should be noted that the element association relationship of the same map element in different frames of the to-be-detected images, that is, the element association relationship between the first map element and the second map element, wherein the first map element and the second map element are the same map element belonging to different frames of the to-be-detected images. The three-dimensional map element is used at least to represent a signboard that needs to be marked in the three-dimensional map.

[0047] As an optional embodiment, taking the element visual reconstruction process of the signboard in the three-dimensional map as an example, as shown in a signboard reconstruction flowchart, FIG. 2 a rolling shutter camera is used to obtain a plurality of frames of to-be-detected images of the plurality of signboards. The obtained frames of to-be-detected images are input into a three-dimensional reconstruction system as input data, and the three-dimensional reconstruction system is used to determine whether the input data, that is, the plurality of frames of to-be-detected images, is processed. If the processing is completed, the signboard data is output as three-dimensional map data and stored. If the processing is not completed, signboard element association, signboard 3D element construction, signboard 3D element optimization and the like are performed. After the processing is completed, the three-dimensional reconstruction system is used again to determine whether the processed data is processed. After the processing is completed, the signboard data is output as three-dimensional map data and stored.

[0048] In an optional embodiment, the three-dimensional map element data of each frame of the to-be-detected images in the plurality of frames of to-be-detected images includes:

[0049] In step S202, image data of each frame of the to-be-detected images is obtained, wherein the image data at least includes: image timestamp, two-dimensional coordinate information of a map element, and map element orientation.

[0050] In step S204, a two-dimensional detection frame corresponding to each frame of the to-be-detected images is determined.

[0051] Step S206, the image data is detected by using the two-dimensional detection box to obtain the three-dimensional map element data, wherein the three-dimensional map element data at least contains the following degrees of freedom: three-dimensional coordinate information of the map element center point, map element orientation and map element length.

[0052] In the embodiments of the present disclosure, the image data of each frame of the to-be-detected image is obtained by using the three-dimensional reconstruction system, and the two-dimensional detection box corresponding to each frame of the to-be-detected image is determined; the image data is detected by using the two-dimensional detection box to obtain the three-dimensional map element data.

[0053] It should be noted that the image data at least contains: image timestamp, two-dimensional coordinate information of the map element, map element orientation; the three-dimensional map element data at least contains the following degrees of freedom: three-dimensional coordinate information of the map element center point, map element orientation and map element length.

[0054] As an optional embodiment, the data input in the process of visual reconstruction mainly contains the timestamp of the to-be-detected image, the position and orientation of the to-be-detected image, and the two-dimensional sign detection box corresponding to the to-be-detected image of the sign in the input data is determined, the two-dimensional sign detection box is detected, and the element data of the sign in the three-dimensional map is output.

[0055] It should be noted that the position and orientation of the to-be-detected image can be obtained by fusion positioning methods such as inertial navigation system IMU and global navigation satellite system GNSS; the two-dimensional detection box of the sign is detected in the deformed image; the element data of the sign in the three-dimensional map contains 6 degrees of freedom, including: x, y, z, yaw, w, h; wherein x, y, z constitute the coordinates (x, y, z) of the center point of the sign, yaw is the orientation of the sign, w is the width of the sign, and h is the height of the sign. In addition, semantic detection can be performed on the image of the sign, and the category of the sign is obtained according to the result of semantic detection. The category is determined according to the classification result of semantic detection, which can be different types such as speed limit value, road sign, etc.

[0056] In an optional embodiment, the element association relationship between the first map element and the second map element is obtained based on a plurality of the three-dimensional map element data, including:

[0057] Step S302, based on a plurality of the three-dimensional map element data, a current frame image containing the first map element and an associated frame image containing the second map element in a plurality of the to-be-detected images are determined;

[0058] Step S304, establishing an element association relationship between the first map element and the second map element according to the current frame image and the associated frame image.

[0059] In the embodiments of the present disclosure, the three-dimensional reconstruction system is used to determine a current frame image containing the first map element and an associated frame image containing the second map element in a plurality of to-be-detected images according to a plurality of three-dimensional map element data, and establish an element association relationship between the first map element and the second map element according to the current frame image and the associated frame image.

[0060] It should be noted that the element association relationship between the first map element and the second map element includes an element association relationship between images and an element association relationship between maps and images.

[0061] As an optional embodiment, the detection result of the signboard image is input into a flow as shown in FIG. 2 To reconstruct subsequent elements, the detection results of different signboard images need to be associated first to find the association relationship of the same signboard in different images (different time stamps).

[0062] In an optional embodiment, the establishing of the element association relationship between the first map element and the second map element according to the current frame image and the associated frame image includes:

[0063] Step S402, obtaining a first pose of the current frame image and a second pose of the associated frame image;

[0064] Step S404, determining an epipolar constraint relationship between the first pose and the second pose;

[0065] Step S406, determining a first position of the first map element in the current frame image and a second position of the second map element in the associated frame image;

[0066] Step S408, determining an element association relationship between the first map element and the second map element based on the epipolar constraint relationship, the first position and the second position.

[0067] As an optional embodiment, as shown in FIG. 3The image-image signboard association relationship diagram shown first acquires the first pose of the signboard in the current frame image and the second pose of the signboard in the associated frame image; determines the epipolar constraint relationship between the first pose and the second pose; determines the first position of the signboard element in the first map in the current frame image and the second position of the corresponding signboard element in the second map in the associated frame image; and determines the element association relationship between the signboard element of the first map and the signboard element of the second map based on the epipolar constraint relationship, the first position and the second position.

[0068] It should be noted that the epipolar constraint relationship refers to that if the pose information of two images is known, the signboard detection center point in one image will correspond to a straight line in the current image, and the equation of the straight line can be obtained by calculation. The intersection point formed by the intersection of the epipolar line is the epipolar point. The distance from the signboard center in the current image to the straight line can be used as a similarity to judge the matching relationship of the detection boxes in the two images, and the matching pair of images is determined according to the matching relationship.

[0069] In an optional embodiment, the element association relationship between the first map element and the second map element is established according to the current frame image and the associated frame image, comprising:

[0070] Step S502, acquiring a first matching matrix of the current frame image and the associated frame image, wherein the first matching matrix is used to represent the similarity between the same map elements in different positions in the current frame image and the associated frame image;

[0071] Step S504, calculating the epipolar line of the map element center point in the current frame image on the associated frame image, and the first relative distance of each map element in the associated frame image to the epipolar line;

[0072] Step S506, establishing the element association relationship between the first map element and the second map element according to the first matching matrix and the first relative distance.

[0073] In the embodiments of the present disclosure, the first matching matrix of the signboard in the current frame image and the associated frame image is first initialized, assuming that the current frame image contains M signboards and the associated frame image contains N signboards, the size of the first matching matrix is M*N; the epipolar line of the signboard center point in the current frame image on the associated frame image is calculated, and the first relative distance (relative position) of the signboard to each epipolar line in the associated frame image is calculated, and the first line pair distance of multiple signboards is used to fill the first matching matrix; the dissimilarity of the mis-matched signboard is set to be the maximum according to the shape, size and type of the signboard; the element association relationship between the first map signboard element and the second map signboard element is established.

[0074] It should be noted that the first matching matrix is used to represent the similarity between the same map elements in different positions in the current frame image and the associated frame image; the element association relationship of the signboard in the first matching matrix can be obtained by using the Hungarian algorithm or the nearest neighbor matching algorithm.

[0075] It should be further noted that the matching pair is used to represent the matching relationship of the detection frame in the two images, and the matching matrix stores the similarity of the two matching pairs in the image, for example, the first image (the current frame image) contains M signboards, and M detection frames can be obtained, the second image (the associated frame image) contains N signboards, and N detection frames can be obtained, then the two detection frames in the image will form a matching pair, and there are M*N matching pairs in total, and a M*N matching matrix is obtained, which stores the similarity between the detection frames; the association relationship between the signboard elements in the matching matrix can be determined by using a method similar to solving bipartite graph matching.

[0076] In an optional embodiment, the element association relationship between the first map element and the second map element is obtained based on the plurality of three-dimensional map element data, comprising:

[0077] In step S602, based on the plurality of three-dimensional map element data, a current frame image containing the first map element and a two-dimensional map containing the second map element in a plurality of detection images are determined.

[0078] In step S604, the element association relationship between the first map element and the second map element is established according to the current frame image and the two-dimensional map.

[0079] In the embodiments of the present disclosure, based on the plurality of three-dimensional map element data, a current frame image containing the first map element and a two-dimensional map containing the second map element are determined from a plurality of above-mentioned to-be-detected images; and an element association relationship between the first map element and the second map element is established according to the current frame image and the two-dimensional map.

[0080] As an optional embodiment, as shown in a map-image signboard association relationship schematic diagram, FIG. 4 the signboard element in the three-dimensional map is projected onto the two-dimensional map of the second map signboard element; the relative distance between the signboard in the three-dimensional map and the signboard center point in each two-dimensional map is calculated; the element association relationship between the first map signboard element and the second map signboard element is established according to the shape, size and type of the signboard.

[0081] In an optional embodiment, the element association relationship between the first map element and the second map element is established according to the current frame image and the two-dimensional map, including:

[0082] In step S702, a second matching matrix of the current frame image and the two-dimensional map is obtained, wherein the second matching matrix is used to represent the similarity between the same map elements at different positions in the current frame image and the two-dimensional map;

[0083] In step S704, the first map element is projected onto the two-dimensional map, and a second relative distance between the map element center point of the first map element and the second map element is calculated;

[0084] In step S706, the element association relationship between the first map element and the second map element is established according to the second matching matrix and the second relative distance.

[0085] In the embodiments of the present disclosure, the number of signboards in the current frame image and the number of signboards in the two-dimensional map are obtained, a second matching matrix is constructed according to the number of signboards, the first map element is projected onto the two-dimensional map, and a second relative distance between the map element center point of the first map element and the second map element is calculated; and the element association relationship between the first map element and the second map element is established according to the second matching matrix and the second relative distance.

[0086] It should be noted that the second matching matrix is used to represent the similarity between the same map elements at different positions in the current frame image and the two-dimensional map.

[0087] In an optional embodiment, the above-mentioned generating a three-dimensional map element in the three-dimensional map based on the element association relationship comprises:

[0088] In step S802, a multi-frame observation estimation algorithm is used to calculate three-dimensional coordinate information of a map element center point of the three-dimensional map element.

[0089] In step S804, three-dimensional element information is obtained according to the three-dimensional coordinate information and two-dimensional element information of the three-dimensional map element in the plurality of frames of the to-be-detected images.

[0090] In step S806, an initial orientation of the three-dimensional map element is determined according to an inverse of a mean value of yaw angles of the plurality of frames of the to-be-detected images.

[0091] In step S808, the three-dimensional map element is generated in the three-dimensional map based on the three-dimensional element information, the initial orientation and the element association relationship.

[0092] In the embodiments of the present disclosure, the three-dimensional coordinate of the sign center point is first calculated by triangulation using a multi-frame observation estimation algorithm; three-dimensional element information is obtained according to the three-dimensional coordinate information and two-dimensional element information of the three-dimensional map element in the plurality of frames of the to-be-detected images, including estimating the depth of the sign and the width and height of the sign in the image, and further calculating the real width and height of the three-dimensional sign; if the observed orientation of the sign is always opposite to the orientation of the vehicle, the inverse of the mean value of the observation frame angle can be used as the initial orientation of the sign; finally, the three-dimensional map element is generated in the three-dimensional map based on the three-dimensional element information, the initial orientation and the element association relationship.

[0093] It should be noted that the three-dimensional coordinate of the sign center point can be obtained by triangulation processing of the sign detection center points in several frames of images according to the matching relationship of the sign; the depth of the sign is used to represent the distance from the three-dimensional coordinate of the sign center point to each image.

[0094] Optionally, a track variable can be maintained for the three-dimensional sign, and the track variable records the two-dimensional sign detection frame in each image in real time. When the three-dimensional sign is associated with a new detection frame, it is first determined whether the observation is consistent with other detection frames in the track variable (reprojection error), and if the observation is successful, the information is added to the track variable, so that the constraint relationship between the three-dimensional sign and the two-dimensional sign detection frame can be fully obtained.

[0095] It should be noted that the constraint relationship is used to represent which detection results of the three-dimensional sign and the image are the same object (which can be obtained through the previous association relationship and filtered through the subsequent process).

[0096] In an optional embodiment, the method further comprises:

[0097] In step S902, the plurality of three-dimensional map element data is subjected to element optimization processing to obtain optimized three-dimensional map element data, wherein the element optimization processing at least includes initial orientation optimization based on map element orientation constraints and projection optimization based on three-dimensional map element reprojection errors.

[0098] In the embodiments of the present disclosure, the plurality of three-dimensional map element data is subjected to element optimization processing, and the element optimization processing at least includes initial orientation optimization based on map element orientation constraints and projection optimization based on three-dimensional map element reprojection errors, to obtain optimized three-dimensional map element data.

[0099] As an optional embodiment, in the optimization process of the actual signboard, as shown in the three-dimensional signboard projection schematic diagram, the projection of the signboard exceeds the boundary of the image, and the projection is not completely aligned with the image axis and has a certain rotation, which will directly lead to the misalignment of the projection and detection results of the signboard. To this end, the minimum circumscribed quadrilateral of the projected signboard in the image area, i.e., the area of the projection result (x_min, y_min, x_max, y_max) in the image, is calculated, and the residual formed by the minimum quadrilateral and the detection frame is optimized. FIG. 5

[0100] As an optional embodiment, compared with global shutter cameras, rolling shutter cameras are more widely used and have lower cost in the prior art, but such cameras have certain problems; each row in the image is not written at the same time, but has a sequence, and the first row writing time and the last row writing time of the camera can differ by 50 ms. If the vehicle speed is calculated at 20 m / s, the front and rear distances will differ by 1 m. Such differences result in the inability to use a single pose to represent all the contents in the image, and ultimately lead to a significant decrease in the reconstruction accuracy.

[0101] In the embodiments of the present disclosure, in order to eliminate the negative effects of the rolling shutter, the pose information of adjacent cameras is used to obtain the accurate pose of each row in the image by using the difference, and the accurate pose is used in the optimization, which greatly improves the accuracy of the signboard reconstruction; for the category of the signboard, the category attribute of the 3D signboard is obtained by weighting processing according to the observation distance according to the two-dimensional detection result associated with the three-dimensional signboard.

[0102] ​According to the embodiment of the present disclosure, a signboard can be reconstructed in real time, laser point cloud is taken as a true value, and the reconstruction accuracy of the signboard obtained by evaluation reaches within 1 m. The updating speed of the high-precision map can be improved, the production cost of the high-precision map can be reduced, and the data return requirement of the crowdsourcing mode can be met. In the embodiment of the present disclosure, the mapping efficiency is high, the time consumption for mapping per kilometer is 10 s, the mapping can be truly scaled, the reconstruction accuracy of the signboard can reach within 1 m, and the service for autonomous driving can be provided to meet the requirements of autonomous driving.

[0103] In the prior art, the traditional high-precision map mainly uses a laser radar to reconstruct road elements, which has a high cost and needs a professional acquisition vehicle, resulting in a long updating cycle of the map. On the other hand, a traditional visual-based reconstruction technology SFM (Structure from motion) needs to rely on multiple images to complete the three-dimensional reconstruction of a scene, needs a large amount of data, and is difficult to scale, and therefore is difficult to apply to the crowdsourcing mode.

[0104] In the embodiment of the present disclosure, professional high-precision map acquisition data is used to acquire three-dimensional data of road elements by a laser radar, semantic information is given to each road element in combination with the result of image perception, and the mapping of the high-precision map is completed; positioning data and image data are used to reconstruct visual sparse point cloud by using a visual three-dimensional reconstruction algorithm. In addition, semantic information in the image is extracted by a perception model and given to the visual point cloud, and the mapping of the map is completed.

[0105] Embodiment 2

[0106] According to the embodiment of the present disclosure, a device embodiment for implementing the three-dimensional map element generation method is also provided, FIG. 6 is a structural schematic diagram of the three-dimensional map element generation device according to the second embodiment of the present disclosure, as FIG. 6 shown, the three-dimensional map element generation device includes a first acquisition module 60, a second acquisition module 62, and a generation module 64, wherein:

[0107] The first acquisition module 60 is configured to acquire three-dimensional map element data of each frame of the to-be-detected image in the multiple frames of to-be-detected images.

[0108] The second acquisition module 62 is configured to acquire an element association relationship between a first map element and a second map element based on the multiple three-dimensional map element data, wherein the first map element and the second map element are the same map element belonging to different frames of to-be-detected images.

[0109] The generation module 64 is configured to generate a three-dimensional map element in a three-dimensional map based on the element association relationship, wherein the three-dimensional map element is at least used to represent a signboard that needs to be marked in the three-dimensional map.

[0110] In the embodiments of the present disclosure, the first obtaining module comprises: an obtaining unit, configured to obtain image data of each frame of the to-be-detected image, wherein the image data at least comprises: an image timestamp, two-dimensional coordinate information of a map element, and a map element orientation; a first determining unit, configured to determine a two-dimensional detection frame corresponding to each frame of the to-be-detected image; and a detection unit, configured to detect the image data by using the two-dimensional detection frame to obtain three-dimensional map element data, wherein the three-dimensional map element data at least comprises the following degrees of freedom: three-dimensional coordinate information of a map element center point, a map element orientation, and a map element length.

[0111] In the embodiments of the present disclosure, the second obtaining module comprises: a second determining unit, configured to determine, based on a plurality of the three-dimensional map element data, a current frame image containing the first map element and an associated frame image containing the second map element in a plurality of the to-be-detected images; and a processing unit, configured to establish an element association relationship between the first map element and the second map element according to the current frame image and the associated frame image.

[0112] In the embodiments of the present disclosure, the processing unit comprises: a first obtaining subunit, configured to obtain a first pose of the current frame image and a second pose of the associated frame image; a first determining subunit, configured to determine an epipolar constraint relationship between the first pose and the second pose; a second determining subunit, configured to determine a first position of the first map element in the current frame image and a second position of the second map element in the associated frame image; and a third determining subunit, configured to determine an element association relationship between the first map element and the second map element based on the epipolar constraint relationship, the first position, and the second position.

[0113] In the embodiments of the present disclosure, the processing unit comprises: a second obtaining subunit, configured to obtain a first matching matrix of the current frame image and the associated frame image, wherein the first matching matrix is used to represent a similarity degree between the same map element in different positions in the current frame image and the associated frame image; a calculating subunit, configured to calculate an epipolar line of a map element center point in the current frame image on the associated frame image and a first relative distance of each map element in the associated frame image to the epipolar line; and a processing subunit, configured to establish an element association relationship between the first map element and the second map element according to the first matching matrix and the first relative distance.

[0114] In the embodiments of the present disclosure, the second obtaining module comprises: a third determining unit configured to determine, based on the plurality of three-dimensional map element data, a current frame image containing the first map element and a two-dimensional map containing the second map element in the plurality of to-be-detected images; and a second processing unit configured to establish an element association relationship between the first map element and the second map element according to the current frame image and the two-dimensional map.

[0115] In the embodiments of the present disclosure, the second processing unit comprises: a third obtaining subunit configured to obtain a second matching matrix of the current frame image and the two-dimensional map, wherein the second matching matrix is used to represent the similarity between the same map element at different positions in the current frame image and the two-dimensional map; a second calculating subunit configured to project the first map element onto the two-dimensional map and calculate a second relative distance between the first map element and a map element center point of the second map element; and a second processing subunit configured to establish the element association relationship between the first map element and the second map element according to the second matching matrix and the second relative distance.

[0116] In the embodiments of the present disclosure, the generating module comprises: a calculating unit configured to calculate three-dimensional coordinate information of a map element center point of the three-dimensional map element by using a multi-frame observation estimation algorithm; a third processing unit configured to obtain three-dimensional element information according to the three-dimensional coordinate information and two-dimensional element information of the three-dimensional map element in the plurality of to-be-detected images; a third determining unit configured to determine an initial orientation of the three-dimensional map element according to the inverse of the average yaw angle of the plurality of to-be-detected images; and a generating unit configured to generate the three-dimensional map element in the three-dimensional map based on the three-dimensional element information, the initial orientation, and the element association relationship.

[0117] In the embodiments of the present disclosure, the apparatus is further configured to perform element optimization processing on the plurality of three-dimensional map element data to obtain optimized three-dimensional map element data, wherein the element optimization processing at least comprises: initial orientation optimization based on a map element orientation constraint, and projection optimization based on a three-dimensional map element re-projection error.

[0118] It should be noted that each of the above modules can be implemented by software or hardware. For example, for the latter, the modules can be located in the same processor, or in different processors in any combination.

[0119] It should be noted that the first obtaining module 60, the second obtaining module 62 and the generating module 64 correspond to steps S102 to S106 in Embodiment 1, and the modules have the same instances and application scenarios as the corresponding steps, but are not limited to the disclosure in Embodiment 1. It should be noted that the modules can be run in a computer terminal as part of the device.

[0120] It should be noted that the optional or preferred embodiments of the present embodiment can refer to the related description in Embodiment 1, which will not be repeated here.

[0121] The three-dimensional map element generation device described above can further include a processor and a memory, and the first obtaining module 60, the second obtaining module 62 and the generating module 64 and the like are stored in the memory as program units, and the processor executes the program units stored in the memory to realize the corresponding functions.

[0122] The processor contains a core, and the core retrieves the corresponding program unit from the memory. The core can be set to one or more. The memory can include a non-permanent memory in a computer readable medium, a random access memory (RAM) and / or a non-volatile memory such as a read-only memory (ROM) or a flash memory (flash RAM), and the memory includes at least one memory chip.

[0123] In the technical solution of the present disclosure, the acquisition, storage and application of user personal information comply with relevant laws and regulations and do not violate public order and good customs.

[0124] According to the embodiments of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium and a computer program product.

[0125] FIG. 7 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular telephones, smart phones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit implementations of the present disclosure described and / or claimed in this document.

[0126] As FIG. 7As shown, the device 800 includes a computing unit 801 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0127] A plurality of components in the device 800 are connected to the I / O interface 805, including an input unit 806 such as a keyboard, a mouse, and the like; an output unit 807 such as various types of displays, speakers, and the like; the storage unit 808 such as a magnetic disk, an optical disk, and the like; and a communication unit 809 such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0128] The computing unit 801 can be various general and / or special-purpose processing components having processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, and the like. The computing unit 801 performs various methods and processes described above, such as the method of obtaining three-dimensional map element data for each of a plurality of frames of to-be-detected images. For example, in some embodiments, the method of obtaining three-dimensional map element data for each of a plurality of frames of to-be-detected images can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the method of obtaining three-dimensional map element data for each of a plurality of frames of to-be-detected images described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform the method of obtaining three-dimensional map element data for each of a plurality of frames of to-be-detected images by any other appropriate means, such as by means of firmware.

[0129] The various embodiments of the systems and techniques described above can be implemented in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a load programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0130] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or the block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, or entirely on a remote machine or server.

[0131] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical conductors, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0132] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0133] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0134] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server is generally established by computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0135] It should be understood that various forms of flow shown above can be used, with steps reordered, added, or removed. For example, steps recited in the present disclosure can be performed in parallel, in series, or in a different order, without limitation, as long as the desired results of the technology of the present disclosure are achieved.

[0136] The specific embodiments described above are not intended to limit the scope of the present disclosure. Those skilled in the art will understand that various modifications, combinations, sub-combinations, and alternatives can be made to the specific embodiments without departing from the spirit and principles of the present disclosure. Any further modifications, equivalent substitutions, improvements, and the like, either alone or in some combination or sub-combination, are intended to be included within the scope of the present disclosure.

Claims

1. A method for generating a three-dimensional map element, comprising: obtaining three-dimensional map element data of each of a plurality of frames of to-be-detected images; based on a plurality of the three-dimensional map element data, obtaining an element association relationship between a first map element and a second map element, wherein the first map element and the second map element are the same map element belonging to different frames of to-be-detected images, and the element association relationship includes an element association relationship between images and an element association relationship between a map and an image; using a multi-frame observation estimation algorithm, calculating three-dimensional coordinate information of a map element center point of the three-dimensional map element; obtaining three-dimensional element information according to the three-dimensional coordinate information and two-dimensional element information of the three-dimensional map element in a plurality of frames of the to-be-detected images; determining an initial orientation of the three-dimensional map element according to an inverse of an average yaw angle of a plurality of frames of the to-be-detected images; and generating the three-dimensional map element in the three-dimensional map based on the three-dimensional element information, the initial orientation, and the element association relationship, wherein the three-dimensional map element is used to at least represent a signboard that needs to be marked in the three-dimensional map.

2. The method of claim 1, wherein, The obtaining of the three-dimensional map element data of each of a plurality of frames of to-be-detected images comprises: obtaining image data of each of the frames of to-be-detected images, wherein the image data at least includes an image timestamp, two-dimensional coordinate information of a map element, and a map element orientation; determining a two-dimensional detection frame corresponding to each of the frames of to-be-detected images; detecting the image data using the two-dimensional detection frame to obtain the three-dimensional map element data, wherein the three-dimensional map element data at least includes the following degrees of freedom: three-dimensional coordinate information of a map element center point, a map element orientation, and a map element length.

3. The method of claim 1, wherein, The obtaining of the element association relationship between the first map element and the second map element based on a plurality of the three-dimensional map element data comprises: based on a plurality of the three-dimensional map element data, determining a current frame image containing the first map element and an associated frame image containing the second map element in a plurality of frames of the to-be-detected images; establishing an element association relationship between the first map element and the second map element according to the current frame image and the associated frame image.

4. The method of claim 3, wherein, The establishing of the element association relationship between the first map element and the second map element according to the current frame image and the associated frame image comprises: obtaining a first pose of the current frame image and a second pose of the associated frame image; determining an epipolar constraint relationship between the first pose and the second pose; determining a first position of the first map element in the current frame image and a second position of the second map element in the associated frame image; based on the epipolar constraint relationship, the first position, and the second position, determining the element association relationship between the first map element and the second map element.

5. The method of claim 3, wherein, The establishing of the element association relationship between the first map element and the second map element according to the current frame image and the associated frame image comprises: obtaining a first matching matrix of the current frame image and the associated frame image, wherein the first matching matrix is used to represent a similarity degree between the same map element in different positions in the current frame image and the associated frame image; calculating an epipolar line of a map element center point in the current frame image on the associated frame image, and a first relative distance of each map element in the associated frame image to the epipolar line; establishing an element association relationship between the first map element and the second map element according to the first matching matrix and the first relative distance.

6. The method of claim 1, wherein, The method further comprises: performing element optimization processing on the plurality of three-dimensional map element data to obtain optimized three-dimensional map element data, wherein the element optimization processing at least includes: initial orientation optimization based on map element orientation constraint, and projection optimization based on three-dimensional map element reprojection error.

9. A three-dimensional map element generation device, comprising:

7. The method of claim 6, wherein, a first obtaining module configured to obtain three-dimensional map element data of each frame of a plurality of frames of to-be-detected images; a second obtaining module configured to obtain an element association relationship between a first map element and a second map element based on a plurality of the three-dimensional map element data, wherein the first map element and the second map element are the same map element belonging to different frames of to-be-detected images, and the element association relationship includes an element association relationship between images and images and an element association relationship between a map and an image. ​ ​ 8. The method of claim 1, wherein, ​ ​ ​ ​ ​ The generating module is configured to calculate three-dimensional coordinate information of a map element center point of the three-dimensional map element by using a multi-frame observation estimation algorithm; obtain three-dimensional element information according to the three-dimensional coordinate information and two-dimensional element information of the three-dimensional map element in the plurality of frames of the to-be-detected images; and determine an initial orientation of the three-dimensional map element according to an inverse number of an average yaw angle of the plurality of frames of the to-be-detected images; and generate the three-dimensional map element in the three-dimensional map based on the three-dimensional element information, the initial orientation and the element association relationship, wherein the three-dimensional map element is used to at least represent a signboard that needs to be marked in the three-dimensional map.

10. The apparatus of claim 9, wherein, The first obtaining module comprises: An obtaining unit is configured to obtain image data of each frame of the to-be-detected images, wherein the image data at least includes: an image timestamp, two-dimensional coordinate information of a map element, and a map element orientation; A first determining unit is configured to determine a two-dimensional detection box corresponding to each frame of the to-be-detected images; A detection unit is configured to detect the image data by using the two-dimensional detection box to obtain three-dimensional map element data, wherein the three-dimensional map element data at least includes the following degrees of freedom: three-dimensional coordinate information of a map element center point, a map element orientation and a map element length.

11. The apparatus of claim 9, wherein, The second obtaining module comprises: A second determining unit is configured to determine, based on a plurality of the three-dimensional map element data, a current frame image containing the first map element and an associated frame image containing the second map element in the plurality of frames of the to-be-detected images; A processing unit is configured to establish an element association relationship between the first map element and the second map element according to the current frame image and the associated frame image.

12. The apparatus of claim 11, wherein, The processing unit comprises: A first obtaining subunit is configured to obtain a first pose of the current frame image and a second pose of the associated frame image; A first determining subunit is configured to determine a epipolar constraint relationship between the first pose and the second pose; A second determining subunit is configured to determine a first position of the first map element in the current frame image and a second position of the second map element in the associated frame image; A third determining subunit is configured to determine an element association relationship between the first map element and the second map element based on the epipolar constraint relationship, the first position and the second position.

13. The apparatus of claim 11, wherein, The processing unit comprises: A second obtaining subunit is configured to obtain a first matching matrix of the current frame image and the associated frame image, wherein the first matching matrix is used to represent a similarity degree between the same map element in different positions in the current frame image and the associated frame image; A calculating subunit is configured to calculate an epipolar line of a map element center point in the current frame image on the associated frame image and a first relative distance of each map element in the associated frame image to the epipolar line; A processing subunit is configured to establish an element association relationship between the first map element and the second map element according to the first matching matrix and the first relative distance.

14. The apparatus of claim 9, wherein, The second obtaining module comprises: a third determining unit, configured to determine, based on the plurality of pieces of three-dimensional map element data, a current frame image containing the first map element and a two-dimensional map containing the second map element in the plurality of pieces of the to-be-detected image; a second processing unit, configured to establish an element association relationship between the first map element and the second map element according to the current frame image and the two-dimensional map.

15. The apparatus of claim 14, wherein, The second processing unit comprises: a third obtaining sub-unit, configured to obtain a second matching matrix of the current frame image and the two-dimensional map, where the second matching matrix is used to represent a similarity degree between the same map element at different positions in the current frame image and the two-dimensional map; a second calculating sub-unit, configured to project the first map element onto the two-dimensional map and calculate a second relative distance between a map element center point of the first map element and a map element center point of the second map element; a second processing sub-unit, configured to establish the element association relationship between the first map element and the second map element according to the second matching matrix and the second relative distance.

16. The apparatus of claim 9, wherein, The apparatus is further configured to perform element optimization processing on the plurality of pieces of three-dimensional map element data to obtain optimized three-dimensional map element data, where the element optimization processing at least includes initial orientation optimization based on a map element orientation constraint and projection optimization based on a three-dimensional map element re-projection error.

17. An electronic device, comprising: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the three-dimensional map element generation method in any one of claims 1-8.

18. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to perform the three-dimensional map element generation method according to any one of claims 1-8.

19. A computer program product, comprising a computer program, which, when executed by a processor, implements the three-dimensional map element generation method according to any one of claims 1-8.

19. A computer program product, comprising a computer program, which, when executed by a processor, implements the three-dimensional map element generation method according to any one of claims 1-8.

Images