High-precision map feature extraction method, device, medium and electronic equipment

Abstract

This application discloses a method, device, medium, and electronic device for extracting high-precision map features, relating to the field of artificial intelligence, particularly computer vision, and specifically autonomous driving, high-precision mapping, and intelligent transportation. A specific implementation scheme comprises: acquiring at least two frames of road images captured from different lanes of the same road area; extracting the original geometric features of road markings from each of the at least two frames of road images; and fusing the at least two original geometric features to obtain the target geometric features of the road markings. By implementing the technical solution provided in this application to extract the geometric features of road markings, more accurate geometric features can be extracted.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence, in particular to the field of computer vision, and specifically to the fields of autonomous driving, high-precision maps, and intelligent transportation. Background Art

[0002] High-precision maps, also known as high-accuracy maps, are used by autonomous vehicles. These maps contain precise vehicle location information and rich road element data, helping vehicles predict complex road conditions, such as slope, curvature, and heading, to better mitigate potential risks. Road markings are prevalent on urban roads. In the processes of autonomous driving, high-precision map production, and intelligent traffic management, road marking extraction is often performed based on camera-captured road images. However, due to the perspective of acquisition, road markings in road images are often missing or distorted to a certain extent, seriously affecting the accuracy of road marking feature extraction. Summary of the Invention

[0003] The present application discloses a high-precision map feature extraction method, device, medium and electronic equipment for extracting geometric features of road surface markings, so as to extract more accurate geometric features.

[0004] According to one aspect of the present application, a method for extracting features from a high-precision map is provided, the method comprising:

[0005] Acquire at least two frames of road images collected from different lanes of the same road area;

[0006] respectively extracting original geometric features of road surface markings in the at least two frames of road images;

[0007] At least two of the original geometric features are fused to obtain target geometric features of the road surface marking.

[0008] According to another aspect of the present application, an electronic device is provided, comprising:

[0009] at least one processor; and

[0010] a memory communicatively connected to the at least one processor; wherein,

[0011] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the high-precision map feature extraction method as described in any one of the embodiments of the present application.

[0012] According to one aspect of the present application, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the high-precision map feature extraction method as described in any one of the embodiments of the present application.

[0013] By implementing the technical solution provided in this application to extract the geometric features of road surface markings, more accurate geometric features can be extracted.

[0014] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present application.

[0016] Figure 1 is a schematic diagram of a high-precision map feature extraction method according to an embodiment of the present application;

[0017] Figure 2 is a schematic diagram of another high-precision map feature extraction method according to an embodiment of the present application;

[0018] Figure 3A is a schematic diagram of another high-precision map feature extraction method according to an embodiment of the present application;

[0019] Figure 3B This is a flow chart of another high-precision map feature extraction method provided in an embodiment of the present application;

[0020] Figure 4 is a schematic diagram of another high-precision map feature extraction method according to an embodiment of the present application;

[0021] Figure 5 is a schematic diagram of another high-precision map feature extraction method according to an embodiment of the present application;

[0022] Figure 6 is a schematic diagram of a high-precision map feature extraction device according to an embodiment of the present application;

[0023] Figure 7 This is a block diagram of an electronic device used to implement a high-precision map feature extraction method in an embodiment of the present application. DETAILED DESCRIPTION

[0024] The following description of exemplary embodiments of the present application is made in conjunction with the accompanying drawings, including various details of the embodiments of the present application to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0025] Figure 1 This is a schematic diagram of a high-precision map feature extraction method according to an embodiment of the present application. This embodiment is applicable to the case of extracting geometric features of road markings. The high-precision map feature extraction method disclosed in this embodiment can be executed by a high-precision map feature extraction device, which can be implemented in software and / or hardware and configured in an electronic device with computing and storage functions. Figure 1 , this embodiment provides a high-precision map feature extraction method, including:

[0026] S110 : Acquire at least two frames of road images captured from different lanes of the same road area.

[0027] The road area refers to an area on the road where ground markings are drawn. For example, the road area may be a road intersection where turn guides, stop lines, or crosswalks are drawn.

[0028] The road image can be collected by a professional image collection vehicle, or by a common vehicle using a dashcam or a handheld device with a camera function, such as a mobile phone or tablet computer. The road image collection method and device are not limited and will be determined based on actual circumstances. The road image can be a front view of the road area.

[0029] Optionally, the at least two frames of road images may be captured by the capture device in different lanes, with at least one frame of road image captured for each lane. For example, a vehicle equipped with the capture device may be controlled to travel in each lane, and while the vehicle is traveling in each lane, the capture device may be controlled to capture road images at a preset frequency (e.g., 10 frames per second), i.e., multiple frames of images are captured for each lane.

[0030] In this embodiment, different road images correspond to different image acquisition positions. As can be seen, since the acquisition position affects the camera's viewing angle, photographing the same road area at different acquisition positions can obtain the road area from different camera viewing angles, thereby capturing the road area from multiple angles.

[0031] S120: Extract original geometric features of road surface markings in the at least two frames of road images respectively.

[0032] Among them, road markings refer to graphics drawn on roads by relevant departments to guide pedestrians or vehicles. For example, road markings can be diamond-shaped markings, lane markings, crosswalks, or stop lines. The original geometric features of road markings are information used to describe the polygons that make up the road markings. For example, if the road marking is a crosswalk, the original geometric features of the crosswalk can be feature data such as the corner points or edges of each polygon that makes up the crosswalk.

[0033] In an optional embodiment, the road surface marking is a road surface marking that crosses lanes. A road surface marking that crosses lanes refers to a road surface marking that crosses at least one lane at the same time. Exemplarily, the road surface marking that crosses lanes is a crosswalk or a stop line. Due to the camera's viewing angle, when using a camera to capture images of road surface markings that cross lanes, there are problems such as incomplete road surface markings and geometric deformation that easily occurs in areas where road surface markings that cross lanes are far from the camera. The present application provides an accurate and effective method for extracting the geometric features of road surface markings that cross lanes.

[0034] The original geometric features of the road surface markings in the road image collected under each lane are extracted respectively. Specifically, one possible implementation method may be to perform semantic segmentation on each road image respectively. First, a semantic segmentation network is used to perform full-element segmentation on the road image, and elements such as signs, vegetation, roads, and pedestrians and vehicles are segmented out in the road image. Then, the ground elements are extracted, and further semantic segmentation is performed on the ground elements to segment out elements such as lane lines, ground arrows, crosswalks, stop lines, and ground speed limits. Next, the road surface markings across the lanes are selected from the segmented ground elements, and the road surface markings across the lanes are processed using the connected domain analysis method, clustering algorithm, and outer contour analysis method to determine the original geometric features of the road surface markings. Optionally, this embodiment may determine the original geometric features of a group of road surface markings for at least one frame of road image collected for each lane; or it may determine the original geometric features of a group of road surface markings for each frame of road image, and this is limited.

[0035] Preferably, since road surface markings are located on the ground, when extracting the original geometric features of road surface markings from road images, this embodiment can first convert at least two frames of collected road images into overhead views, and then extract the original geometric features of road surface markings based on the overhead views to improve the accuracy of feature extraction.

[0036] Another possible implementation method may be to input at least two frames of road images into a pre-trained feature recognition model respectively, and extract the original geometric features of the road surface markings in each frame of road image through the feature recognition model.

[0037] S130: Fusing at least two of the original geometric features to obtain target geometric features of the road surface marking.

[0038] Among them, the original geometric features refer to the unprocessed geometric features extracted from a single road image. Due to the problem of camera perspective, when using a camera to capture images of cross-lane road markings, the areas of the cross-lane road markings that are farther away from the camera are prone to geometric deformation. Therefore, the original geometric features may not truly reflect the geometric characteristics of the road markings. The target geometric features are the feature fusion results obtained by fusing at least two original geometric features. Since the original geometric features are extracted from a single road image, the extracted geometric features are incomplete. The target geometric features fuse the original geometric features captured from at least two different lanes. Compared with a single original geometric feature, the target geometric features are more accurate and complete.

[0039] The at least two original geometric features are fused to obtain the target geometric features of the road surface marking. Specifically, the at least two original geometric features may be superimposed and then modified in combination with the standard geometric features of the road surface marking to obtain the target geometric features. Alternatively, the at least two original geometric features may be input into a pre-built feature fusion model, and the original geometric features are fused by the feature fusion model to output the target geometric features. Alternatively, the target geometric features may be obtained by performing feature fusion based on the regional intersection relationship between the original geometric features.

[0040] The technical solution of an embodiment of the present application obtains at least two frames of road images captured from different lanes of the same road area; extracts the original geometric features of road markings from each lane's road image; and fuses the at least two original geometric features to obtain target geometric features of the road markings. This fusion of at least two original geometric features effectively integrates geometric features captured from different camera perspectives, resulting in more accurate target geometric features that include more information about the road marking's geometric characteristics. The technical solution provided by this application can effectively address the issue of inaccurate geometric features of road markings extracted from road images, which can be caused by incomplete and distorted geometric features of road markings in road images.

[0041] Figure 2 This is a schematic diagram of another HD map feature extraction method according to an embodiment of the present application; this embodiment is an optional solution based on the above embodiment. Specifically, it refines the operation of "fusing at least two of the original geometric features to obtain the target geometric features of the road surface marking."

[0042] See also Figure 2, the high-precision map feature extraction method provided in this embodiment includes:

[0043] S210 : Acquire at least two frames of road images captured from different lanes of the same road area.

[0044] S220: Extract original geometric features of road surface markings in the at least two frames of road images respectively.

[0045] Specifically, at least one set of original geometric features of road surface markings is extracted from each frame of road image.

[0046] S230: Determine an intersection area and a non-intersection area of ​​at least two of the original geometric features.

[0047] The intersection area of ​​primitive geometric features refers to the image area where primitive geometric features describing the same part of road markings in different road images are located. In contrast, the image area where other primitive geometric features in the road image are located is the non-intersection area of ​​primitive geometric features.

[0048] For example, if the road marking is a crosswalk, the extracted raw geometric features are the polygonal outlines that make up the crosswalk. Because crosswalks are road markings that span lanes, portions of the crosswalk may be missing from the road image due to acquisition location issues, and the missing crosswalk portions may differ between different road images. The overlapping portions of the crosswalks in the road images—that is, the portion of the crosswalk included in all road images—are the intersection of the raw geometric features.

[0049] S240: Extract candidate local features of at least two of the original geometric features in the intersection area, and determine a first local feature of the intersection area according to the confidence levels of the candidate local features in the intersection area.

[0050] Candidate local features refer to the subset of original geometric features located within the intersection region. Confidence refers to the accuracy of the original geometric features within the intersection region. Generally speaking, areas of a road image with minimal deformation of road markings have higher confidence levels than other areas. Confidence can be determined through network model analysis, by comparing candidate local features based on standard geometric features, or by determining the distance between the road marking and the acquisition location. The method for determining confidence is not specified here; it will be determined based on actual circumstances.

[0051] The first local feature refers to a candidate local feature used to generate the target geometric feature.

[0052] Candidate local features refer to original geometric features located in the intersection area of ​​different road images. Due to the differences in the degree of deformation of candidate local features, the confidence levels corresponding to the candidate local features are different.

[0053] Candidate local features are extracted from the intersection area of ​​at least two original geometric features. Based on the confidence level of the candidate local features in the intersection area, a first local feature of the intersection area is determined. Specifically, candidate local features are extracted from the intersection area of ​​at least two original geometric features, the confidence level corresponding to each candidate local feature is calculated, and the candidate local feature with the highest confidence level is selected as the first local feature. This allows candidate local features to be screened based on confidence level even when there is redundancy in the geometric features of road markings, resulting in the selection of relatively accurate geometric features and ensuring the accuracy of the geometric features.

[0054] In an optional embodiment, the confidence level of the candidate local feature in the intersection area is determined based on the distance between the intersection area position and the acquisition position of the candidate local feature.

[0055] The location where the candidate local features are collected can be the location where the road image is collected. It is known that when a camera captures an image, due to the camera's viewing angle, the farther away from the capture location, the more severe the deformation. This is especially true for ground markings that cross lanes.

[0056] Road images are images of the same road area captured at different acquisition locations. The intersection area in different road images varies in distance from the acquisition location of the candidate local feature. The confidence level of the candidate local feature in the intersection area is determined based on the distance between the two. Specifically, the Euclidean distance between the center of the intersection area and the acquisition location of the road image can be calculated in the world coordinate system to determine the confidence level of the candidate local feature in the sub-area. Generally speaking, the distance value and confidence level are negatively correlated; that is, the larger the distance value, the lower the confidence level.

[0057] The present application improves the accuracy of extracting geometric features corresponding to road markings by determining the confidence of candidate local features in the intersection area based on the distance value between the intersection area position and the collection position of the candidate local features.

[0058] S250: Extract second local features of at least two of the original geometric features in the non-intersection area.

[0059] The original geometric features located in the non-intersecting area describe the geometric characteristics of different parts of the road sign. To ensure the integrity of the road sign information, the original geometric features located in the non-intersecting area are extracted as the second local features. The second local features refer to the original geometric features located in the non-intersecting area. The second local features are used together with the first local features to generate the target geometric features.

[0060] S260: Determine target geometric features of the road surface marking according to the first local features and the second local features.

[0061] The first local feature is generated in the intersection area of ​​the original geometric features, and the second local feature is generated in the non-intersection area of ​​the original geometric features. The target geometric feature of the road marking is determined based on the first and second local features. Specifically, the target geometric feature is obtained by concatenating the first and second local features.

[0062] The technical solution of the embodiment of the present application distinguishes between the intersection and non-intersection areas of the original geometric features. In the intersection area where the original geometric features are redundant, the candidate local features are screened based on the confidence level to select a relatively accurate first local feature, thereby ensuring the accuracy of the target geometric feature. The present application extracts the second local feature in the non-intersection area of ​​at least two original geometric features and determines the target geometric feature of the road sign based on the first and second local features, thereby ensuring the integrity of the road sign information.

[0063] Figure 3A This is a schematic diagram of another HD map feature extraction method according to an embodiment of the present application; this embodiment is an optional solution based on the above embodiment. Specifically, it refines the operation of "determining the first local feature of the intersection area based on the confidence level of the candidate local feature in the intersection area."

[0064] See also Figure 3A , the high-precision map feature extraction method provided in this embodiment includes:

[0065] S310: Acquire at least two frames of road images collected from the same road area in different lanes.

[0066] S320: Extract original geometric features of road surface markings in the at least two frames of road images respectively.

[0067] S330: Determine an intersection area and a non-intersection area of ​​at least two of the original geometric features.

[0068] S340: Extract candidate local features of at least two of the original geometric features in the intersection area, and divide the intersection area into at least two sub-areas.

[0069] The intersection area is divided into at least two sub-areas. Specifically, the vehicle driving direction is taken as a reference direction, and the intersection area corresponding to the road surface markings perpendicular to the reference direction is divided.

[0070] Optionally, the intersection area is divided equally along straight lines parallel to the reference direction to obtain at least two sub-areas.

[0071] S350: Determine a first local feature of the intersection region according to the confidence of the candidate local feature in each sub-region.

[0072] Different sub-regions correspond to different confidence levels. The confidence level of each sub-region is calculated separately to determine the confidence level of the candidate local features located in the sub-region. Based on the confidence level of the candidate local features, the candidate local features located in the intersection region are screened and the first local feature is selected from the intersection region.

[0073] Specifically, the confidences of the candidate local features in each sub-region are sorted in a certain order, and the candidate local feature corresponding to the maximum confidence is selected as the first local feature.

[0074] In an optional embodiment, the confidence of the candidate local feature in the sub-region is determined based on the distance value between the sub-region position and the collection position of the candidate local feature. The collection position of the candidate local feature can be the collection position of the road image. Specifically, the distance value between the sub-region position and the collection position of the candidate local feature can be calculated in the world coordinate system by calculating the Euclidean distance between the sub-region center position and the collection position of the road image to determine the confidence of the candidate local feature in the sub-region. Generally speaking, the distance value is negatively correlated with the confidence, that is, the larger the distance value, the smaller the confidence. The present application improves the extraction accuracy of the geometric features corresponding to the road surface markings by determining the confidence of the candidate local feature in the sub-region based on the distance value between the sub-region position and the collection position of the candidate local feature.

[0075] S360: Extract second local features of at least two of the original geometric features in the non-intersection area.

[0076] S370: Determine target geometric features of the road surface marking according to the first local features and the second local features.

[0077] The technical solution of this embodiment of the application divides the intersection area into at least two sub-areas and determines the first local feature of the intersection area based on the confidence level of the candidate local features in each sub-area. This implements fine-grained calculation of the confidence levels corresponding to the candidate local features, and screens the candidate local features based on the confidence levels, further improving the accuracy of the first local feature and thus ensuring the accuracy of the extraction of the geometric features corresponding to the road markings.

[0078] For ease of understanding, Figure 3B This is a flow chart of another high-precision map feature extraction method provided in the embodiment of this application. Figure 3BAs shown, first, the original geometric features A of the road markings in the road image (such as the top view of the road image) a and the original geometric features B of the road markings in the road image (such as the top view of the road image) b are extracted. Then, the intersection area C and the non-intersection areas D1 and D2 of the original geometric features A and B are determined. Next, the intersection area C is divided into at least two sub-areas, C i is a sub-region in the intersection region C, A i and B i The original geometric features A and B are respectively in sub-region C i According to the candidate local features A i and B i In sub-area C i The confidence level of B i The first local feature of the intersection area C is determined. The second local feature of the original geometric feature A located in the non-intersection area D1 is extracted, and the second local feature of the original geometric feature B located in the non-intersection area D2 is extracted. According to the first local feature B i and the second local features D1 and D2 to determine the target geometric feature E of the road surface marking.

[0079] Figure 4 This is a schematic diagram of another HD map feature extraction method according to an embodiment of the present application; this embodiment is an alternative solution based on the above embodiment. Specifically, when the road area contains at least two road surface markings, the operation of "fusing at least two original geometric features to obtain target geometric features of the road surface markings" is refined.

[0080] See also Figure 4 , the high-precision map feature extraction method provided in this embodiment includes:

[0081] S410: Acquire at least two frames of road images collected from different lanes of the same road area.

[0082] S420: Extract original geometric features of road surface markings in the at least two frames of road images respectively.

[0083] S430: If the road area includes at least two road surface markings, determine at least two original geometric features associated with the same road surface marking according to an intersection-and-union ratio relationship between the original geometric features.

[0084] A road region including at least two road markings means that at least two road markings of the same type exist within the same road region. For example, if the road region is an intersection formed by the intersection of two roads, the road region typically includes four crosswalks. When a road region includes at least two road markings, it is necessary to determine the primitive geometric features corresponding to the road markings that belong to the same road marking.

[0085] Among them, the intersection over union relationship is used to describe the overlap between the original geometric features. The intersection over union relationship between the original geometric features can be determined using the intersection over union (IOU) strategy in the target detection task, which will not be further expanded here.

[0086] At least two original geometric features associated with the same road surface marking are determined based on the intersection-and-union (IU) relationship between the original geometric features. Specifically, the road surface marking corresponding to each original geometric feature is determined based on the IU relationship between the original geometric features, and the original geometric features are associated with the corresponding road surface marking. For example, at least two original geometric features whose IU ratio is greater than a preset threshold are considered the original geometric features corresponding to the same road surface marking.

[0087] S440: Fusing at least two original geometric features associated with the same road surface marking to obtain target geometric features of the at least two road surface markings.

[0088] The original geometric features corresponding to the same road sign are fused to obtain the target geometric features corresponding to the road sign. Optionally, when the road area contains at least two road signs, the original geometric features associated with each road sign are fused separately to obtain the target geometric features corresponding to each road sign.

[0089] The technical solution of an embodiment of the present application, when a road area contains at least two road surface markings, determines at least two original geometric features associated with the same road surface marking based on the intersection-over-union relationship between the original geometric features; then fuses the at least two original geometric features associated with the same road surface marking to obtain target geometric features of the at least two road surface markings. By determining the original geometric features associated with the road surface markings and then fusing them, the present application ensures the accuracy of extracting the geometric features corresponding to the road surface markings.

[0090] Figure 5 This is a schematic diagram of another HD map feature extraction method according to an embodiment of the present application; this embodiment is an optional solution based on the above embodiment. Specifically, it refines the operation of "respectively extracting the original geometric features of the road surface markings in the at least two frames of road imagery."

[0091] See also Figure 5 , the high-precision map feature extraction method provided in this embodiment includes:

[0092] S510: Acquire at least two frames of road images collected from the same road area in different lanes.

[0093] S520: Determine a road top view of the at least two frames of road images.

[0094] Since road images are generally front views, and road signs are graphics drawn on the road surface, extracting the geometric features of road signs from the top view of the road has higher accuracy than extracting geometric features from the formal map.

[0095] Specifically, when determining the road top view, the road front view may be converted into the road top view based on the principles of perspective transformation and affine transformation. Alternatively, the road top view may be generated by combining the point cloud data of the road area with the front view of the road area. In this embodiment, the road image top view is preferably generated by combining the point cloud data.

[0096] In an optional embodiment, determining the road top view of the at least two frames of road images includes: determining the road top view corresponding to the road area under each lane based on at least two frames of road images and at least two frames of point cloud data collected for the road area in each lane.

[0097] Among them, the point cloud data and road images of the road area are collected from the same road area in different lanes by the laser radar and camera configured on the image acquisition vehicle.

[0098] Specifically, for the point cloud data and road image data collected under each lane, semantic segmentation can be performed on each frame of the collected road image to determine the ground object area in the image, and ground point cloud detection can be performed on the point cloud data; then, based on the calibration relationship between the lidar and the camera, the detected multiple frames of ground point clouds are projected into each frame of the road image, and it is determined whether each projection point is the target projection point corresponding to the ground object area after semantic segmentation; if so, the best frame is selected from the multiple frames of ground object area corresponding to each target projection point, and fused to generate a road top view. That is, for the data collected for each lane, a frame of road top view is determined. The embodiment of the present application generates a road top view through the point cloud data of the road area in combination with the road image, and determines the position of the ground based on the point cloud data, which can improve the accuracy of feature extraction.

[0099] It should be noted that when determining the top-down view of at least two road image frames in this embodiment, one road image top-down view can be determined for each image frame, or one road image top-down view can be determined for each road image frame collected in the same lane. To ensure the accuracy of the top-down view, this embodiment prefers the second method.

[0100] S530: Extract road marking areas from the at least two frames of road top view respectively, and perform feature extraction on the road marking areas to obtain original geometric features of the road markings in the at least two frames of road images.

[0101] Typically, captured road images contain not only the road area but also its surroundings, such as vehicles, pedestrians, and vegetation. In this embodiment, the road marking area serves as a region of interest (ROI), and this region of interest needs to be extracted from the road image. Feature extraction is performed on the road marking area, extracting the features of the polygons that make up the road markings as the original geometric features of the road markings in the road image.

[0102] S540: Fusing at least two of the original geometric features to obtain target geometric features of the road surface marking.

[0103] The technical solution of the embodiments of the present application determines a road top view from at least two frames of road images, extracts road marking areas from each of the at least two frames of road top view, and performs feature extraction on the road marking areas to obtain the original geometric features of the road markings in the at least two frames of road images. The at least two original geometric features are then fused to obtain the target geometric features of the road markings. Road signs are graphics drawn on the road surface. The embodiments of the present application extract the geometric features of road signs from the road top view, effectively improving the accuracy of extracting the corresponding geometric features of road signs.

[0104] Figure 6 is a schematic diagram of a high-precision map feature extraction device according to an embodiment of the present application; see Figure 6 , an embodiment of the present application discloses a high-precision map feature extraction device 600, which may include: a road image acquisition module 610, a raw geometric feature extraction module 620 and a raw geometric feature fusion module 630.

[0105] A road image acquisition module 610 is configured to acquire at least two frames of road images captured from different lanes of the same road area;

[0106] The original geometric feature extraction module 620 is used to extract the original geometric features of the road surface markings in the at least two frames of road images respectively;

[0107] The original geometric feature fusion module 630 is used to fuse at least two of the original geometric features to obtain the target geometric features of the road surface marking.

[0108] The technical solution of an embodiment of the present application obtains at least two frames of road images captured from different lanes of the same road area; extracts the original geometric features of road markings from each lane's road image; and fuses the at least two original geometric features to obtain target geometric features of the road markings. This fusion of at least two original geometric features effectively integrates geometric features captured from different camera perspectives, resulting in more accurate target geometric features that include more information about the road marking's geometric characteristics. The technical solution provided by this application can effectively address the issue of inaccurate geometric features of road markings extracted from road images, which can be caused by incomplete and distorted geometric features of road markings in road images.

[0109] Optionally, the original geometric feature fusion module 630 includes: an intersection area determination submodule, used to determine the intersection area and non-intersection area of ​​at least two of the original geometric features; a first local feature determination submodule, used to extract candidate local features of at least two of the original geometric features in the intersection area, and determine the first local feature of the intersection area based on the confidence of the candidate local features in the intersection area; a second local feature extraction submodule, used to extract the second local features of at least two of the original geometric features in the non-intersection area; and a target geometric feature determination submodule, used to determine the target geometric features of the road surface marking based on the first local features and the second local features.

[0110] Optionally, the first local feature determination submodule includes: a sub-region division unit, used to divide the intersection region into at least two sub-regions; and a first local feature determination unit, used to determine the first local feature of the intersection region based on the confidence of the candidate local feature in each sub-region.

[0111] Optionally, the device also includes: a confidence determination module, which is specifically used to determine the confidence of the candidate local feature in the intersection area or sub-area based on the distance value between the intersection area position or sub-area position and the collection position of the candidate local feature.

[0112] Optionally, if the road area contains at least two road surface markings, the original geometric feature fusion module 630 includes: an associated original geometric feature determination submodule, used to determine at least two original geometric features associated with the same road surface marking based on the intersection-and-union ratio relationship between the original geometric features; and a target geometric feature determination submodule, used to fuse the at least two original geometric features associated with the same road surface marking to obtain the target geometric features of the at least two road surface markings.

[0113] Optionally, the original geometric feature extraction module 620 includes: a road top view determination submodule, used to determine the road top view of the at least two frames of road images; a road surface marking area feature extraction submodule, used to extract road surface marking areas from the at least two frames of road top views respectively, and perform feature extraction on the road surface marking areas to obtain the original geometric features of the road surface markings in the at least two frames of road images.

[0114] Optionally, the road top view determination submodule is specifically used to determine the road top view corresponding to the road area in each lane based on at least two frames of road images and at least two frames of point cloud data collected for the road area in each lane.

[0115] Optionally, the road surface marking is a road surface marking across lanes.

[0116] The high-precision map feature extraction device provided in the embodiment of the present application can execute the high-precision map feature extraction method provided in any embodiment of the present application, and has the corresponding functional modules and beneficial effects for executing the high-precision map feature extraction method.

[0117] In the technical solution of this application, the acquisition, storage and application of relevant data of any application involved (such as the application's authorization code, application identifier and application authorization parameters, etc.), relevant data of the open platform (such as historical access records) and relevant data of third-party organizations (such as the target organization and other organizations, etc.) shall comply with the provisions of relevant laws and regulations and shall not violate public order and good morals.

[0118] In the technical solution of this application, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0119] “It should be noted that the head model in this embodiment is not a head model for a specific user and cannot reflect the personal information of a specific user.”

[0120] “It should be noted that the two-dimensional facial images in this embodiment come from a public dataset” etc.

[0121] According to an embodiment of the present application, the present application also provides an electronic device, a readable storage medium and a computer program product.

[0122] Figure 7A schematic block diagram of an example electronic device 700 that can be used to implement an embodiment of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or claimed herein.

[0123] like Figure 7 As shown, the device 700 includes a computing unit 701, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 702 or a computer program loaded from a storage unit 708 into a random access memory (RAM) 703. Various programs and data required for the operation of the device 700 can also be stored in the RAM 703. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0124] Various components in device 700 are connected to I / O interface 705, including an input unit 706, such as a keyboard, mouse, etc.; an output unit 707, such as various types of displays, speakers, etc.; a storage unit 708, such as a magnetic disk, optical disk, etc.; and a communication unit 709, such as a network card, modem, wireless communication transceiver, etc. The communication unit 709 allows device 700 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0125] The computing unit 701 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 701 performs the various methods and processes described above, such as the HD map feature extraction method. For example, in some embodiments, the HD map feature extraction method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the HD map feature extraction method described above can be performed. Alternatively, in other embodiments, the computing unit 701 can be configured to perform the HD map feature extraction method by any other suitable means (e.g., via firmware).

[0126] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0127] The program code for implementing the methods of the present application can be written in any combination of one or more programming languages. Such program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the program code is executed by the processor or controller, the functions / operations specified in the flow charts and / or block diagrams are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0128] In the context of the present application, a machine-readable medium can be a tangible medium that can contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0129] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0130] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0131] A computer system may include a client and a server. The client and server are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services. The server may also be a server in a distributed system or a server integrated with blockchain.

[0132] Artificial intelligence (AI) is the study of how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, specialized AI chips, cloud computing, distributed storage, and big data processing. AI software technologies primarily encompass computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graphs.

[0133] Cloud computing refers to a technology system that provides network access to elastically scalable shared pools of physical or virtual resources. These resources can include servers, operating systems, networks, software, applications, and storage devices, and can be deployed and managed on-demand in a self-service manner. Cloud computing technology provides efficient and powerful data processing capabilities for the application of technologies such as artificial intelligence and blockchain, as well as for model training.

[0134] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this application can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this application can be achieved. This is not a limitation herein.

[0135] The above specific embodiments do not constitute a limitation on the scope of protection of this application. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application shall be included within the scope of protection of this application.

Claims

1. A high-precision map feature extraction method, comprising: Acquire at least two frames of road images collected from different lanes of the same road area; respectively extracting original geometric features of road surface markings in the at least two frames of road images; fusing at least two of the original geometric features to obtain target geometric features of the road surface marking; The method of fusing at least two of the original geometric features to obtain the target geometric features of the road surface marking includes: determining the intersection area and non-intersection area of ​​at least two of the original geometric features; extracting candidate local features of at least two of the original geometric features in the intersection area, and determining the first local feature of the intersection area based on the confidence of the candidate local features in the intersection area; extracting the second local feature of at least two of the original geometric features in the non-intersection area; and determining the target geometric features of the road surface marking based on the first local feature and the second local feature.

2. The method according to claim 1, wherein The determining, based on the confidence level of the candidate local feature in the intersection area, the first local feature of the intersection area includes: Dividing the intersection area into at least two sub-areas; A first local feature of the intersection area is determined according to the confidence of the candidate local feature in each sub-area.

3. The method according to claim 1 or 2, further comprising: The confidence level of the candidate local feature in the intersection region or sub-region is determined based on the distance between the intersection region position or sub-region position and the acquisition position of the candidate local feature.

4. The method according to claim 1, wherein If the road area includes at least two road surface markings, the at least two original geometric features are fused to obtain target geometric features of the road surface markings, including: Determining at least two original geometric features associated with the same road surface marking according to an intersection-and-union ratio relationship between the original geometric features; At least two original geometric features associated with the same road surface marking are fused to obtain target geometric features of the at least two road surface markings.

5. The method according to claim 1, wherein Extracting original geometric features of road surface markings in the at least two frames of road images respectively includes: Determining a road top view of the at least two frames of road images; Road surface marking areas are respectively extracted from the at least two frames of road overhead view, and features are extracted from the road surface marking areas to obtain original geometric features of the road surface markings in the at least two frames of road image.

6. The method according to claim 5, wherein: Determining the road top view of the at least two frames of road images includes: A road top view corresponding to the road area under each lane is determined based on at least two frames of road images and at least two frames of point cloud data collected for the road area in each lane.

7. The method according to any one of claims 1 to 6, wherein The road surface marking is a road surface marking across lanes.

8. A high-precision map feature extraction device, comprising: A road image acquisition module, configured to acquire at least two frames of road images collected from different lanes of the same road area; an original geometric feature extraction module, configured to extract original geometric features of road surface markings in the at least two frames of road images respectively; an original geometric feature fusion module, configured to fuse at least two of the original geometric features to obtain a target geometric feature of the road surface marking; Among them, the original geometric feature fusion module includes: an intersection area determination submodule, which is used to determine the intersection area and non-intersection area of ​​at least two of the original geometric features; a first local feature determination submodule, which is used to extract candidate local features of at least two of the original geometric features in the intersection area, and determine the first local feature of the intersection area based on the confidence of the candidate local features in the intersection area; a second local feature extraction submodule, which is used to extract the second local features of at least two of the original geometric features in the non-intersection area; and a target geometric feature determination submodule, which is used to determine the target geometric features of the road surface marking based on the first local features and the second local features.

9. The device according to claim 8, wherein The first local feature determination submodule includes: a sub-region division unit, configured to divide the intersection region into at least two sub-regions; The first local feature determination unit is configured to determine the first local feature of the intersection region according to the confidence of the candidate local feature in each sub-region.

10. The device according to claim 8 or 9, further comprising: The confidence determination module is specifically used to determine the confidence of the candidate local feature in the intersection area or sub-area based on the distance value between the intersection area position or sub-area position and the collection position of the candidate local feature.

11. The device according to claim 8, wherein If the road area contains at least two road surface markings, the original geometric feature fusion module includes: The associated original geometric feature determination submodule is used to determine at least two original geometric features associated with the same road surface mark based on the intersection-and-union ratio relationship between the original geometric features; The target geometric feature determination submodule is used to fuse at least two original geometric features associated with the same road surface marking to obtain target geometric features of the at least two road surface markings.

12. The device according to claim 8, wherein Original geometric feature extraction module, including: A road top view determining submodule, configured to determine a road top view of the at least two frames of road images; The road marking area feature extraction submodule is used to extract road marking areas from the at least two frames of road top view respectively, and perform feature extraction on the road marking areas to obtain original geometric features of the road markings in the at least two frames of road images.

13. The device according to claim 12, wherein The road top view determination submodule is specifically configured to determine a road top view corresponding to the road area in each lane based on at least two frames of road images and at least two frames of point cloud data collected for the road area in each lane.

14. The device according to any one of claims 8 to 13, wherein: The road surface marking is a road surface marking across lanes.

15. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 7.

17. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 7.

Images