High-precision map road sign pattern data solving method and device, equipment and storage medium

Abstract

The application relates to a high-precision map road sign figure data solving method, device and equipment and a storage medium, and relates to the technical field of high-precision maps. The method first receives multiple to-be-solved road sign images, screens the multiple to-be-solved road sign images based on a screening rule to obtain an image set comprising a first target image, takes an image with the largest pixel area of the first target image in the image set as an initial image, selects a second target image from the image set based on the initial image, and solves the pose of the road sign based on the second target image. Since the first target image is obtained through the screening rule, unqualified images are screened out, and the second target image is selected according to a preset collection interval, the second target image is taken as a road sign standard image for solving, and all images are not directly solved, so that the problems of distortion in the images, inaccurate size of the recognition box and the like are solved, and the solving precision is improved.

Description

Technical Field

[0001] This application relates to the field of graphic data reading and processing technology, and in particular to a method, apparatus, device and storage medium for calculating high-precision map road sign graphic data. Background Technology

[0002] High-precision maps, also known as high-resolution maps, are a key technology in autonomous driving, demanding extremely high accuracy. The interpretation of road signs is a crucial step in the mapping process. Therefore, the accuracy of interpreting road sign and other graphic data becomes a vital prerequisite for improving mapping accuracy.

[0003] In related technologies, although the accuracy of image processing can be improved by improving the solution algorithm, since the improvement is made to the solution algorithm itself, when the solution object is a complex road sign image, there are still cases of inaccurate feature point matching, which causes misidentification of the road sign image and thus results in insufficient solution accuracy of the road sign. Summary of the Invention

[0004] To overcome the problems existing in related technologies, this application provides a high-precision map road sign graphic data processing method, which, without changing the processing algorithm, optimizes and filters road sign images based on preset collection distance and preset collection quantity, removes unnecessary interference, improves recognition accuracy, reduces the computational load of the algorithm, and improves the efficiency of road sign recognition.

[0005] The first aspect of this application provides a method for solving high-precision map road sign graphic data, including:

[0006] Receive multiple road sign images to be solved;

[0007] Based on the filtering rules, the multiple road sign images to be solved are filtered to obtain an image set including a first target image; the first number of the first target images in the image set is at least greater than a preset recognition number; the preset recognition number is used to characterize the minimum number of road sign poses that can be solved; wherein, the filtering rules are used to retain images that meet the recognition conditions;

[0008] The image with the largest pixel area of ​​the first target image in the image set is taken as the initial image, and a second target image is selected from the image set based on the initial image. The acquisition interval between the first target image and the second target image is a preset distance, and the acquisition interval between two adjacent second target images is the preset distance.

[0009] The pose of the road sign is calculated based on the second target image.

[0010] Optionally, the filtering from the multiple road sign images to be solved based on the filtering rules includes:

[0011] The multiple road sign images to be solved are identified to obtain the identification results;

[0012] Based on the identification results, the corresponding filtering rules are determined;

[0013] The first target image set is determined from the multiple road sign images to be solved based on the corresponding filtering rules.

[0014] Optionally, the filtering rules include:

[0015] Confidence filtering rules, shape recognition filtering rules, area recognition filtering rules, and / or frame ratio filtering rules.

[0016] Optionally, if the filtering rules include confidence-based filtering rules, then determining the first target image from the multiple road sign images to be solved includes:

[0017] Receive the road sign image to be solved;

[0018] Extract the feature information of the road sign image to be solved;

[0019] The confidence level of the feature information is calculated to obtain the current confidence level data;

[0020] The current confidence level data is compared with the preset confidence level data, and the road sign image to be solved is determined to be the first target image when the current confidence level data is greater than the preset confidence level data.

[0021] And / or, if the filtering rules include shape recognition filtering rules, then determining the first target image from the multiple road sign images to be solved includes:

[0022] Receive the road sign image to be solved;

[0023] The road sign image to be solved is detected based on an edge detection algorithm to obtain the detection result;

[0024] The rectangular road sign image to be solved in the detection results is identified as the first target image;

[0025] And / or, if the filtering rules include an area filtering rule, then determining the first target image from the plurality of road sign images to be solved includes:

[0026] Receive the road sign image to be solved;

[0027] Calculate the pixel area of ​​the road sign in the image of the road sign to be solved to obtain the current road sign area;

[0028] The current road sign area is compared with the preset area to obtain the comparison result;

[0029] The road sign image whose current road sign area is greater than the preset area in the comparison results is identified as the first target image;

[0030] And / or, if the filtering rules include a bounding box filtering rule, then determining the first target image from the plurality of road sign images to be solved includes:

[0031] Receive the road sign image to be solved;

[0032] The contour detection algorithm is used to identify the contour of the road sign in the image to be solved, and the contour pixel area of ​​the road sign is calculated.

[0033] Calculate the ratio of the area of ​​the outline pixels to the bounding rectangle of the road sign image to be solved, and obtain the ratio result;

[0034] The road sign images to be solved that are greater than the preset ratio in the ratio results are identified as the first target images.

[0035] Optional, also includes:

[0036] Receive the road sign image to be solved;

[0037] Read the distortion parameters from the road sign image to be solved;

[0038] The road sign image to be solved is distorted using an iterative distortion correction algorithm to obtain an undistorted road sign image.

[0039] Optionally, selecting the second target image from the image set based on the initial image includes:

[0040] Obtain the initial image;

[0041] Based on the initial image, a first target image with a sampling distance from the initial image is determined as the second target image;

[0042] Set the second target image as the initial image, and return to determine the first target image with a sampling distance of the initial image as the second target image based on the initial image, until the selection is terminated when the termination condition is met;

[0043] The termination condition includes reaching the preset target number of samples.

[0044] Optionally, the process of determining the acquisition interval includes:

[0045] Receive the first target image;

[0046] Query the trajectory point information of the acquisition vehicle for the first target image to obtain the acquisition time point of the first target image;

[0047] Determine the target latitude and longitude corresponding to the data collection time point;

[0048] Based on the target's latitude and longitude, calculate the acquisition distance between the two first target images.

[0049] The second aspect of this application provides a high-precision map road sign graphic data processing device, comprising:

[0050] The receiving module receives multiple images of road signs to be solved;

[0051] The filtering module filters from the multiple road sign images to be solved based on filtering rules to obtain an image set including a first target image; the first number of the first target images in the image set is at least greater than a preset recognition number; the preset recognition number is used to characterize the minimum number of road sign poses that can be solved; wherein, the filtering rules are used to retain images that meet the recognition conditions;

[0052] The acquisition module takes the image with the largest pixel area of ​​the first target image in the image set as the initial image, and selects a second target image from the image set based on the initial image. The acquisition interval between the first target image and the second target image is a preset distance, and the acquisition interval between two adjacent second target images is the preset distance.

[0053] The solution module calculates the pose of the road sign based on the second target image.

[0054] A third aspect of this application provides an electronic device, comprising:

[0055] Processor; and

[0056] A memory that stores executable code, which, when executed by the processor, causes the processor to perform the method described above.

[0057] A fourth aspect of this application provides a non-transitory machine-readable storage medium having executable code stored thereon, which, when executed by a processor of an electronic device, causes the processor to perform the method described above.

[0058] Therefore, the high-precision map road sign graphic data processing method, apparatus, device, and storage medium provided in this application first receive multiple road sign images to be processed, and then filter them based on filtering rules to obtain an image set including a first target image. The first number of first target images in the image set is at least greater than a preset recognition number, which is used to characterize the minimum number of road sign poses that can be calculated. The filtering rules are used to retain images that meet the recognition conditions. The image with the largest pixel area of ​​the first target image in the image set is taken as the initial image, and a second target image is selected from the image set based on the initial image. The acquisition distance between the first target image and the second target image is a preset distance, and the acquisition distance between two adjacent second target images is also a preset distance. The pose of the road sign is calculated based on the second target image. Since the first target image is obtained first through the filtering rules, unqualified images are filtered out, and then the second target image is selected and used as the standard image for road sign processing, instead of directly processing all images, the method solves problems such as image distortion and inaccurate recognition box size, thereby improving the processing accuracy.

[0059] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0060] The above and other objects, features and advantages of this application will become more apparent from the more detailed description of exemplary embodiments thereof in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments thereof.

[0061] Figure 1 This is a flowchart illustrating the high-precision map road sign graphic data processing method shown in the embodiments of this application.

[0062] Figure 2 This is a schematic diagram of the structure of a rectangular traffic sign as shown in an embodiment of this application.

[0063] Figure 3 This is a schematic diagram of the structure of the high-precision map road sign graphic data processing device shown in the embodiments of this application.

[0064] Figure 4 This is a schematic diagram of the structure of an electronic device shown in an embodiment of this application. Detailed Implementation

[0065] Preferred embodiments of the present application will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0066] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0067] It should be understood that although the terms "first," "second," "third," etc., may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0068] This application primarily focuses on high-precision map scenarios, specifically relating to a method, apparatus, device, and storage medium for calculating high-precision map road sign graphic data. High-precision maps contain rich road information, including road signs, lane lines, and traffic signs. Road sign information is particularly crucial for autonomous driving systems, as it provides essential navigation, traffic rules, and speed limits. Therefore, the accuracy of road sign calculation in high-precision maps is critical. While related technologies improve the calculation algorithm to enhance road sign accuracy, these methods, by using all collected road sign images as the calculation targets, still suffer from distortion and other issues affecting accuracy and leading to misidentification, ultimately resulting in insufficient road sign calculation accuracy.

[0069] Based on this, in this embodiment of the application, the road sign image to be solved is first filtered by filtering rules, the first target image obtained is selected according to a preset distance, and the second target image obtained is used as the standard image of the road sign for solving, thereby improving the accuracy of road sign solving.

[0070] See Figure 1 , Figure 1This is a flowchart illustrating a method for calculating high-precision map road sign graphic data, as shown in an embodiment of this application. This application discloses a method for calculating high-precision map road sign graphic data, including:

[0071] S10: Receive multiple road sign images to be solved.

[0072] In this embodiment of the application, the road sign image can be understood as the image collected by the data acquisition vehicle during the high-precision map making process. The image includes at least one road sign image and environmental images other than road signs.

[0073] In this embodiment of the application, the collected images are grouped according to the road sign at each actual coordinate position. The images in the same group all contain the image of the road sign. These images can be collected under different distances, angles, or lighting conditions and contain the same road sign.

[0074] It is understandable that the multiple road sign images to be solved belong to the same group and are used for subsequent pose calculation of the road signs.

[0075] S20. Based on the filtering rules, filter from the multiple road sign images to be solved to obtain an image set including a first target image; the first number of the first target images in the image set is at least greater than a preset recognition number; the preset recognition number is used to characterize the minimum number of road sign poses that can be solved; wherein, the filtering rules are used to retain images that meet the recognition conditions.

[0076] In this embodiment, the filtering rules can perform preliminary filtering on multiple road sign images to be solved, retaining images that meet the preset standards as the first target images. The first target images obtained after filtering are more conducive to subsequent processing and analysis, thereby improving the accuracy and reliability of road sign pose calculation.

[0077] In practical use, the system identifies the categories of multiple road sign images to be solved within a group. The identification results are then filtered according to selection rules, and images that meet these rules are used as the first target images to construct a new image set. For example, when identifying images in a group of speed limit road signs, the numbers on the signs are first identified through text recognition. This can be done automatically using OCR or manually. The identified numbers are then statistically analyzed, and the frequency of each number is calculated for filtering. If the number 30 appears most frequently, the number on the speed limit road sign can be changed to 30.

[0078] It can be understood that the preset recognition quantity refers to the minimum number of images that can be used to calculate the pose of the road sign. The first number after the image set is filtered needs to be at least greater than the preset recognition quantity in order to meet the image quantity requirements of subsequent steps.

[0079] S30. Take the image with the largest pixel area of ​​the first target image in the image set as the initial image, and select a second target image from the image set based on the initial image, wherein the acquisition distance between the first target image and the second target image is a preset distance, and the acquisition distance between two adjacent second target images is the preset distance.

[0080] In this embodiment of the application, the pixel area refers to the pixel area of ​​the road sign recognition box in the first target image. The road sign recognition box can locate a part of the road sign image in the image and use the road sign recognition box to frame the road sign. The road sign recognition box can be a rectangular recognition box, and the area of ​​the rectangular recognition box can be the minimum area that includes all road signs.

[0081] In this embodiment of the application, the pixel area of ​​the first target image in the image set is calculated. The pixel area can be obtained by calculating the width and height of the rectangle.

[0082] The pixel areas of all the first target images in the image set are compared, and the image with the largest pixel area is selected as the initial image. The pixel area calculation formula is defined as follows:

[0083] (1)

[0084] In equation (1), This refers to the number of gray levels or color levels contained in a pixel in a road sign image; it can also be called the bit depth or color depth of the image. This represents the horizontal resolution of the image, that is, the number of pixels in the horizontal direction. This represents the resolution of the image in the vertical direction, i.e., the number of pixels in the vertical direction. This represents the actual area of ​​each pixel.

[0085] In this embodiment, the acquisition interval is the actual coordinate distance between two road sign images when the actual acquisition vehicle acquires the road sign images. The GPS latitude and longitude of the two points before and after the timestamp of each road sign image in the group are determined. The GPS latitude and longitude of the two points before and after the timestamp of the acquired road sign image are found using a binary search method. Based on the ratio of the time deviation of the road sign image timestamp between the two points and the GPS latitude and longitude of the two points, the latitude and longitude coordinates corresponding to the timestamp of the road sign image are calculated.

[0086] Based on the above embodiments, the metric distance between two road sign images can be calculated using the latitude and longitude coordinates of at least two road sign images.

[0087] S40. Calculate the pose of the road sign based on the second target image.

[0088] In this embodiment, the first target image is obtained through a filtering rule, unqualified images are filtered out, and then the second target image is selected through a preset acquisition interval. The second target image is used as the standard image for road sign calculation, instead of directly calculating all images. This solves problems such as image distortion and inaccurate recognition box size, thereby improving the calculation accuracy.

[0089] The above embodiments describe the process of filtering and selecting road sign images during high-precision map making. This application embodiment will provide a detailed description of the filtering process.

[0090] In this embodiment of the application, filtering is performed from the multiple road sign images to be solved based on filtering rules, including:

[0091] S201. Recognize the multiple road sign images to be solved to obtain recognition results;

[0092] In this embodiment, the road sign image to be solved is input into the road sign recognition model, which can identify the target road sign image in the road sign image to be solved, and mark the road sign using the road sign recognition box. This can accurately and efficiently mark the road sign image in the road sign image to be solved.

[0093] The road sign recognition model can be established using methods such as convolutional neural networks and machine learning.

[0094] S202. Determine the corresponding filtering rules based on the identification results;

[0095] In this embodiment of the application, the identification result can determine the category of the road sign by the content of the road sign marking, such as a no-stopping road sign or a speed limit road sign, or it can determine the category of the road sign by the shape of the road sign, such as a rectangular road sign or a circular road sign.

[0096] Based on the category of the road sign, select the corresponding filtering rules for filtering. For example, during the filtering process, if the recognition result is a rectangular shape and the road sign identification content is a directional sign, the road sign type can be determined to be a rectangular traffic directional sign. Therefore, the filtering rules corresponding to the rectangular traffic directional sign road sign are called for filtering.

[0097] S203. Based on the corresponding filtering rules, determine the image set of the first target image from the multiple road sign images to be solved.

[0098] In this embodiment of the application, the road sign type of the road sign image to be solved is confirmed according to the recognition result, and the corresponding filtering rules are selected for filtering. The resulting image set of the first target image can remove misidentified road sign images from the road sign images to be solved, so as to avoid the road sign solving accuracy being affected by the misidentified road sign images in the subsequent solving process.

[0099] To further improve the accuracy of selecting the corresponding filtering rules, the filtering rules in the above embodiments are further defined in this application embodiment.

[0100] The filtering rules include:

[0101] Confidence filtering rules, shape recognition filtering rules, area recognition filtering rules, and / or frame ratio filtering rules.

[0102] The above embodiments introduced four filtering rules. The filtering methods of the four filtering rules will be described in detail below.

[0103] In this embodiment of the application, when the filtering rule is determined to be a confidence-based filtering rule, the first target image is determined from the plurality of road sign images to be solved, including:

[0104] Receive the road sign image to be solved.

[0105] Extract the feature information of the road sign image to be solved.

[0106] The confidence level of the feature information is calculated to obtain the current confidence level data.

[0107] The current confidence level data is compared with the preset confidence level data, and the road sign image to be solved is determined to be the first target image when the current confidence level data is greater than the preset confidence level data.

[0108] In this embodiment of the application, the confidence level filtering rule refers to filtering road sign images whose confidence level for road sign recognition is greater than a certain threshold.

[0109] In this embodiment of the application, the road sign image is input into the feature extraction model, wherein the feature extraction model can be the object detection model of YOLOv5, or the features of the road sign image can be extracted through a convolutional neural network.

[0110] In this embodiment of the application, the feature information of the road sign image is input into a classifier connected to the model. The classifier classifies the feature information of the road sign image and outputs the probability score of each category.

[0111] Typically, the soft max function can be used to convert the probability scores output by the classifier into a probability distribution, ensuring that the sum of the probabilities is 1.

[0112] Based on the above embodiments, for the probability distribution output by the classifier, the maximum probability method can be used to select the category with the highest probability as the classification result, and this probability can be used as the current confidence level data.

[0113] In this embodiment of the application, the current confidence data is compared with the preset confidence data, and road sign images whose current confidence data is greater than the preset confidence data are retained. That is, the images are determined to be correct road sign images, and the road sign images retained after filtering are used as the first target images.

[0114] Therefore, it can be seen that by filtering with confidence level rules, misidentified road signs can be removed, thereby improving the accuracy of subsequent road sign calculations.

[0115] In this embodiment of the application, when the filtering rule is determined to be a shape recognition filtering rule, the first target image is determined from the plurality of road sign images to be solved, including:

[0116] Receive the road sign image to be solved.

[0117] The image of the road sign to be solved is detected based on the edge detection algorithm, and the detection result is obtained.

[0118] The rectangular road sign image to be solved in the detection results is identified as the first target image.

[0119] In this embodiment of the application, the shape recognition filtering rule refers to filtering road sign images whose shape is rectangular.

[0120] In this embodiment of the application, the shape of the road sign image is detected. The outline of the road sign image can be obtained by an edge detection algorithm, or it can be determined by judging the pixel coordinates of the four corner points.

[0121] In this embodiment of the application, it is determined whether the processed road sign image conforms to the characteristics of a rectangle, and the road sign images that are rectangular in the detection results are retained. The road sign images retained after filtering are used as the first target images.

[0122] See Figure 2 , Figure 2 This is a schematic diagram of the structure of a rectangular traffic sign as shown in an embodiment of this application.

[0123] In practical applications, the road sign image to be processed is a rectangular traffic sign image. HRNET can be used to optimize the rectangular traffic sign image and determine its pixel coordinates. These pixel coordinates are the pixel coordinates of the four corner points (top left, top right, bottom left, and bottom right). The shape formed by the four corner pixel coordinates is then determined. As shown in Figure x, the four corner pixel coordinates form a rectangle, and this road sign image is retained as the first target image.

[0124] Therefore, it can be seen that by filtering according to shape recognition rules, misidentified road signs can be removed, thereby improving the accuracy of subsequent road sign calculations.

[0125] In this embodiment of the application, when the filtering rule is determined to be the recognition area filtering rule, the first target image is determined from the plurality of road sign images to be solved, including:

[0126] Receive the road sign image to be solved.

[0127] Calculate the pixel area of ​​the road sign in the image of the road sign to be solved to obtain the current road sign area;

[0128] The current road sign area is compared with the preset area to obtain the comparison result.

[0129] The road sign image whose current area is greater than the preset area in the comparison results is identified as the first target image.

[0130] In this embodiment of the application, the recognition area filtering rule refers to filtering road sign images whose recognition area is greater than a certain threshold.

[0131] In this embodiment, the pixel area of ​​the road sign in the road sign image can be calculated using contour detection algorithms and image processing functions from the OpenCV library.

[0132] In practical use, the received road sign images to be solved are converted to grayscale, and then the grayscale images are binarized to obtain binarized road sign images. Contour detection is performed using the find Contours function in the OpenCV library to obtain the contour information of all road sign images. The pixel area of ​​each contour is calculated and compared with a preset area threshold. The road sign image whose area is greater than the preset area is determined as the first target image.

[0133] As can be seen, after filtering by the area recognition rule, the remaining road sign images have larger pixel areas, thus more feature points can be extracted from the road signs, which is beneficial to improving the accuracy of subsequent calculations.

[0134] In this embodiment of the application, when the filtering rule is determined to be a recognition box filtering rule, the first target image is determined from the plurality of road sign images to be solved, including:

[0135] Receive the road sign image to be solved.

[0136] The contour detection algorithm is used to identify the contour of the road sign in the image to be solved, and the contour pixel area of ​​the road sign is calculated.

[0137] The ratio of the area of ​​the outline pixels to the bounding rectangle of the road sign image to be solved is calculated to obtain the ratio result.

[0138] The road sign images to be solved that are greater than the preset ratio in the ratio results are identified as the first target images.

[0139] In this embodiment, the identification box filtering rule refers to the ratio of the outline area of ​​the road sign identification box to the circumscribed rectangle of the road sign identification box being greater than a certain threshold.

[0140] In this embodiment of the application, the contour of the road sign image to be solved can be identified by a contour detection algorithm.

[0141] In this embodiment of the application, in order to evaluate the size of the road sign outline relative to the entire road sign image, the pixel area of ​​the road sign outline can be processed as the pixel area of ​​the bounding rectangle of the road sign image to be solved.

[0142] In this embodiment, the preset ratio can be set according to actual needs and scenarios. When the ratio result of the road sign image to be solved is greater than the preset ratio, the image is determined to be the first target image.

[0143] In this embodiment, by setting the recognition box filtering rules, images of road signs that meet the standards are retained in the road sign images to be solved, so as to avoid the outer rectangle of the road sign recognition box being too large, which would result in the road sign image content being too small and affect the solution accuracy.

[0144] This application embodiment also includes:

[0145] Receive the road sign image to be solved.

[0146] Read the distortion parameters from the road sign image to be solved.

[0147] The road sign image to be solved is distorted using an iterative distortion correction algorithm to obtain an undistorted road sign image.

[0148] In this embodiment, the distortion parameters typically include radial distortion and tangential distortion, and the distortion parameter in the road sign image to be solved refers to radial distortion.

[0149] It is understandable that, since multiple images of the same road sign are collected from different positions and angles, and the road sign is not located at the center of the entire image in most images, the pixel position at the center of the image is relatively accurate in radial distortion, but the distortion is more severe for pixels farther away from the center. Therefore, some images have severe radial distortion, which greatly affects the accuracy of subsequent road sign image processing.

[0150] In this embodiment of the application, the method for removing distortion from the road sign image to be solved can use the cv2.undistortPoints function in the OpenCV library, which is an iterative distortion removal algorithm.

[0151] In practical use, the cv2.cvtColor function is used to convert the road sign image into a grayscale image, and the cv2.findChessboardCorners function is used to detect the checkerboard corner points in the grayscale image to obtain the original first point coordinate array. The cv2.undistorPoints function is used to distort the second point coordinate data to obtain the distorted second point coordinate data. The second point coordinate data is then plotted on the original grayscale image, and the distortion removal effect is observed.

[0152] In this embodiment of the application, selecting a second target image from the image set based on the initial image includes:

[0153] Obtain the initial image;

[0154] Based on the initial image, a first target image with a sampling distance from the initial image is determined as the second target image;

[0155] Set the second target image as the initial image, and return to determine the first target image with a sampling distance of the initial image as the second target image based on the initial image, until the selection is terminated when the termination condition is met;

[0156] The termination condition includes reaching the preset target number of samples.

[0157] In this embodiment of the application, the initial image is used as the starting point for selection, and the second target image is selected from the first target image according to a preset acquisition interval.

[0158] The first target image can be sorted according to the order of the position of the acquisition vehicle when the images are acquired. Therefore, the acquisition interval actually refers to the distance that the acquisition vehicle moves between acquiring two images.

[0159] In this embodiment of the application, the acquisition interval refers to the distance traversed from the initial image to the first target image.

[0160] In actual use, the image with the largest pixel area of ​​the road sign recognition frame is used as the initial image. One first target image is selected forward at a 1m interval as the second target image. Then, the second target image is used as the initial image, and one first target image is selected in the same direction as the second target image. This operation is repeated until the number of selected images is 7.

[0161] Based on this, if the number of selected images is less than 7, the interval is reduced to 0.8 times the original interval, that is, one first target image is selected at an interval of 0.8m forward as the second target image, and the second target image is used as the initial image to select one first target image in the same direction as the second target image. This operation is repeated until the number of selected images is 7.

[0162] Based on this, if the number of selected images still does not reach 7, the interval is repeatedly reduced to 0.8 times the original interval, for a maximum of 8 iterations, until the minimum selection interval is 0.2m, or 0.8. 7 =0.2m.

[0163] If the number of selected images is still less than 7, the image selection direction is changed to select one first target image backward as the second target image, and this second target image is used as the initial image to select one first target image in the same direction as the second target image. This operation is repeated until the number of selected images reaches 7.

[0164] Based on this, if the number of selected images still does not reach 7, the interval is repeatedly reduced to 0.8 times the original interval, for a maximum of 8 iterations, until the minimum selection interval is 0.2m, or 0.8. 7 =0.2m.

[0165] If the number of images selected is still less than 7, the image selection direction will be changed to select images forward first, until the minimum interval distance of 0.2m is no longer met, and then the remaining images will be selected backward until 7 images are selected.

[0166] In this embodiment, the optimal acquisition spacing can be 1m. If the acquisition spacing is large, errors will occur in the later feature point matching process because the position of the road sign in the two images is different, the number of matching feature points will be reduced, resulting in larger errors in the rotation matrix and translation matrix.

[0167] In addition, due to the large acquisition interval, the shape and texture of road signs may vary greatly in different images, which may have an adverse effect on feature point matching.

[0168] When the acquisition interval is small, the error in the feature point matching process will decrease. However, due to interference from factors such as pixel error and camera error, the error in triangulation for solving the road sign pose will also increase.

[0169] In addition, if the image interval is too small, the changes in the rotation and translation matrices of the two images will be small, which will result in insufficient accuracy in the road sign pose calculation.

[0170] In this embodiment of the application, during the high-precision map creation process, road signs need to be identified and located, requiring multiple images to determine their pose. Typically, a basis-analysis (BA) optimization method can be used to improve the accuracy of pose estimation, that is, using two consecutive images to estimate the road sign's pose.

[0171] The effectiveness of the estimated road sign pose can be evaluated through error comparison. Other evaluation strategies can also be used, such as comparing the road sign backprojection error. This involves backprojecting the road sign pose onto two images and calculating the error between the projected point of the road sign on the images and the actual position of the road sign. By comparing the magnitude of the errors of different road sign poses, the best-performing road sign pose can be determined.

[0172] In the above embodiment, seven images are selected as the best road sign images to determine the pose of the road sign.

[0173] In this embodiment of the application, the process of determining the acquisition interval includes:

[0174] Receive the first target image.

[0175] The acquisition time point of the first target image is obtained by querying the trajectory point information of the acquisition vehicle for the first target image.

[0176] Determine the target latitude and longitude corresponding to the data collection time point.

[0177] Based on the target's latitude and longitude, calculate the acquisition distance between the two first target images.

[0178] In this embodiment of the application, the information on the vehicle trajectory points can be the latitude and longitude coordinates at which each road sign image is captured, as well as the time point at which the road sign image is captured.

[0179] In this embodiment, the latitude and longitude coordinates of two points before and after the timestamp of the captured road sign image are obtained based on a binary search algorithm. The time difference between the two timestamps is calculated, and the latitude and longitude coordinates of the first target image are obtained based on the time difference and the latitude and longitude coordinates of the two points. The latitude and longitude coordinates of the two first target images are calculated respectively, and the metric distance between the two first target images is obtained according to the distance formula.

[0180] In practical applications, based on the binary search algorithm, the first timestamp closest to the timestamp t of the road sign image is found. Second timestamp ,in, , satisfy The first timestamp is determined based on the vehicle trajectory information. Second timestamp The corresponding first coordinate Second coordinate The latitude and longitude of the road sign image are estimated based on the first distance formula.

[0181] The first distance formula is defined as follows:

[0182] (2)

[0183] In equation (2), For timestamps The coordinates are lon, which is the abbreviation for longitude. The longitude value of the first coordinate. The longitude value is the second coordinate. express Compared to Time difference, This represents the total length of the time difference. Therefore, The proportion representing the time difference indicates the percentage of longitude values ​​that occur within a given time period. Multiply this proportion by... This represents the change in longitude value over a given time period, and is finally added to the initial longitude. Time The longitude value at that time.

[0184] Calculate the metric distance between the two road sign images based on their latitude and longitude coordinates.

[0185] Corresponding to the aforementioned application function implementation method embodiments, this application also provides an embodiment of a high-precision map road sign graphic data processing device.

[0186] See Figure 3 , Figure 3 This is a schematic diagram of the structure of a high-precision map road sign graphic data processing device shown in an embodiment of this application.

[0187] This application embodiment illustrates a high-precision map road sign graphic data processing device, comprising:

[0188] Receiver module 1 receives multiple images of road signs to be solved;

[0189] The filtering module 2 filters from the multiple road sign images to be solved based on filtering rules to obtain an image set including a first target image; the first number of the first target images in the image set is at least greater than a preset recognition number; the preset recognition number is used to characterize the minimum number of road sign poses that can be solved; wherein, the filtering rules are used to retain images that meet the recognition conditions;

[0190] The acquisition module 3 takes the image with the largest pixel area of ​​the first target image in the image set as the initial image, and selects a second target image from the image set based on the initial image. The acquisition interval between the first target image and the second target image is a preset distance, and the acquisition interval between two adjacent second target images is the preset distance.

[0191] The calculation module 4 calculates the pose of the road sign based on the second target image.

[0192] In this embodiment of the application, the filtering from the multiple road sign images to be solved based on filtering rules includes:

[0193] The multiple road sign images to be solved are identified to obtain the identification results;

[0194] Based on the identification results, the corresponding filtering rules are determined;

[0195] The first target image set is determined from the multiple road sign images to be solved based on the corresponding filtering rules.

[0196] In this embodiment of the application, the filtering rules include:

[0197] Confidence filtering rules, shape recognition filtering rules, area recognition filtering rules, and / or frame ratio filtering rules.

[0198] In this embodiment of the application, the filtering rules include confidence-based filtering rules. Therefore, determining the first target image from the plurality of road sign images to be solved includes:

[0199] Receive the road sign image to be solved;

[0200] Extract the feature information of the road sign image to be solved;

[0201] The confidence level of the feature information is calculated to obtain the current confidence level data;

[0202] The current confidence level data is compared with the preset confidence level data, and the road sign image to be solved is determined to be the first target image when the current confidence level data is greater than the preset confidence level data.

[0203] And / or, if the filtering rules include shape recognition filtering rules, then determining the first target image from the multiple road sign images to be solved includes:

[0204] Receive the road sign image to be solved;

[0205] The image of the road sign to be solved is detected based on an edge detection algorithm to obtain a first detection result;

[0206] The rectangular road sign image to be solved in the detection results is identified as the first target image;

[0207] And / or, if the filtering rules include an area filtering rule, then determining the first target image from the plurality of road sign images to be solved includes:

[0208] Receive the road sign image to be solved;

[0209] Calculate the pixel area of ​​the road sign in the image of the road sign to be solved to obtain the current road sign area;

[0210] The current road sign area is compared with the preset area to obtain the comparison result;

[0211] The road sign image whose current road sign area is greater than the preset area in the comparison results is identified as the first target image;

[0212] And / or, if the filtering rules include a bounding box filtering rule, then determining the first target image from the plurality of road sign images to be solved includes:

[0213] Receive the road sign image to be solved;

[0214] The contour detection algorithm is used to identify the contour of the road sign in the image to be solved, and the contour pixel area of ​​the road sign is calculated.

[0215] Calculate the ratio of the area of ​​the outline pixels to the bounding rectangle of the road sign image to be solved, and obtain the ratio result;

[0216] The road sign images to be solved that are greater than the preset ratio in the ratio results are identified as the first target images.

[0217] This application embodiment also includes:

[0218] Receive the road sign image to be solved;

[0219] Read the distortion parameters from the road sign image to be solved;

[0220] The road sign image to be solved is distorted using an iterative distortion correction algorithm to obtain an undistorted road sign image.

[0221] In this embodiment of the application, selecting a second target image from the image set based on the initial image includes:

[0222] Obtain the initial image;

[0223] Based on the initial image, a first target image with a sampling distance from the initial image is determined as the second target image;

[0224] Set the second target image as the initial image, and return the first target image with a sampling distance of the initial image as the second target image, until the termination condition is met to terminate the selection;

[0225] The termination conditions include the preset number of iterations, the preset distance, or the preset target number of extractions.

[0226] In this embodiment of the application, the process of determining the acquisition interval includes:

[0227] Receive the first target image;

[0228] Query the trajectory point information of the acquisition vehicle for the first target image to obtain the acquisition time point of the first target image;

[0229] Determine the target latitude and longitude corresponding to the data collection time point;

[0230] Based on the target's latitude and longitude, calculate the acquisition distance between the two first target images.

[0231] In this embodiment, the device can, during the high-precision map creation process, first filter multiple road sign images to be solved based on filtering rules, removing unqualified images, and then select a second target image according to a preset acquisition interval. This second target image is then used as the standard road sign image for solution. Because multiple road sign images to be solved are first filtered to remove misidentified road sign images, and then a second target image is selected according to the preset acquisition interval, the matching of feature points between adjacent road sign images is more accurate, ensuring the accuracy of subsequent solutions.

[0232] See Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device shown in an embodiment of this application. The electronic device 500 includes a memory 510 and a processor 520.

[0233] The processor 520 can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor.

[0234] Memory 510 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage devices. ROM may store static data or instructions required by the processor 520 or other modules of the computer. Permanent storage devices may be read-write storage devices. Permanent storage devices may be non-volatile storage devices that retain stored instructions and data even when the computer is powered off. In some embodiments, permanent storage devices use mass storage devices (e.g., magnetic or optical disks, flash memory) as permanent storage devices. In other embodiments, permanent storage devices may be removable storage devices (e.g., floppy disks, optical drives). System memory may be a read-write storage device or a volatile read-write storage device, such as dynamic random access memory. System memory may store some or all of the instructions and data required by the processor during operation. Furthermore, memory 510 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), and disks and / or optical disks may also be used. In some implementations, memory 510 may include a removable storage device that is readable and / or writable, such as a laser disc (CD), a read-only digital multifunction optical disc (e.g., DVD-ROM, dual-layer DVD-ROM), a read-only Blu-ray disc, an ultra-high density optical disc, a flash memory card (e.g., SD card, mini SD card, Micro-SD card, etc.), a magnetic floppy disk, etc. Computer-readable storage media do not contain carrier waves or transient electronic signals transmitted wirelessly or via wired connections.

[0235] The memory 510 stores executable code, which, when processed by the processor 520, can cause the processor 520 to execute part or all of the methods described above.

[0236] The solution of this application has been described in detail above with reference to the accompanying drawings. In the above embodiments, the descriptions of each embodiment have different emphases; parts not described in detail in a certain embodiment can be referred to in the relevant descriptions of other embodiments. Those skilled in the art should also understand that the actions and modules involved in the specification are not necessarily essential to this application. Furthermore, it is understood that the steps in the method of this application embodiment can be adjusted, combined, and deleted according to actual needs, and the modules in the device of this application embodiment can be combined, divided, and deleted according to actual needs.

[0237] Furthermore, the method according to this application can also be implemented as a computer program or computer program product, which includes computer program code instructions for performing some or all of the steps in the method described above.

[0238] Alternatively, this application may be implemented as a non-transitory machine-readable storage medium (or computer-readable storage medium, or machine-readable storage medium) storing executable code (or computer program, or computer instruction code) thereon, which, when executed by a processor of an electronic device (or electronic device, server, etc.), causes the processor to perform part or all of the steps of the methods described above according to this application.

[0239] Those skilled in the art will also understand that the various exemplary logic blocks, modules, circuits, and algorithm steps described in connection with the present application can be implemented as electronic hardware, computer software, or a combination of both.

[0240] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems and methods according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0241] The various embodiments of this application have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for processing graphic data of road signs on high-precision maps, characterized in that, include: Receive multiple road sign images to be solved; Based on the filtering rules, the multiple road sign images to be solved are filtered to obtain an image set including the first target image; The first number of target images in the image set is at least greater than a preset recognition number; the preset recognition number is used to characterize the minimum number of road sign poses that can be calculated; wherein, the filtering rule is used to retain images that meet the recognition conditions; The image with the largest pixel area among the first target images in the image set is taken as the initial image. Based on the initial image, a first target image with a sampling distance equal to the initial image is determined as the second target image. The second target image is set as the initial image, and the process of determining the first target image with a sampling distance equal to the initial image as the second target image is repeated until a termination condition is met, which terminates the selection. The termination condition includes reaching a preset number of targets. The sampling distance between the first target image and the second target image is a preset distance, and the sampling distance between two adjacent second target images is the preset distance. The pose of the road sign is calculated based on the second target image.

2. The method for solving high-precision map road sign graphic data according to claim 1, characterized in that, The filtering process based on filtering rules from the multiple road sign images to be solved includes: The multiple road sign images to be solved are identified to obtain the identification results; Based on the identification results, the corresponding filtering rules are determined; The first target image set is determined from the multiple road sign images to be solved based on the corresponding filtering rules.

3. The method for calculating high-precision map road sign graphic data according to claim 2, characterized in that, The filtering rules include: Confidence filtering rules, shape recognition filtering rules, area recognition filtering rules, and / or frame ratio filtering rules.

4. The method for calculating high-precision map road sign graphic data according to claim 3, characterized in that, The filtering rules include confidence filtering rules, and the first target image is determined from the multiple road sign images to be solved, including: Receive the road sign image to be solved; Extract the feature information of the road sign image to be solved; The confidence level of the feature information is calculated to obtain the current confidence level data; The current confidence level data is compared with the preset confidence level data, and the road sign image to be solved is determined to be the first target image when the current confidence level data is greater than the preset confidence level data. And / or, if the filtering rules include shape recognition filtering rules, then determining the first target image from the multiple road sign images to be solved includes: Receive the road sign image to be solved; The road sign image to be solved is detected based on an edge detection algorithm to obtain the detection result; The rectangular road sign image to be solved in the detection results is identified as the first target image; And / or, if the filtering rules include an area filtering rule, then determining the first target image from the plurality of road sign images to be solved includes: Receive the road sign image to be solved; Calculate the pixel area of ​​the road sign in the image of the road sign to be solved to obtain the current road sign area; The current road sign area is compared with the preset area to obtain the comparison result; The road sign image whose current road sign area is greater than the preset area in the comparison results is identified as the first target image; And / or, if the filtering rules include a bounding box filtering rule, then determining the first target image from the plurality of road sign images to be solved includes: Receive the road sign image to be solved; The contour detection algorithm is used to identify the contour of the road sign in the image to be solved, and the contour pixel area of ​​the road sign is calculated. Calculate the ratio of the area of ​​the outline pixels to the bounding rectangle of the road sign image to be solved, and obtain the ratio result; The road sign images to be solved that are greater than a preset ratio in the ratio results are identified as the first target images.

5. The method for calculating high-precision map road sign graphic data according to claim 1, characterized in that, Also includes: Receive the road sign image to be solved; Read the distortion parameters from the road sign image to be solved; The road sign image to be solved is distorted using an iterative distortion correction algorithm to obtain an undistorted road sign image.

6. The method for calculating high-precision map road sign graphic data according to claim 1, characterized in that, The process of determining the acquisition interval includes: Receive the first target image; Query the trajectory point information of the acquisition vehicle for the first target image to obtain the acquisition time point of the first target image; Determine the target latitude and longitude corresponding to the data collection time point; Based on the target's latitude and longitude, calculate the acquisition distance between the two first target images.

7. A device for processing graphic data of high-precision map road signs, characterized in that, include: The receiving module receives multiple images of road signs to be solved; The filtering module filters from the multiple road sign images to be solved based on filtering rules to obtain an image set including the first target image; The first number of target images in the image set is at least greater than a preset recognition number; the preset recognition number is used to characterize the minimum number of road sign poses that can be calculated; wherein, the filtering rule is used to retain images that meet the recognition conditions; The acquisition module takes the image with the largest pixel area of ​​the first target image in the image set as the initial image, and based on the initial image, determines a first target image with an acquisition distance equal to the initial image as the second target image. It then sets the second target image as the initial image and returns to the previous state, determining a second target image with an acquisition distance equal to the initial image as the second target image. The selection process continues until a termination condition is met, including reaching a preset number of targets. The acquisition distance between the first target image and the second target image is a preset distance, and the acquisition distance between two adjacent second target images is also the preset distance. The solution module calculates the pose of the road sign based on the second target image.

8. An electronic device, characterized in that, include: processor; as well as A memory having executable code stored thereon, which, when executed by the processor, causes the processor to perform the method as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, It stores executable code that, when executed by a processor of an electronic device, causes the processor to perform the method as described in any one of claims 1-6.

Images