Road sign generation method and device based on spatial clustering

Abstract

The present application relates to a method and device for generating road signs based on spatial clustering. The method includes: acquiring multiple consecutive frames of road images, generating feature points of road signs in the same coordinate system for the multiple frames of road images; clustering the feature points based on a preset clustering algorithm, selecting clustering results that meet preset conditions, performing three-dimensional plane fitting, and obtaining a first three-dimensional plane equation; projecting the pixel points of the road sign corner points onto the first three-dimensional plane equation to obtain the three-dimensional coordinate points of the road sign corner points for each frame of road image; performing three-dimensional plane fitting on the three-dimensional coordinate points of the road sign corner points to obtain a second three-dimensional plane equation; averaging the three-dimensional coordinate points of the road sign corner points based on a preset order of the road sign corner points to obtain the three-dimensional coordinate points of candidate corner points, and projecting the three-dimensional coordinates of the candidate corner points onto the second three-dimensional plane equation to obtain the coordinates of the target road sign corner points. The present application can improve the accuracy of road sign generation results.

Description

Technical Field

[0001] The present application relates to the field of map image processing, and in particular to a method and device for generating road signs based on spatial clustering. Background Art

[0002] In the field of autonomous driving, simultaneous localization and mapping (SLAM) refers to the use of a single visual sensor (such as a camera) by an autonomous vehicle to create a map consistent with the real environment and simultaneously determine its own position in the map.

[0003] Existing technology requires at least two images to create a SLAM map in order to calculate its 3D information. However, image data measured by a single visual sensor only provides the relative distances between objects, not their true 3D spatial information. However, using only two images as the SLAM map solution can result in significant accuracy errors (due to poor visual accuracy). To address this accuracy issue, multiple images are typically used. The conventional approach involves averaging the results from multiple frames. While this can suppress fluctuations in individual frames, it lacks the ability to remove outliers, so erroneous data can still be included in the results, leading to inaccurate results.

[0004] Therefore, in the process of generating road signs in the SLAM map, there is a problem of low accuracy of the road sign generation results. Summary of the Invention

[0005] In order to solve or partially solve the problems existing in the related art, the present application provides a road sign generation method and device based on spatial clustering, which improves the accuracy of road sign generation results during the road sign generation process of the SLAM map.

[0006] In a first aspect of the present application, a method for generating road signs based on spatial clustering is provided, comprising:

[0007] Acquire multiple frames of continuous road images and generate feature points of road signs in the multiple frames of road images in the same coordinate system;

[0008] Clustering the feature points based on a preset clustering algorithm, selecting clustering results that meet preset conditions for three-dimensional plane fitting, and obtaining a first three-dimensional plane equation;

[0009] Projecting the pixel points of the road sign corners onto the first three-dimensional plane equation to obtain the three-dimensional coordinate points of the road sign corners in each frame of the road image;

[0010] Perform three-dimensional plane fitting on the three-dimensional coordinate points of the road sign corners to obtain the second three-dimensional plane equation;

[0011] Based on the preset order of road sign corner points, the three-dimensional coordinate points of the road sign corner points are averaged to obtain the three-dimensional coordinate points of the candidate corner points, and the three-dimensional coordinates of the candidate corner points are projected onto the second three-dimensional plane equation to obtain the coordinates of the target road sign corner points.

[0012] Optionally, acquiring multiple frames of continuous road images and generating feature points of road signs in the multiple frames of road images in the same coordinate system includes:

[0013] Preprocessing the continuous multi-frame road images to obtain the first coordinate point of the road sign pixels in the continuous multi-frame road images in the camera coordinate system;

[0014] Obtain a first mapping relationship between the camera coordinate system and the world coordinate system, and generate a feature point of the first coordinate point under the SLAM map according to the first coordinate point and the first mapping relationship.

[0015] Optionally, obtaining a first mapping relationship between the camera coordinate system and the world coordinate system includes:

[0016] A pose sequence of a vehicle-mounted camera is obtained, and a first mapping relationship is generated based on the movement distance and the pose sequence of multiple consecutive frames of road images.

[0017] Optionally, clustering the feature points based on a preset clustering algorithm includes:

[0018] Clustering the feature points based on a preset clustering algorithm to obtain at least three groups of clustering results;

[0019] And / or two groups of clustering results are obtained, wherein at least one group of clustering results includes at least half the number of feature points.

[0020] Optionally, clustering the feature points based on a preset clustering algorithm includes:

[0021] All feature points are clustered based on the DBSCAN algorithm according to the pixel threshold from high to low.

[0022] Optionally, clustering results that meet preset conditions are selected for three-dimensional plane fitting, including:

[0023] Obtaining a quality score for the clustering result, which is used to indicate the quality of the street sign image restored by the clustering result;

[0024] The characteristic points of the clustering results with the highest quality score are selected for three-dimensional plane fitting.

[0025] Optionally, based on a preset order of road sign corner points, averaging the three-dimensional coordinate points of the road sign corner points to obtain the three-dimensional coordinate points of the candidate corner points, and projecting the three-dimensional coordinates of the candidate corner points onto the second three-dimensional plane equation to obtain the coordinates of the target road sign corner point, including:

[0026] Classify the road sign corner points based on their positions at the four corners of the road sign and the preset order of the road sign corner points;

[0027] Calculate the average of the three-dimensional coordinates of each type of road sign corner point to obtain the three-dimensional coordinate points of the candidate corner points;

[0028] Based on the preset order of street sign corner points, the candidate corner points are projected onto the second three-dimensional plane equation.

[0029] In a second aspect of the present application, a road sign generation device based on spatial clustering is provided, comprising:

[0030] An acquisition unit, configured to acquire a plurality of consecutive road image frames and generate feature points of road signs in the plurality of road image frames in the same camera coordinate system;

[0031] A clustering unit is used to cluster the feature points based on a preset clustering algorithm, select clustering results that meet preset conditions to perform three-dimensional plane fitting, and obtain a first three-dimensional plane equation;

[0032] A projection unit, configured to project pixel points of the road sign corners onto a first three-dimensional plane equation to obtain three-dimensional coordinate points of the road sign corners in each frame of the road image;

[0033] A fitting unit, used for performing three-dimensional plane fitting on the three-dimensional coordinate points of the road sign corner points to obtain a second three-dimensional plane equation;

[0034] The solving unit is used to average the three-dimensional coordinate points of the road sign corner points based on a preset road sign corner point sequence to obtain the three-dimensional coordinate points of the candidate corner points, and project the three-dimensional coordinates of the candidate corner points onto the second three-dimensional plane equation to obtain the coordinates of the target road sign corner points.

[0035] A third aspect of the present application provides an electronic device, including:

[0036] processor; and

[0037] The memory stores executable codes thereon, and when the executable codes are executed by the processor, the processor is caused to execute the method described above.

[0038] A fourth aspect of the present application provides a computer-readable storage medium having executable code stored thereon. When the executable code is executed by a processor of an electronic device, the processor is caused to execute the method described above.

[0039] The technical solution provided by this application may include the following beneficial effects: This application relates to a method and device for generating road signs based on spatial clustering. The method includes: obtaining multiple consecutive frames of road images, generating feature points of road signs in the multiple frames of road images in the same coordinate system; clustering the feature points based on a preset clustering algorithm, selecting clustering results that meet preset conditions for three-dimensional plane fitting, and obtaining a first three-dimensional plane equation; obtaining pixel points of the road sign corner points in each frame of road image, projecting the road sign corner point pixel points onto the first three-dimensional plane equation, and obtaining the three-dimensional coordinate points of the road sign corner points in each frame of road image; performing three-dimensional plane fitting on the three-dimensional coordinate points of the road sign corner points, and obtaining a second three-dimensional plane equation; averaging the three-dimensional coordinate points of the road sign corner points based on a preset order of road sign corner points to obtain candidate three-dimensional coordinate points, and projecting the three-dimensional coordinates of the candidate corner points onto the second three-dimensional plane equation to obtain the target road sign corner point coordinates. This application uses a clustering method to eliminate error points of road sign pixel points in multiple frames of road images, thereby improving the accuracy of generating road sign images. At the same time, by fitting the three-dimensional plane equation twice, the final fitting result is closer to reality.

[0040] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The above and other objects, features and advantages of the present application will become more apparent by describing in more detail exemplary embodiments of the present application in conjunction with the accompanying drawings, wherein the same reference numerals generally represent the same components in the exemplary embodiments of the present application.

[0042] Figure 1 1 is a flow chart of a method for generating road signs based on spatial clustering according to an embodiment of the present application;

[0043] Figure 2 Schematic diagram of the structure of a road sign generation device based on spatial clustering shown in an embodiment of the present application;

[0044] Figure 3 It is a structural diagram of an electronic device shown in an embodiment of the present application. DETAILED DESCRIPTION

[0045] The following describes embodiments of the present application in more detail with reference to the accompanying drawings. Although the accompanying drawings illustrate embodiments of the present application, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. Rather, these embodiments are provided to make the present application more thorough and complete, and to fully convey the scope of the present application to those skilled in the art.

[0046] The terms used in this application are for the purpose of describing specific embodiments only and are not intended to limit this application. As used in this application and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0047] It should be understood that although the terms "first", "second", "third", etc. may be used in this application to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from each other. For example, without departing from the scope of this application, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0048] In order to help those skilled in the art better understand the technical solution of this application, the terms involved in this application are introduced below.

[0049] SLAM (Simultaneous Localization and Mapping) refers to the process by which an autonomous vehicle uses a single visual sensor (such as a camera) to create a map consistent with the real environment and simultaneously determine its own position within the map. Clustering algorithms, in which cluster analysis consists of several patterns, typically a vector of measurements or a point in multidimensional space. Cluster analysis is based on similarity; patterns within a cluster are more similar than patterns in different clusters. The DBSCAN algorithm (Density-Based Spatial Clustering of Applications with Noise) is a density-based clustering algorithm. Unlike partitioning and hierarchical clustering algorithms, it defines a cluster as the largest set of density-connected points. It partitions areas with sufficiently high density into clusters and can discover clusters of arbitrary shapes in noisy spatial databases. It partitions areas with sufficient density into clusters and can discover clusters of arbitrary shapes in noisy spatial databases. It defines a cluster as the largest set of density-connected points. The algorithm primarily determines two parameters: epsilon: the radius of the neighborhood around a point, and minPts: the minimum number of points within the neighborhood. An arbitrary point (not assigned to a class or designated as an outlier) is selected and its NBHD (p.epsilon) is calculated to determine whether it is a core point. If so, a class is established around the point; otherwise, it is designated as an outlier. The algorithm iterates through the remaining points until a class is established. Directly reachable points are added to the class, followed by density-reachable points. If a point marked as an outlier is added, its status is changed to an edge point. Steps 1 and 2 are repeated until all points meet the criteria of being in a class (core or edge) or an outlier. The algorithm is evaluated using a contour function to determine the optimal parameters. In the field of autonomous driving, simultaneous localization and mapping (SLAM) involves the autonomous vehicle using a single visual sensor (such as a camera) to create a map consistent with the real world and simultaneously determine its own position within the map.

[0050] Existing technology requires at least two images to create a SLAM map in order to calculate its 3D information. However, image data measured by a single visual sensor only provides the relative distances between objects, not their true 3D spatial information. However, using only two images as the SLAM map solution can result in significant accuracy errors (due to poor visual accuracy). To address this accuracy issue, multiple images are typically used. The conventional approach involves averaging the results from multiple frames. While this can suppress fluctuations in individual frames, it lacks the ability to remove outliers, so erroneous data can still be included in the results, leading to inaccurate results.

[0051] Therefore, when the SLAM map is used to locate the vehicle, there is a problem of low positioning accuracy.

[0052] To address the above problems, an embodiment of the present application provides a road sign generation method based on spatial clustering to improve the accuracy of road signs in SLAM maps.

[0053] The technical solutions of the embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0054] See also Figure 1 , is a flowchart of a method for generating road signs based on spatial clustering according to an embodiment of the present application, which mainly includes steps S101 to S104, as described below:

[0055] Step S101 : acquiring multiple frames of road images and generating feature points of road signs in the multiple frames of road images in the same coordinate system.

[0056] The multi-frame road image is an image captured by the same camera device within a continuous time period, or the multi-frame road image is an image generated by extracting frames from the same vehicle-mounted camera. The camera device may include but is not limited to a driving recorder, a camera, or a driver's mobile phone installed on the vehicle and other devices with a camera function. The camera device may be a monocular camera device. The camera device may be set at the front of the vehicle to record the road signs in front of the vehicle, thereby obtaining continuous video images containing the road signs, so as to obtain the continuous multi-frame road images in this step. The video data containing the road signs obtained while the vehicle is driving is extracted. Generally, the frame rate of the video is 30 frames per second. The video can be extracted according to preset rules, for example, 10 frames, 15 frames, 20 frames or other values ​​per second. The time interval between two adjacent frames of the image is the frame extraction time interval, thereby obtaining multiple captured images.

[0057] In this embodiment, the road sign is a square sign, and in a road image, a road sign is usually a polygon.

[0058] Step S101 requires that the road sign pixels of the multiple frames of road images be unified into the same coordinate system after the rotation matrix and translation vector between the multiple frames of road images are obtained, so as to facilitate clustering of all road sign pixels in the same coordinate system.

[0059] In one embodiment, obtaining a continuous multi-frame road image and generating feature points of road signs in the multi-frame road image in the same coordinate system include: preprocessing the continuous multi-frame road image to obtain a first coordinate point of road sign pixels in the continuous multi-frame road image in the camera coordinate system; obtaining a first mapping relationship between the camera coordinate system and the world coordinate system, and generating a feature point of the first coordinate point in the SLAM map based on the first coordinate point and the first mapping relationship.

[0060] In this embodiment, preprocessing multiple frames of road images includes acquiring road images that meet preset conditions, binarizing the road images, and identifying road sign pixels. Identifying road sign pixels can be performed based on the square imaging characteristics of road signs. The road images that meet the preset conditions are used to determine the clarity of the acquired road images.

[0061] In this embodiment, the road images need to be unified into the same coordinate system based on a first mapping relationship. In one embodiment, obtaining the first mapping relationship between the camera coordinate system and the world coordinate system includes: obtaining a pose sequence of the vehicle-mounted camera, and generating the first mapping relationship based on the movement distance and pose sequence of multiple consecutive frames of road images.

[0062] In this embodiment, the pixel depth of the pixel coordinates is restored based on the moving distance and posture sequence of multiple consecutive frames of road images, thereby obtaining the three-dimensional point of the road sign pixel on the SLAM map.

[0063] In some embodiments, the coordinate system of any one frame of the continuous multiple frames of road images may be set as the camera coordinate system.

[0064] In one embodiment, the coordinate system of the last frame of the road image in the continuous multi-frame road image can be set as the camera coordinate system to obtain a first coordinate point. Then, based on the internal and external parameters of the last frame of the road image and the vehicle-mounted camera, a first mapping relationship between the same camera coordinate system and the world coordinate system is obtained. According to the first mapping relationship and the first coordinate, the three-dimensional point of the road sign pixel on the SLAM map is obtained.

[0065] Step S102 : clustering the feature points based on a preset clustering algorithm, selecting clustering results that meet preset conditions to perform three-dimensional plane fitting, and obtaining a first three-dimensional plane equation.

[0066] In step S102, the points with large fluctuations are eliminated from the feature points through the clustering method, and only high-confidence results are retained to improve the accuracy of the plane results. At the same time, a preset clustering algorithm is used and multi-level / dynamic thresholds are set to improve the robustness of the algorithm.

[0067] In one embodiment, clustering the feature points based on a preset clustering algorithm includes: clustering the feature points based on a preset clustering algorithm to obtain at least three groups of clustering results; and / or obtaining two groups of clustering results, wherein at least one group of clustering results includes at least half the number of feature points.

[0068] In this embodiment, clustering is stopped if more than three distance results are generated during the clustering process, and the clustering result with the most cluster feature points is selected as the clustering result that meets the preset conditions. When two clustering results are obtained, the two clustering results are scored, and the clustering result with the better score is selected as the final clustering result.

[0069] In one embodiment, clustering the feature points based on a preset clustering algorithm includes: clustering all feature points based on the DBSCAN algorithm according to pixel thresholds from high to low. Specifically, the clustering includes:

[0070] First, considering the uneven distribution density of the minority class samples within the class, a set of EPS values ​​based on the distribution density of the feature point set can be obtained ( , used to describe the density of sample distribution in the neighborhood); then, the average distances of each minority class sample point obtained by calculation are combined into a distance vector array, and these average distances are used as the original data set, and distance clustering is performed on this data set; after clustering this distance array into N clusters, the sum of all distances in each cluster is calculated and averaged, and the obtained average value is used as the neighborhood threshold of this cluster. By calculating the average values ​​of these N clusters separately, N neighborhood thresholds EPSi, i = 1, 2, ..., N can be obtained; next, these N domain thresholds are sorted in ascending order and saved in an array; in the following clustering algorithm, the smallest one in the threshold array is first selected as the EPS value of the DBSCAN algorithm, and then all minority class samples are clustered, and then the next threshold in the threshold array is used to continue DBSCAN clustering on the minority class samples marked as noise sample points, and some cluster clusters and remaining noise sample points are also obtained. Finally, repeat the above operation. After all minority class samples are clustered by different EPS values, all clustering operations of minority class samples are completed. Those data that are not classified into any cluster in the end are noise data.

[0071] In this embodiment, the number of feature points in each group is sorted from largest to smallest, and then the traversal is performed. When the cumulative number of feature points in the traversal is greater than 50%, or the number of traversed groups is greater than 3, the traversal is stopped. In this embodiment, the clustering result that meets the preset conditions is the group with the most feature points when the traversal group is greater than 3, or the clustering result with feature points greater than 50% is directly selected as the clustering result that meets the preset conditions.

[0072] In one embodiment, clustering results that meet preset conditions are selected for three-dimensional plane fitting, including: obtaining a quality score of the clustering result, where the quality score is used to indicate the quality of the street sign image restored by the clustering result; and selecting feature points of the clustering result with the highest quality score for three-dimensional plane fitting.

[0073] In this embodiment, the quality score of the clustering results is positively correlated with the quality of the clustered feature points. The quality score can be obtained by evaluating the clustering results based on the imaging characteristics of the road sign. Alternatively, the distance results of the road sign can be restored to the road sign image and the score can be assigned based on the degree of restoration of the road sign image. Alternatively, feature points can be selected from the clustering results and the similarity between them can be calculated. Feature points with higher similarity will have higher quality scores corresponding to the clustering results.

[0074] In this embodiment, the feature points of the clustering result with the highest quality score are selected to perform three-dimensional plane fitting, including: substituting all pixel points in the clustering result into the three-dimensional plane equation for solution to obtain the final fitting equation, the first three-dimensional plane equation.

[0075] Step S103 : Projecting the pixel points of the road sign corners onto the first three-dimensional plane equation to obtain the three-dimensional coordinate points of the road sign corners in each frame of the road image.

[0076] In this embodiment, before step S103, it is necessary to obtain the pixel points of the road sign corners for each frame of the road image. This embodiment obtains a first vector from the origin O of the camera coordinate system to the first three-dimensional plane equation. Based on the first vector and the first coordinates of the road sign corner pixels, the projection points of the road sign corner pixels on the first three-dimensional plane equation are obtained.

[0077] Step S104: performing three-dimensional plane fitting on the three-dimensional coordinate points of the road sign corner points to obtain a second three-dimensional plane equation.

[0078] Step S104: Substitute the three-dimensional coordinates of the corner points of the road sign into the three-dimensional plane equation to solve it, and obtain the final fitting equation, which is the second three-dimensional plane equation. The second three-dimensional plane is used to further verify the reliability of the three-dimensional coordinates.

[0079] Step S105 , based on a preset order of road sign corner points, average the three-dimensional coordinate points of the road sign corner points to obtain the three-dimensional coordinate points of the candidate corner points, and project the three-dimensional coordinates of the candidate corner points onto the second three-dimensional plane equation to obtain the coordinates of the target road sign corner points.

[0080] In step S105, numerical averaging is performed in order of the corner points, and the average coordinates are projected onto the second three-dimensional plane equation as the final coordinates of the road sign corner points.

[0081] In one embodiment, based on a preset order of road sign corner points, the three-dimensional coordinates of the road sign corner points are averaged to obtain candidate three-dimensional coordinates of the corner points, and the three-dimensional coordinates of the candidate corner points are projected onto a second three-dimensional plane equation to obtain the coordinates of the target road sign corner point. This method includes: classifying the road sign corner points based on their positions at the four corners of the road sign and the preset order of the road sign corner points; averaging the three-dimensional coordinates of each type of road sign corner point to obtain the three-dimensional coordinates of the candidate corner points; and projecting the candidate corner points onto the second three-dimensional plane equation based on the preset order of the road sign corner points. The order of the road sign corner points includes selecting the four corner points of the road sign in a counterclockwise or clockwise order.

[0082] In step S105, after obtaining the coordinates of the corner points of the target road sign, the center point of the four corner points is obtained, and the world coordinates corresponding to the center point of the four corner points are stored in the high-precision map as the map coordinates of the road sign.

[0083] Corresponding to the aforementioned embodiment of the method for realizing the application function, the present application also provides a road sign generation device based on spatial clustering, an electronic device and corresponding embodiments.

[0084] Figure 2 1 is a schematic diagram of the structure of a road sign generation device based on spatial clustering according to an embodiment of the present application. For ease of explanation, only the parts related to the embodiment of the present application are shown. Figure 2 The exemplary apparatus mainly includes an acquisition unit 201, a clustering unit 202, a projection unit 203, a fitting unit 204, and a solution unit 205, wherein:

[0085] An acquisition unit 201 is configured to acquire multiple frames of road images and generate feature points of road signs in the multiple frames of road images in the same camera coordinate system.

[0086] A clustering unit 202 is configured to cluster the feature points based on a preset clustering algorithm, select clustering results that meet preset conditions, perform three-dimensional plane fitting, and obtain a first three-dimensional plane equation;

[0087] A projection unit 203 is configured to project the pixel points of the road sign corners onto the first three-dimensional plane equation to obtain three-dimensional coordinate points of the road sign corners in each frame of the road image;

[0088] A fitting unit 204 is configured to perform three-dimensional plane fitting on the three-dimensional coordinate points of the road sign corner points to obtain a second three-dimensional plane equation;

[0089] The solving unit 205 is used to average the three-dimensional coordinate points of the road sign corner points based on a preset road sign corner point order to obtain the three-dimensional coordinate points of the candidate corner points, and project the three-dimensional coordinates of the candidate corner points onto the second three-dimensional plane equation to obtain the coordinates of the target road sign corner points.

[0090] In one embodiment, obtaining a continuous multi-frame road image and generating feature points of road signs in the multi-frame road image in the same coordinate system include: preprocessing the continuous multi-frame road image to obtain a first coordinate point of road sign pixels in the continuous multi-frame road image in the camera coordinate system; obtaining a first mapping relationship between the camera coordinate system and the world coordinate system, and generating a feature point of the first coordinate point in the SLAM map based on the first coordinate point and the first mapping relationship.

[0091] In one embodiment, obtaining a first mapping relationship between a camera coordinate system and a world coordinate system includes: obtaining a posture sequence of a vehicle-mounted camera, and generating the first mapping relationship based on a movement distance and a posture sequence of a continuous plurality of frames of road images.

[0092] In one embodiment, clustering the feature points based on a preset clustering algorithm includes: clustering the feature points based on a preset clustering algorithm to obtain at least three groups of clustering results; and / or obtaining two groups of clustering results, wherein at least one group of clustering results includes at least half the number of feature points.

[0093] In one embodiment, clustering the feature points based on a preset clustering algorithm includes: clustering all feature points based on a DBSCAN algorithm according to pixel thresholds from high to low.

[0094] In one embodiment, clustering results that meet preset conditions are selected for three-dimensional plane fitting, including: obtaining a quality score of the clustering result, where the quality score is used to indicate the quality of the street sign image restored by the clustering result; and selecting feature points of the clustering result with the highest quality score for three-dimensional plane fitting.

[0095] In one embodiment, based on a preset order of road sign corner points, the three-dimensional coordinate points of the road sign corner points are averaged to obtain the three-dimensional coordinate points of the candidate corner points, and the three-dimensional coordinates of the candidate corner points are projected onto a second three-dimensional plane equation to obtain the coordinates of the target road sign corner points, including: classifying the road sign corner points based on the positions of the road sign corner points at the four corners of the road sign and the preset order of road sign corner points; calculating the average value of the three-dimensional coordinates of each type of road sign corner points to obtain the three-dimensional coordinate points of the candidate corner points; and projecting the candidate corner points onto the second three-dimensional plane equation based on the preset order of road sign corner points.

[0096] The technical solution provided by this application may include the following beneficial effects: This application relates to a method and device for generating road signs based on spatial clustering. The method includes: obtaining multiple consecutive frames of road images, generating feature points of road signs in the multiple frames of road images in the same coordinate system; clustering the feature points based on a preset clustering algorithm, selecting clustering results that meet preset conditions for three-dimensional plane fitting, and obtaining a first three-dimensional plane equation; obtaining pixel points of the road sign corner points in each frame of road image, projecting the road sign corner point pixel points onto the first three-dimensional plane equation, and obtaining the three-dimensional coordinate points of the road sign corner points in each frame of road image; performing three-dimensional plane fitting on the three-dimensional coordinate points of the road sign corner points, and obtaining a second three-dimensional plane equation; averaging the three-dimensional coordinate points of the road sign corner points based on a preset order of road sign corner points to obtain candidate three-dimensional coordinate points, and projecting the three-dimensional coordinates of the candidate corner points onto the second three-dimensional plane equation to obtain the target road sign corner point coordinates. This application uses a clustering method to eliminate error points of road sign pixel points in multiple frames of road images, thereby improving the accuracy of generating road sign images. At the same time, by fitting the three-dimensional plane equation twice, the final fitting result is closer to reality.

[0097] See also Figure 3 , is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. The electronic device 300 includes a memory 310 and a processor 320 .

[0098] The processor 320 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0099] Memory 310 may include various types of storage units, such as system memory, read-only memory (ROM), and permanent storage. ROM may store static data or instructions required by processor 320 or other computer modules. Permanent storage may be a readable and writable storage device. Permanent storage may be a non-volatile storage device that retains stored instructions and data even when the computer is powered off. In some embodiments, the permanent storage device utilizes a mass storage device (e.g., a magnetic or optical disk, flash memory). In other embodiments, the permanent storage device may be a removable storage device (e.g., a floppy disk, optical drive). System memory may be a readable and writable storage device or a volatile readable and writable storage device, such as dynamic random access memory (DRAM). System memory may store some or all instructions and data required by the processor during operation. Furthermore, memory 310 may include any combination of computer-readable storage media, including various types of semiconductor memory chips (e.g., DRAM, SRAM, SDRAM, flash memory, programmable read-only memory), as well as magnetic disks and / or optical disks. In some embodiments, the memory 310 may include a readable and / or writable removable storage device, such as a compact disc (CD), a read-only digital versatile disc (e.g., DVD-ROM, double-layer DVD-ROM), a read-only Blu-ray disc, an ultra-density optical disc, a flash memory card (e.g., SD card, mini SD card, Micro-SD card, etc.), a magnetic floppy disk, etc. Computer-readable storage media do not contain carrier waves and transient electronic signals transmitted wirelessly or wired.

[0100] The memory 310 stores executable codes. When the executable codes are processed by the processor 320 , the processor 320 may execute part or all of the above-mentioned methods.

[0101] In addition, the method according to the present application may also be implemented as a computer program or a computer program product, which includes computer program code instructions for executing some or all of the steps in the above method of the present application.

[0102] Alternatively, the present application can also be implemented as a computer-readable storage medium (or non-transitory machine-readable storage medium or machine-readable storage medium) on which executable code (or computer program or computer instruction code) is stored. When the executable code (or computer program or computer instruction code) is executed by a processor of a vehicle (or server, etc.), the processor executes part or all of the steps of the above-mentioned method according to the present application.

[0103] The embodiments of the present application have been described above. The above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is selected to best explain the principles of the embodiments, their practical applications, or improvements to the technology in the market, or to enable other persons skilled in the art to understand the embodiments disclosed herein.

Claims

1. A road sign generation method based on spatial clustering, characterized in that: The method comprises: Acquire multiple frames of continuous road images and generate feature points of road signs in the multiple frames of road images in the same coordinate system; Clustering the feature points based on a preset clustering algorithm, selecting clustering results that meet preset conditions for three-dimensional plane fitting, and obtaining a first three-dimensional plane equation; Projecting the pixel points of the road sign corners onto the first three-dimensional plane equation to obtain the three-dimensional coordinate points of the road sign corners in each frame of the road image; Performing three-dimensional plane fitting on the three-dimensional coordinate points of the road sign corner points to obtain a second three-dimensional plane equation; Based on a preset order of road sign corner points, the three-dimensional coordinate points of the road sign corner points are averaged to obtain the three-dimensional coordinate points of candidate corner points, and the three-dimensional coordinates of the candidate corner points are projected onto the second three-dimensional plane equation to obtain the coordinates of the target road sign corner point.

2. The method according to claim 1, characterized in that The step of acquiring multiple frames of continuous road images and generating feature points of road signs in the multiple frames of road images in the same coordinate system includes: Preprocessing the continuous multi-frame road images to obtain a first coordinate point of a road sign pixel in the continuous multi-frame road images in a camera coordinate system; A first mapping relationship between a camera coordinate system and a world coordinate system is obtained, and a feature point of the first coordinate point under a SLAM map is generated according to the first coordinate point and the first mapping relationship.

3. The method according to claim 2, characterized in that The obtaining of a first mapping relationship between the camera coordinate system and the world coordinate system includes: A posture sequence of a vehicle-mounted camera is obtained, and the first mapping relationship is generated based on the movement distance of the continuous multi-frame road image and the posture sequence.

4. The method according to claim 1, wherein The clustering of the feature points based on a preset clustering algorithm includes: Clustering the feature points based on the preset clustering algorithm to obtain at least three groups of clustering results; And / or two groups of clustering results are obtained, wherein at least one group of clustering results includes at least half the number of feature points.

5. The method according to claim 4, characterized in that The clustering of the feature points based on the preset clustering algorithm includes: The feature points are clustered based on the DBSCAN algorithm according to pixel thresholds from high to low.

6. The method according to claim 4, characterized in that The selecting of clustering results that meet preset conditions for three-dimensional plane fitting includes: Obtaining a quality score of the clustering result, where the quality score is used to indicate the quality of the road sign image restored by the clustering result; The characteristic points of the clustering results with the highest quality score are selected for three-dimensional plane fitting.

7. The method according to claim 1, characterized in that The method includes: averaging the three-dimensional coordinates of the road sign corner points based on a preset order of road sign corner points to obtain three-dimensional coordinates of candidate corner points; and projecting the three-dimensional coordinates of the candidate corner points onto the second three-dimensional plane equation to obtain the coordinates of the target road sign corner point. Classifying the road sign corner points based on the positions of the road sign corner points at the four corners of the road sign and the preset order of the road sign corner points; Calculate the average of the three-dimensional coordinates of each type of road sign corner point to obtain the three-dimensional coordinate points of the candidate corner points; Based on the preset road sign corner point sequence, the three-dimensional coordinates of the candidate corner points are projected onto the second three-dimensional plane equation to obtain the target road sign angle coordinates.

8. A road sign generation device based on spatial clustering, characterized in that: The device comprises: An acquisition unit, configured to acquire a plurality of consecutive road image frames and generate feature points of road signs in the plurality of road image frames in the same camera coordinate system; A clustering unit, configured to cluster the feature points based on a preset clustering algorithm, select clustering results that meet preset conditions, perform three-dimensional plane fitting, and obtain a first three-dimensional plane equation; A projection unit, configured to project pixel points of the road sign corners onto the first three-dimensional plane equation to obtain three-dimensional coordinate points of the road sign corners in each frame of the road image; A fitting unit, configured to perform three-dimensional plane fitting on the three-dimensional coordinate points of the road sign corner points to obtain a second three-dimensional plane equation; A solving unit is used to average the three-dimensional coordinate points of the road sign corner points based on a preset road sign corner point sequence to obtain the three-dimensional coordinate points of the candidate corner points, and project the three-dimensional coordinates of the candidate corner points onto the second three-dimensional plane equation to obtain the coordinates of the target road sign corner points.

9. An electronic device, characterized in that: include: processor; as well as A memory having executable codes stored thereon, which, when executed by the processor, causes the processor to perform the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that Executable codes are stored thereon, and when the executable codes are executed by a processor of an electronic device, the processor is caused to execute the method according to any one of claims 1 to 7.

Images