Traffic sign board extraction method and system, electronic equipment and storage medium
By automatically extracting traffic signs in high-precision maps, using RGB image data and laser point cloud data, combined with target detection network and instance segmentation method, the problem of low efficiency in traffic sign extraction in the existing technology is solved, and high precision is achieved Quick production of maps.
Patent Information
- Application Number
- CN202410918398.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-10
- Publication Date
- 2024-11-26
AI Technical Summary
The extraction of traffic signs in existing high-precision maps requires a lot of labor and time costs, resulting in low efficiency in making high-precision maps.
By obtaining RGB image data and laser point cloud data in road data, using pre-trained target detection network and instance segmentation method, the geometric shapes and key point coordinates of traffic signs are extracted to realize the automatic extraction of traffic signs.
It reduces labor and time cost consumption in high-precision map production, improves production efficiency, and quickly locates traffic signs through object detection algorithms, reducing calculation and time consumption.
Smart Images

Figure CN119027909A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of high-precision map production, and more specifically, to a traffic sign extraction method, system, electronic equipment and storage medium. Background Art
[0002] A high-precision map (HD map) is a high-precision map used for autonomous driving, which contains map elements such as road shapes, road markings, traffic signs, and obstacles. The map accuracy can be down to the centimeter level.
[0003] In autonomous driving, the driving trajectory is predicted by visual perception. Due to uncontrollable factors such as weather and environment, the perception results may have errors, which may lead to traffic accidents. High-precision maps can effectively assist autonomous driving, obtain road conditions in advance, and make up for the shortcomings of visual perception. In the production of high-precision map data, traffic signs can guide vehicle driving planning, provide road direction information, and remind vehicles, etc., which is an indispensable part. However, the extraction of traffic signs in existing high-precision maps requires a lot of labor costs and time costs. Therefore, how to improve the efficiency of high-precision map production is an urgent problem to be solved. Summary of the invention
[0004] In view of the technical problems existing in the prior art, the present invention provides a traffic sign extraction method, system, electronic device and storage medium, which can realize the automatic extraction of traffic signs in high-precision maps, reduce the consumption of labor costs and time costs, and thus solve the problem of how to improve the production efficiency of high-precision maps.
[0005] A first aspect of the present invention provides a method for extracting traffic signs, comprising:
[0006] Acquire road data of a target road section, wherein the road data includes laser point cloud data, RGB image data, and trajectory data;
[0007] Extracting a first traffic sign of the target road section from the RGB image data;
[0008] According to the trajectory data, acquiring the three-dimensional point cloud data closest to the first traffic sign from the laser point cloud data;
[0009] Performing forward projection on the three-dimensional point cloud data to obtain a corresponding two-dimensional point cloud projection image;
[0010] Segmenting the two-dimensional point cloud projection image using an instance segmentation method to obtain a second traffic sign;
[0011] According to the geometric shape of the second traffic sign, obtaining the coordinates of key points thereof;
[0012] Calculating the three-dimensional traffic sign coordinates of the second traffic sign in the three-dimensional point cloud data according to the geometric shape and the key point coordinates;
[0013] All traffic signs on the target road section are traversed to obtain the three-dimensional traffic sign coordinates corresponding to each traffic sign, thereby completing the extraction of traffic signs on the target road section.
[0014] Based on the above technical solution, the present invention can also make the following improvements.
[0015] Preferably, the step of extracting the first traffic sign of the target road section from the RGB image data comprises:
[0016] Using a pre-trained target detection network, extracting a third traffic sign from a target area of each frame of the RGB image data to obtain a plurality of third traffic signs;
[0017] The target similarity algorithm and the Hungarian algorithm are used to associate multiple third traffic signs, and duplicate third traffic signs are deleted to obtain the first traffic sign.
[0018] Preferably, the step of performing forward projection on the three-dimensional point cloud data to obtain its corresponding two-dimensional point cloud projection image includes:
[0019] The three-dimensional point cloud data is segmented into preset sizes, and the segmented three-dimensional point cloud data is forward projected to obtain its corresponding two-dimensional point cloud projection image and a mapping relationship between the three-dimensional point cloud data and the two-dimensional point cloud projection image.
[0020] Preferably, the step of segmenting the three-dimensional point cloud data into preset sizes and performing forward projection on the segmented three-dimensional point cloud data comprises:
[0021] Taking the up and down directions of the road and the signboard on the right side of the road as the center and the preset size, the three-dimensional point cloud data is segmented for the first time to obtain multiple point cloud blocks;
[0022] Dividing each point cloud block for the second time according to the preset size along the vehicle driving direction in the trajectory data;
[0023] The point cloud blocks obtained by the second segmentation are forward projected along the driving direction of the vehicle to obtain a two-dimensional point cloud projection image.
[0024] Preferably, the step of segmenting the two-dimensional point cloud projection image using an instance segmentation method to obtain a second traffic sign comprises:
[0025] Annotating the two-dimensional point cloud projection image according to rectangles, circles, triangles, rhombuses, pentagons, hexagons and octagons;
[0026] The labeled data is input into the pre-trained instance segmentation network for classification to obtain the second traffic sign.
[0027] Preferably, the step of obtaining the coordinates of key points of the second traffic sign according to its geometric shape comprises:
[0028] The coordinates of each contour point of the second traffic sign are acquired through the geometric shape of the second traffic sign, and the coordinates of other related points are calculated according to the geometric relationship to obtain the coordinates of the key points of the second traffic sign.
[0029] Preferably, the step of calculating the three-dimensional traffic sign coordinates of the second traffic sign in the three-dimensional point cloud data according to the geometric shape and the key point coordinates further includes:
[0030] When there are multiple three-dimensional traffic sign coordinates, the multiple three-dimensional traffic sign coordinates are filtered according to the aspect ratio threshold and the area threshold, and the multiple sign outlines obtained by filtering are merged and deduplicated to obtain the optimal single three-dimensional traffic sign coordinate.
[0031] A second aspect of the present invention provides a traffic sign extraction system, comprising:
[0032] A data acquisition module is used to acquire road data of a target road section, wherein the road data includes laser point cloud data, RGB image data and trajectory data;
[0033] An image extraction module, used for extracting the first traffic sign of the target road section from the RGB image data;
[0034] A point cloud extraction module, used for acquiring three-dimensional point cloud data closest to the first traffic sign from the laser point cloud data according to the trajectory data;
[0035] A point cloud projection module is used to perform forward projection on the three-dimensional point cloud data to obtain a corresponding two-dimensional point cloud projection image;
[0036] A projection segmentation module, used for segmenting the two-dimensional point cloud projection image by using an instance segmentation method to obtain a second traffic sign;
[0037] A coordinate acquisition module, used for acquiring the coordinates of key points of the second traffic sign according to the geometric shape of the second traffic sign;
[0038] A coordinate inverse calculation module, used for calculating the three-dimensional traffic sign coordinates of the second traffic sign in the three-dimensional point cloud data according to the geometric shape and the key point coordinates;
[0039] The loop extraction module is used to traverse all traffic signs on the target road section, obtain the three-dimensional traffic sign coordinates corresponding to each traffic sign, and complete the extraction of traffic signs on the target road section.
[0040] A third aspect of the present invention provides an electronic device, comprising a memory and a processor, wherein the processor is used to implement the steps of any traffic sign extraction method in the first aspect when executing a computer management program stored in the memory.
[0041] According to a fourth aspect of the present invention, a computer-readable storage medium is provided, on which a computer management program is stored. When the computer management program is executed by a processor, the steps of any traffic sign extraction method in the first aspect are implemented.
[0042] The present invention provides a traffic sign extraction method, system, electronic device and storage medium. The method obtains a first traffic sign of a target road section from RGB image data, obtains three-dimensional point cloud data closest to the first traffic sign from laser point cloud data, performs forward projection on the three-dimensional point cloud data to obtain a two-dimensional point cloud projection map, and then segments the two-dimensional point cloud projection map to obtain a second traffic sign. The key point coordinates of the second traffic sign are calculated according to the geometric shape of the second traffic sign, and the three-dimensional traffic sign coordinates corresponding to the second traffic sign are obtained by reverse calculation according to the geometric shape and key point coordinates, so as to realize the three-dimensional traffic sign extraction of the target road section; the automatic extraction of traffic sign elements of the target road section is realized, the consumption of labor and time costs in high-precision map production is reduced, and the efficiency of high-precision map production is improved; and the traffic signs are screened in the collected RGB image data through a target detection algorithm, so that the point cloud area containing the traffic signs in the three-dimensional point cloud can be quickly located, the amount of calculation and time consumption are effectively reduced, and the automatic extraction efficiency and map production speed are improved. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] Figure 1 A flow chart of a traffic sign extraction method provided by the present invention;
[0044] Figure 2 A detailed flow chart of the data of the traffic sign extraction method provided by the present invention;
[0045] Figure 3 A schematic diagram of the structure of a traffic sign extraction system provided by the present invention;
[0046] Figure 4 A schematic diagram of the hardware structure of a possible electronic device provided by the present invention;
[0047] Figure 5A schematic diagram of the hardware structure of a possible computer-readable storage medium provided by the present invention. DETAILED DESCRIPTION
[0048] The specific implementation of the present invention is further described in detail below in conjunction with the accompanying drawings and examples. The following examples are used to illustrate the present invention, but are not intended to limit the scope of the present invention.
[0049] Figure 1 A flow chart of a traffic sign extraction method provided by the present invention, such as Figure 1 As shown, the method includes:
[0050] Step S100: Acquire road data of a target road section, wherein the road data includes laser point cloud data, RGB image data and trajectory data;
[0051] It should be noted that the execution subject of the method of this embodiment can be a computer terminal device with data processing, network communication and program running functions, such as a computer, a tablet computer, etc.; it can also be a server device with the same or similar functions, or a cloud server with similar functions, and this embodiment does not limit this. For ease of understanding, this embodiment and the following embodiments will be described using a server device as an example.
[0052] It is understandable that the above road data can be collected from the target road section in real time by a road data collection vehicle, or obtained from a historical database or a real-time database, or obtained in real time through a third-party plug-in, and this embodiment does not impose any restrictions on this.
[0053] It should be understood that the above-mentioned trajectory data may be the driving trajectory data of the vehicle.
[0054] Step S200: extracting the first traffic sign of the target road section from the RGB image data;
[0055] Furthermore, the step of extracting the first traffic sign includes:
[0056] Step S201: using a pre-trained target detection network, extracting a third traffic sign from a target area of each frame of the RGB image data to obtain a plurality of third traffic signs;
[0057] It is understandable that the above target detection network may be trained in advance, and it may be trained by a YOLO system algorithm based on deep learning.
[0058] Specifically, since multiple frames may be collected for the same area in RGB image data, multiple traffic signs may be obtained when the first traffic sign is extracted. In this embodiment, the multiple traffic signs need to be filtered to obtain a unique first traffic sign.
[0059] In this example, the traffic signs are screened in the collected RGB images through the target detection algorithm, which can quickly locate the point cloud area containing the traffic signs in the 3D point cloud, effectively reduce the amount of calculation and time consumption, and improve the efficiency of automated extraction and the speed of producing high-precision maps.
[0060] Step S202: using the target similarity algorithm and the Hungarian algorithm to associate multiple third traffic signs, deleting duplicate third traffic signs, and obtaining a first traffic sign.
[0061] Specifically, a group of continuous frame images containing the first traffic sign is obtained, and for the same target in the continuous frame images, target similarity (SSIM or cosine similarity) is used for target comparison and the Hungarian algorithm is used for target association, and images containing duplicate targets are deleted, so as to obtain a uniquely determined first traffic sign.
[0062] Step S300: acquiring three-dimensional point cloud data closest to the first traffic sign from the laser point cloud data according to the trajectory data;
[0063] Specifically, according to the driving direction in the vehicle driving trajectory data and the mapping relationship between the laser point cloud data and the RGB image, the nearest trajectory point containing the first traffic sign in the point cloud data is obtained, that is, the corresponding three-dimensional point cloud data.
[0064] Step S400: performing forward projection on the three-dimensional point cloud data to obtain a corresponding two-dimensional point cloud projection image;
[0065] Specifically, the three-dimensional point cloud data is segmented according to a fixed size, and the point cloud blocks obtained by segmentation are two-dimensionally projected, and the mapping relationship between the three-dimensional point cloud and the two-dimensional projection is saved.
[0066] Furthermore, the step of obtaining the two-dimensional point cloud projection image includes:
[0067] Step S401: segmenting the three-dimensional point cloud data into preset sizes, and forward-projecting the segmented three-dimensional point cloud data to obtain its corresponding two-dimensional point cloud projection image and a mapping relationship between the three-dimensional point cloud data and the two-dimensional point cloud projection image.
[0068] Furthermore, the above step of segmenting the three-dimensional point cloud data and performing forward projection on the segmented three-dimensional point cloud data to obtain a two-dimensional point cloud projection image includes:
[0069] Step S4011: taking the up and down directions of the road and the road right side sign as the center and the preset size, performing a first segmentation on the three-dimensional point cloud data to obtain a plurality of point cloud blocks;
[0070] Step S4012: dividing each point cloud block for the second time according to the preset size along the vehicle driving direction in the trajectory data;
[0071] Step S4013: forward-project the point cloud blocks obtained by the second segmentation along the driving direction of the vehicle to obtain a two-dimensional point cloud projection image.
[0072] Specifically, first, the three-dimensional point cloud data is divided for the first time with a fixed size along the up and down directions of the road with the signboard on the right side of the road as the center. Then, based on a single point cloud block, the direction of vehicle travel is selected for collection, and the point cloud block is divided for the second time with a fixed size along the trajectory points. In the same direction, the divided point cloud blocks are projected into a two-dimensional point cloud projection map through a forward view along the trajectory points. At the same time, the mapping relationship between the point cloud trajectory coordinates and the two-dimensional point cloud projection map coordinates is saved for subsequent coordinate inverse calculation.
[0073] Step S500: segmenting the two-dimensional point cloud projection image using an instance segmentation method to obtain a second traffic sign;
[0074] Specifically, on the two-dimensional point cloud projection image, instance segmentation is used to predict the traffic signs in the projection image according to the geometric shapes of various types of signs, and the mask images corresponding to the traffic signs and the coordinates of the key points corresponding to the geometric shapes are obtained.
[0075] In this embodiment, there is overlap and aggregation of traffic signs at intersections on urban roads. The instance segmentation algorithm can effectively solve this adhesion problem and improve the accuracy of making high-precision maps.
[0076] Furthermore, the step of obtaining the second traffic sign comprises:
[0077] Step S501: annotating the two-dimensional point cloud projection image according to rectangles, circles, triangles, rhombuses, pentagons, hexagons and octagons;
[0078] Step S502: input the labeled data into a pre-trained instance segmentation network for classification to obtain a second traffic sign.
[0079] It can be understood that the training method of the above instance segmentation network includes: annotating the traffic signs in the two-dimensional point cloud projection image according to geometric shapes such as rectangles, circles, triangles, diamonds, pentagons, hexagons and octagons. The annotated data is used as the input of the instance segmentation network, and after sufficient training, an image instance segmentation model is obtained.
[0080] In this embodiment, the instance segmentation method is used to perform instance segmentation and extraction on the point cloud projection image, thereby solving the problem of overlapping signboards in the two-dimensional projection image and severe adhesion in semantic segmentation.
[0081] Furthermore, the above-mentioned step of obtaining the coordinates of specific key points based on the geometric characteristics of the traffic sign specifically includes: according to the result of the sign classification by the model, searching for the coordinate points where the contour points are concentrated through the geometric shape of the sign, and at the same time judging the directions of the triangular sign and the pentagonal sign, and calculating the coordinates of other related points based on the geometric relationship to obtain the coordinates of all the key points of the sign.
[0082] Furthermore, since the two-dimensional projection of the wheel is similar to a circular sign, there are a large number of false detections, which are filtered by calculating the distance between the detection sign and the ground. The specific implementation method is: the ground in the point cloud projection image is a certain distance away from the lower edge of the image, so it is necessary to find the coordinates of the ground and then determine the height of the sign from the ground. Since there are holes in the point cloud projection image, the image width is used as the horizontal coordinate and the image height is used as the vertical coordinate. A point is taken every 200 pixels at the lower edge of the image to eliminate the influence of the hole. Take these points as the starting coordinates and continue to traverse upward pixel by pixel until there are pixels for 10 consecutive traversals. It is considered that this is the ground. For all the search results, the highest and lowest values are removed and the average is taken to obtain the ground coordinates. Calculate the height of the lowest point in the sign from the ground, set the height threshold to 150 pixels, filter out targets with a height less than 150 pixels, and thus filter out wheels and other ground false detection targets. For other false detection targets such as elevated bridge projections and truck projections, filter them by the area and aspect ratio thresholds of the sign.
[0083] Step S600: acquiring the coordinates of key points of the second traffic sign according to its geometric shape;
[0084] Furthermore, the step of obtaining the key point coordinates includes:
[0085] Step S601: Obtain the coordinates of each contour point of the second traffic sign through the geometric shape of the second traffic sign, and calculate the coordinates of other related points according to the geometric relationship to obtain the coordinates of the key points of the second traffic sign.
[0086] Specifically, based on the mask image and key point coordinates obtained by segmenting the signboard using the above-mentioned example segmentation model, the three-dimensional coordinates of the signboard in the laser point cloud are obtained by inverse calculation of the two-dimensional coordinates.
[0087] Step S700: calculating the three-dimensional traffic sign coordinates of the second traffic sign in the three-dimensional point cloud data according to the geometric shape and the key point coordinates;
[0088] Furthermore, the step of obtaining the coordinates of the three-dimensional traffic sign also includes:
[0089] Step S701: When there are multiple three-dimensional traffic sign coordinates, the multiple three-dimensional traffic sign coordinates are filtered according to the aspect ratio threshold and the area threshold, and the multiple sign outlines obtained by filtering are merged and deduplicated to obtain the optimal single three-dimensional traffic sign coordinate.
[0090] Specifically, the false detection targets are filtered out by using the aspect ratio and area thresholds, and the three-dimensional coordinates are obtained by inverse calculation of the two-dimensional coordinates of the sign. At the same time, the contours of multiple signboards after three-dimensional inverse calculation are merged and deduplicated to obtain the three-dimensional coordinates of the best single signboard contour.
[0091] Step S800: traverse all traffic signs on the target road section, obtain the three-dimensional traffic sign coordinates corresponding to each traffic sign, and complete the extraction of traffic signs on the target road section.
[0092] In this embodiment, the three-dimensional coordinates of a single traffic sign are extracted, and all traffic signs of the target road section are extracted cyclically. The detailed extraction process is shown in FIG. Figure 2 , thereby realizing the extraction of all traffic signs on the target road section; at the same time, in order to improve the extraction efficiency of traffic signs, they can also be extracted synchronously and in parallel, which is not limited in this embodiment.
[0093] In this embodiment, deep learning is combined with traditional algorithms, so that the method of this embodiment has good generalization ability and is applicable to various traffic signs on urban roads. The error of the extracted traffic signs is within 5 cm, ensuring the accuracy of making high-precision maps.
[0094] It can be understood that based on the defects in the background technology, the embodiment of the present invention proposes a method for extracting traffic signs. The present invention obtains the first traffic sign of the target road section from the RGB image data, obtains the three-dimensional point cloud data closest to the first traffic sign from the laser point cloud data, and performs forward projection on the three-dimensional point cloud data to obtain a two-dimensional point cloud projection map, and then segments the two-dimensional point cloud projection map to obtain a second traffic sign, and calculates its key point coordinates according to the geometric shape of the second traffic sign, and inversely calculates the three-dimensional traffic sign coordinates corresponding to the second traffic sign according to its geometric shape and key point coordinates, so as to realize the three-dimensional traffic sign extraction of the target road section; realizes the automatic extraction of the traffic sign elements of the target road section, reduces the consumption of labor and time costs in the production of high-precision maps, improves the efficiency of making high-precision maps, and screens traffic signs in the collected RGB image data through the target detection algorithm, can quickly locate the point cloud area containing traffic signs in the three-dimensional point cloud, effectively reduces the amount of calculation and time consumption, and improves the efficiency of automatic extraction and the speed of map production.
[0095] See also Figure 3 , Figure 3 A schematic diagram of a traffic sign extraction system provided by an embodiment of the present invention is shown in FIG. Figure 3 As shown, a traffic sign extraction system includes a data acquisition module 100, an image extraction module 200, a point cloud extraction module 300, a point cloud projection module 400, a projection segmentation module 500, a coordinate acquisition module 600, a coordinate inverse calculation module 700 and a cycle extraction module 800, wherein:
[0096] The data acquisition module 100 is used to acquire the road data of the target road section, wherein the road data includes laser point cloud data, RGB image data and trajectory data; the image extraction module 200 is used to extract the first traffic sign of the target road section from the RGB image data; the point cloud extraction module 300 is used to acquire the three-dimensional point cloud data closest to the first traffic sign from the laser point cloud data according to the trajectory data; the point cloud projection module 400 is used to perform forward projection on the three-dimensional point cloud data to obtain its corresponding two-dimensional point cloud projection map; the projection segmentation module 500 is used to The two-dimensional point cloud projection image is segmented by an instance segmentation method to obtain a second traffic sign; a coordinate acquisition module 600 is used to obtain the key point coordinates of the second traffic sign according to the geometric shape; a coordinate inversion module 700 is used to calculate the three-dimensional traffic sign coordinates of the second traffic sign in the three-dimensional point cloud data according to the geometric shape and the key point coordinates; a loop extraction module 800 is used to traverse all traffic signs on the target road section, obtain the three-dimensional traffic sign coordinates corresponding to each traffic sign, and complete the traffic sign extraction of the target road section.
[0097] It can be understood that the traffic sign extraction system provided by the present invention corresponds to the traffic sign extraction method provided by the aforementioned embodiments. The relevant technical features of the traffic sign extraction system can refer to the relevant technical features of the traffic sign extraction method, which will not be repeated here.
[0098] See also Figure 4 , Figure 4 Schematic diagram of an electronic device provided by an embodiment of the present invention. Figure 4 As shown, an embodiment of the present invention provides an electronic device, including a memory 1310, a processor 1320, and a computer program 1311 stored in the memory 1310 and executable on the processor 1320. When the processor 1320 executes the computer program 1311, the following steps are implemented:
[0099] Acquire road data of a target road section, the road data comprising laser point cloud data, RGB image data and trajectory data; extract a first traffic sign of the target road section from the RGB image data; acquire three-dimensional point cloud data closest to the first traffic sign from the laser point cloud data according to the trajectory data; perform forward projection on the three-dimensional point cloud data to obtain a corresponding two-dimensional point cloud projection map; segment the two-dimensional point cloud projection map using an instance segmentation method to obtain a second traffic sign; acquire key point coordinates of the second traffic sign according to its geometric shape; calculate the three-dimensional traffic sign coordinates of the second traffic sign in the three-dimensional point cloud data according to the geometric shape and the key point coordinates; traverse all traffic signs of the target road section to obtain the three-dimensional traffic sign coordinates corresponding to each traffic sign, and complete the extraction of traffic signs of the target road section.
[0100] See also Figure 5 , Figure 5 Schematic diagram of an embodiment of a computer-readable storage medium provided by the present invention. Figure 5 As shown, this embodiment provides a computer-readable storage medium 1400, on which a computer program 1411 is stored. When the computer program 1411 is executed by a processor, the following steps are implemented:
[0101] Acquire road data of a target road section, the road data comprising laser point cloud data, RGB image data and trajectory data; extract a first traffic sign of the target road section from the RGB image data; acquire three-dimensional point cloud data closest to the first traffic sign from the laser point cloud data according to the trajectory data; perform forward projection on the three-dimensional point cloud data to obtain a corresponding two-dimensional point cloud projection map; segment the two-dimensional point cloud projection map using an instance segmentation method to obtain a second traffic sign; acquire key point coordinates of the second traffic sign according to its geometric shape; calculate the three-dimensional traffic sign coordinates of the second traffic sign in the three-dimensional point cloud data according to the geometric shape and the key point coordinates; traverse all traffic signs of the target road section to obtain the three-dimensional traffic sign coordinates corresponding to each traffic sign, and complete the extraction of traffic signs of the target road section.
[0102] A traffic sign extraction method, system, electronic device and storage medium provided by an embodiment of the present invention include: obtaining road data of a target road section, the road data including laser point cloud data, RGB image data and trajectory data; extracting a first traffic sign of the target road section from the RGB image data; obtaining three-dimensional point cloud data closest to the first traffic sign from the laser point cloud data according to the trajectory data; performing forward projection on the three-dimensional point cloud data to obtain a two-dimensional point cloud projection map corresponding thereto; segmenting the two-dimensional point cloud projection map using an instance segmentation method to obtain a second traffic sign; obtaining key point coordinates of the second traffic sign according to its geometric shape; calculating the three-dimensional traffic sign coordinates of the second traffic sign in the three-dimensional point cloud data according to the geometric shape and the key point coordinates; traversing all traffic signs of the target road section to obtain the three-dimensional traffic sign coordinates corresponding to each traffic sign, and completing the traffic sign extraction of the target road section. The present invention obtains the first traffic sign of the target road section from RGB image data, obtains the three-dimensional point cloud data closest to the first traffic sign from the laser point cloud data, and performs forward projection on the three-dimensional point cloud data to obtain a two-dimensional point cloud projection map, and then segments the two-dimensional point cloud projection map to obtain a second traffic sign, and calculates its key point coordinates according to the geometric shape of the second traffic sign, and reversely calculates the three-dimensional traffic sign coordinates corresponding to the second traffic sign according to its geometric shape and key point coordinates, thereby realizing the three-dimensional traffic sign extraction of the target road section; realizing the automatic extraction of traffic sign elements of the target road section, reducing the consumption of labor and time costs in high-precision map production, and improving the efficiency of high-precision map production, and screening traffic signs in the collected RGB image data through a target detection algorithm, can quickly locate the point cloud area containing traffic signs in the three-dimensional point cloud, effectively reduce the amount of calculation and time consumption, and improve the automatic extraction efficiency and map production speed.
[0103] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.
[0104] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0105] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0106] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.
[0107] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0108] Although the preferred embodiments of the present invention have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.
[0109] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.
Claims
1. A method for extracting traffic signs, characterized in that: The method comprises: Acquire road data of a target road section, wherein the road data includes laser point cloud data, RGB image data, and trajectory data; Extracting a first traffic sign of the target road section from the RGB image data; According to the trajectory data, acquiring the three-dimensional point cloud data closest to the first traffic sign from the laser point cloud data; Performing forward projection on the three-dimensional point cloud data to obtain a corresponding two-dimensional point cloud projection image; Segmenting the two-dimensional point cloud projection image using an instance segmentation method to obtain a second traffic sign; According to the geometric shape of the second traffic sign, obtaining the coordinates of key points thereof; Calculating the three-dimensional traffic sign coordinates of the second traffic sign in the three-dimensional point cloud data according to the geometric shape and the key point coordinates; All traffic signs on the target road section are traversed to obtain the three-dimensional traffic sign coordinates corresponding to each traffic sign, thereby completing the extraction of traffic signs on the target road section.
2. The method for extracting traffic signs according to claim 1, characterized in that: The step of extracting the first traffic sign of the target road section from the RGB image data comprises: Using a pre-trained target detection network, extracting a third traffic sign from a target area of each frame of the RGB image data to obtain a plurality of third traffic signs; The target similarity algorithm and the Hungarian algorithm are used to associate multiple third traffic signs, and duplicate third traffic signs are deleted to obtain the first traffic sign.
3. The method for extracting traffic signs according to claim 1, characterized in that: The step of performing forward projection on the three-dimensional point cloud data to obtain a corresponding two-dimensional point cloud projection image includes: The three-dimensional point cloud data is segmented into preset sizes, and the segmented three-dimensional point cloud data is forward projected to obtain its corresponding two-dimensional point cloud projection image and a mapping relationship between the three-dimensional point cloud data and the two-dimensional point cloud projection image.
4. The method for extracting traffic signs according to claim 3, characterized in that: The step of dividing the three-dimensional point cloud data into preset sizes and performing forward projection on the divided three-dimensional point cloud data comprises: Taking the up and down directions of the road and the signboard on the right side of the road as the center and the preset size, the three-dimensional point cloud data is segmented for the first time to obtain multiple point cloud blocks; Dividing each point cloud block for the second time according to the preset size along the vehicle driving direction in the trajectory data; The point cloud blocks obtained by the second segmentation are forward projected along the driving direction of the vehicle to obtain a two-dimensional point cloud projection image.
5. The method for extracting traffic signs according to claim 1, characterized in that: The step of segmenting the two-dimensional point cloud projection image using an instance segmentation method to obtain a second traffic sign comprises: Annotating the two-dimensional point cloud projection image according to rectangles, circles, triangles, rhombuses, pentagons, hexagons and octagons; The labeled data is input into the pre-trained instance segmentation network for classification to obtain the second traffic sign.
6. The method for extracting traffic signs according to claim 5, characterized in that: The step of obtaining the coordinates of key points of the second traffic sign according to its geometric shape comprises: The coordinates of each contour point of the second traffic sign are obtained through the geometric shape of the second traffic sign, and the coordinates of other related points are calculated according to the geometric relationship to obtain the coordinates of the key points of the second traffic sign.
7. The method for extracting traffic signs according to claim 1, characterized in that: The step of calculating the three-dimensional traffic sign coordinates of the second traffic sign in the three-dimensional point cloud data according to the geometric shape and the key point coordinates also includes: When there are multiple three-dimensional traffic sign coordinates, the multiple three-dimensional traffic sign coordinates are filtered according to the aspect ratio threshold and the area threshold, and the multiple sign outlines obtained by filtering are merged and deduplicated to obtain the optimal single three-dimensional traffic sign coordinate.
8. A traffic sign extraction system, characterized in that: include: A data acquisition module is used to acquire road data of a target road section, wherein the road data includes laser point cloud data, RGB image data and trajectory data; An image extraction module, used for extracting the first traffic sign of the target road section from the RGB image data; A point cloud extraction module, used for acquiring three-dimensional point cloud data closest to the first traffic sign from the laser point cloud data according to the trajectory data; A point cloud projection module is used to perform forward projection on the three-dimensional point cloud data to obtain a corresponding two-dimensional point cloud projection image; A projection segmentation module, used for segmenting the two-dimensional point cloud projection image by using an instance segmentation method to obtain a second traffic sign; A coordinate acquisition module, used for acquiring the coordinates of key points of the second traffic sign according to the geometric shape of the second traffic sign; A coordinate inverse calculation module, used for calculating the three-dimensional traffic sign coordinates of the second traffic sign in the three-dimensional point cloud data according to the geometric shape and the key point coordinates; The loop extraction module is used to traverse all traffic signs on the target road section, obtain the three-dimensional traffic sign coordinates corresponding to each traffic sign, and complete the extraction of traffic signs on the target road section.
9. An electronic device, characterized in that: It comprises a memory and a processor, wherein the processor is used to implement the steps of the traffic sign extraction method as described in any one of claims 1 to 7 when executing a computer management program stored in the memory.
10. A computer-readable storage medium, characterized in that: A computer management program is stored thereon, and when the computer management program is executed by a processor, the steps of the traffic sign extraction method as described in any one of claims 1 to 7 are implemented.
Citation Information
Patent Citations
High-precision direction signboard target extraction method based on point cloud
CN111695486A
Rapid segmentation method and system for traffic signs
CN113887492A
Traffic sign board extraction method and system, electronic equipment and storage medium
CN114067288A
Method for automatically extracting digital limit information of signboard based on high-precision map acquisition
CN114463754A
Three-dimensional image entity segmentation method and device, equipment, storage medium and vehicle
CN115760878A
