Automatic Extraction Method and System for Dashed Lane Lines in Road Traffic Maps Collected by UAVs

Abstract

The present invention provides an automatic extraction method and system for dashed lane lines in a road traffic map collected by a drone. The method includes: obtaining three-dimensional point cloud data and RGB two-dimensional images of a traffic road collected by the drone; extracting lane information from consecutive two-dimensional images; mapping the road trajectory information extracted from the two-dimensional images into the three-dimensional point cloud data to obtain a virtual traffic road trajectory; cutting the three-dimensional point cloud map in the form of a sliding window to obtain three-dimensional point cloud map blocks; performing two-dimensional mapping on each three-dimensional point cloud map block, extracting the dashed lane line segments in each two-dimensional mapping map block, and reflecting the dashed lane line segments back into the three-dimensional point cloud map to obtain the three-dimensional coordinate information of the dashed lane line segments. The present invention can realize the automatic extraction of dashed lane lines in three-dimensional point cloud data, reduce the manual operation of dashed lane lines in map operations, and improve the map production efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of lane line extraction, and more specifically, to an automatic extraction method and system for dotted lane lines in a road traffic map collected by an unmanned aerial vehicle (UAV). Background Art

[0002] High-precision navigation electronic maps play an important auxiliary role in driverless and intelligent driving. High-precision map operations usually require collecting road traffic information before operation. Traditional methods mainly come from mobile measurement of road information by a collection vehicle equipped with a lidar and a camera. However, for road conditions in complex scenarios such as urban intersections, highway entrances and exits, and loop interchanges, traditional collection vehicles need to spend a large amount of cost on repeated collection, resulting in high costs. To reduce costs, UAVs are usually combined with traditional surveying and mapping to collect traffic road information through UAVs. In the traffic road operations collected by UAVs, it is necessary to manually operate on each small block of the dotted lane lines on the road, which is extremely time-consuming and laborious. Summary of the Invention

[0003] The present invention provides an automatic extraction method and system for dotted lane lines in a road traffic map collected by an unmanned aerial vehicle (UAV) in view of the technical problems existing in the prior art.

[0004] According to a first aspect of the present invention, there is provided an automatic extraction method for dotted lane lines in a road traffic map collected by an unmanned aerial vehicle (UAV), including:

[0005] Obtaining three-dimensional point cloud data and RGB two-dimensional images of a traffic road collected by an unmanned aerial vehicle (UAV);

[0006] Extracting lane information from consecutive RGB two-dimensional images, where the lane information includes lane width information, lane contour information, and road trajectory information;

[0007] Coarsely registering the three-dimensional point cloud data and the RGB two-dimensional images, mapping the road trajectory information extracted from the RGB two-dimensional images into the three-dimensional point cloud data to obtain a virtual traffic road trajectory, removing the three-dimensional point cloud data far from the virtual traffic road trajectory, and obtaining a local three-dimensional point cloud map after removal;

[0008] Cutting the local three-dimensional point cloud map in a sliding window manner with the key points of the virtual traffic road trajectory as the center to obtain three-dimensional point cloud patches;

[0009] Performing two-dimensional mapping on each three-dimensional point cloud patch to obtain a two-dimensional mapping patch, extracting the dotted lane line segments in each two-dimensional mapping patch, reflecting the dotted lane line segments into the three-dimensional point cloud map, obtaining the three-dimensional coordinate information of the dotted lane line segments, and connecting each dotted lane line segment to complete the extraction of the dotted lane lines in the three-dimensional point cloud data.

[0010] Based on the above technical solutions, the present invention can also be improved as follows.

[0011] Optionally, extracting lane information from continuous RGB two-dimensional images, where the lane information includes lane width information, lane contour information, and road trajectory information, includes:

[0012] Performing lane semantic segmentation on the traffic road in the continuous RGB two-dimensional image based on the transformer semantic segmentation neural network to obtain lane information in the RGB two-dimensional image, where the lane information includes lane width information, lane contour information, and similar key point information in the continuous RGB two-dimensional image;

[0013] Filtering and connecting similar key points in the continuous RGB two-dimensional image to form road trajectory points of the entire road traffic and saving the road width information.

[0014] Optionally, performing rough registration on the three-dimensional point cloud data and the RGB two-dimensional image, including:

[0015] Based on the three-dimensional point cloud data and the RGB two-dimensional image, obtaining the parameters for image and point cloud conversion through the calibration plate plane in different poses , as follows:

[0016] ;

[0017] Image coordinate system is represented as, three-dimensional point cloud is represented as, are the focal lengths in the horizontal and vertical directions, is the center point of the image plane, is the rotation matrix, is the translation vector.

[0018] Optionally, centering on the key points of the virtual traffic road trajectory and cutting the local three-dimensional point cloud map in the form of a sliding window to obtain three-dimensional point cloud patches, including:

[0019] Setting the sliding window size, centering on the key points of the virtual traffic road trajectory, and cutting the local three-dimensional point cloud map on the virtual traffic road trajectory from a top-down perspective within a fixed size range of the sliding window size to obtain three-dimensional point cloud patches.

[0020] Optionally, performing two-dimensional mapping on each three-dimensional point cloud patch to obtain a two-dimensional mapped patch and extracting the dashed lane segments in each two-dimensional mapped patch, including:

[0021] Performing two-dimensional mapping on the three-dimensional point cloud patch from a vertical angle to obtain a two-dimensional mapped patch corresponding to each three-dimensional point cloud patch;

[0022] The Transformer-based semantic segmentation model segments the traffic road dashed lane line segments in each two-dimensional mapped tile to obtain the contour information of the dashed lane line segments.

[0023] Optionally, after mapping each three-dimensional point cloud tile into a two-dimensional mapped tile, extracting the dashed lane line segments in each two-dimensional mapped tile, the following steps are further included:

[0024] Based on the obtained contour information of the dashed lane line segments, after extracting the skeleton line of the binary image, the center line of the dashed lane line segments is obtained through smoothing and line fitting.

[0025] The width of each dashed lane line segment is obtained by matching the average distance between the contour of the dashed lane line segment and the center line of the dashed lane line segment with the standard road dashed lane line width value.

[0026] The dashed lane line segments that are too long, too short, or too thick are filtered out, and according to the characteristics of the road dashed lane line segments, the dashed lane line segments that are too far or too close to the average lane line or have an inconsistent aspect ratio are filtered out to obtain the filtered dashed lane line segments.

[0027] Optionally, mapping the dashed lane line segments back to the three-dimensional point cloud map to obtain the three-dimensional coordinate information of the dashed lane line segments, including:

[0028] Mapping the coordinates and width information of the dashed lane line segments in the extracted two-dimensional mapped tile back to the three-dimensional point cloud map to obtain the x and y axis information of the dashed lane line segments in the three-dimensional point cloud map.

[0029] The z axis information of the dashed lane line segments is obtained through the ground coordinate information to obtain the three-dimensional coordinate information and width information of the dashed lane line segments.

[0030] Optionally, connecting each dashed lane line segment to complete the extraction of the dashed lane lines in the three-dimensional point cloud data, including:

[0031] According to the principle of finding nearby in a local large area, the dashed lane line segments that are close are connected, the dashed lane line segments that are far apart are not connected, and the locally overlapping dashed lane line segments are removed to obtain the position information of the entire dashed lane line.

[0032] Based on the center line and width information of the dashed lane line, four corner point information is obtained. Based on the four corner point information, the side line points on the left and right of the dashed lane line are connected to form a long dashed lane line, and the extraction of the dashed lane lines in the three-dimensional point cloud data is completed.

[0033] According to a second aspect of the present invention, there is provided an automatic extraction system for dashed lane lines in a road traffic map collected by a drone, including:

[0034] An acquisition module, configured to acquire three-dimensional point cloud data and RGB two-dimensional images of a traffic road collected by the drone;

[0035] A first extraction module, configured to extract lane information from consecutive RGB two-dimensional images, where the lane information includes lane width information, lane contour information, and road trajectory information;

[0036] A mapping module, configured to perform rough registration on the three-dimensional point cloud data and the RGB two-dimensional images, map the road trajectory information extracted from the RGB two-dimensional images into the three-dimensional point cloud data to obtain a virtual traffic road trajectory, remove the three-dimensional point cloud data far from the virtual traffic road trajectory, and obtain a local three-dimensional point cloud map after removal;

[0037] A cutting module, configured to cut the local three-dimensional point cloud map in a sliding window manner with key points of the virtual traffic road trajectory as the center to obtain three-dimensional point cloud map blocks;

[0038] A second extraction module, configured to perform two-dimensional mapping on each three-dimensional point cloud map block to obtain a two-dimensional mapped map block, and extract dashed lane line segments in each two-dimensional mapped map block;

[0039] A connection module, configured to reflect the dashed lane line segments back into the three-dimensional point cloud map to obtain three-dimensional coordinate information of the dashed lane line segments, and connect each dashed lane line segment to complete the extraction of the dashed lane lines in the three-dimensional point cloud data.

[0040] According to a third aspect of the present invention, there is provided an electronic device, including a memory and a processor, where the processor is configured to implement the steps of the method for automatically extracting dashed lane lines in a road traffic map collected by a drone when executing a computer management program stored in the memory.

[0041] According to a fourth aspect of the present invention, there is provided a computer-readable storage medium, on which a computer management program is stored, and the computer management program is configured to implement the steps of the method for automatically extracting dashed lane lines in a road traffic map collected by a drone when executed by a processor.

[0042] An automatic extraction method and system for dashed lane lines in a road traffic map collected by an unmanned aerial vehicle provided by the present invention obtain three-dimensional point cloud data and RGB two-dimensional images of a traffic road collected by the unmanned aerial vehicle; extract lane information from consecutive two-dimensional images; map the road trajectory information extracted from the two-dimensional images to the three-dimensional point cloud data to obtain a virtual traffic road trajectory; cut the three-dimensional point cloud map in the form of a sliding window to obtain three-dimensional point cloud map blocks; perform two-dimensional mapping on each three-dimensional point cloud map block, extract the dashed lane line segments in each two-dimensional mapping map block, and map the dashed lane line segments back to the three-dimensional point cloud map to obtain the three-dimensional coordinate information of the dashed lane line segments. The present invention can realize the automatic extraction of dashed lane lines in three-dimensional point cloud data, reduce the manual operation of dashed lane lines in map operations, and improve the map production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a flowchart of an automatic extraction method for dashed lane lines in a road traffic map collected by an unmanned aerial vehicle provided by the present invention;

[0044] Figure 2 It is a schematic structural diagram of a semantic segmentation network based on a transformer;

[0045] Figure 3 It is a three-dimensional point cloud map of a traffic road collected by an unmanned aerial vehicle;

[0046] Figure 4 It is a local road map after roughly removing the point cloud near non-road traffic;

[0047] Figure 5 It is a diagram after extracting and connecting the dashed lane lines after two-dimensional mapping of the cut point cloud, where (a) is the two-dimensional mapping diagram after cutting, (b) is a schematic diagram of the extracted dashed lane lines, and (c) is a diagram of connecting the front and back of the dashed lane lines;

[0048] Figure 6 It is a schematic diagram of the overall process of an automatic extraction method for dashed lane lines in a road traffic map collected by an unmanned aerial vehicle;

[0049] Figure 7 It is a schematic structural diagram of an automatic extraction system for dashed lane lines in a road traffic map collected by an unmanned aerial vehicle provided by the present invention;

[0050] Figure 8 It is a schematic hardware structure diagram of a possible electronic device provided by the present invention;

[0051] Figure 9 It is a schematic hardware structure diagram of a possible computer-readable storage medium provided by the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0052] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are some, but not all, of the embodiments of the present invention. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present invention without creative efforts shall fall within the protection scope of the present invention. In addition, the technical features in each embodiment or single embodiment provided by the present invention can be combined with each other arbitrarily to form a feasible technical solution. This combination is not restricted by the order of steps and / or the mode of structural composition, but must be based on the ability of those of ordinary skill in the art to implement. When the combination of technical solutions results in contradictions or cannot be implemented, it should be considered that such a combination of technical solutions does not exist and is not within the protection scope required by the present invention.

[0053] The present invention is mainly used for making road traffic road elements collected by drones in a high-precision map production system and is used for automatically extracting dashed lane lines of roads. In the manual operation of high-precision maps, when extracting dashed lane lines, it is necessary to manually extract the four corner points of each small piece of the dashed lane line along the road direction and connect the corner points on both sides into a string, and the error accuracy is within three centimeters. The automatic extraction system for dashed lane lines of roads collected by drones proposed by the present invention reduces manual interaction and improves the efficiency of map operation.

[0054] Figure 1 The following is a flowchart of a method for automatically extracting dashed lane lines in a road traffic map collected by a drone provided by the present invention. As Figure 1 shown, the method includes:

[0055] S1. Obtain the three-dimensional point cloud data and RGB two-dimensional images of the traffic road collected by the drone.

[0056] It can be understood that data information such as the three-dimensional point cloud data and RGB two-dimensional images of road traffic collected by the drone in the map operation is obtained.

[0057] S2. Extract lane information from consecutive RGB two-dimensional images, where the lane information includes lane width information, lane contour information, and road trajectory information.

[0058] It can be understood that the lane trajectory of the three-dimensional point cloud data collected by the virtual drone. Since the road trajectory information cannot be included in the road traffic road information collected by the drone, in order to reduce the processing amount of the point cloud information of buildings and trees around the traffic road, the present invention virtualizes the road trajectory of the point cloud data collected by the drone to eliminate the non-road point cloud data around the road.

[0059] First, lane extraction is performed on consecutive RGB two-dimensional images corresponding to the three-dimensional point cloud data. A Transformer semantic segmentation neural network is used to perform lane semantic segmentation on the traffic roads in the consecutive RGB two-dimensional images. The structural diagram of the semantic segmentation neural network is as shown in Figure 2 shown. It mainly includes four parts. First, the backbone structure uses an attention module based on Transformer to extract features from the sequence of RGB two-dimensional images collected by the drone acquisition vehicle. Secondly, through an encoder-decoder module, continuous features in the sequence of images collected by the drone are extracted. Then, a deformable Transformer is used to match the similarity features in the drone sequence images to obtain key point features. Finally, through an encoder-decoder, the segmentation of lane targets is achieved. Through this process, information such as the width and contour of the lane and similar key points in the drone continuous images can be obtained. Connect the feature points in the drone sequence images, filter out similar key points, and form the lane trajectory points of the entire road traffic and save information such as the road width.

[0060] S3. Coarsely register the three-dimensional point cloud data and the RGB two-dimensional images, map the road trajectory information extracted from the RGB two-dimensional images into the three-dimensional point cloud data to obtain a virtual traffic road trajectory, and remove the three-dimensional point cloud data far from the virtual traffic road trajectory to obtain a local three-dimensional point cloud map after removal.

[0061] Among them, Figure 3 is the local road map of the three-dimensional point cloud of the road traffic. After extracting lane and road information from the RGB two-dimensional images, coarsely approve the three-dimensional point cloud data and the RGB two-dimensional images.

[0062] The conversion formula between the three-dimensional point cloud and the image coordinate system collected by the drone is as follows. The image coordinate system is represented, and the three-dimensional point cloud is represented. are the focal lengths in the horizontal and vertical directions, is the center point of the image plane, is the rotation matrix, is the translation vector. Through the calibration plate plane in different poses, the parameters for the conversion between the image and the point cloud can be obtained.

[0063] .

[0064] Through the mapping relationship between the RGB two-dimensional image and the three-dimensional point cloud data, the road trajectory information in the RGB two-dimensional image is mapped into the three-dimensional point cloud data to obtain a virtual traffic road trajectory. Along the direction of the virtual traffic road trajectory, according to the width information of the road, the three-dimensional point cloud data far from the traffic road trajectory points is removed, that is, the three-dimensional point cloud data far from the road traffic such as buildings, flowers and trees, etc. is removed, which greatly reduces the number of point cloud processing. As Figure 4 shown, it is a three-dimensional point cloud map of the road after roughly removing buildings, flowers and trees far from the road.

[0065] S4, centering on the key points of the virtual traffic road trajectory, the local three-dimensional point cloud map is cut in the way of a sliding window to obtain three-dimensional point cloud map blocks.

[0066] It can be understood that the filtered three-dimensional point cloud map of the road is cut to obtain the cut three-dimensional point cloud map blocks. Specifically, the cutting method is to cut the three-dimensional point cloud data along the virtual road traffic data in the way of a sliding window. Centering on the key points of the virtual traffic road trajectory, on the virtual traffic road trajectory from a top view angle, the three-dimensional point cloud map of the road is cut within a fixed size range of the sliding window size to obtain three-dimensional point cloud map blocks.

[0067] S5, each three-dimensional point cloud map block is two-dimensionally mapped to obtain a two-dimensional mapped block, the dotted lane line segments in each two-dimensional mapped block are extracted, the dotted lane line segments are reflected into the three-dimensional point cloud map, the three-dimensional coordinate information of the dotted lane line segments is obtained, and each dotted lane line segment is connected to complete the extraction of the dotted lane lines in the three-dimensional point cloud data.

[0068] As an embodiment, the step of two-dimensionally mapping each three-dimensional point cloud map block to obtain a two-dimensional mapped block and extracting the dotted lane line segments in each two-dimensional mapped block includes: two-dimensionally mapping the three-dimensional point cloud map block at a vertical angle to obtain a two-dimensional mapped block corresponding to each three-dimensional point cloud map block; using a semantic segmentation model based on Transformer to segment the traffic road dotted lane line segments in each two-dimensional mapped block to obtain the contour information of the dotted lane line segments.

[0069] It can be understood that after the three-dimensional point cloud data is segmented to obtain three-dimensional point cloud map blocks, the three-dimensional point cloud is two-dimensionally mapped at a vertical angle, the maximum intensity value is taken as the brightness of the two-dimensional mapped image for projection, and the mapping relationship from the three-dimensional point cloud to the two-dimensional is retained to obtain the two-dimensional mapped blocks after two-dimensional mapping of each three-dimensional point cloud map block.

[0070] Extract the dotted lane line segments from each two-dimensional mapped block. First, use a semantic segmentation model based on Transformer, that is Figure 2The network structure in [description] is used to segment the dashed lane segments in the projection map to obtain the contour information of the dashed lane segments. Secondly, the midline of the contour of the dashed lane segment is extracted. This is mainly achieved by extracting the skeleton line of the binary image, followed by smoothing and line fitting to obtain the midline of the dashed lane segment. Then, the average distance between the contour of the dashed lane segment and its midline is matched with the width value of the road dashed lane line stipulated by the state to obtain the width of a single dashed lane segment. Next, the extracted dashed lane segments are filtered. Lane segments that are too long, too short, or too thick are filtered out. At the same time, according to the characteristics of the road dashed lane segments, those that are too far or too close to the average lane line or have an inconsistent aspect ratio are filtered out.

[0071] The information such as the coordinates and width of the dashed lane segments in the extracted two-dimensional mapping diagram are reflected onto the three-dimensional point cloud to obtain the x and y axis information of the dashed lane segments in the three-dimensional point cloud, and the z axis information is obtained through the ground coordinate information, etc. Thus, the three-dimensional coordinate information and width, etc. of the dashed lane segments can be obtained.

[0072] For multiple three-dimensional dashed lane segments, through the principle of local large area (a range of dozens of meters before and after) and proximity search, the dashed line segments that are close enough in distance are connected, those that are far apart are not connected, and the locally overlapping dashed line segments are removed. In this way, the position information of the entire dashed lane line can be obtained. Based on the midline and width information of the dashed lane line, the four corner point information of the dashed lane line is obtained, and the side line points on the left and right of the dashed lane line are connected to form a long dashed lane line. In this way, the extraction of the dashed lane line in the entire traffic road data collected by the UAV can be completed. For example, Figure 5 is a diagram of the extraction and connection of the dashed lane line in the three-dimensional point cloud after being cut into blocks. Among them, Figure 5 in [description], (a) is the two-dimensional mapping diagram after cutting, (b) is the schematic diagram of the extracted dashed lane line, and (c) is the connection diagram of the dashed lane line before and after.

[0073] See Figure 6, which is the overall flowchart of an automatic extraction method for dashed lane lines in a road traffic map collected by an unmanned aerial vehicle (UAV) provided by the present invention, mainly includes the following steps: First, a lane in a two-dimensional image corresponding to the three-dimensional point cloud is roughly extracted through a segmentation network, the center line of the lane is obtained, and points on the center line of the lane are saved at a fixed distance as points of the lane trajectory. At the same time, due to equipment errors, only a rough calibration matching is required to register the three-dimensional point cloud and the two-dimensional RGB image, map the virtual lane trajectory information to the three-dimensional point cloud, and connect them to form a road traffic network. Second, according to the position information of the traffic road trajectory in the three-dimensional point cloud, the three-dimensional point cloud that is far from the left and right directions of the trajectory is removed. Since the state has standards for the width of each motor vehicle lane on the road, the width of each lane on the urban highway is about 2 to 4 meters. Therefore, the three-dimensional point cloud more than thirty or forty meters to the left and right of the trajectory center line needs to be removed to reduce the computational amount of the three-dimensional point cloud of buildings, flowers, and trees. Then, according to the obtained lane trajectory points, the three-dimensional point cloud is cut in a top-down sliding block manner along the road direction, and a top-down two-dimensional mapping is performed. The small pieces of dashed lane lines in the two-dimensional mapping image are segmented and extracted through a transformer segmentation network. The center line, width, and the obtained left and right corner point information of the small pieces of dashed lane lines are saved, and the x-axis and y-axis coordinate information of the dashed lane lines in the three-dimensional point cloud is obtained through inverse mapping, and the z-axis coordinate information is obtained by getting the lowest average value of the ground. Finally, through a principle of local large area proximity search, the dashed lane lines with similar distances and slopes are merged, screened, and filtered to obtain the unique coordinate width information of the dashed lane lines; and the corner points of the dashed lane lines in the same lane are connected into strings according to the slope, distance, etc. of the dashed lane lines. Thus, the automatic extraction of the dashed lane lines in the three-dimensional point cloud data collected by the UAV is completed.

[0074] See Figure 7 , which is the structural schematic diagram of an automatic extraction system for dashed lane lines in a road traffic map collected by an unmanned aerial vehicle (UAV) provided by the present invention, includes an acquisition module 701, a first extraction module 702, a mapping module 703, a cutting module 704, a second extraction module 705, and a connection module 706, where:

[0075] The acquisition module 701 is used to acquire the three-dimensional point cloud data and the RGB two-dimensional image of the traffic road collected by the UAV;

[0076] The first extraction module 702 is used to extract lane information from consecutive RGB two-dimensional images, and the lane information includes lane width information, lane contour information, and road trajectory information;

[0077] The mapping module 703 is configured to perform rough registration on the three-dimensional point cloud data and the RGB two-dimensional image, map the road trajectory information extracted from the RGB two-dimensional image into the three-dimensional point cloud data to obtain a virtual traffic road trajectory, remove the three-dimensional point cloud data far from the virtual traffic road trajectory, and obtain a local three-dimensional point cloud map after removal;

[0078] The cutting module 704 is configured to cut the local three-dimensional point cloud map in a sliding window manner with the key points of the virtual traffic road trajectory as the center to obtain three-dimensional point cloud map blocks;

[0079] The second extraction module 705 is configured to perform two-dimensional mapping on each three-dimensional point cloud map block to obtain a two-dimensional mapped map block, and extract the dashed lane line segments in each two-dimensional mapped map block;

[0080] The connection module 706 is configured to reflect the dashed lane line segments back into the three-dimensional point cloud map to obtain the three-dimensional coordinate information of the dashed lane line segments, and connect each dashed lane line segment to complete the extraction of the dashed lane lines in the three-dimensional point cloud data.

[0081] It can be understood that an automatic extraction system for dashed lane lines in a road traffic map collected by an unmanned aerial vehicle provided by the present invention corresponds to the automatic extraction method for dashed lane lines in a road traffic map collected by an unmanned aerial vehicle provided in the foregoing embodiments. The relevant technical features of the automatic extraction system for dashed lane lines in a road traffic map collected by an unmanned aerial vehicle can refer to the relevant technical features of the automatic extraction method for dashed lane lines in a road traffic map collected by an unmanned aerial vehicle, which will not be elaborated herein.

[0082] Please refer to Figure 8 , Figure 8 which is a schematic diagram of an embodiment of an electronic device provided by an embodiment of the present invention. As Figure 8 shown, an embodiment of the present invention provides an electronic device 800, including a memory 810, a processor 820, and a computer program 811 stored on the memory 810 and executable on the processor 820. When the processor 820 executes the computer program 811, the steps of the automatic extraction method for dashed lane lines in a road traffic map collected by an unmanned aerial vehicle are implemented.

[0083] Please refer to Figure 9 , Figure 9 which is a schematic diagram of an embodiment of a computer-readable storage medium provided by the present invention. As Figure 9 shown, this embodiment provides a computer-readable storage medium 900, on which a computer program 911 is stored. When the computer program 911 is executed by a processor, the steps of the automatic extraction method for dashed lane lines in a road traffic map collected by an unmanned aerial vehicle are implemented.

[0084] An automatic extraction method and system for dashed lane lines in a road traffic map collected by a drone provided by an embodiment of the present invention has the following beneficial effects:

[0085] (1) The high-precision map drone collects the three-dimensional point cloud and two-dimensional RGB images of the traffic road. Due to the equipment error (meter-level error) between image acquisition and point cloud acquisition, it is very difficult to directly extract the high-precision road dashed lane lines in the two-dimensional RGB images. Therefore, the two-dimensional mapping diagram of the three-dimensional point cloud is used for extracting the dashed lane lines, which can not only meet the error requirement within centimeters, but also speed up the extraction speed. Figure 5 The error requirement within centimeters can be met, and the extraction speed can be accelerated.

[0086] (2) The traffic road collected by the drone cannot verify the lane for sequential acquisition and cannot contain the road traffic trajectory information. Therefore, the present invention detects the lanes in the RGB two-dimensional map through a semantic segmentation network based on transformer to obtain the road traffic topological map of lane driving. By removing the three-dimensional point clouds of buildings and trees far from the lane, the processing amount of the three-dimensional point cloud can be greatly reduced, and the detection speed can be accelerated.

[0087] (3) The present invention obtains the contour information of the dashed lane lines through the semantic segmentation network of transformer, and then obtains the center line and smoothness through the skeleton line, etc., so as to accurately obtain the coordinate information of the four corner points of the dashed lane lines. Compared with the traditional corner detection method, through the segmentation network plus refinement, it can adapt to more scenarios and has higher accuracy.

[0088] It should be noted that in the above embodiments, the descriptions of each embodiment have their own emphases. For the parts not detailedly described in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0089] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0090] The present invention will be described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present invention. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0091] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device that implements the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0092] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one flow Figure 1 one flow or multiple flows and / or blocks Figure 1 or multiple blocks.

[0093] Although the preferred embodiments of the present invention have been described, those skilled in the art can make additional changes and modifications once they learn the basic creative concepts. Therefore, the appended claims are intended to be construed to include the preferred embodiments and all changes and modifications falling within the scope of the present invention.

[0094] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these changes and modifications.

Claims

1. An automatic extraction method for dashed lane lines in a road traffic map collected by an unmanned aerial vehicle, characterized in that, Including: Obtain the three-dimensional point cloud data and RGB two-dimensional images of the traffic road collected by the drone; Extract lane information from consecutive RGB two-dimensional images, where the lane information includes lane width information, lane contour information, and road trajectory information; Coarsely register the three-dimensional point cloud data and RGB two-dimensional images, map the road trajectory information extracted from the RGB two-dimensional images into the three-dimensional point cloud data to obtain a virtual traffic road trajectory, remove the three-dimensional point cloud data far from the virtual traffic road trajectory, and obtain a local three-dimensional point cloud map after removal; Centering on the key points of the virtual traffic road trajectory, cut the local three-dimensional point cloud map in a sliding window manner to obtain three-dimensional point cloud patches; Perform two-dimensional mapping on each three-dimensional point cloud patch to obtain a two-dimensional mapped patch, extract the dashed lane segments in each two-dimensional mapped patch, reflect the dashed lane segments into the three-dimensional point cloud map, obtain the three-dimensional coordinate information of the dashed lane segments, and connect each dashed lane segment to complete the extraction of the dashed lane lines in the three-dimensional point cloud data.

2. The automatic extraction method for dashed lane lines according to claim 1, characterized in that, The extracting lane information from consecutive RGB two-dimensional images, where the lane information includes lane width information, lane contour information, and road trajectory information, includes: Based on the transformer semantic segmentation neural network, perform lane semantic segmentation on the traffic road in consecutive RGB two-dimensional images to obtain lane information in the RGB two-dimensional images, where the lane information includes lane width information, lane contour information, and similar key point information in consecutive RGB two-dimensional images; Filter and connect the similar key points in consecutive RGB two-dimensional images to form the road trajectory points of the entire road traffic and save the road width information.

3. The automatic extraction method for dashed lane lines according to claim 1, characterized in that, The coarsely registering the three-dimensional point cloud data and RGB two-dimensional images includes: Based on the three-dimensional point cloud data and the RGB two-dimensional image, the parameters for the conversion between the image and the point cloud are obtained through the calibration of the calibration plate plane in different poses , as follows: ； Image coordinate system represents a three-dimensional point cloud represents where \(f_x\) and \(f_y\) are the focal lengths in the horizontal and vertical directions \((u_0, v_0)\) is the center point of the image plane \(R\) is the rotation matrix \(t\) is the translation vector 4. The automatic extraction method for dashed lane lines according to claim 1, characterized in that, Centering on the key points of the virtual traffic road trajectory, cutting the local three-dimensional point cloud map in a sliding window manner to obtain three-dimensional point cloud patches, including: Set the sliding window size, and centering on the key points of the virtual traffic road trajectory, cut the local three-dimensional point cloud map in a fixed size range of the sliding window size on the virtual traffic road trajectory from a top-down perspective to obtain three-dimensional point cloud patches.

5. The automatic extraction method for dashed lane lines according to claim 1 or 4, characterized in that, The performing two-dimensional mapping on each three-dimensional point cloud patch to obtain a two-dimensional mapped patch and extracting the dashed lane segments in each two-dimensional mapped patch includes: Perform two-dimensional mapping on the three-dimensional point cloud patch at a vertical angle to obtain a two-dimensional mapped patch corresponding to each three-dimensional point cloud patch; Based on the semantic segmentation model of Transformer, segment the traffic road dashed lane segments in each two-dimensional mapped patch to obtain the contour information of the dashed lane segments.

6. The automatic extraction method for dashed lane lines according to claim 5, characterized in that, After performing two-dimensional mapping on each three-dimensional point cloud patch to obtain a two-dimensional mapped patch and extracting the dashed lane segments in each two-dimensional mapped patch, it further includes: Based on the obtained contour information of the dashed lane segments, extract the skeleton line of the binary image, and then obtain the center line of the dashed lane segments through smoothing and line fitting. The average distance between the contour of the dashed lane line segment and the midline of the dashed lane line segment is matched with the standard road dashed lane line width value to obtain the width of each dashed lane line segment. The dashed lane line segments that are too long, too short, or too thick are filtered out, and according to the characteristics of the road dashed lane line segments, the dashed lane line segments that are too far or too close to the average lane line or have an inconsistent aspect ratio are filtered out to obtain the filtered dashed lane line segments.

7. The automatic extraction method for dashed lane lines according to claim 1, characterized in that, The step of reflecting the dashed lane line segment into the three-dimensional point cloud map to obtain the three-dimensional coordinate information of the dashed lane line segment includes: The coordinates and width information of the dashed lane line segment in the extracted two-dimensional mapping map are reflected into the three-dimensional point cloud map to obtain the x and y axis information of the dashed lane line segment in the three-dimensional point cloud map. The z-axis information of the dashed lane line segment is obtained through the ground coordinate information to obtain the three-dimensional coordinate information and width information of the dashed lane line segment.

8. The automatic extraction method for dashed lane lines according to claim 7, characterized in that, The step of connecting each dashed lane line segment to complete the extraction of the dashed lane line in the three-dimensional point cloud data includes: Through the principle of finding nearby in a local large area, the dashed lane line segments that are close are connected, the dashed lane line segments that are far away are not connected, and the locally overlapping dashed lane line segments are removed to obtain the position information of the entire dashed lane line. Four corner point information is obtained according to the midline and width information of the dashed lane line. Based on the four corner point information, the side line points on the left and right of the dashed lane line are connected to form a long dashed lane line, completing the extraction of the dashed lane line in the three-dimensional point cloud data.

9. An automatic extraction system for dashed lane lines in a road traffic map collected by an unmanned aerial vehicle, characterized in that, It includes: An acquisition module for acquiring the three-dimensional point cloud data and RGB two-dimensional images of the traffic road collected by the unmanned aerial vehicle. A first extraction module for extracting lane information in consecutive RGB two-dimensional images, where the lane information includes the width information of the lane, the contour information of the lane, and the road trajectory information. A mapping module for performing rough registration on the three-dimensional point cloud data and the RGB two-dimensional images, mapping the road trajectory information extracted from the RGB two-dimensional images into the three-dimensional point cloud data to obtain a virtual traffic road trajectory, removing the three-dimensional point cloud data far from the virtual traffic road trajectory, and obtaining a local three-dimensional point cloud map after removal. A cutting module for cutting the local three-dimensional point cloud map in a sliding window manner with the key points of the virtual traffic road trajectory as the center to obtain three-dimensional point cloud map blocks. A second extraction module for performing two-dimensional mapping on each three-dimensional point cloud map block to obtain a two-dimensional mapping map block, and extracting the dashed lane line segments in each two-dimensional mapping map block. A connection module for reflecting the dashed lane line segments into the three-dimensional point cloud map to obtain the three-dimensional coordinate information of the dashed lane line segments, connecting each dashed lane line segment, and completing the extraction of the dashed lane line in the three-dimensional point cloud data.

10. A computer-readable storage medium, characterized in that, It stores a computer management program, and when the computer management program is executed by a processor, it implements the steps of the method for automatically extracting the dashed lane line in the road traffic map collected by the unmanned aerial vehicle as described in any one of claims 1-8.

Images