Urban structured road real-time detection method and system

Through the flat processing of lidar point cloud data and the application of the deep dual-resolution network DDRNet, efficient real-time detection of urban structured roads is achieved, and the problem of difficulty in detection of single sensors and high cost of high precision maps is solved, achieving high accuracy and High efficiency detection effect.

CN119091300APending Publication Date: 2024-12-06WUHAN UNIV OF SCI & TECH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411185710.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-08-27
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

In the real-time detection of urban structured roads, single sensors are difficult to complete the task, high-precision maps lead to high development costs, and multi-sensor fusion has problems such as data fusion and low generalization capabilities. When extracting features based on point cloud aerial view It is difficult to take into account both algorithm efficiency and accuracy.

Method used

Lidar point cloud data is used for flattening, a two-dimensional bird's eye view is generated, and model training is carried out in the deep dual-resolution network DDRNet to realize real-time detection of urban structured roads.

Benefits of technology

Achieve 95.2% detection accuracy and 70.42 frames/second inference speed, meet the real-time and accuracy requirements of road detection by autonomous vehicles, significantly improve detection accuracy and processing efficiency, and solve the low data processing efficiency in the existing technology. and insufficient detection accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119091300A_ABST
    Figure CN119091300A_ABST
Patent Text Reader

Abstract

The invention belongs to but not limited to the technical field of intelligent driving, and particularly relates to an urban structured road real-time detection method and system, and the method comprises the steps: S1, collecting the laser radar point cloud data of an urban road through a real vehicle, and flattening the laser radar point cloud data into a 2D aerial view; s2, marking urban structured roads in the BEV data and generating a data set; and S3, performing model training in the deep double-resolution network DDRNet, and generating a prediction picture of the urban structured road in real time. Experimental results show that the algorithm precision of the urban structured road detection model provided by the invention reaches 95.2%, the reasoning speed of the whole network reaches 70.42 frame / s, and the requirements of autonomous vehicles on the real-time performance and accuracy of urban structured road detection are met.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to but is not limited to the field of intelligent driving technology, and in particular relates to a method and system for real-time detection of urban structured roads. Background Art

[0002] Real-time detection of the road ahead is one of the essential functions of driverless cars. If the perception of urban structured roads needs to rely on high-precision maps, it will greatly increase the cost of enterprise research and development. It is difficult to complete the real-time detection task of urban roads by relying on a single sensor such as a camera or radar. The use of high-precision maps will lead to high development costs. Multi-sensor fusion will face difficulties in data fusion and low generalization ability. Extracting feature information based on point cloud bird's-eye view is more effective, but it will face the problem of difficulty in balancing algorithm efficiency and accuracy.

[0003] The three-dimensional point cloud is constructed into a bird's-eye view and input into the two-dimensional target detection network. The deep fusion network constructed by LIANG M et al. extracts features from images and point clouds respectively, establishes the projection relationship of the laser radar point cloud source data, and extracts fusion features, but the computational efficiency is low. The BirdNet network constructed by BELTRAN J et al. processes the three-dimensional point cloud information into a bird's-eye view and inputs it into the Fast Region Convolutional Neural Network (Fast R-CNN) to complete the detection task, but there is still a problem of low reasoning speed. The BEV-DetNet network proposed by MOHAPATRA S et al. uses key points to detect the center of the target and performs detection on a relatively simple bird's-eye view. The detection time per frame is 60ms, and the detection speed is still slow. The FS23D network constructed by WANG B] et al. uses grid foreground segmentation for the sparse feature information after the bird's-eye view projection, and the detection speed reaches 55.1 frames / s, but its algorithm accuracy is low.

[0004] In view of the above analysis, the existing technologies have the following technical problems that need to be solved urgently: it is difficult to complete the real-time detection task of urban structured roads by relying on a single sensor such as a camera or a radar, and the use of high-precision maps will lead to high development costs. Multi-sensor fusion will face difficulties in data fusion and low generalization ability. Extracting feature information based on point cloud bird's-eye view is more effective, but it will face the problem of difficulty in balancing algorithm efficiency and accuracy. Summary of the invention

[0005] In view of the problems existing in the prior art, the present invention provides a method and system for real-time detection of urban structured roads. The real-time detection algorithm of urban roads is used to perceive and detect the road area ahead in real time.

[0006] The present invention is implemented as follows: a real-time detection method for urban structured roads, comprising:

[0007] S1, collects LiDAR point cloud data of urban roads through real vehicles and flattens it into a 2D bird's-eye view (BEV);

[0008] S2, annotate the urban roads in the BEV data and generate a dataset;

[0009] S3, performs model training in the deep dual-resolution network DDRNet and generates predicted images of urban structured road areas in real time.

[0010] Furthermore, S1 specifically includes:

[0011] Point cloud data collection: Use a collection vehicle equipped with a Hesai pandar64 laser radar rotating scanner to collect laser radar point cloud data on urban roads and highways;

[0012] Point cloud data flattening processing: Point cloud data flattening processing refers to the dimensionality reduction operation on the three-dimensional point cloud, so that the disordered and irregular point cloud data is mapped to the two-dimensional BEV image, thereby highlighting the semantic information of the data; the point cloud flattening design principle is: retain the x, y, z information of the point cloud coordinates, remove the point cloud reflection intensity i, and determine whether each point of the point cloud is within the area of ​​interest; perform image coordinate conversion on the point cloud data, fill in the pixel value and perform binarization processing, and finally generate an 8-bit grayscale BEV image.

[0013] Point cloud data coordinate conversion: Because the final output data is a BEV image, and the image coordinate system and the point cloud coordinate system are essentially different, the point cloud data is converted into image coordinates;

[0014] Pixel value filling and binarization processing: The points in the point cloud area of ​​interest are filtered out and pixel value filling and binarization processing is performed. The point position is mapped to the pixel position. The pixel value of the area in the area of ​​interest but without point cloud is set to 0, and the pixel value of the area with point cloud is set to 255.

[0015] Generate BEV image: In order to retain more image semantic details and improve the recognition accuracy of the model, this study sets the width of each pixel to 0.075 meters; by mapping the point cloud area of ​​interest to a BEV image, an 8-bit grayscale image with a resolution of 1280*640 can be obtained.

[0016] Furthermore, S2 specifically includes:

[0017] BEV data annotation specifically includes: generating an urban road dataset from a BEV perspective by manual annotation, annotating the BEV data using labelme, annotating the urban structured roads in the BEV data and generating a BEV dataset.

[0018] Furthermore, the DDRNet-23-slim algorithm specifically includes:

[0019] DDRNet-23-slim is a dual-path structure. The backbone network uses residual blocks for connection. The overall network structure of DDRNet-23-slim consists of two parallel branches with different resolutions to extract features. The two branches perform multiple spatial and semantic information fusions at different stages to obtain two feature maps with different resolutions. The fusion obtains an output feature map with rich semantic information. In addition to the low-resolution branch obtaining semantic information, another high-resolution shallow path provides rich spatial details as a supplement.

[0020] In order to strike a balance between resolution and inference speed, the high-resolution branch creates a feature map with a resolution of 1 / 8 of the input image and appends it to the end of the conv3 stage; the high-resolution branch does not contain any downsampling operations and has a one-to-one correspondence with the low-resolution branch to form a deep high-resolution representation; multiple bilateral feature fusions are then performed at different stages to fully fuse spatial and semantic information; bilateral fusion includes fusing the high-resolution branch to the low-resolution branch (high-to-low fusion) and fusing the low-resolution branch to the high-resolution branch (low-to-high fusion). For high-to-low fusion, the high-resolution feature map is downsampled by a sequence of 3×3 convolutions (with a step size of 2) before point-by-point summation; for low-resolution to high-resolution, the low-resolution feature map is first compressed by a 1×1 convolution and then upsampled by bilinear interpolation;

[0021] The DAPPM module is used to further extract contextual information from low-resolution feature maps. It takes a feature map with an image resolution of 1 / 64 as input, uses an exponentially large pooling kernel, generates feature maps with image resolutions of 1 / 128, 1 / 256, and 1 / 512, uses global average pooling to generate input feature maps and image-level information, and upsamples the feature maps. Then, more 3×3 convolutions are used to fuse contextual information of different scales in a hierarchical residual manner; finally, 1×1 convolutions are used to connect and compress all feature maps. Although DAPPM contains more convolutional layers and more complex fusion strategies, it has little impact on inference speed because the input resolution is only 1 / 64 of the image resolution.

[0022] Another object of the present invention is to provide a system for implementing the urban structured road real-time detection method, comprising:

[0023] The point cloud data acquisition module collects the laser radar point cloud data of urban roads through real vehicles and flattens it into a 2D bird's-eye view;

[0024] A data set generation module is used to annotate the urban structured roads in the BEV data and generate a data set;

[0025] The prediction image generation module performs model training in the deep dual-resolution network DDRNet and generates prediction images of urban structured roads in real time.

[0026] Another object of the present invention is to provide a computer device, the computer device includes a memory and a processor, the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the real-time detection method of urban structured roads.

[0027] Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to execute the steps of the real-time detection method for urban structured roads.

[0028] Another object of the present invention is to provide an information data processing terminal, which includes the urban structured road real-time detection system.

[0029] In combination with the above technical solutions and the technical problems solved, the advantages and positive effects of the technical solutions to be protected by the present invention are as follows:

[0030] First, the experimental results show that the algorithm accuracy of the urban structured road detection model proposed in the present invention reaches 95.2%, and the reasoning speed of the overall network reaches 70.42 frames / s, which meets the requirements of autonomous driving vehicles for real-time and accuracy of road detection.

[0031] Second, the technical solution of the present invention addresses the deficiencies of the real-time urban road detection methods in the prior art and proposes an improved method based on LiDAR point cloud data processing and deep learning. Existing urban road detection technologies usually face problems such as low data processing efficiency, insufficient detection accuracy, and poor real-time performance in complex urban environments. Traditional methods often rely on two-dimensional images or GPS data, which are prone to misjudgment or missed detection when dealing with the diversity and complexity of urban roads, especially when detecting structured roads, where accuracy is difficult to guarantee.

[0032] The present invention significantly improves the accuracy and reliability of urban road detection by introducing laser radar point cloud data. First, the present invention uses a laser radar collection vehicle to obtain high-precision three-dimensional point cloud data, and generates a two-dimensional bird's-eye view (BEV image) that retains the key geometric features of the road through flattening of the point cloud data. Compared with traditional two-dimensional image processing methods, this point cloud-based flattening process can more accurately capture the structural information of urban roads, thereby reducing detection errors.

[0033] Secondly, the present invention uses a deep dual-resolution network (DDRNet) for real-time detection of urban structured roads. The DDRNet model has significant advantages in high-resolution data processing and can better capture detailed features in complex road environments. By training on the annotated BEV dataset, DDRNet can quickly and accurately generate road prediction images in real-time applications, solving the problem of insufficient real-time performance in the prior art.

[0034] In addition, the present invention realizes real-time analysis and prediction of a large amount of point cloud data through an efficient data processing process, greatly improving the processing efficiency of the system. Compared with traditional methods, the technical solution of the present invention not only improves the detection accuracy, but also ensures the real-time and robustness of the system, and can cope with complex and changeable urban road environments.

[0035] Through the above improvements, the present invention has achieved significant technical progress in real-time detection of urban structured roads, solved key problems in the prior art, and provided reliable data support and technical guarantee for applications such as autonomous driving systems and intelligent traffic management. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 is a flow chart of a real-time detection method for urban structured roads provided by an embodiment of the present invention;

[0037] Figure 2 is a structural diagram of a real-time detection system for urban structured roads provided by an embodiment of the present invention;

[0038] Figure 3 is a schematic diagram of a BEV scenario at an urban intersection provided by an embodiment of the present invention;

[0039] Figure 4 This is a schematic diagram of a BEV scenario on an ordinary road in an urban area provided by an embodiment of the present invention;

[0040] Figure 5 is a schematic diagram of a highway BEV scenario provided by an embodiment of the present invention;

[0041] Figure 6 : is a schematic diagram of BEV image annotation provided by an embodiment of the present invention; wherein (a) is an original unlabeled BEV image; (b) is a labeled BEV image, wherein the red area is a structured urban road (road) and the black area is a background (background); (c) is the final generated BEV image label label;

[0042] Figure 7 It is a schematic diagram of the overall structure of DDRNet-23-slim provided by an embodiment of the present invention;

[0043] Figure 8is a bilateral feature fusion flow chart provided by an embodiment of the present invention;

[0044] Fig. 9 is an internal structure diagram of a DAPPM provided by an embodiment of the present invention;

[0045] Fig.10 is a schematic diagram of segmentation of a BEV scene on a common road provided by an embodiment of the present invention;

[0046] Fig.11 is a schematic diagram of highway BEV scene segmentation provided by an embodiment of the present invention;

[0047] Fig.12 It is a schematic diagram of BEV scene segmentation at an intersection provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0049] The working principle of the present invention is based on the processing of LiDAR point cloud data and the application of deep learning models to achieve real-time detection of urban structured roads. First, in actual operation, a collection vehicle equipped with a Hesai Pandar64 LiDAR rotating scanner is used to scan urban roads and highways to obtain high-precision three-dimensional point cloud data. These point cloud data contain detailed geometric information of the roads, but due to their disorder and irregularity, they are difficult to be directly used for the detection and analysis of urban roads.

[0050] To solve this problem, the point cloud data is flattened after collection to reduce the dimensionality of the three-dimensional point cloud data to a two-dimensional plane. Specifically, the flattening process retains the x, y, and z coordinate information in the point cloud data, while removing the reflection intensity i of the point cloud. By determining whether each point is within the area of ​​interest, performing image coordinate conversion and pixel value filling, a binary 8-bit grayscale bird's-eye view (BEV image) is finally generated. This image not only retains the key geometric features of the road, but also converts the point cloud data into a form suitable for deep learning model input.

[0051] Next, the generated BEV image dataset is used to train a deep dual-resolution network (DDRNet). DDRNet has a strong ability to process high-resolution input data and can capture the detailed features of urban structured roads. By annotating urban structured roads and generating datasets, the DDRNet model can learn the structural patterns of urban roads during training, so that it can quickly and accurately perform real-time predictions in practical applications.

[0052] Finally, after sufficient training, the DDRNet model is applied to real-time detection tasks. After receiving new lidar point cloud data, the model first flattens the data into a BEV image, and then performs real-time analysis through the DDRNet model to generate a predicted image of the city's structured roads. This predicted image can accurately reflect the structural characteristics of the current urban roads, providing key real-time data support for autonomous driving systems and urban traffic management.

[0053] like Figure 1 As shown, the real-time detection method of urban structured roads provided by the embodiment of the present invention includes:

[0054] S1, collects LiDAR point cloud data of urban roads through real vehicles and flattens it into a 2D bird's-eye view (BEV);

[0055] S2, annotate the urban structured roads in the BEV data and generate a dataset;

[0056] S3, performs model training in the deep dual-resolution network DDRNet and generates predicted images of urban structured roads in real time.

[0057] like Figure 2 As shown, the urban structured road real-time detection system provided by the embodiment of the present invention includes:

[0058] The point cloud data acquisition module collects the laser radar point cloud data of urban roads through real vehicles and flattens it into a 2D bird's-eye view;

[0059] A data set generation module is used to annotate the urban structured roads in the BEV data and generate a data set;

[0060] The prediction image generation module performs model training in the deep dual-resolution network DDRNet and generates prediction images of urban structured roads in real time.

[0061] 1 Point cloud data collection and BEV dataset generation

[0062] 1.1 Point cloud data collection

[0063] A collection vehicle equipped with a Hesai pandar64 lidar rotating scanner is used to collect lidar point cloud data on urban roads and highways.

[0064] 1.2 Point cloud data flattening processing

[0065] Point cloud data flattening refers to the dimensionality reduction operation of three-dimensional point clouds, so that disordered and irregular point cloud data can be mapped to two-dimensional BEV images, thereby highlighting the semantic information of the data for subsequent content research. The design principle of point cloud flattening is: retain the x, y, and z information of the point cloud coordinates, remove the point cloud reflection intensity i, and determine whether each point of the point cloud is within the area of ​​interest; perform image coordinate conversion on the point cloud data, fill in the pixel value and perform binarization processing, and finally generate an 8-bit grayscale BEV image.

[0066] 1.3 Point cloud data coordinate conversion

[0067] Because the final output data is a BEV image, and the image coordinate system and the point cloud coordinate system are essentially different, the point cloud data needs to be converted into image coordinates.

[0068] 1.4 Pixel value filling and binarization processing

[0069] The points in the point cloud region of interest are filtered out and the pixel values ​​are filled and binarized. The point positions are mapped to pixel positions, and the pixel values ​​of the areas in the region of interest but without point clouds are set to 0, and the pixel values ​​of the areas with point clouds are set to 255.

[0070] 1.5 Generate BEV images

[0071] In order to retain more image semantic details and improve the recognition accuracy of the model, this study sets the width of each pixel to 0.075 meters. By mapping the point cloud region of interest to a BEV image, an 8-bit grayscale image with a resolution of 1280*640 can be obtained. The following is a BEV image generated after the point cloud is flattened. Figure 3-5 They are BEV scenario at urban intersection, BEV scenario on ordinary urban road, and BEV scenario on highway.

[0072] 1.6BEV Data Annotation

[0073] Since there is no public dataset for single detection of urban roads from the BEV perspective, a manual annotation method is used to generate an urban road dataset from the BEV perspective. The open source image annotation software labelme is used as the annotation tool to annotate the BEV data. The annotation categories of this study are shown in Table 1.

[0074] Figure 6 Schematic diagram of BEV image annotation: (a) is the original unlabeled BEV image; (b) is the labeled BEV image, where the red area is the urban structured road (road) and the black area is the background (background); (c) is the final generated BEV image label label. This study only annotates urban structured roads, and sidewalks and other non-motorized vehicle lanes are not within the scope of this study.

[0075] Table 1 Marking categories

[0076]

[0077] 2DDRNet-23-slim Algorithm Introduction

[0078] DDRNet-23-slim is a dual-path structure, and the backbone network uses residual blocks for connection. In order to solve the partial information loss caused by the repeated downsampling process of the previous encoder-decoder structure model, the overall network structure of DDRNet-23-slim mainly extracts features by two parallel branches of different resolutions. The two branches perform multiple spatial and semantic information fusions at different stages to obtain two-part feature maps of different resolutions, which are then fused to obtain output feature maps containing rich semantic information. In addition to the low-resolution branch obtaining semantic information, another high-resolution shallow path provides rich spatial details as a supplement. The overall structure of DDRNet-23-slim is shown below. Figure 7 shown.

[0079] In order to strike a balance between resolution and inference speed, the high-resolution branch creates a feature map with a resolution of 1 / 8 of the input image and appends it to the end of the conv3 stage. The high-resolution branch does not contain any downsampling operations and has a one-to-one correspondence with the low-resolution branch to form a deep high-resolution representation. Multiple bilateral feature fusions are then performed at different stages to fully fuse spatial and semantic information. Bilateral fusion includes fusing the high-resolution branch to the low-resolution branch (high to low fusion) and fusing the low-resolution branch to the high-resolution branch (low to high fusion). For high-to-low fusion, the high-resolution feature map is downsampled by a sequence of 3×3 convolutions (with a stride of 2) before point-by-point summation. For low-resolution to high-resolution, the low-resolution feature map is first compressed by a 1×1 convolution and then upsampled by bilinear interpolation. As Figure 8 The figure is the flowchart of bilateral feature fusion.

[0080] The DAPPM module is used to further extract contextual information from the low-resolution feature map, such as Fig. 9The internal structure diagram of DAPPM. It takes the feature map of 1 / 64 image resolution as input, uses exponentially large pooling kernels to generate feature maps of 1 / 128, 1 / 256, and 1 / 512 image resolutions, and uses global average pooling to generate input feature maps and image-level information. The feature map is upsampled, and then more 3×3 convolutions are used to fuse contextual information of different scales in a hierarchical residual manner. Finally, all feature maps are connected and compressed using 1×1 convolutions. Although DAPPM contains more convolutional layers and more complex fusion strategies, since the input resolution is only 1 / 64 of the image resolution, it hardly affects the inference speed.

[0081] The experiment uses single card training. The resolution of all input images is 1280*640. The batch size (batch-size) used for training is 4, the training iteration cycle epoch is 500, the optimization algorithm is the stochastic gradient descent algorithm (sgd), the loss function is the cross entropy loss function, the learning rate is 0.01, the weight decay (wd) is 0.0005, and the momentum size (momentum) is 0.9

[0082] In the following experiments, pixel accuracy PA (Pixel Accuracy), mean intersection over union MIou (MeanIntersection overUnion), mean accuracy MA (MeanAccuracy) and frames per second FPS (Frames PerSecond) are mainly used to evaluate network performance. k represents the total number of categories, P ij represents the number of pixels that belong to class i but are predicted to be class j, P ii Indicates the number of pixels predicted correctly, P ji Represents the number of pixels that belong to class j but are predicted to be class i.

[0083] PA represents the ratio of correctly predicted pixels to all pixels. The calculation formula of PA value is as follows:

[0084]

[0085] MIou means calculating the IOU of (intersection-over-union ratio of the true label and the predicted result) for each category, and then averaging the IOU of all categories. The calculation formula of MIou is as follows:

[0086]

[0087] MA means calculating the average accuracy of each category. Where n is the number of categories, and Acc_i represents the accuracy of the i-th category. The calculation formula of MA value is as follows:

[0088]

[0089] FPS indicates how many frames of BEV image data the model can process per second. The higher the FPS value, the faster the model segmentation. After experimental calculation, the algorithm takes 14.2ms to process each image, which means the model's inference speed is 70.42 frames / s.

[0090] Table 2 shows the evaluation indicators of the model.

[0091] Table 2 Evaluation indicators of the model

[0092]

[0093] Visualization of model segmentation effect

[0094] The following is a demonstration of the effect of DDRNet-23-slim segmenting urban structured roads. Fig.10 Schematic diagram of BEV scene segmentation on ordinary roads. Fig.11 Schematic diagram of highway BEV scene segmentation. Fig.12 The following is a schematic diagram of the segmentation of the BEV scene at the intersection. As can be seen from the segmentation diagram, the DDRNet-23-slim model has a good segmentation effect for the BEV scene on ordinary urban roads and highways, and the prediction results are accurate. For the more complex BEV scene at the intersection, the details of obstacles such as flower beds and curbs can be accurately segmented. There is a small amount of overfitting at the intersection with non-motor vehicles. Overall, it can be seen that the DDRNet-23-slim model still has a good segmentation effect in complex road scenes.

[0095] An application embodiment of the present invention provides a computer device, which includes a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor executes the steps of the real-time detection method of urban structured roads.

[0096] An application embodiment of the present invention provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the steps of a real-time detection method for urban structured roads.

[0097] An application embodiment of the present invention provides an information data processing terminal, which includes a real-time detection system for urban structured roads.

[0098] The real-time detection system for urban structured roads of the present invention first uses a point cloud data acquisition module to collect laser radar point cloud data of urban roads using a real vehicle. After processing, these point cloud data are flattened into a 2D bird's-eye view, so that the structure of urban roads can be more intuitively presented. This step ensures the high accuracy and integrity of the original data, providing a reliable basis for subsequent processing.

[0099] Next, the dataset generation module annotates the generated BEV (bird's eye view) data, marks the features of the urban structured roads in detail and generates the corresponding dataset. This dataset contains various road types and their related feature information, which can provide rich and diverse data support for subsequent model training.

[0100] In the third step, the prediction image generation module uses the deep dual-resolution network (DDRNet) to train the model on the labeled data set. During the training process, DDRNet can effectively capture the multi-scale characteristics of urban roads and generate a high-precision prediction model. After training, the module can process the newly collected point cloud data in real time and generate the corresponding urban structured road prediction images.

[0101] Finally, the system applies the generated prediction images to actual scenarios, and performs real-time detection and analysis of the structured information of urban roads. This real-time detection function enables the system to update and reflect road conditions in a timely manner and provide them to autonomous vehicles or intelligent transportation systems, thereby improving the intelligence level and safety of urban traffic management.

[0102] The following are two practical application examples of the present invention in industry:

[0103] Example 1: Urban road detection in autonomous vehicles

[0104] In the application of autonomous driving vehicles, real-time and accurate urban road detection is crucial. The present invention can be integrated into the autonomous driving system to generate structured information of urban roads in real time using point cloud data collected by LiDAR. Through the deep dual-resolution network (DDRNet), the system can quickly identify structured features such as road boundaries, lane lines and obstacles in complex urban environments, and provide high-precision road maps. This real-time detection capability greatly improves the driving safety and path planning efficiency of autonomous vehicles, especially in complex intersections, congested sections and construction areas in cities.

[0105] Example 2: Intelligent Traffic Management System

[0106] In intelligent traffic management systems, real-time monitoring and management of the status of urban roads is crucial for optimizing traffic flow, reducing congestion and improving road safety. The present invention can be applied to urban traffic management centers. Through the laser radar equipment installed at key locations on the road, road point cloud data can be collected in real time and flattened. Using the DDRNet model, the system can generate structured information of urban roads in real time, helping traffic managers to quickly identify traffic accidents, road damage, illegal parking and other situations. This application not only improves the real-time and accuracy of traffic management, but also provides efficient data support for intelligent transportation systems, which helps to achieve more intelligent urban traffic management.

[0107] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0108] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modification, equivalent substitution and improvement made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. A real-time detection method for urban structured roads, characterized in that: include: S1, collects LiDAR point cloud data of urban roads through real vehicles and flattens it into a 2D bird's-eye view; S2, annotate the urban structured roads in the BEV data and generate a dataset; S3, performs model training in the deep dual-resolution network DDRNet and generates predicted images of urban structured roads in real time.

2. The real-time detection method for urban structured roads according to claim 1, characterized in that: S1 specifically includes: Point cloud data collection: Use a collection vehicle equipped with a Hesai pandar64 laser radar rotating scanner to collect laser radar point cloud data on urban roads and highways; Point cloud data flattening processing: Point cloud data flattening processing refers to the dimensionality reduction operation on the three-dimensional point cloud, so that the disordered and irregular point cloud data is mapped to the two-dimensional BEV image, thereby highlighting the semantic information of the data; the point cloud flattening design principle is: retain the x, y, z information of the point cloud coordinates, remove the point cloud reflection intensity i, and determine whether each point of the point cloud is within the area of ​​interest; perform image coordinate conversion on the point cloud data, fill in the pixel value and perform binarization processing, and finally generate an 8-bit grayscale BEV image.

3. The real-time detection method for urban structured roads according to claim 1, characterized in that: S2 specifically includes: Point cloud data coordinate conversion: Because the final output data is a BEV image, and the image coordinate system and the point cloud coordinate system are essentially different, the point cloud data is converted into image coordinates; Pixel value filling and binarization processing: The points in the point cloud area of ​​interest are filtered out and pixel value filling and binarization processing is performed. The point position is mapped to the pixel position. The pixel value of the area in the area of ​​interest but without point cloud is set to 0, and the pixel value of the area with point cloud is set to 255. Generate BEV image: In order to retain more image semantic details and improve the recognition accuracy of the model, this study sets the width of each pixel to 0.075 meters; by mapping the point cloud area of ​​interest to a BEV image, an 8-bit grayscale image with a resolution of 1280*640 can be obtained.

4. The real-time detection method for urban structured roads according to claim 1, characterized in that: BEV data annotation specifically includes: generating an urban road dataset from a BEV perspective by manual annotation, using labelme to annotate BEV data, annotating urban structured roads in BEV data, and defining urban structured roads as: all road areas that the collection vehicle can currently reach, without considering the impact of other vehicles and pedestrian obstacles in the area.

5. The real-time detection method for urban structured roads according to claim 1, characterized in that: The DDRNet-23-slim algorithm adopts a deep neural network with a dual-path structure. The backbone network is connected by residual blocks, and the overall network structure includes two parallel branches with different resolutions for feature extraction. The two branches fuse spatial information and semantic information multiple times at different stages to generate feature maps of different resolutions, and obtain output feature maps containing rich semantic information through fusion; among them, the low-resolution branch is responsible for acquiring semantic information, and the high-resolution branch provides rich spatial details as a supplement, ensuring that the feature map contains both semantic information and accurate spatial representation.

6. The real-time detection method for urban structured roads according to claim 5, characterized in that: The DDRNet-23-slim algorithm realizes information interaction between high-resolution and low-resolution branches through multiple bilateral feature fusion operations; Among them, the high-resolution branch creates a feature map with a resolution of 1 / 8 of the input image, does not include downsampling operations, and forms a one-to-one correspondence with the low-resolution branch; bilateral fusion includes downsampling fusion from the high-resolution branch to the low-resolution branch and upsampling fusion from the low-resolution branch to the high-resolution branch, ensuring the full fusion of spatial information and semantic information; in addition, the DAPPM module further extracts contextual information from the low-resolution feature map, generates feature maps through multi-scale pooling, and connects and compresses them through 1×1 convolution, fusing contextual information in a hierarchical residual manner, ultimately optimizing feature representation while maintaining inference speed.

7. A system for real-time detection of urban structured roads for implementing the method for real-time detection of urban structured roads according to any one of claims 1 to 6, characterized in that: include: The point cloud data acquisition module collects the laser radar point cloud data of urban roads through real vehicles and flattens it into a 2D bird's-eye view; A data set generation module is used to annotate the urban structured roads in the BEV data and generate a data set; The prediction image generation module performs model training in the deep dual-resolution network DDRNet and generates prediction images of urban structured roads in real time.

8. A computer device, comprising a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the real-time detection method for urban structured roads as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the real-time detection method for urban structured roads as claimed in any one of claims 1 to 6.

10. An information data processing terminal, comprising the urban structured road real-time detection system according to claim 7.