Urban traffic scene extraction method, device, system and storage medium

Abstract

This invention discloses a method, device, system, and storage medium for extracting urban traffic scenes, belonging to the field of intelligent interpretation technology. The method comprises: obtaining 3D point cloud data and image data of an urban traffic scene; extracting static elements of the urban traffic scene using a trained KPConv network based on the 3D point cloud data, obtaining the static elements of the urban traffic scene, and parameterizing the static elements of the urban traffic scene; extracting dynamic objects in the urban traffic scene using a trained YOLOv7-W6 network based on the image data, obtaining the dynamic objects in the urban traffic scene, and mapping the dynamic objects in the urban traffic scene to the coordinate system of the 3D point cloud data, thereby completing the urban traffic scene extraction. This method can effectively extract static elements and dynamic objects in urban traffic scenes.

Description

Technical Field

[0001] The present invention relates to a method, device, system and storage medium for extracting urban traffic scenes, and belongs to the technical field of intelligent interpretation. Background Art

[0002] With the continuous development of information technology and the continuous improvement of urban informatization applications, smart city construction has emerged. Intelligently sensing urban traffic systems are one of the most critical infrastructure components of smart cities. They can intelligently calculate information such as traffic flow, congestion, and vehicle speed, assisting traffic police in rapid response and effective handling, and promoting better urban planning and traffic operations. Therefore, research is needed to extract and model intelligent urban traffic scenarios.

[0003] Urban traffic scenes are mainly composed of main and secondary roads and their intersections. Due to the presence of a large number of static elements and dynamic targets, the complexity of their extraction is a difficulty in the perception of urban traffic systems.

[0004] Traditional urban traffic scene extraction methods mostly rely on manually designed features, which are limited by point cloud quality and classifier parameters, and face challenges such as poor results, low efficiency and large memory usage. Summary of the Invention

[0005] The purpose of the present invention is to provide a method, device, system and storage medium for extracting urban traffic scenes, which can effectively extract static elements and dynamic targets in urban traffic scenes.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a method for extracting urban traffic scenes, comprising:

[0008] Obtain 3D point cloud data and image data of urban traffic scenes;

[0009] Based on the 3D point cloud data, extracting static elements of the urban traffic scene using the trained KPConv network, obtaining the static elements of the urban traffic scene, and parameterizing the static elements of the urban traffic scene;

[0010] Based on the image data, the trained YOLOv7-W6 network is used to extract dynamic targets in urban traffic scenes, obtain dynamic targets in urban traffic scenes, and map the dynamic targets in urban traffic scenes to the coordinate system of the 3D point cloud data to complete the urban traffic scene extraction.

[0011] In combination with the first aspect, further, based on the 3D point cloud data, the trained KPConv network is used to extract static elements of the urban traffic scene, and obtaining the static elements of the urban traffic scene includes:

[0012] Preprocessing the 3D point cloud data to obtain preprocessed 3D point cloud data;

[0013] Vectorize the pre-processed 3D point cloud data to obtain 3D point cloud vectors;

[0014] Input the 3D point cloud vector into the trained KPConv network for semantic segmentation to obtain the 3D point cloud category code, and based on the category code, post-process the 3D point cloud to obtain the static elements of the urban traffic scene;

[0015] The category codes include background, trees and man-made rods.

[0016] In combination with the first aspect, further, preprocessing the 3D point cloud data includes:

[0017] Based on statistical methods, outlier removal is performed on the 3D point cloud data;

[0018] Based on the local minimum elevation method, removing ground points from the 3D point cloud data;

[0019] The outlier removal from the 3D point cloud data based on a statistical method includes:

[0020] Finding neighboring points for each point in the 3D point cloud data based on a preset number of neighboring points and a preset standard deviation ratio;

[0021] For each point in the 3D point cloud data, calculate the average value and standard deviation of the distance from the point to each of its corresponding neighboring points, and set a first threshold based on the average value and standard deviation;

[0022] For each point in the 3D point cloud data, determine the neighboring points corresponding to the point whose distance to the point is greater than a first threshold as outliers, and remove the outliers;

[0023] Based on the local minimum elevation method, removing ground points from the 3D point cloud data includes:

[0024] Correcting the coordinate system of the 3D point cloud data so that the Z axis of the coordinate system of the 3D point cloud data is perpendicular to the ground, and using the Z axis coordinate of the 3D point cloud data as the elevation;

[0025] Gridding the 3D point cloud data, and performing elevation filtering on the 3D point cloud data based on the grid to eliminate outliers;

[0026] The elevation of the lowest point in each grid is selected as the local minimum elevation of each grid, and the points whose elevation difference with the local minimum elevation is less than or equal to the second threshold are determined as ground points, and the ground points are removed.

[0027] In combination with the first aspect, further, vectorizing the preprocessed 3D point cloud data includes: performing 6-dimensional vectorization on the preprocessed 3D point cloud data based on the X-axis coordinate, Y-axis coordinate, Z-axis coordinate, red color value, green color value and blue color value corresponding to each point in the preprocessed 3D point cloud data.

[0028] In combination with the first aspect, further, based on the category coding, post-processing of the 3D point cloud includes: denoising, category correction and horizontal clustering merging of the 3D point cloud with category coding as trees and artificial poles, further dividing the 3D point cloud with category coding as artificial poles into street lights, surveillance, road signs or telephone poles, and based on an adaptive growth strategy, separating individual tree trunks of the 3D point cloud with category coding as trees.

[0029] In combination with the first aspect, further parameterizing the static elements of the urban traffic scene includes:

[0030] Based on the principal component analysis method, the shape of the 3D point cloud coded as trees and artificial poles is judged to obtain the 3D point cloud shape;

[0031] Based on the shape of the 3D point cloud, segmenting the 3D point cloud classified as trees and artificial rods into rod trunks and rod components to obtain the rod trunks and rod components;

[0032] Calculating the height and radius of the rod-shaped trunk and parameterizing the rod-shaped trunk;

[0033] Calculating the length, width, height and maximum radius of the rod-shaped component, and parameterizing the rod-shaped component;

[0034] Calculating the radius of the rod-shaped trunk includes: obtaining the radius of the rod-shaped trunk layer by layer in the XOY coordinate system, and calculating the average value of the radii of each layer of the rod-shaped trunk in the XOY coordinate system as the radius of the rod-shaped trunk;

[0035] Calculating the length, width, and height of the rod-shaped component includes: calculating a covariance matrix of the rod-shaped component, and multiplying the covariance matrix of the rod-shaped component by the coordinates of the 3D point cloud corresponding to the rod-shaped component to obtain the spacing of the rod-shaped component in the X-axis, Y-axis, and Z-axis directions as the length, width, and height of the rod-shaped component;

[0036] Calculating the maximum radius of the rod-shaped component includes: obtaining the radius of the rod-shaped component in the XOY coordinate system layer by layer, and calculating the maximum value of the radius of each layer of the rod-shaped component in the XOY coordinate system as the maximum radius of the rod-shaped component.

[0037] In combination with the first aspect, further, based on the principal component analysis method, shape judgment is performed on the 3D point cloud coded as trees and artificial poles. Obtaining the shape of the 3D point cloud includes:

[0038] Compute the geometric center, covariance matrix, and matrix eigenvectors of 3D point clouds whose categories are trees and man-made poles;

[0039] Based on the geometric center, the covariance matrix, and the matrix eigenvectors, eigenvector decomposition is performed on the 3D point cloud classified as trees and artificial rods to obtain the shape of the 3D point cloud;

[0040] The 3D point cloud shapes include linear, planar and three-dimensional shapes.

[0041] In combination with the first aspect, further, based on the image data, using the trained YOLOv7-W6 network to extract dynamic targets in urban traffic scenes, obtaining dynamic targets in urban traffic scenes includes:

[0042] The image data is input into the trained YOLOv7-W6 network for frame extraction recognition to obtain dynamic targets in urban traffic scenes;

[0043] The dynamic targets in the urban traffic scene include pedestrians, bicycles, motorcycles, cars, trucks and buses.

[0044] In combination with the first aspect, further, mapping the dynamic target of the urban traffic scene to the coordinate system of the 3D point cloud data includes:

[0045] Obtaining the center pixel coordinates of the dynamic target in the urban traffic scene;

[0046] The central pixel coordinates of the dynamic target in the urban traffic scene are mapped to the coordinate system of the 3D point cloud data.

[0047] In combination with the first aspect, further, the KPConv network is trained based on pre-acquired 3D point cloud training data in combination with a cross-validation method; the YOLOv7-W6 network is trained based on pre-acquired image training data in combination with a label assignment strategy and a cross-validation method.

[0048] In a second aspect, the present invention provides an urban traffic scene extraction device, comprising:

[0049] Data acquisition module, used to obtain 3D point cloud data and image data of urban traffic scenes;

[0050] A static element extraction module is used to extract static elements of urban traffic scenes based on the 3D point cloud data using a trained KPConv network, obtain static elements of urban traffic scenes, and parameterize the static elements of urban traffic scenes;

[0051] The dynamic target extraction module is used to extract dynamic targets in urban traffic scenes based on the image data using the trained YOLOv7-W6 network, obtain dynamic targets in urban traffic scenes, and map the dynamic targets in urban traffic scenes to the coordinate system of the 3D point cloud data to complete the extraction of urban traffic scenes.

[0052] In a third aspect, the present invention provides a computer system comprising:

[0053] Storage medium: used to store computer programs;

[0054] Processor: used to execute the computer program to implement the steps of the urban traffic scene extraction method described in any one of the first aspects.

[0055] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the urban traffic scene extraction method described in any one of the first aspects.

[0056] Fifthly, the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the urban traffic scene extraction method described in any one of the first aspects.

[0057] Compared with the prior art, the present invention has the following beneficial effects:

[0058] The urban traffic scene extraction method provided by this invention utilizes a trained KPConv network to extract static elements of urban traffic scenes based on 3D point cloud data, and a trained YOLOv7-W6 network to extract dynamic objects in urban traffic scenes based on image data. This method effectively extracts both static and dynamic objects from urban traffic scenes by combining the advantages of 3D point cloud and image data. Parameterizing static elements in urban traffic scenes provides a basis for constructing urban traffic scene models. Mapping dynamic objects in urban traffic scenes to the coordinate system of the 3D point cloud data provides a motion display interface for these dynamic objects.

[0059] Removing outliers and ground points from 3D point cloud data can accurately utilize the effective information of 3D point cloud data to improve the accuracy of static feature extraction.

[0060] When post-processing 3D point clouds, denoising and category correction can solve the problem of background being misidentified as trees or artificial poles, and the problem of tree perigees being misidentified as artificial poles; horizontal clustering and merging can solve the problem of monitored cantilever being truncated; and the adaptive growth strategy can solve the problem of the crowns of adjacent trees sticking to each other. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 This is a flow chart of the urban traffic scene extraction method provided by an embodiment of the present invention;

[0062] Figure 2 This is a flowchart of extracting static elements from urban traffic scenes provided by an embodiment of the present invention;

[0063] Figure 3 This is a flow chart of parameterization of static elements of urban traffic scenes provided by an embodiment of the present invention;

[0064] Figure 4 This is a flow chart of dynamic target extraction and mapping in urban traffic scenarios provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0065] The technical solution of the present application will be further described in detail below in conjunction with specific implementation methods.

[0066] The embodiments of the present application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application and are not to be construed as limiting the present application. The embodiments of the present application and the technical features in the embodiments may be combined with each other unless there is a conflict.

[0067] Example 1:

[0068] Figure 1 This is a flow chart of a method for extracting urban traffic scenes provided by this embodiment. This flow chart only shows the logical sequence of the method of this embodiment. Different methods can be used without conflict. Figure 1 The steps shown or described are accomplished in the order shown.

[0069] The urban traffic scene extraction method provided in this embodiment can be applied to a terminal and can be executed by an urban traffic scene extraction device, which can be implemented by software and / or hardware. The device can be integrated into a terminal, for example: any tablet computer or computer device with communication function.

[0070] See also Figure 1 The urban traffic scene extraction method of this embodiment specifically includes the following steps:

[0071] Step 1: Obtain 3D point cloud data and image data of urban traffic scenes;

[0072] In this embodiment, the 3D point cloud data of the urban traffic scene is obtained based on the laser radar, and the image data of the urban traffic scene is obtained based on the camera.

[0073] Step 2: Based on the 3D point cloud data, the trained KPConv network is used to extract the static elements of the urban traffic scene, obtain the static elements of the urban traffic scene, and parameterize the static elements of the urban traffic scene;

[0074] In this embodiment, Figure 2 As shown in the figure, based on 3D point cloud data, the trained KPConv network is used to extract static elements of urban traffic scenes. The specific steps of obtaining static elements of urban traffic scenes include:

[0075] Step 1: Preprocess the 3D point cloud data to obtain preprocessed 3D point cloud data;

[0076] In this embodiment, preprocessing the 3D point cloud data specifically includes the following steps:

[0077] Step I: Remove outliers from 3D point cloud data based on statistical methods;

[0078] In this embodiment, based on statistical methods, outlier removal of 3D point cloud data specifically includes the following steps:

[0079] Step 1: Find the neighboring points of each point in the 3D point cloud data based on the preset number of neighboring points and the preset standard deviation ratio;

[0080] Step 2: For each point in the 3D point cloud data, calculate the average and standard deviation of the distances from it to its corresponding neighboring points, and set a first threshold based on the average and standard deviation;

[0081] Step ③: For each point in the 3D point cloud data, the neighboring points corresponding to the point whose distance to the point is greater than a first threshold are determined as outliers, and the outliers are removed.

[0082] Specifically, find the neighboring points of each point in the 3D point cloud data, traverse each point in the 3D point cloud data, calculate the distance from the neighboring point corresponding to the current point to the current point, and find the average value , and find the standard deviation , if the distance from a neighboring point corresponding to the current point to the current point is greater than , then the neighboring point is determined as an outlier and removed, where Indicates the preset coefficient.

[0083] Step II: Based on the local minimum elevation method, remove ground points from the 3D point cloud data.

[0084] In this embodiment, the ground point removal process for 3D point cloud data based on the local minimum elevation method specifically includes the following steps:

[0085] Step 1: Correct the coordinate system of the 3D point cloud data so that the Z axis of the coordinate system is perpendicular to the ground, and use the Z axis coordinate of the 3D point cloud data as the elevation;

[0086] Step 2: Grid the 3D point cloud data and perform elevation filtering on the 3D point cloud data based on the grid to eliminate outliers;

[0087] Step 3: Select the elevation of the lowest point in each grid as the local minimum elevation of each grid, determine the points whose elevation difference between the elevation in each grid and the local minimum elevation is less than or equal to the second threshold as ground points, and remove the ground points.

[0088] Specifically, the elevation of the lowest point in each grid is selected as the local minimum elevation of each grid. , the difference between the elevation of each grid and the local minimum elevation is less than or equal to the second threshold The points are determined to be ground points and removed.

[0089] In this embodiment, the grid division is based on two-dimensional grid division from a bird's-eye view, and both the elevation filtering and the local minimum elevation are based on the grid as a basic unit.

[0090] Step 2: Vectorize the pre-processed 3D point cloud data to obtain 3D point cloud vectors;

[0091] In this embodiment, vectorizing the preprocessed 3D point cloud data specifically includes: performing 6-dimensional vectorization on the preprocessed 3D point cloud data based on the X-axis coordinate, Y-axis coordinate, Z-axis coordinate, red color value, green color value and blue color value corresponding to each point in the preprocessed 3D point cloud data.

[0092] Specifically, the 3D point cloud data is converted into a 3D point cloud vector with N rows and 6 columns, where N represents the total number of 3D point clouds, and the 6 columns correspond to the X-axis coordinate, Y-axis coordinate, Z-axis coordinate, red color value, green color value, and blue color value of the 3D point cloud, respectively.

[0093] Step 3: Input the 3D point cloud vector into the trained KPConv network for semantic segmentation, obtain the 3D point cloud category code, and post-process the 3D point cloud based on the category code to obtain the static elements of the urban traffic scene.

[0094] In this embodiment, the KPConv network is trained based on pre-acquired 3D point cloud training data in combination with a cross-validation method.

[0095] The 3D point cloud vector is input into the trained KPConv network for semantic segmentation to obtain the 3D point cloud category encoding, where the category encoding includes background, trees, and artificial rods.

[0096] Specifically, 21 collected 3D point clouds of traffic scenes are selected and input into the KPConv network for training. During the training process, 14 collected 3D point clouds of traffic scenes are selected for cross-validation to obtain the trained KPConv network.

[0097] Use the trained KPConv network to perform semantic segmentation on the traffic scene of Bayi Road-Guangba Road, and assign corresponding category codes to each point.

[0098] In this embodiment, post-processing of the 3D point cloud based on category coding specifically includes: denoising, category correction and horizontal clustering merging of the 3D point cloud with category coding as trees and artificial poles, further dividing the 3D point cloud with category coding as artificial poles into street lights, surveillance, road signs or telephone poles, and based on the adaptive growth strategy, separating the individual tree trunks of the 3D point cloud with category coding as trees.

[0099] In this embodiment, denoising and category correction specifically include: point number threshold setting, height threshold setting, noise removal and misclassification identification, tree extraction and adhesion processing, and distance threshold matching correction.

[0100] Specifically, the point count threshold is used to identify small clusters, i.e., noise points; the height threshold is used to identify floating noise points, i.e., points whose height does not match the height of the ground or typical objects; noise removal and misclassification recognition are based on the point cloud threshold and height threshold to remove floating points identified as noise, and identify clusters that are misclassified as artificial rods according to the point cloud threshold; tree extraction and adhesion processing processes the adhered trees into multiple trunks by cutting off the lower half of the tree cluster as the trunk.

[0101] In this embodiment, horizontal cluster merging specifically includes: height threshold screening, orientation calculation and horizontal rod identification, horizontal distance projection calculation, and cluster merging.

[0102] Specifically, the height threshold screening is performed by calculating the Z-axis coordinate of the lowest point of each artificial rod cluster to screen out horizontal cantilevers with higher heights; the orientation calculation and horizontal rod identification are determined by calculating the direction vector when the Z-axis component is close to 0; the horizontal distance projection calculation and cluster merging are performed by merging horizontal rods based on the horizontal projection distance.

[0103] In this embodiment, the pseudo code of the algorithm of the adaptive growth strategy is shown in Algorithm 1.

[0104]

[0105] In this embodiment, Figure 3 As shown in Figure 1, parameterizing the static elements of urban traffic scenes specifically includes the following steps:

[0106] Step 1: Based on the principal component analysis method, the 3D point cloud coded as trees and artificial poles is subjected to shape judgment to obtain the 3D point cloud shape;

[0107] In this embodiment, based on the principal component analysis (PCA) method, shape determination is performed on 3D point clouds coded as trees and artificial poles. The steps of obtaining the shape of the 3D point clouds are as follows:

[0108] Step 1: Calculate the geometric center, covariance matrix, and matrix eigenvectors of the 3D point cloud whose categories are trees and artificial poles;

[0109] In this embodiment, the geometric center of the 3D point cloud is obtained by summing and averaging the coordinates of the 3D point cloud.

[0110] The covariance matrix and matrix eigenvector of the 3D point cloud are:

[0111] ;

[0112] in, represents the covariance matrix, represents the matrix eigenvector, Represents the matrix eigenvalue.

[0113] Step 2: Based on the geometric center, covariance matrix, and matrix eigenvectors, perform eigenvector decomposition on the 3D point cloud coded as trees and artificial poles to obtain the 3D point cloud shape.

[0114] In this embodiment, the 3D point cloud shapes include linear, planar, and solid shapes.

[0115] Specifically, the shape discrimination parameters of 3D point cloud are:

[0116] ;

[0117] in, 、 、 Indicates the possibility that the 3D point cloud shape is linear, planar, or solid.

[0118] Step 2: Based on the shape of the 3D point cloud, segment the 3D point cloud classified as trees and artificial rods into rod trunks and rod parts to obtain the rod trunks and rod parts;

[0119] In this embodiment, the linear 3D point cloud is taken as the trunk of the rod-shaped object, and the 3D point clouds of other shapes are taken as the rod-shaped object parts.

[0120] Specifically, points with denser distribution and relatively lower positions are identified as tree trunk points. Based on the characteristics that the metal poles of road signs and traffic lights vary in thickness and have horizontal connecting poles, the vertical poles of road signs and traffic lights are extracted.

[0121] In this embodiment, based on the characteristics of dense distribution and relatively low position of 3D point cloud, the algorithm for extracting tree trunk points is shown in Algorithm 2.

[0122]

[0123] Step 3: Calculate the height and radius of the rod-shaped trunk and parameterize the rod-shaped trunk;

[0124] In this embodiment, the step of calculating the height of the rod trunk is the same as the step of calculating the height of the rod component.

[0125] Calculating the radius of the rod-shaped trunk specifically includes: obtaining the radius of the rod-shaped trunk in the XOY coordinate system layer by layer, and calculating the average value of the radii of each layer of the rod-shaped trunk in the XOY coordinate system as the radius of the rod-shaped trunk.

[0126] Step 4: Calculate the length, width, height, and maximum radius of the rod-shaped component and parameterize the rod-shaped component.

[0127] In this embodiment, calculating the length, width, and height of the rod-shaped component specifically includes: calculating the covariance matrix of the rod-shaped component, and multiplying the covariance matrix of the rod-shaped component with the coordinates of the 3D point cloud corresponding to the rod-shaped component to obtain the spacing of the rod-shaped component in the X-axis, Y-axis, and Z-axis directions as the length, width, and height of the rod-shaped component.

[0128] Calculating the maximum radius of the rod-shaped component specifically includes: obtaining the radius of the rod-shaped component in the XOY coordinate system layer by layer, and calculating the maximum value of the radius of each layer of the rod-shaped component in the XOY coordinate system as the maximum radius of the rod-shaped component.

[0129] Step 3: Based on the image data, use the trained YOLOv7-W6 network to extract dynamic targets in urban traffic scenes, obtain dynamic targets in urban traffic scenes, and map the dynamic targets in urban traffic scenes to the coordinate system of the 3D point cloud data to complete the urban traffic scene extraction.

[0130] In this embodiment, the YOLOv7-W6 network is trained based on pre-acquired image training data in combination with a label assignment strategy and a cross-validation method.

[0131] The image data is input into the trained YOLOv7-W6 network for frame extraction recognition to obtain dynamic targets in urban traffic scenes, where dynamic targets in urban traffic scenes include pedestrians, bicycles, motorcycles, cars, trucks and buses.

[0132] Specifically, such as Figure 4 As shown, dynamic objects in urban traffic scenes are first classified into six categories: pedestrians, bicycles, motorcycles, cars, trucks, and buses. Then, a YOLOv7-W6 network with an auxiliary head is trained using the training set from the COCO dataset, combining a label assignment strategy. Cross-validation is performed using the validation set from the COCO dataset during training. Image data captured by the camera attached to the Lingwei Titian-M1 lidar is then fed into the YOLOv7-W6 network for frame-by-frame recognition. The network outputs information such as confidence scores and class probabilities. Finally, the YOLOv7-W6 network outputs all detection bounding boxes for the current image or video frame, including class number, class confidence, bounding box center pixel coordinates, and bounding box width and height. The results are then inferred, and the average detection time, frames per second (FPS), non-maximum suppression (NMS) processing time, and total processing time for all images are evaluated.

[0133] In this embodiment, mapping the dynamic target of the urban traffic scene to the coordinate system of the 3D point cloud data specifically includes the following steps:

[0134] Step 1: Obtain the center pixel coordinates of the dynamic target in the urban traffic scene;

[0135] Step 2: Map the central pixel coordinates of the dynamic target in the urban traffic scene to the coordinate system of the 3D point cloud data.

[0136] In this embodiment, the road surface points in the urban traffic scene are approximately regarded as a plane, and the 3D coordinates of the dynamic target are Considered as a plane coordinate, the pixel coordinate on the image plane is marked as , the transformation relationship between two different coordinates can be constructed through the homography matrix. Based on the transformation relationship, the central pixel coordinates of the dynamic target in the urban traffic scene are mapped to the coordinate system of the 3D point cloud data.

[0137] Example 2:

[0138] This embodiment provides an urban traffic scene extraction device, including:

[0139] Data acquisition module, used to obtain 3D point cloud data and image data of urban traffic scenes;

[0140] The static feature extraction module is used to extract static features of urban traffic scenes based on 3D point cloud data using the trained KPConv network, obtain static features of urban traffic scenes, and parameterize them;

[0141] The dynamic target extraction module is used to extract dynamic targets in urban traffic scenes based on image data using the trained YOLOv7-W6 network, obtain dynamic targets in urban traffic scenes, and map them to the coordinate system of the 3D point cloud data to complete urban traffic scene extraction.

[0142] The urban traffic scene extraction device provided in the embodiment of the present application can execute the urban traffic scene extraction method provided in Example 1 of the present application, and has the corresponding functional modules and beneficial effects of the execution method.

[0143] Example 3:

[0144] This embodiment provides a computer system, including:

[0145] Storage medium: used to store computer programs;

[0146] Processor: used to execute computer programs to implement the steps of the urban traffic scene extraction method in Example 1.

[0147] Example 4:

[0148] This embodiment provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the urban traffic scene extraction method in Embodiment 1 are implemented.

[0149] Example 5:

[0150] This embodiment provides a computer program product, including a computer program, which implements the steps of the urban traffic scene extraction method in Example 1 when executed by a processor.

[0151] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0152] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0153] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0154] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0155] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for extracting urban traffic scenes, characterized in that: include: Obtain 3D point cloud data and image data of urban traffic scenes; Based on 3D point cloud data, the trained KPConv network is used to extract and parameterize the static elements of urban traffic scenes. Based on image data, the trained YOLOv7-W6 network is used to extract dynamic targets in urban traffic scenes, obtain dynamic targets in urban traffic scenes, and map them to the coordinate system of 3D point cloud data to complete urban traffic scene extraction; Based on 3D point cloud data, the trained KPConv network is used to extract static elements of urban traffic scenes. The static elements of urban traffic scenes include: Preprocessing the 3D point cloud data to obtain preprocessed 3D point cloud data; Vectorize the pre-processed 3D point cloud data to obtain 3D point cloud vectors; Input the 3D point cloud vector into the trained KPConv network for semantic segmentation to obtain the 3D point cloud category code. Based on the category code, the 3D point cloud is post-processed to obtain the static elements of the urban traffic scene. Among them, the category codes include background, trees, and artificial poles; Preprocessing of 3D point cloud data includes: Based on statistical methods, outliers are removed from 3D point cloud data; Based on the local minimum elevation method, ground points are removed from 3D point cloud data; Among them, based on statistical methods, outlier removal of 3D point cloud data includes: Find the neighboring points of each point in the 3D point cloud data based on the preset number of neighboring points and the preset standard deviation ratio; For each point in the 3D point cloud data, calculate the average and standard deviation of the distances from the point to each of its corresponding neighboring points, and set a first threshold based on the average and standard deviation; For each point in the 3D point cloud data, the points whose corresponding neighboring points are greater than a first threshold are determined as outliers, and the outliers are removed; Based on the local minimum elevation method, ground point removal of 3D point cloud data includes: Correct the coordinate system of the 3D point cloud data so that the Z axis of the coordinate system is perpendicular to the ground, and use the Z axis coordinate of the 3D point cloud data as the elevation; Grid the 3D point cloud data and perform elevation filtering on the 3D point cloud data based on the grid to eliminate outliers; The elevation of the lowest point in each grid is selected as the local minimum elevation of each grid, and the points whose elevation difference with the local minimum elevation is less than or equal to the second threshold are determined as ground points, and the ground points are removed.

2. The urban traffic scene extraction method according to claim 1, characterized in that: The vectorization of the preprocessed 3D point cloud data includes: performing 6-dimensional vectorization on the preprocessed 3D point cloud data based on the X-axis coordinate, Y-axis coordinate, Z-axis coordinate, red color value, green color value and blue color value corresponding to each point in the preprocessed 3D point cloud data.

3. The urban traffic scene extraction method according to claim 1, characterized in that: Based on the category coding, post-processing of the 3D point cloud includes: denoising, category correction and horizontal clustering merging of the 3D point cloud coded as trees and artificial poles, further dividing the 3D point cloud coded as artificial poles into street lights, surveillance, road signs or telephone poles, and based on an adaptive growth strategy, separating individual tree trunks of the 3D point cloud coded as trees.

4. The urban traffic scene extraction method according to claim 1, characterized in that: Parameterizing the static elements of the urban traffic scene includes: Based on the principal component analysis method, the shape of the 3D point cloud coded as trees and artificial poles is judged to obtain the 3D point cloud shape; Based on the shape of the 3D point cloud, segmenting the 3D point cloud classified as trees and artificial rods into rod trunks and rod components to obtain the rod trunks and rod components; Calculating the height and radius of the rod-shaped trunk and parameterizing the rod-shaped trunk; Calculating the length, width, height and maximum radius of the rod-shaped component, and parameterizing the rod-shaped component; Calculating the radius of the rod-shaped trunk includes: obtaining the radius of the rod-shaped trunk layer by layer in the XOY coordinate system, and calculating the average value of the radii of each layer of the rod-shaped trunk in the XOY coordinate system as the radius of the rod-shaped trunk; Calculating the length, width, and height of the rod-shaped component includes: calculating a covariance matrix of the rod-shaped component, and multiplying the covariance matrix of the rod-shaped component by the coordinates of the 3D point cloud corresponding to the rod-shaped component to obtain the spacing of the rod-shaped component in the X-axis, Y-axis, and Z-axis directions as the length, width, and height of the rod-shaped component; Calculating the maximum radius of the rod-shaped component includes: obtaining the radius of the rod-shaped component in the XOY coordinate system layer by layer, and calculating the maximum value of the radius of each layer of the rod-shaped component in the XOY coordinate system as the maximum radius of the rod-shaped component.

5. The urban traffic scene extraction method according to claim 4, characterized in that: Based on the principal component analysis method, the shape of the 3D point cloud coded as trees and artificial poles is judged. The shape of the 3D point cloud is obtained including: Compute the geometric center, covariance matrix, and matrix eigenvectors of 3D point clouds whose categories are trees and man-made poles; Based on the geometric center, the covariance matrix, and the matrix eigenvectors, eigenvector decomposition is performed on the 3D point cloud classified as trees and artificial rods to obtain the shape of the 3D point cloud; The 3D point cloud shapes include linear, planar and three-dimensional shapes.

6. The urban traffic scene extraction method according to claim 1, characterized in that: Based on the image data, the trained YOLOv7-W6 network is used to extract dynamic targets in urban traffic scenes. The dynamic targets in urban traffic scenes are obtained by: The image data is input into the trained YOLOv7-W6 network for frame extraction recognition to obtain dynamic targets in urban traffic scenes; The dynamic targets in the urban traffic scene include pedestrians, bicycles, motorcycles, cars, trucks and buses.

7. The urban traffic scene extraction method according to claim 1, characterized in that: Mapping the urban traffic scene dynamic target to the coordinate system of the 3D point cloud data includes: Obtaining the center pixel coordinates of the dynamic target in the urban traffic scene; The central pixel coordinates of the dynamic target in the urban traffic scene are mapped to the coordinate system of the 3D point cloud data.

8. The urban traffic scene extraction method according to claim 1, characterized in that: The KPConv network is trained based on pre-acquired 3D point cloud training data in combination with a cross-validation method; the YOLOv7-W6 network is trained based on pre-acquired image training data in combination with a label assignment strategy and a cross-validation method.

Images