Urban traffic scene extraction method, device and system and storage medium

Abstract

The invention discloses an urban traffic scene extraction method, device and system and a storage medium, and belongs to the technical field of intelligent interpretation, and the method comprises the steps: obtaining 3D point cloud data and image data of an urban traffic scene; based on the 3D point cloud data, utilizing a trained KPConv network to extract static elements of an urban traffic scene, acquiring the static elements of the urban traffic scene, and parameterizing the static elements of the urban traffic scene; and based on the image data, utilizing a trained YOLOv7-W6 network to carry out urban traffic scene dynamic target extraction to obtain an urban traffic scene dynamic target, and mapping the urban traffic scene dynamic target to a coordinate system where the 3D point cloud data is located to complete urban traffic scene extraction. The method can realize effective extraction of static elements and dynamic targets of the urban traffic scene.

Description

Technical Field

[0001] The invention relates to a method, device, system and storage medium for extracting urban traffic scenes, and belongs to the technical field of intelligent interpretation. Background Art

[0002] With the continuous development of information technology, the level of urban information application has been continuously improved, and the construction of smart cities has emerged. The intelligent perception urban traffic system is one of the most critical infrastructure components of smart cities. It can intelligently calculate information such as traffic flow, congestion and vehicle speed, assist traffic police to respond quickly and effectively, and promote better urban planning and traffic operations. Therefore, it is necessary to carry out research on intelligent urban traffic scene extraction and model construction.

[0003] Urban traffic scenes are mainly composed of main and secondary roads and their intersections. Due to the existence of a large number of static elements and dynamic targets, the complexity of their extraction is a difficulty in the perception of urban traffic systems.

[0004] Traditional urban traffic scene extraction methods mostly rely on manually designed features, which are limited by point cloud quality and classifier parameters, and face the challenges of poor results, low efficiency and large memory usage. Summary of the invention

[0005] The purpose of the present invention is to provide a method, device, system and storage medium for extracting urban traffic scenes, which can effectively extract static elements and dynamic targets of urban traffic scenes.

[0006] In order to achieve the above object, the present invention provides the following technical solutions: In a first aspect, the present invention provides a method for extracting urban traffic scenes, comprising: Obtain 3D point cloud data and image data of urban traffic scenes; Based on the 3D point cloud data, using the trained KPConv network to extract static elements of the urban traffic scene, obtain the static elements of the urban traffic scene, and parameterize the static elements of the urban traffic scene; Based on the image data, the trained YOLOv7-W6 network is used to extract dynamic targets in urban traffic scenes, obtain dynamic targets in urban traffic scenes, and map the dynamic targets in urban traffic scenes to the coordinate system where the 3D point cloud data is located, thereby completing the extraction of urban traffic scenes.

[0007] In combination with the first aspect, further, based on the 3D point cloud data, the trained KPConv network is used to extract static elements of the urban traffic scene, and obtaining the static elements of the urban traffic scene includes: Preprocessing the 3D point cloud data to obtain preprocessed 3D point cloud data; Vectorize the preprocessed 3D point cloud data to obtain 3D point cloud vectors; Input the 3D point cloud vector into the trained KPConv network for semantic segmentation, obtain the 3D point cloud category code, and post-process the 3D point cloud based on the category code to obtain the static elements of the urban traffic scene; The category codes include background, trees and artificial rods.

[0008] In combination with the first aspect, further, preprocessing the 3D point cloud data includes: Based on statistical methods, outlier points are removed from the 3D point cloud data; Based on the local minimum elevation method, removing ground points from the 3D point cloud data; Wherein, based on the statistical method, removing outliers from the 3D point cloud data includes: Finding neighboring points of each point in the 3D point cloud data based on a preset number of neighboring points and a preset standard deviation ratio; For each point in the 3D point cloud data, calculate the average value and standard deviation of the distance from the point to each of its corresponding neighboring points, and set a first threshold based on the average value and standard deviation; For each point in the 3D point cloud data, determine the neighboring points corresponding to the point whose distance to the point is greater than a first threshold as outliers, and remove the outliers; Based on the local minimum elevation method, removing ground points from the 3D point cloud data includes: Correcting the coordinate system of the 3D point cloud data so that the Z axis of the coordinate system of the 3D point cloud data is perpendicular to the ground, and using the Z axis coordinate of the 3D point cloud data as the elevation; Gridding the 3D point cloud data, and performing elevation filtering on the 3D point cloud data based on the grid to eliminate outliers; The elevation of the lowest point in each grid is selected as the local minimum elevation of each grid, and the points whose difference between the elevation in each grid and the local minimum elevation is less than or equal to the second threshold are determined as ground points, and the ground points are removed.

[0009] In combination with the first aspect, further, vectorizing the preprocessed 3D point cloud data includes: performing 6-dimensional vectorization on the preprocessed 3D point cloud data based on the X-axis coordinate, Y-axis coordinate, Z-axis coordinate, red color value, green color value and blue color value corresponding to each point in the preprocessed 3D point cloud data.

[0010] In combination with the first aspect, further, based on the category coding, post-processing of the 3D point cloud includes: denoising, category correction and horizontal clustering merging of 3D point clouds with category coding as trees and artificial poles, further dividing the 3D point cloud with category coding as artificial poles into street lights, surveillance, road signs or telephone poles, and based on an adaptive growth strategy, separating individual tree trunks of the 3D point cloud with category coding as trees.

[0011] In combination with the first aspect, further, parameterizing the static elements of the urban traffic scene includes: Based on the principal component analysis method, the shape of the 3D point cloud coded as trees and artificial rods is judged to obtain the shape of the 3D point cloud; Based on the shape of the 3D point cloud, segmenting the 3D point cloud with the category code of trees and artificial rods into rod trunks and rod parts to obtain the rod trunks and rod parts; Calculating the height and radius of the main trunk of the rod-shaped object, and parameterizing the main trunk of the rod-shaped object; Calculating the length, width, height and maximum radius of the rod-shaped component, and parameterizing the rod-shaped component; Wherein, calculating the radius of the rod-shaped trunk includes: obtaining the radius of the rod-shaped trunk in the XOY coordinate system layer by layer, and calculating the average value of the radii of each layer of the rod-shaped trunk in the XOY coordinate system as the radius of the rod-shaped trunk; Calculating the length, width, and height of the rod-shaped component includes: calculating a covariance matrix of the rod-shaped component, and multiplying the covariance matrix of the rod-shaped component by the coordinates of the 3D point cloud corresponding to the rod-shaped component to obtain the spacing of the rod-shaped component in the X-axis, Y-axis, and Z-axis directions as the length, width, and height of the rod-shaped component; Calculating the maximum radius of the rod-shaped component includes: obtaining the radius of the rod-shaped component in the XOY coordinate system layer by layer, and calculating the maximum value of the radius of each layer of the rod-shaped component in the XOY coordinate system as the maximum radius of the rod-shaped component.

[0012] In combination with the first aspect, further, based on the principal component analysis method, the 3D point cloud with category coding as trees and artificial rods is subjected to shape judgment, and the shape of the 3D point cloud is obtained, including: Compute the geometric center, covariance matrix, and matrix eigenvectors of 3D point clouds whose categories are encoded as trees and artificial poles; Based on the geometric center, the covariance matrix and the matrix eigenvector, the 3D point cloud with the category code of trees and artificial rods is decomposed by eigenvector to obtain the shape of the 3D point cloud; The 3D point cloud shapes include linear, planar and solid shapes.

[0013] In combination with the first aspect, further, based on the image data, the trained YOLOv7-W6 network is used to extract dynamic targets in urban traffic scenes, and obtaining dynamic targets in urban traffic scenes includes: The image data is input into the trained YOLOv7-W6 network for frame extraction recognition to obtain dynamic targets in urban traffic scenes; The dynamic targets in the urban traffic scene include pedestrians, bicycles, motorcycles, cars, trucks and buses.

[0014] In combination with the first aspect, further, mapping the urban traffic scene dynamic target to the coordinate system where the 3D point cloud data is located includes: Obtaining the central pixel coordinates of the dynamic target in the urban traffic scene; The central pixel coordinates of the dynamic target in the urban traffic scene are mapped to the coordinate system where the 3D point cloud data is located.

[0015] In combination with the first aspect, further, the KPConv network is trained based on pre-acquired 3D point cloud training data in combination with a cross-validation method; the YOLOv7-W6 network is trained based on pre-acquired image training data in combination with a label allocation strategy and a cross-validation method.

[0016] In a second aspect, the present invention provides an urban traffic scene extraction device, comprising: Data acquisition module, used to obtain 3D point cloud data and image data of urban traffic scenes; A static element extraction module is used to extract static elements of urban traffic scenes based on the 3D point cloud data using a trained KPConv network, obtain static elements of urban traffic scenes, and parameterize the static elements of urban traffic scenes; The dynamic target extraction module is used to extract dynamic targets in urban traffic scenes based on the image data using the trained YOLOv7-W6 network, obtain dynamic targets in urban traffic scenes, and map the dynamic targets in urban traffic scenes to the coordinate system where the 3D point cloud data is located, thereby completing the extraction of urban traffic scenes.

[0017] In a third aspect, the present invention provides a computer system, comprising: Storage medium: used to store computer programs; Processor: used to execute the computer program to implement the steps of the urban traffic scene extraction method described in any one of the first aspects.

[0018] In a fourth aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the urban traffic scene extraction method described in any one of the first aspects.

[0019] Fifthly, the present invention provides a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the urban traffic scene extraction method described in any one of the first aspects.

[0020] Compared with the prior art, the present invention has the following beneficial effects: The urban traffic scene extraction method provided by the present invention is based on 3D point cloud data and uses a trained KPConv network to extract static elements of urban traffic scenes. Based on image data, it uses a trained YOLOv7-W6 network to extract dynamic targets of urban traffic scenes. It can combine the advantages of 3D point cloud data and image data to achieve effective extraction of static elements and dynamic targets of urban traffic scenes. Parameterizing the static elements of urban traffic scenes can provide a basis for building urban traffic scene models. Mapping the dynamic targets of urban traffic scenes to the coordinate system where the 3D point cloud data is located can provide a motion display interface for dynamic targets.

[0021] Removing outliers and ground points from 3D point cloud data can accurately utilize the effective information of 3D point cloud data to improve the accuracy of static feature extraction.

[0022] When post-processing the 3D point cloud, denoising and category correction can solve the problem of the background being misidentified as trees or artificial poles, and the problem of the near-ground of trees being misidentified as artificial poles; horizontal clustering and merging can solve the problem of the monitored cantilever being truncated; and the adaptive growth strategy can solve the problem of the crowns of adjacent trees sticking to each other. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a flow chart of a method for extracting urban traffic scenes provided by an embodiment of the present invention; Figure 2 is a flow chart of extracting static elements from urban traffic scenes provided by an embodiment of the present invention; Figure 3 It is a parameterization flow chart of static elements of urban traffic scenes provided by an embodiment of the present invention; Figure 4 It is a flow chart of dynamic target extraction and mapping in urban traffic scenes provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The technical solution of the present application is further described in detail below in conjunction with specific implementation methods.

[0025] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements with the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and should not be construed as limitations on the present application. In the absence of conflict, the embodiments of the present application and the technical features in the embodiments may be combined with each other.

[0026] Embodiment 1: Figure 1 is a flow chart of a method for extracting urban traffic scenes provided by this embodiment. This flow chart only shows the logical sequence of the method of this embodiment. Different methods can be used without conflict. Figure 1 The steps shown or described are accomplished in the order shown.

[0027] The urban traffic scene extraction method provided in this embodiment can be applied to a terminal and can be executed by an urban traffic scene extraction device, which can be implemented by software and / or hardware, and the device can be integrated in a terminal, for example: any tablet computer or computer device with communication function.

[0028] See also Figure 1 The urban traffic scene extraction method of this embodiment specifically includes the following steps: Step 1: Obtain 3D point cloud data and image data of urban traffic scenes; In this embodiment, the 3D point cloud data of the urban traffic scene is acquired based on the laser radar, and the image data of the urban traffic scene is acquired based on the camera.

[0029] Step 2: Based on the 3D point cloud data, the trained KPConv network is used to extract the static elements of the urban traffic scene, obtain the static elements of the urban traffic scene, and parameterize the static elements of the urban traffic scene; In this embodiment, Figure 2 As shown in the figure, based on 3D point cloud data, the trained KPConv network is used to extract static elements of urban traffic scenes. The specific steps of obtaining static elements of urban traffic scenes include: Step 1: Preprocess the 3D point cloud data to obtain the preprocessed 3D point cloud data; In this embodiment, preprocessing the 3D point cloud data specifically includes the following steps: Step Ⅰ: Remove outliers from 3D point cloud data based on statistical methods; In this embodiment, based on the statistical method, outlier removal of 3D point cloud data specifically includes the following steps: Step ①: Find the neighboring points of each point in the 3D point cloud data based on the preset number of neighboring points and the preset standard deviation ratio; Step ②: For each point in the 3D point cloud data, calculate the average value and standard deviation of the distance from the point to each of its corresponding neighboring points, and set a first threshold based on the average value and standard deviation; Step ③: For each point in the 3D point cloud data, the neighboring points corresponding to the point whose distance to the point is greater than the first threshold are determined as outliers, and the outliers are removed.

[0030] Specifically, find the neighboring points of each point in the 3D point cloud data, traverse each point in the 3D point cloud data, calculate the distance from the neighboring point corresponding to the current point to the current point, and find the average value , and find the standard deviation , if the distance from a neighboring point corresponding to the current point to the current point is greater than , then the neighboring point is determined as an outlier and removed, where Indicates the preset coefficient.

[0031] Step II: Based on the local minimum elevation method, remove ground points from the 3D point cloud data.

[0032] In this embodiment, based on the local minimum elevation method, removing ground points from 3D point cloud data specifically includes the following steps: Step 1: Correct the coordinate system of the 3D point cloud data so that the Z axis of the coordinate system of the 3D point cloud data is perpendicular to the ground, and use the Z axis coordinate of the 3D point cloud data as the elevation; Step ②: Grid the 3D point cloud data, and perform elevation filtering on the 3D point cloud data based on the grid to eliminate outliers; Step ③: Select the elevation of the lowest point in each grid as the local minimum elevation of each grid, determine the points whose elevation difference between the elevation in each grid and the local minimum elevation is less than or equal to the second threshold as ground points, and remove the ground points.

[0033] Specifically, the elevation of the lowest point in each grid is selected as the local minimum elevation of each grid. , the difference between the elevation in each grid and the local minimum elevation is less than or equal to the second threshold The points are determined as ground points and removed.

[0034] In this embodiment, the grid division is based on two-dimensional grid division from a bird's-eye view perspective, and both the elevation filtering and the local minimum elevation are based on the grid as a basic unit.

[0035] Step 2: Vectorize the preprocessed 3D point cloud data to obtain 3D point cloud vectors; In this embodiment, vectorizing the preprocessed 3D point cloud data specifically includes: performing 6-dimensional vectorization on the preprocessed 3D point cloud data based on the X-axis coordinate, Y-axis coordinate, Z-axis coordinate, red color value, green color value and blue color value corresponding to each point in the preprocessed 3D point cloud data.

[0036] Specifically, the 3D point cloud data is converted into a 3D point cloud vector of N rows and 6 columns, where N represents the total number of 3D point clouds, and the 6 columns correspond to the X-axis coordinate, Y-axis coordinate, Z-axis coordinate, red color value, green color value, and blue color value of the 3D point cloud, respectively.

[0037] Step 3: Input the 3D point cloud vector into the trained KPConv network for semantic segmentation, obtain the 3D point cloud category coding, and post-process the 3D point cloud based on the category coding to obtain the static elements of the urban traffic scene.

[0038] In this embodiment, the KPConv network is trained based on pre-acquired 3D point cloud training data in combination with a cross-validation method.

[0039] The 3D point cloud vector is input into the trained KPConv network for semantic segmentation to obtain the 3D point cloud category coding, where the category coding includes background, trees and artificial rods.

[0040] Specifically, 21 collected 3D point clouds of traffic scenes are selected and input into the KPConv network for training, and during the training process, 14 collected 3D point clouds of traffic scenes are selected for cross-validation to obtain the trained KPConv network.

[0041] Use the trained KPConv network to perform semantic segmentation on the traffic scene of Bayi Road-Guangba Road, and assign corresponding category codes to each point.

[0042] In this embodiment, post-processing of the 3D point cloud based on category coding specifically includes: denoising, category correction and horizontal clustering merging of 3D point clouds with category coding as trees and artificial poles, further dividing the 3D point cloud with category coding as artificial poles into street lights, surveillance, road signs or telephone poles, and based on an adaptive growth strategy, separating individual tree trunks of the 3D point cloud with category coding as trees.

[0043] In this embodiment, denoising and category correction specifically include: point count threshold setting, height threshold setting, noise point removal and misclassification recognition, tree extraction and adhesion processing, and distance threshold matching correction.

[0044] Specifically, the point count threshold is used to identify small clusters, i.e., noise points; the height threshold is used to identify floating noise points, i.e., points whose height does not match the height of the ground or typical objects; noise removal and misclassification recognition are based on the point cloud threshold and the height threshold to remove floating points identified as noise, and to identify clusters that are misclassified as artificial rods according to the point cloud threshold; tree extraction and adhesion processing processes the adhered trees as multiple trunks by cutting off the lower half of the tree cluster as the trunk.

[0045] In this embodiment, the horizontal cluster merging specifically includes: height threshold screening, orientation calculation horizontal rod identification, horizontal distance projection calculation and cluster merging.

[0046] Specifically, the height threshold screening is performed by calculating the Z-axis coordinate of the lowest point of each artificial rod cluster to screen out horizontal cantilevers with higher heights; the orientation calculation horizontal rod identification is performed by calculating the direction vector when the Z-axis component is close to 0; the horizontal distance projection calculation and cluster merging are performed by merging horizontal rods through the projection distance in the horizontal direction.

[0047] In this embodiment, the algorithm pseudo code of the adaptive growth strategy is shown in Algorithm 1.

[0048]

[0049] In this embodiment, Figure 3 As shown in the figure, parameterizing the static elements of urban traffic scenes specifically includes the following steps: Step 1: Based on the principal component analysis method, the 3D point cloud with category coding as trees and artificial rods is judged to obtain the shape of the 3D point cloud; In this embodiment, based on the principal component analysis (PCA) method, the shape of the 3D point cloud coded as trees and artificial rods is judged, and the shape of the 3D point cloud is obtained, which specifically includes the following steps: Step 1: Calculate the geometric center, covariance matrix and matrix eigenvector of the 3D point cloud whose category codes are trees and artificial rods; In this embodiment, the geometric center of the 3D point cloud is calculated by summing and averaging the coordinates of the 3D point cloud.

[0050] The covariance matrix and matrix eigenvector of the 3D point cloud are: ; in, represents the covariance matrix, represents the matrix eigenvector, Represents the matrix eigenvalue.

[0051] Step ②: Based on the geometric center, covariance matrix and matrix eigenvector, the 3D point cloud with category encoding as trees and artificial rods is decomposed into eigenvectors to obtain the 3D point cloud shape.

[0052] In this embodiment, the 3D point cloud shapes include linear, planar and solid shapes.

[0053] Specifically, the shape discrimination parameters of 3D point cloud are: ; in, , , Indicates the possibility of 3D point cloud shape being linear, planar, or solid.

[0054] Step 2: Based on the shape of the 3D point cloud, the 3D point cloud with the category code of trees and artificial rods is segmented into the rod trunk and rod parts to obtain the rod trunk and rod parts; In this embodiment, a linear 3D point cloud is taken as the trunk of the rod-shaped object, and 3D point clouds of other shapes are taken as the rod-shaped object parts.

[0055] Specifically, points with denser distribution and relatively lower positions are determined as tree trunk points, and the vertical poles of road signs and traffic lights are extracted based on the characteristics that the metal poles of road signs and traffic lights are of different thicknesses and have horizontal connecting poles.

[0056] In this embodiment, according to the characteristics of dense distribution and relatively low position of 3D point cloud, the algorithm for extracting tree trunk points is shown in Algorithm 2.

[0057]

[0058] Step 3: Calculate the height and radius of the rod-shaped trunk and parameterize the rod-shaped trunk; In this embodiment, the step of calculating the height of the rod trunk is the same as the step of calculating the height of the rod component.

[0059] Calculating the radius of the rod-shaped trunk specifically includes: obtaining the radius of the rod-shaped trunk in the XOY coordinate system layer by layer, and calculating the average value of the radii of each layer of the rod-shaped trunk in the XOY coordinate system as the radius of the rod-shaped trunk.

[0060] Step 4: Calculate the length, width, height and maximum radius of the rod-shaped component, and parameterize the rod-shaped component.

[0061] In this embodiment, calculating the length, width, and height of the rod-shaped component specifically includes: calculating the covariance matrix of the rod-shaped component, and multiplying the covariance matrix of the rod-shaped component with the coordinates of the 3D point cloud corresponding to the rod-shaped component, and obtaining the spacing of the rod-shaped component in the X-axis, Y-axis, and Z-axis directions as the length, width, and height of the rod-shaped component.

[0062] Calculating the maximum radius of the rod-shaped component specifically includes: obtaining the radius of the rod-shaped component in the XOY coordinate system layer by layer, and calculating the maximum value of the radius of each layer of the rod-shaped component in the XOY coordinate system as the maximum radius of the rod-shaped component.

[0063] Step 3: Based on the image data, use the trained YOLOv7-W6 network to extract dynamic targets in urban traffic scenes, obtain dynamic targets in urban traffic scenes, and map the dynamic targets in urban traffic scenes to the coordinate system of the 3D point cloud data to complete the extraction of urban traffic scenes.

[0064] In this embodiment, the YOLOv7-W6 network is trained based on pre-acquired image training data in combination with a label assignment strategy and a cross-validation method.

[0065] The image data is input into the trained YOLOv7-W6 network for frame extraction recognition to obtain dynamic targets in urban traffic scenes, where dynamic targets in urban traffic scenes include pedestrians, bicycles, motorcycles, cars, trucks and buses.

[0066] Specifically, Figure 4 As shown in the figure, the dynamic targets in the urban traffic scene are first divided into six categories: pedestrians, bicycles, motorcycles, cars, trucks, and buses. Then, the training set in the COCO dataset is used in combination with the label assignment strategy to train the YOLOv7-W6 network with an auxiliary head, and the validation set in the COCO dataset is used for cross-validation during the training process. Then, the image data collected by the camera attached to the Titian-M1 laser radar of Lingwei is input into the YOLOv7-W6 network for frame extraction recognition, and the confidence, category probability and other information output by the YOLOv7-W6 network are comprehensively considered. Finally, the YOLOv7-W6 network outputs all the detection frame information of the current picture or video frame, including the category number, category confidence, frame center pixel coordinates, frame width and height, and the result reasoning is performed to evaluate the average detection time of a single picture, frames per second (Frames Per Second, FPS), non-maximum suppression (NMS) processing time, and total processing time of all pictures.

[0067] In this embodiment, mapping the dynamic target of the urban traffic scene to the coordinate system where the 3D point cloud data is located specifically includes the following steps: Step 1: Obtain the central pixel coordinates of the dynamic target in the urban traffic scene; Step 2: Map the central pixel coordinates of the dynamic target in the urban traffic scene to the coordinate system of the 3D point cloud data.

[0068] In this embodiment, the road surface points in the urban traffic scene are approximately regarded as a plane, and the 3D coordinates of the dynamic target are Considered as a plane coordinate, the pixel coordinate on the image plane is marked as , the transformation relationship between two different coordinates can be constructed through the homography matrix. Based on the transformation relationship, the central pixel coordinates of the dynamic target in the urban traffic scene are mapped to the coordinate system where the 3D point cloud data is located.

[0069] Embodiment 2: This embodiment provides a device for extracting urban traffic scenes, including: Data acquisition module, used to obtain 3D point cloud data and image data of urban traffic scenes; The static feature extraction module is used to extract the static features of urban traffic scenes based on 3D point cloud data using the trained KPConv network, obtain the static features of urban traffic scenes, and parameterize the static features of urban traffic scenes; The dynamic target extraction module is used to extract dynamic targets in urban traffic scenes based on image data using the trained YOLOv7-W6 network, obtain dynamic targets in urban traffic scenes, and map the dynamic targets in urban traffic scenes to the coordinate system of 3D point cloud data to complete the extraction of urban traffic scenes.

[0070] The urban traffic scene extraction device provided in the embodiment of the present application can execute the urban traffic scene extraction method provided in Example 1 of the present application, and has the corresponding functional modules and beneficial effects of the execution method.

[0071] Embodiment 3: This embodiment provides a computer system, including: Storage medium: used to store computer programs; Processor: used to execute computer programs to implement the steps of the urban traffic scene extraction method in Example 1.

[0072] Embodiment 4: This embodiment provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of the urban traffic scene extraction method in Embodiment 1 are implemented.

[0073] Embodiment 5: This embodiment provides a computer program product, including a computer program, which implements the steps of the urban traffic scene extraction method in Embodiment 1 when executed by a processor.

[0074] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0075] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0076] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0077] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0078] The above are only preferred implementations of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A method for extracting urban traffic scenes, characterized in that: include: Obtain 3D point cloud data and image data of urban traffic scenes; Based on the 3D point cloud data, using the trained KPConv network to extract static elements of the urban traffic scene, obtain the static elements of the urban traffic scene, and parameterize the static elements of the urban traffic scene; Based on the image data, the trained YOLOv7-W6 network is used to extract dynamic targets in urban traffic scenes, obtain dynamic targets in urban traffic scenes, and map the dynamic targets in urban traffic scenes to the coordinate system where the 3D point cloud data is located, thereby completing the extraction of urban traffic scenes.

2. The urban traffic scene extraction method according to claim 1, characterized in that: Based on the 3D point cloud data, the trained KPConv network is used to extract static elements of urban traffic scenes, and the static elements of urban traffic scenes are obtained, including: Preprocessing the 3D point cloud data to obtain preprocessed 3D point cloud data; Vectorize the preprocessed 3D point cloud data to obtain 3D point cloud vectors; Input the 3D point cloud vector into the trained KPConv network for semantic segmentation, obtain the 3D point cloud category code, and post-process the 3D point cloud based on the category code to obtain the static elements of the urban traffic scene; The category codes include background, trees and artificial rods.

3. The urban traffic scene extraction method according to claim 2, characterized in that: Preprocessing the 3D point cloud data includes: Based on statistical methods, outlier points are removed from the 3D point cloud data; Based on the local minimum elevation method, removing ground points from the 3D point cloud data; Wherein, based on the statistical method, removing outliers from the 3D point cloud data includes: Finding neighboring points of each point in the 3D point cloud data based on a preset number of neighboring points and a preset standard deviation ratio; For each point in the 3D point cloud data, calculate the average value and standard deviation of the distance from the point to each of its corresponding neighboring points, and set a first threshold based on the average value and standard deviation; For each point in the 3D point cloud data, determine the neighboring points corresponding to the point whose distance to the point is greater than a first threshold as outliers, and remove the outliers; Based on the local minimum elevation method, removing ground points from the 3D point cloud data includes: Correcting the coordinate system of the 3D point cloud data so that the Z axis of the coordinate system of the 3D point cloud data is perpendicular to the ground, and using the Z axis coordinate of the 3D point cloud data as the elevation; Gridding the 3D point cloud data, and performing elevation filtering on the 3D point cloud data based on the grid to eliminate outliers; The elevation of the lowest point in each grid is selected as the local minimum elevation of each grid, and the points whose difference between the elevation in each grid and the local minimum elevation is less than or equal to the second threshold are determined as ground points, and the ground points are removed.

4. The urban traffic scene extraction method according to claim 2, characterized in that: The vectorization of the preprocessed 3D point cloud data includes: performing 6-dimensional vectorization on the preprocessed 3D point cloud data based on the X-axis coordinate, Y-axis coordinate, Z-axis coordinate, red color value, green color value and blue color value corresponding to each point in the preprocessed 3D point cloud data.

5. The urban traffic scene extraction method according to claim 2, characterized in that: Based on the category coding, post-processing of the 3D point cloud includes: denoising, category correction and horizontal clustering merging of the 3D point cloud with category coding as trees and artificial poles, further dividing the 3D point cloud with category coding as artificial poles into street lights, surveillance, road signs or telephone poles, and based on an adaptive growth strategy, separating individual tree trunks of the 3D point cloud with category coding as trees.

6. The urban traffic scene extraction method according to claim 2, characterized in that: Parameterizing the static elements of the urban traffic scene includes: Based on the principal component analysis method, the shape of the 3D point cloud coded as trees and artificial rods is judged to obtain the shape of the 3D point cloud; Based on the shape of the 3D point cloud, segmenting the 3D point cloud with the category code of trees and artificial rods into rod trunks and rod parts to obtain the rod trunks and rod parts; Calculating the height and radius of the main trunk of the rod-shaped object, and parameterizing the main trunk of the rod-shaped object; Calculating the length, width, height and maximum radius of the rod-shaped component, and parameterizing the rod-shaped component; Wherein, calculating the radius of the rod-shaped trunk includes: obtaining the radius of the rod-shaped trunk in the XOY coordinate system layer by layer, and calculating the average value of the radii of each layer of the rod-shaped trunk in the XOY coordinate system as the radius of the rod-shaped trunk; Calculating the length, width, and height of the rod-shaped component includes: calculating a covariance matrix of the rod-shaped component, and multiplying the covariance matrix of the rod-shaped component by the coordinates of the 3D point cloud corresponding to the rod-shaped component to obtain the spacing of the rod-shaped component in the X-axis, Y-axis, and Z-axis directions as the length, width, and height of the rod-shaped component; Calculating the maximum radius of the rod-shaped component includes: obtaining the radius of the rod-shaped component in the XOY coordinate system layer by layer, and calculating the maximum value of the radius of each layer of the rod-shaped component in the XOY coordinate system as the maximum radius of the rod-shaped component.

7. The urban traffic scene extraction method according to claim 6, characterized in that: Based on the principal component analysis method, the shape of the 3D point cloud coded as trees and artificial rods is judged, and the shape of the 3D point cloud is obtained, including: Compute the geometric center, covariance matrix, and matrix eigenvectors of 3D point clouds whose categories are encoded as trees and artificial poles; Based on the geometric center, the covariance matrix and the matrix eigenvector, the 3D point cloud with the category code of trees and artificial rods is decomposed by eigenvector to obtain the shape of the 3D point cloud; The 3D point cloud shapes include linear, planar and solid shapes.

8. The two-way space partitioning method for vector data according to claim 1, characterized in that: Based on the image data, the trained YOLOv7-W6 network is used to extract dynamic targets in urban traffic scenes. The dynamic targets in urban traffic scenes are obtained by: The image data is input into the trained YOLOv7-W6 network for frame extraction recognition to obtain dynamic targets in urban traffic scenes; The dynamic targets in the urban traffic scene include pedestrians, bicycles, motorcycles, cars, trucks and buses.

9. The two-way space partitioning method for vector data according to claim 1, characterized in that: Mapping the urban traffic scene dynamic target to the coordinate system where the 3D point cloud data is located includes: Obtaining the central pixel coordinates of the dynamic target in the urban traffic scene; The central pixel coordinates of the dynamic target in the urban traffic scene are mapped to the coordinate system where the 3D point cloud data is located.

10. The two-way space partitioning method for vector data according to claim 1, characterized in that: The KPConv network is trained based on pre-acquired 3D point cloud training data in combination with a cross-validation method; the YOLOv7-W6 network is trained based on pre-acquired image training data in combination with a label assignment strategy and a cross-validation method.

Images