Method, device, electronic device and storage medium for classifying objects in road point cloud

Abstract

The present application provides a classification method, device, electronic device and storage medium for objects in a road point cloud, the classification method comprising: collecting initial point cloud data; dividing the initial point cloud data according to a first preset division interval to obtain a point cloud section; determining a target two-dimensional image corresponding to the point cloud section according to the number of point clouds in each pixel included in the point cloud section and the mapping relationship with the grayscale value; slicing the target two-dimensional image along a preset direction according to a second preset division interval, and determining the road surface layer by the number of point clouds included in each slice layer; denoising and classifying the target two-dimensional image according to the road surface layer and a preset height distance to determine the classification information of road objects during vehicle driving. The technical solution provided by the present application uses a two-dimensional point cloud section of a certain width as a processing unit, and when the volume of vehicle-mounted point cloud data is huge, the operating space is guaranteed and the processing efficiency is improved.

Description

Technical Field

[0001] The present application relates to the field of point cloud data processing, and in particular to a classification method, device, electronic device and storage medium for objects in a road point cloud. Background Art

[0002] The vehicle-mounted mobile measurement system is a new type of measurement equipment composed of a variety of high-performance sensors such as laser scanners, inertial navigation, and panoramic cameras. It can be directly mounted on mobile carriers such as cars. Using a non-contact measurement method, the three-dimensional coordinates and attribute information of a large number of points in a large range can be directly obtained during the vehicle's driving process. The acquisition speed is fast and the measurement accuracy is high. The output point cloud results are matched and integrated with the panoramic pictures taken by the panoramic camera, which can restore the city streetscape to the greatest extent. However, the urban road scene is complex, and it is inevitable that there are situations where objects are blocked from each other and difficult to distinguish. The effects of strong sunlight, high reflectivity objects, and too close distance measurement will also appear in the point cloud in the form of noise. Therefore, before applying the vehicle-mounted point cloud results, the point cloud must first be de-noised to improve the accuracy of the point cloud, and the point cloud must be further classified to restore the characteristics of road objects.

[0003] One of the advantages of the vehicle-mounted mobile measurement system is its wide scanning range and dense point cloud data. However, the amount of point cloud data collected by the vehicle-mounted mobile measurement system is large, making it difficult to process it in a unified manner. In addition, the collected point cloud also includes noise points that affect subsequent processing and need to be removed. Existing point cloud denoising technologies are mostly used in three-dimensional scenes. To process point clouds directly in three-dimensional scenes, you must first compare them within a 360-degree range and select a suitable observation direction for processing. At the same time, in three-dimensional scenes, the composition of point cloud data is also relatively complex, and the amount of data that needs to be processed is relatively large. Therefore, the collection and processing of point cloud data in three-dimensional scenes are relatively complex, resulting in slow processing of point clouds, which in turn affects the efficiency of point cloud data processing. Summary of the invention

[0004] In view of this, the purpose of the present application is to provide a method, device, electronic device and storage medium for classifying objects in a road point cloud, which can realize point cloud denoising and classification in a two-dimensional image composed of point cloud sections, and use a two-dimensional point cloud section of a certain width as a processing unit, which simplifies the complexity of the point cloud data, reduces the data processing steps, and helps to improve the efficiency of point cloud data processing.

[0005] This application mainly includes the following aspects:

[0006] In a first aspect, an embodiment of the present application provides a method for classifying objects in a road point cloud, the classification method comprising:

[0007] Collect initial point cloud data during vehicle driving;

[0008] Dividing the initial point cloud data according to a first preset division interval to obtain a point cloud section;

[0009] Determine a target two-dimensional image corresponding to the point cloud section according to the number of point clouds in each pixel included in the point cloud section and a mapping relationship between the number of point clouds and grayscale color values;

[0010] Slicing the target two-dimensional image along a preset direction according to the second division preset interval to obtain at least one slice layer, and determining the road surface layer according to the number of point clouds included in each slice layer;

[0011] The target two-dimensional image is denoised and classified according to the road surface layer and the preset height distance to determine the classification information of road surface objects during vehicle driving.

[0012] Further, the point cloud profile is determined by the following steps:

[0013] Determining a plurality of driving nodes and a driving time corresponding to each driving node based on a total driving distance of the vehicle driving process and a first preset division interval;

[0014] The point cloud section is determined based on a plurality of point clouds corresponding to a driving time period between every two driving moments.

[0015] Further, the target two-dimensional image corresponding to the point cloud section is determined by the following steps:

[0016] Dividing the horizontal length and the vertical length of the point cloud section according to a preset pixel size to determine a plurality of pixels;

[0017] According to the number of point clouds in each pixel, determine the gray value corresponding to each pixel;

[0018] According to the determined interval position of each pixel, the target two-dimensional image color value of each pixel is determined, and the target two-dimensional image of each point cloud section is established; wherein the interval position of each pixel is determined based on the gray value of the pixel.

[0019] Further, the pavement layer is determined by the following steps:

[0020] Slicing the target two-dimensional image along a preset direction according to the second division preset interval to obtain at least one slice layer;

[0021] Obtain the number of point clouds of each slice layer;

[0022] A slice layer whose point cloud number is higher than the point cloud number of each layer except this layer is determined as the pavement layer.

[0023] Furthermore, the target two-dimensional image is denoised by the following steps:

[0024] Determine the point clouds located below the road surface layer and above the road surface layer in the target two-dimensional image and at a distance greater than a preset height of the road surface layer as noise points;

[0025] Determine a target area between the road surface layer and a plane at a preset height distance from the road surface layer, and an average point density in the target area;

[0026] Determine the point density threshold of the target area according to a preset scale factor and an average point density;

[0027] The pixel whose point cloud number is lower than the point density threshold is determined as a candidate noise pixel; if the pixels adjacent to the candidate noise pixel satisfy the conditions that no pixel point density is higher than the point density threshold and more than half of the pixel point densities are lower than the point density threshold, the candidate noise pixel is determined to be a noise pixel, and the point cloud within the noise pixel is determined to be a noise point;

[0028] The determined noise points are removed from the target two-dimensional image.

[0029] Furthermore, the target two-dimensional image is classified through the following steps to determine classification information of road objects during driving:

[0030] Taking the road surface layer as a reference, determining the area between the denoised target two-dimensional image road surface layer and a plane at a preset height distance from the road surface layer as the target area;

[0031] In the target area, road objects are classified according to the different heights corresponding to different types of road objects during vehicle driving.

[0032] In a second aspect, an embodiment of the present application further provides a classification device for objects in a road point cloud, the classification device comprising:

[0033] The acquisition module is used to collect the initial point cloud data during the vehicle driving process;

[0034] A division module, used for dividing the initial point cloud data according to a first preset division interval to obtain a point cloud section;

[0035] A conversion module, used to determine a target two-dimensional image corresponding to the point cloud section according to the number of point clouds in each pixel included in the point cloud section and a mapping relationship between the number of point clouds and grayscale color values;

[0036] An analysis module, configured to slice the target two-dimensional image along a preset direction according to the second division preset interval to obtain at least one slice layer, and determine a road surface layer according to the number of point clouds included in each slice layer;

[0037] The processing module is used to denoise and classify the target two-dimensional image according to the road surface layer and the preset height distance, and determine the classification information of the road surface objects during the vehicle driving process.

[0038] Furthermore, when the segmentation module is used to determine the point cloud section, the segmentation module is used to:

[0039] Determining a plurality of driving nodes and a driving time corresponding to each driving node based on a total driving distance of the vehicle driving process and a first preset division interval;

[0040] The point cloud section is determined based on a plurality of point clouds corresponding to a driving time period between every two driving moments.

[0041] In a third aspect, an embodiment of the present application further provides an electronic device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and when the machine-readable instructions are executed by the processor, the steps of the method for classifying objects in the road point cloud as described above are performed.

[0042] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for classifying objects in a road point cloud as described above are executed.

[0043] The embodiments of the present application provide a method, device, electronic device and storage medium for classifying objects in a road point cloud, which collect initial point cloud data during vehicle driving; divide the initial point cloud data according to a first preset division interval to obtain a point cloud profile; determine a target two-dimensional image corresponding to the point cloud profile based on the number of point clouds in each pixel included in the point cloud profile and the mapping relationship between the number of point clouds and grayscale color values; slice the target two-dimensional image along a preset direction according to a second preset division interval to obtain at least one slice layer, and determine a road surface layer through the number of point clouds included in each slice layer; denoise and classify the target two-dimensional image based on the road surface layer and a preset height distance to determine classification information of road objects during vehicle driving.

[0044] In this way, the technical solution provided by the present application can realize point cloud denoising and classification in a two-dimensional image composed of point cloud sections. Before denoising and classification, the road surface layer is first extracted, which not only reduces the amount of data, but also reduces the impact of road surface points on classification. At the same time, the road surface layer can also be used as a reference for denoising and classification. When performing denoising and classification, a two-dimensional point cloud section of a certain width is used as a processing unit, which simplifies the complexity of point cloud data, reduces data processing steps, and helps to improve the efficiency of point cloud data processing.

[0045] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0047] Figure 1 A flow chart of a method for classifying objects in a road point cloud provided by an embodiment of the present application is shown;

[0048] Figure 2 A flow chart of determining a target two-dimensional image corresponding to a point cloud section provided in an embodiment of the present application is shown;

[0049] Figure 3 A flowchart of denoising a target two-dimensional image provided by an embodiment of the present application is shown;

[0050] Figure 4 A schematic diagram of the structure of a device for classifying objects in a road point cloud provided by an embodiment of the present application is shown;

[0051] Figure 5 A schematic structural diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0052] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. It should be understood that the drawings in the present application only serve the purpose of explanation and description and are not used to limit the scope of protection of the present application. In addition, it should be understood that the schematic drawings are not drawn in real proportion. The flowchart used in this application shows the operations implemented according to some embodiments of the present application. It should be understood that the operations of the flowchart can be implemented out of sequence, and the steps without logical context can be reversed in order or implemented simultaneously. In addition, those skilled in the art, under the guidance of the content of the present application, can add one or more other operations to the flowchart, or remove one or more operations from the flowchart.

[0053] In addition, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present application.

[0054] In order to enable those skilled in the art to use the contents of this application, the following implementation is given in combination with a specific application scenario "classification of objects in road point clouds". For those skilled in the art, the general principles defined here can be applied to other embodiments and application scenarios without departing from the spirit and scope of this application.

[0055] The following methods, devices, electronic devices or computer-readable storage media of the embodiments of the present application can be applied to any scenario that requires classification of road surface objects in a road point cloud. The embodiments of the present application are not limited to specific application scenarios. Any solution using a method, device, electronic device and storage medium for classifying objects in a road point cloud provided by the embodiments of the present application is within the protection scope of the present application.

[0056] It is worth noting that the vehicle-mounted mobile measurement system is a new type of measurement equipment composed of a variety of high-performance sensors such as laser scanners, inertial navigation, panoramic cameras, etc., which can be directly mounted on mobile carriers such as cars. Using a non-contact measurement method, the three-dimensional coordinates and attribute information of a large number of points in a large range can be directly obtained during the vehicle's driving process. The acquisition speed is fast and the measurement accuracy is high. The output point cloud results are matched and integrated with the panoramic pictures taken by the panoramic camera, which can restore the city streetscape to the greatest extent. Whether it is the field process or the indoor results, it is more efficient and convenient than traditional surveying and mapping methods such as total stations and RTK that only support single-point measurement at a time. Today, when urban construction is increasingly inclined towards digitalization and informatization, the point cloud results output by the vehicle-mounted mobile measurement system can be widely used in urban construction, such as road management and maintenance, asset census, urban three-dimensional modeling, etc. Among them, the road point cloud can also empower autonomous driving, make high-precision electronic maps based on the road point cloud, and update the map base map. However, urban road scenes are complex, and inevitably there are situations where objects block each other and are difficult to distinguish. The effects of strong sunlight, high-reflectivity objects, and too close distance measurement will also appear in the point cloud in the form of noise. Therefore, before applying the vehicle-mounted point cloud results, the point cloud must first be de-noised to improve the accuracy of the point cloud. Usually, it is far from enough to remove the noise. In order to use the point cloud results more efficiently, the point cloud will be further classified to restore the characteristics of road objects. Therefore, for the application of road point clouds, noise removal and rough classification of objects are very necessary.

[0057] At present, one of the advantages of mobile measurement systems is the wide scanning range and dense point cloud data. In the existing technology, point cloud denoising technology is mostly used in three-dimensional scenes. The amount of point cloud data collected by the vehicle-mounted mobile measurement system is large and difficult to process uniformly. Processing point clouds in three-dimensional scenes also requires comparing and selecting observation directions in 360 degrees. The selection of point cloud observation direction affects the judgment of the shape of road objects, resulting in complex operations and slow processing speeds.

[0058] Based on this, the present application proposes a classification method, device, electronic device and storage medium for objects in road point clouds, which performs point cloud denoising and classification in a two-dimensional image composed of point cloud sections. Before denoising and classification, the road surface layer is first extracted, which not only reduces the amount of data, but also reduces the impact of road surface points on classification. At the same time, the road surface layer can also be used as a reference for denoising and rough classification. When performing denoising and classification, a two-dimensional point cloud section of a certain width is used as a processing unit, which simplifies the complexity of point cloud data, reduces data processing steps, and helps to improve the efficiency of point cloud data processing.

[0059] To facilitate the understanding of the present application, the technical solution provided by the present application is described in detail below in conjunction with specific embodiments.

[0060] See also Figure 1 , Figure 1 The following is a flow chart of a method for classifying objects in a road point cloud provided in an embodiment of the present application. Figure 1 As shown in, the classification method includes:

[0061] S101, collecting initial point cloud data during vehicle driving;

[0062] In this step, the vehicle's driving trajectory and initial point cloud data, including the laser scanning time and position information of the point cloud, are collected through the laser scanner, inertial navigation, panoramic camera and other equipment in the vehicle-mounted mobile measurement system. The vehicle-mounted mobile measurement system is a special detection instrument used in the fields of earth science, surveying and mapping science and technology, and water conservancy engineering. The initial point cloud data is obtained by scanning the objects around the driving trajectory, including road surfaces, street lights, green belts, vehicles, and bridges, with the vehicle as the center of the circle, through the vehicle-mounted mobile measurement system during the driving process.

[0063] S102, dividing the initial point cloud data according to a first preset division interval to obtain a point cloud section;

[0064] In this step, the initial point cloud data is segmented and cut according to the first preset division interval to establish a point cloud profile for each segment of the point cloud. The purpose of establishing the point cloud profile is to process the point cloud data in a two-dimensional plane, simplify the complexity of the point cloud data, reduce the data processing steps, and improve the efficiency of point cloud data processing.

[0065] As an example, the point cloud profile may be constructed by the following steps:

[0066] Determining a plurality of driving nodes and a driving time corresponding to each driving node based on a total driving distance of the vehicle driving process and a first preset division interval;

[0067] The point cloud section is determined based on a plurality of point clouds corresponding to a driving time period between every two driving moments.

[0068] It should be noted that the total driving distance during the vehicle's driving process, that is, the total driving trajectory, is clipped according to the first preset division interval, so as to determine the clipped trajectory node and the driving time corresponding to the node; the area between two adjacent trajectory nodes, that is, the point cloud data corresponding to the driving time period between the two trajectory nodes, takes the direction of the vehicle's driving as the point cloud observation direction, and determines the point cloud profile of each time period; for example, check the starting time T1 of the point cloud, determine the trajectory point P1 corresponding to the driving trajectory at time T1, take the trajectory point P1 as the starting point, set the first preset division interval d, and the trajectory point P2 at a position d meters forward of P1 as the end point. After determining the trajectory point P2, continue to determine the moment T2 of P2, and intercept the point cloud data of the T1-T2 time period in the point cloud. Take the direction from P1 to P2 as the point cloud observation direction, and determine the profile of the point cloud in the T1-T2 time period. Continue to use P2 as the starting point and move forward d meters to find the trajectory point P3 to determine the second section of the point cloud profile. Use P3 as the starting point and move forward d meters to find the trajectory point P4 to determine the third section of the point cloud profile, until all the collected point clouds are divided, thus completing the establishment of the profiles of all the point clouds.

[0069] S103, determining a target two-dimensional image corresponding to the point cloud section according to the number of point clouds in each pixel included in the point cloud section and a mapping relationship between the number of point clouds and grayscale color values;

[0070] In this step, each point cloud section determined in step S102 can be converted into a target two-dimensional image. For the step of determining the target two-dimensional image corresponding to the point cloud section, please refer to Figure 2 , Figure 2 This is a flow chart of determining the target two-dimensional image corresponding to the point cloud section provided in the embodiment of the present application. Figure 2 As shown in , the method for determining a target two-dimensional image corresponding to a point cloud section provided in an embodiment of the present application includes:

[0071] S201, dividing the horizontal length and the vertical length of the point cloud section according to a preset pixel size to determine a plurality of pixels;

[0072] In this step, the horizontal length and the vertical length of the point cloud section are determined according to the coordinates of each point cloud in the point cloud section, that is, according to the coordinates of each point cloud in the point cloud section, the coordinate range included in the point cloud section is determined, and the grid network is determined according to the coordinate range of the point cloud section and the preset pixel size; wherein the grid network is composed of grids organized in rows and columns, and each grid is the smallest unit pixel of the grid; illustratively, the coordinate range X of the midpoint of the point cloud section is determined min , X max , Y min , Y max , calculate △X(X max-X min )、△Y(Y max -Y min ), set the appropriate pixel size s, take △X as the length and △Y as the width, and establish a grid with △X / s rows and △Y / s columns.

[0073] S202, determining the grayscale value corresponding to each pixel according to the number of point clouds in each pixel;

[0074] In this step, according to the number of point clouds in each pixel, the maximum and minimum values ​​of the point clouds are counted, and the intermediate value V and the gray value G of each pixel are calculated by the following formula to create a grayscale image.

[0075]

[0076] G = 255 × (1-V);

[0077] In the above formula, V is the middle value of the current pixel, N is the number of point clouds in the current pixel, and N is the number of point clouds in the current pixel. min is the minimum number of point clouds in the pixel in the current point cloud profile image, N max It is the maximum number of point clouds in a pixel in the current point cloud profile image.

[0078] It should be noted that the number of point clouds N in each pixel is counted to determine the range of the number of point clouds [N max , N min 】, corresponding to the color range of the grayscale image [255, 0]. According to the number of point clouds N in the pixel, the intermediate value V and the grayscale value G of each pixel are calculated according to the above formula to establish a grayscale image. The fewer the number of point clouds in the pixel, the brighter the pixel.

[0079] S203, determining the target two-dimensional image color value of each pixel according to the determined interval position to which each pixel belongs, and establishing the target two-dimensional image of each point cloud section; wherein the interval position to which each pixel belongs is determined based on the grayscale value of the pixel.

[0080] In this step, based on the grayscale value of each pixel determined in step S202, the grayscale range [G max , G min 】, calculate the position L of each gray value according to the following formula;

[0081]

[0082] In the above formula, L is the interval position of the current pixel, G is the gray value of the current pixel, and G min is the minimum grayscale value in the current grayscale image, G max is the maximum grayscale value in the current grayscale image.

[0083] Secondly, determine the color interval L where the gray value is located x And the color values ​​of the upper and lower boundary points of the color interval [C a , C b 】. The color value when the grayscale image is converted into the target two-dimensional image, that is, the RGB image, is calculated according to the following formula, and the RGB image corresponding to the point cloud profile is established.

[0084] C=C a +(C b -C a )×(LL x );

[0085] In the above formula, C is the RGB image color value of the current pixel, C a is the upper boundary color value of the interval where the current pixel is located, C b is the lower boundary color value of the interval where the current pixel is located, L is the interval position where the current pixel is located, and L x It is the interval number to which the current pixel belongs, that is, the Xth interval.

[0086] Exemplarily, the grayscale value 0-255 is divided into four equal parts to obtain four color intervals L1, L2, L3, L4 and five interval boundary points C1, C2, C3, C4, C5, [0, 63] is the first interval L1, [64, 127] is the second interval L2, [128, 191] is the third interval L3, and [192, 255] is the fourth interval L4. In order to enhance the display effect of the target two-dimensional image RGB image, RGB is selected respectively. The five colors in the image, red (255, 0, 0), yellow (255, 255, 0), green (0, 255, 0), cyan (0, 255, 255), and blue (0, 0, 255), are the colors of the boundary points of the C1-C5 interval. L1 [0, 63] is red-yellow, L2 [64, 127] is yellow-green, L3 [128, 191] is green-cyan, and L4 [192, 255] is cyan-blue.

[0087] S104, slicing the target two-dimensional image along a preset direction according to the second division preset interval to obtain at least one slice layer, and determining a road surface layer according to the number of point clouds included in each slice layer;

[0088] In this step, the laser scanner performs a 360° scan, and the point cloud is evenly distributed within the 360° scanning range, with the angle interval between any two points being equal. There are usually no objects in the sky, so there are generally no point clouds in the sky. The point cloud obtained by rotating the scanner 360° will have at least 90° of the range on the ground, that is, the point cloud on the ground accounts for at least 25% of all point clouds.

[0089] As an example, the pavement layer may be obtained by the following steps:

[0090] Slicing the target two-dimensional image along a preset direction according to the second division preset interval to obtain at least one slice layer;

[0091] Obtain the number of point clouds of each slice layer;

[0092] A slice layer whose point cloud number is higher than the point cloud number of each layer except this layer is determined as the pavement layer.

[0093] Exemplarily, each target two-dimensional image obtained in step S203, that is, the RGB image, is sliced ​​along a preset direction according to the second division preset interval △H, wherein the second division preset interval △H is a division height perpendicular to the ground, that is, the distance from the ground to the sky is equally divided at a height of △H, and the preset direction is the direction of vehicle travel. The RGB image is sliced ​​along the direction of vehicle travel according to the height interval of △H to obtain at least one slice layer, and each slice layer obtained is parallel to the ground. The number of pixels in each slice layer is first counted, and then the number of point clouds in each slice layer is counted. The proportion of any layer of point cloud will be less than 1 / 4 (90° / 360°), therefore, the layer of slice with the largest number of point clouds is defined as the road surface and classified into the ground layer.

[0094] S105: De-noise and classify the target two-dimensional image according to the road surface layer and the preset height distance to determine classification information of road objects during vehicle driving.

[0095] In this step, the target two-dimensional image is denoised according to the road surface layer and the preset height distance. Noise points have different manifestations and patterns, some are gathered in one place, and some are scattered and isolated. For the flowchart of denoising, please refer to Figure 3 , Figure 3 This is a flow chart of denoising a target two-dimensional image provided by an embodiment of the present application. Figure 3 As shown in , the steps of denoising the target two-dimensional image include:

[0096] This application mainly uses the following two methods to remove noise:

[0097] The first denoising method: using elevation filtering to remove noise;

[0098] S301, determining point clouds located below the road surface layer and above the road surface layer in the target two-dimensional image and at a distance greater than a preset height of the road surface layer as noise points;

[0099] In this step, the pavement layer has been determined in S104, and the point clouds below the pavement layer in the target two-dimensional image and above the pavement layer at a preset height distance from the pavement layer are determined as noise points; illustratively, the point clouds in the area mapped in the RGB image below the pavement layer and 30 meters above the pavement layer are all regarded as noise points.

[0100] The second denoising method: using density threshold method to remove noise;

[0101] S302, determining a target area between the road surface layer and a plane at a preset height distance from the road surface layer, and an average point density in the target area;

[0102] In this step, the target area between the pavement layer and a plane at a preset height distance from the pavement layer is determined, the number of point clouds in the pixels in the target area is counted, and the average point density of the target area is calculated; wherein the average point density of the target area is the number of point clouds in the pixels in the target area divided by the area of ​​the target area.

[0103] Exemplarily, step S301 has determined that the point clouds below the pavement layer and 30 meters above the pavement layer are noise points. The target area in step S302 is the area within 30 meters from the pavement layer to the pavement layer. The average point density of this area is the sum of the number of point clouds in all pixels in this area divided by the area of ​​this area, where the area of ​​this area is the pixel size multiplied by the number of pixels in the area.

[0104] S303, determining a point density threshold of the target area according to a preset proportionality coefficient and an average point density;

[0105] In this step, a proportional coefficient is set, and the average point density obtained in S302 is multiplied by the set proportional coefficient to obtain the point density threshold of the target area in each target two-dimensional image. Exemplarily, the proportional coefficient f is set, and the average point density d is multiplied by the proportional coefficient f to obtain the point density threshold ρ of the target area, that is, the point density threshold ρ in the area mapped in the RGB image within 30 meters from the road surface layer to the road surface layer.

[0106] S304, determining pixels whose point cloud number is lower than the point density threshold as candidate noise pixels; if the pixels adjacent to the candidate noise pixels satisfy both that no pixel point density is higher than the point density threshold and that more than half of the pixel point densities are lower than the point density threshold, the candidate noise pixel is determined to be a noise pixel, and the point cloud within the noise pixel is determined to be a noise point;

[0107] Exemplarily, pixels whose point cloud number is lower than the point density threshold ρ are determined as candidate noise pixels, and the final output of this pixel is determined according to the values ​​of the pixels surrounding the candidate noise pixel. The point density of each pixel in the n×n neighborhood of the candidate noise pixel is calculated separately. If the neighborhood pixels simultaneously satisfy the conditions that no pixel point density is higher than ρ and more than half of the pixel point densities are less than ρ, then the candidate noise pixel is confirmed to be a noise pixel, and the point cloud within the candidate noise pixel is a noise point.

[0108] S305: Eliminate the determined noise points from the target two-dimensional image.

[0109] In this step, the noise points determined by step S301 and step S302 to step S304 are removed from the target two-dimensional image. Exemplarily, the first denoising method removes the point cloud located below the pavement layer and 30 meters above the pavement layer obtained in step S301 from the RGB image; the second denoising method removes the noise points obtained in step S302 to step S304 from the RGB image.

[0110] S105: De-noise and classify the target two-dimensional image according to the road surface layer and the preset height distance to determine classification information of road objects during vehicle driving.

[0111] In this step, the target two-dimensional image is classified according to the road surface layer and the preset height distance to determine the classification information of the road surface objects during the vehicle driving process.

[0112] As an example, the target two-dimensional image is classified to determine the classification information of road objects during driving, which can be constructed by the following steps:

[0113] Taking the road surface layer as a reference, determining the area between the denoised target two-dimensional image road surface layer and a plane at a preset height distance from the road surface layer as the target area;

[0114] In the target area, road objects are classified according to the different heights corresponding to different types of road objects during vehicle driving.

[0115] It should be noted that before classifying the target two-dimensional image, it is necessary to extract the two-dimensional shape features of typical road objects from a large number of RGB images of point cloud sections, and establish a road object template library based on the features. Common objects on the road include bridges, street lights, signal lights, traffic signs, trees, etc. The pavement layer has been determined in step S104, and the noise points have been determined in steps S301 and S304. The noise points are uniformly divided into the noise layer, and the area between the denoised target two-dimensional image pavement layer and the plane at a preset height distance from the pavement layer is determined as the target area; for example, the area between the denoised RGB image pavement layer and the plane 30 meters away from the pavement layer is determined as the target area; the objects within the range of h1 meters above the pavement layer in the target area are mostly moving motor vehicles, non-motor vehicles, pedestrians, low green belts, etc. These objects are temporarily classified as others and divided into the Others layer. The Others layer is first divided, and then the unprocessed pixels of different heights corresponding to different types of pavement objects in the target area are classified in order from low to high, and the pavement objects during vehicle driving are classified.

[0116] For example, the Others layer is divided first: in the RGB image, there will be a clear boundary in the vertical direction of the road green belt section. When there is a green belt in the middle of the road, there will be a clear gap in the middle of the road surface layer. The point cloud in the pixels above the gap in the road surface layer in the Others layer is clustered, and if it meets the characteristics of the green belt, it is classified as the Green Belt class. The remaining points in the Others layer are clustered and classified into the Car layer.

[0117] For example, bridges are classified into the Bridge layer: According to the conventional settings and driving regulations of vehicles, the minimum clear height of motor vehicle lanes is 4.5m, and point clouds within the range of h2 meters above the road surface layer are searched for clustering. The bridge profile has the characteristics of vertical continuity and horizontal connection in space, and the pixel points are corroded and expanded to extract the bridge in the profile, accurately match it in the road surface object template library, obtain the bridge, and classify it into the Bridge layer.

[0118] Exemplarily, traffic signs are classified into the traffic sign (Road Facility) layer: point clouds within unclassified pixels within a range of h3 meters above the road surface layer are searched for clustering. The profile of rod-shaped objects such as traffic signs is usually perpendicular to the road surface layer, and has the characteristics of continuous vertical upward and horizontal extension at high altitudes. The pixels are corroded and expanded to extract the traffic signs in the profile, and templates that meet the characteristics are matched in the road object template library to obtain traffic signs, which are classified into the traffic sign (Road Facility) layer. Traffic signs are further divided, and geometric shapes such as triangles, circles, and rectangles are identified using feature extraction algorithms in graphics processing. Signs containing triangles are classified into the R1 layer, signs containing rectangles are classified into the R2 layer, and signs containing circles are classified into the R3 layer.

[0119] Exemplarily, traffic lights are classified into the Traffic Light classification layer: According to the road traffic light setting and installation specifications, the installation height of lane traffic lights is 5.5-7m, and the point cloud within the unclassified pixels within the range of h4 meters above the road surface layer is searched for clustering. Traffic lights meet the typical characteristics of the rod-shaped terrain profile, and there are several circular figures in the horizontal direction. The pixel points are corroded and expanded, and the traffic lights in the profile are extracted according to the features. On actual roads, there may be trees blocking the middle of the traffic lights. First, use PCA analysis to remove the tree points to obtain a clean traffic light pole, and then match it in the road surface object template library to obtain the traffic light, which is classified into the Traffic Light classification layer.

[0120] For example, street lights are classified into the street light layer: According to the urban road lighting design standards, lamps are usually installed on lamp poles with a height of less than 15m. The point cloud within the unclassified pixels within the range of h5 meters above the road surface layer is searched for clustering. Street lights meet the profile characteristics of pole-shaped objects. The pixel points are corroded and expanded to extract street lights. On actual roads, there may be trees blocking the middle of the street lights. First, use PCA analysis to remove tree points to obtain clean street light poles, and then match them in the road surface object template library to obtain street lights and classify them into the street light layer.

[0121] For example, trees are classified into the tree layer: the tree profile can be divided into two parts: the trunk and the crown. The trunk part is vertically upward, and the crown part is connected to the trunk and diverges to both sides with the trunk as the center. The unprocessed pixel point cloud is clustered, and points with both trunk and crown features are extracted. They are matched in the road object template library to obtain trees and extracted to the tree layer.

[0122] It should be noted that the distance height of h1-h5 described in the above embodiment is: 0

[0123] A method for classifying objects in a road point cloud provided in an embodiment of the present application collects initial point cloud data during vehicle driving; divides the initial point cloud data according to a first preset division interval to obtain a point cloud section; determines a target two-dimensional image corresponding to the point cloud section based on the number of point clouds in each pixel included in the point cloud section and a mapping relationship between the number of point clouds and grayscale color values; slices the target two-dimensional image along a preset direction according to a second preset division interval to obtain at least one slice layer, and determines a road surface layer through the number of point clouds included in each slice layer; denoises and classifies the target two-dimensional image based on the road surface layer and a preset height distance to determine classification information of road objects during vehicle driving.

[0124] In this way, the technical solution provided by the present application can realize point cloud denoising and classification in a two-dimensional image composed of point cloud sections. Before denoising and classification, the road surface layer is first extracted, which not only reduces the amount of data, but also reduces the impact of road surface points on classification. At the same time, the road surface layer can also be used as a reference for denoising and rough classification. When performing denoising and classification, a two-dimensional point cloud section of a certain width is used as a processing unit, which simplifies the complexity of point cloud data, reduces data processing steps, and helps to improve the efficiency of point cloud data processing.

[0125] Based on the same application concept, the embodiment of the present application also provides a device for classifying objects in a road point cloud corresponding to the method for classifying objects in a road point cloud provided in the above embodiment. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the method for classifying objects in a road point cloud provided in the above embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0126] See also Figure 4 , Figure 4 A schematic diagram of a device for classifying objects in a road point cloud provided in an embodiment of the present application is shown in FIG. Figure 4 As shown in , the classification device 410 includes:

[0127] The acquisition module 411 is used to acquire initial point cloud data during the vehicle driving process;

[0128] A division module 412, configured to divide the initial point cloud data according to a first preset division interval to obtain a point cloud section;

[0129] ​The conversion module 413 is used to determine the target two-dimensional image corresponding to the point cloud section according to the number of point clouds in each pixel included in the point cloud section and the mapping relationship between the number of point clouds and the grayscale color value;

[0130] The analysis module 414 is used to slice the target two-dimensional image along a preset direction according to the second division preset interval to obtain at least one slice layer, and determine the road surface layer according to the number of point clouds included in each slice layer;

[0131] The processing module 415 is used to denoise and classify the target two-dimensional image according to the road surface layer and the preset height distance, and determine the classification information of the road surface objects during the vehicle driving process.

[0132] Optionally, when the division module 412 is used to determine the point cloud section, the division module 412 is used to:

[0133] Determining a plurality of driving nodes and a driving time corresponding to each driving node based on a total driving distance of the vehicle driving process and a first preset division interval;

[0134] The point cloud section is determined based on a plurality of point clouds corresponding to a driving time period between every two driving moments.

[0135] Optionally, when the conversion module 413 is used to determine the target two-dimensional image corresponding to the point cloud section, the conversion module 413 is used to:

[0136] Dividing the horizontal length and the vertical length of the point cloud section according to a preset pixel size to determine a plurality of pixels;

[0137] According to the number of point clouds in each pixel, determine the gray value corresponding to each pixel;

[0138] According to the determined interval position of each pixel, the target two-dimensional image color value of each pixel is determined, and the target two-dimensional image of each point cloud section is established; wherein the interval position of each pixel is determined based on the gray value of the pixel.

[0139] Optionally, when the analysis module 414 is used to determine the pavement layer, the analysis module 414 is used to:

[0140] Slicing the target two-dimensional image along a preset direction according to the second division preset interval to obtain at least one slice layer;

[0141] Obtain the number of point clouds of each slice layer;

[0142] A slice layer whose point cloud number is higher than the point cloud number of each layer except this layer is determined as the pavement layer.

[0143] Optionally, when the processing module 415 is used to denoise the target two-dimensional image, the processing module 415 is used to:

[0144] Determine the point clouds located below the road surface layer and above the road surface layer in the target two-dimensional image and at a distance greater than a preset height of the road surface layer as noise points;

[0145] Determine a target area between the road surface layer and a plane at a preset height distance from the road surface layer, and an average point density in the target area;

[0146] Determine the point density threshold of the target area according to a preset scale factor and an average point density;

[0147] The pixel whose point cloud number is lower than the point density threshold is determined as a candidate noise pixel; if the pixels adjacent to the candidate noise pixel satisfy the conditions that no pixel point density is higher than the point density threshold and more than half of the pixel point densities are lower than the point density threshold, the candidate noise pixel is determined to be a noise pixel, and the point cloud within the noise pixel is determined to be a noise point;

[0148] The determined noise points are removed from the target two-dimensional image.

[0149] Optionally, when the processing module 415 is used to classify the target two-dimensional image and determine the classification information of the road object during driving, the processing module 415 is used to:

[0150] Taking the road surface layer as a reference, determining the area between the denoised target two-dimensional image road surface layer and a plane at a preset height distance from the road surface layer as the target area;

[0151] In the target area, road objects are classified according to the different heights corresponding to different types of road objects during vehicle driving.

[0152] An embodiment of the present application provides a device for classifying objects in a road point cloud, comprising: an acquisition module for acquiring initial point cloud data during vehicle driving; a division module for dividing the initial point cloud data according to a first preset division interval to obtain a point cloud profile; a conversion module for determining a target two-dimensional image corresponding to the point cloud profile based on the number of point clouds in each pixel included in the point cloud profile and a mapping relationship between the number of point clouds and grayscale color values; an analysis module for slicing the target two-dimensional image along a preset direction according to a second preset division interval to obtain at least one slice layer, and determining a road surface layer through the number of point clouds included in each slice layer; and a processing module for denoising and classifying the target two-dimensional image based on the road surface layer and a preset height distance to determine classification information of road objects during vehicle driving.

[0153] In this way, the technical solution provided by the present application can realize point cloud denoising and classification in a two-dimensional image composed of point cloud sections. Before denoising and classification, the road surface layer is first extracted, which not only reduces the amount of data, but also reduces the impact of road surface points on classification. At the same time, the road surface layer can also be used as a reference for denoising and rough classification. When performing denoising and classification, a two-dimensional point cloud section of a certain width is used as a processing unit, which simplifies the complexity of point cloud data, reduces data processing steps, and helps to improve the efficiency of point cloud data processing.

[0154] See also Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 5 As shown in , the electronic device 500 includes a processor 510 , a memory 520 and a bus 530 .

[0155] The memory 520 stores machine-readable instructions executable by the processor 510. When the electronic device 500 is running, the processor 510 communicates with the memory 520 via the bus 530. When the machine-readable instructions are executed by the processor 510, the above-mentioned Figures 1 to 3 The steps of the method for classifying objects in the road point cloud in the method embodiment shown are specifically implemented in the method embodiment and will not be described in detail here.

[0156] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figures 1 to 3 The steps of the method for classifying objects in the road point cloud in the method embodiment shown are specifically implemented in the method embodiment and will not be described in detail here.

[0157] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0158] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0159] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0160] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0161] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application can essentially be embodied in the form of a software product, or in other words, the part that contributes to the prior art or the part of the technical solution. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0162] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for classifying objects in a road point cloud, characterized in that: The classification method includes: Collecting initial point cloud data during vehicle driving; the initial point cloud data is point cloud data obtained by scanning objects around the driving track at 360 degrees during vehicle driving; Based on the total driving distance of the vehicle during driving and a first preset division interval, the initial point cloud data is divided to obtain at least one point cloud section; For each point cloud section, according to the number of point clouds in each pixel included in the point cloud section and the mapping relationship between the number of point clouds and the grayscale color value, determine the target two-dimensional image corresponding to the point cloud section; Slicing the target two-dimensional image along the direction of vehicle travel according to a second preset division interval to obtain at least one slice layer, and determining the road surface layer according to the number of point clouds included in each slice layer; the second preset division interval is a division height perpendicular to the ground; Denoising and classifying the target two-dimensional image according to the road surface layer and the preset height distance to determine classification information of road objects during vehicle driving; The target two-dimensional image corresponding to the point cloud section is determined by the following steps: Dividing the horizontal length and the vertical length of the point cloud section according to a preset pixel size to determine a plurality of pixels; wherein the horizontal length and the vertical length of the point cloud section are determined according to the coordinate range of the point cloud included in the point cloud section; According to the number of point clouds in each pixel, determine the gray value corresponding to each pixel; According to the determined interval position of each pixel, the target two-dimensional image color value of each pixel is determined, and the target two-dimensional image of each point cloud section is established; wherein the interval position of each pixel is determined based on the gray value of the pixel; and the target two-dimensional image is an RGB image; The target two-dimensional image is denoised by the following steps: Determine the point clouds located below the road surface layer and above the road surface layer in the target two-dimensional image and at a distance greater than a preset height of the road surface layer as noise points; Determine a target area between the road surface layer and a plane at a preset height distance from the road surface layer, and an average point density in the target area; Determine the point density threshold of the target area according to a preset scale factor and an average point density; The pixel whose point cloud number is lower than the point density threshold is determined as a candidate noise pixel; if the pixels adjacent to the candidate noise pixel satisfy the conditions that no pixel point density is higher than the point density threshold and more than half of the pixel point densities are lower than the point density threshold, the candidate noise pixel is determined to be a noise pixel, and the point cloud within the noise pixel is determined to be a noise point; Eliminating the determined noise points from the target two-dimensional image; The gray value corresponding to each pixel is determined by the following formula: G = 255 × (1-V) Among them, G is the gray value of the pixel, V is the median value of the current pixel, and N is the number of point clouds in the current pixel. is the minimum value of the number of point clouds in the pixel in the current point cloud profile image. It is the maximum number of point clouds in a pixel in the current point cloud profile image.

2. The classification method according to claim 1, characterized in that: Determine the point cloud section by following these steps: Determining a plurality of driving nodes and a driving time corresponding to each driving node based on a total driving distance of the vehicle driving process and a first preset division interval; The point cloud section is determined based on a plurality of point clouds corresponding to a driving time period between every two driving moments.

3. The classification method according to claim 1, characterized in that: The pavement layers are determined by the following steps: Slicing the target two-dimensional image along a preset direction according to the second division preset interval to obtain at least one slice layer; Obtain the number of point clouds of each slice layer; A slice layer whose point cloud number is higher than the point cloud number of each layer except this layer is determined as the pavement layer.

4. The classification method according to claim 1, characterized in that: The target two-dimensional image is classified by the following steps to determine the classification information of the road object during driving: Taking the road surface layer as a reference, determining the area between the denoised target two-dimensional image road surface layer and a plane at a preset height distance from the road surface layer as the target area; In the target area, road objects are classified according to the different heights corresponding to different types of road objects during vehicle driving.

5. A device for classifying objects in a road point cloud, characterized in that: The classification device comprises: The acquisition module is used to acquire initial point cloud data during the vehicle's driving process; the initial point cloud data is point cloud data obtained by scanning objects around the driving track at 360 degrees during the vehicle's driving process; A division module, configured to divide the initial point cloud data based on the total travel distance of the vehicle during the travel process and a first preset division interval to obtain at least one point cloud section; A conversion module, for determining, for each point cloud section, a target two-dimensional image corresponding to the point cloud section according to the number of point clouds in each pixel included in the point cloud section and a mapping relationship between the number of point clouds and grayscale color values; an analysis module, configured to slice the target two-dimensional image along the direction of vehicle travel according to a second division preset interval to obtain at least one slice layer, and determine a road surface layer according to the number of point clouds included in each slice layer; the second division preset interval is a division height perpendicular to the ground; A processing module, used for denoising and classifying the target two-dimensional image according to the road surface layer and the preset height distance, and determining the classification information of the road surface object during the vehicle driving process; When the conversion module is used to determine the target two-dimensional image corresponding to the point cloud section, the conversion module is used to: Dividing the horizontal length and the vertical length of the point cloud section according to a preset pixel size to determine a plurality of pixels; wherein the horizontal length and the vertical length of the point cloud section are determined according to the coordinate range of the point cloud included in the point cloud section; According to the number of point clouds in each pixel, determine the gray value corresponding to each pixel; According to the determined interval position of each pixel, the target two-dimensional image color value of each pixel is determined, and the target two-dimensional image of each point cloud section is established; wherein the interval position of each pixel is determined based on the gray value of the pixel; and the target two-dimensional image is an RGB image; When the processing module is used to denoise the target two-dimensional image, the processing module is used to: Determine the point clouds located below the road surface layer and above the road surface layer in the target two-dimensional image and at a distance greater than a preset height of the road surface layer as noise points; Determine a target area between the road surface layer and a plane at a preset height distance from the road surface layer, and an average point density in the target area; Determine the point density threshold of the target area according to a preset scale factor and an average point density; The pixel whose point cloud number is lower than the point density threshold is determined as a candidate noise pixel; if the pixels adjacent to the candidate noise pixel satisfy the conditions that no pixel point density is higher than the point density threshold and more than half of the pixel point densities are lower than the point density threshold, the candidate noise pixel is determined to be a noise pixel, and the point cloud within the noise pixel is determined to be a noise point; Eliminating the determined noise points from the target two-dimensional image; The gray value corresponding to each pixel is determined by the following formula: G = 255 × (1-V) Among them, G is the gray value of the pixel, V is the median value of the current pixel, and N is the number of point clouds in the current pixel. is the minimum value of the number of point clouds in the pixel in the current point cloud profile image. It is the maximum number of point clouds in a pixel in the current point cloud profile image.

6. The classification device according to claim 5, characterized in that: When the division module is used to determine the point cloud section, the division module is used to: Determining a plurality of driving nodes and a driving time corresponding to each driving node based on a total driving distance of the vehicle driving process and a first preset division interval; The point cloud section is determined based on a plurality of point clouds corresponding to a driving time period between every two driving moments.

7. An electronic device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the electronic device is running, the processor and the memory communicate through the bus, and the machine-readable instructions are executed by the processor to execute the steps of the classification method as described in any one of claims 1 to 4.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the classification method according to any one of claims 1 to 4 are executed.

Images