Target recognition method, point cloud data processing method, vehicle and medium

Abstract

The present disclosure relates to a target identification method, a point cloud data processing method, a vehicle and a medium, which can be applied to the field of intelligent driving of vehicles. The target identification method comprises: determining an object in environment perception data based on the environment perception data obtained by a collection device of a vehicle; the object corresponds to first point cloud data, and the first point cloud data is obtained according to the environment perception data; performing target identification based on second point cloud data; the second point cloud data is generated after point cloud supplement and / or point cloud sparsification according to the density of point cloud points in the first point cloud data. The generated second point cloud data contains rich geometric information, providing high-reliability data input for subsequent 3D target detection networks, thereby facilitating the improvement of 3D target detection accuracy.

Description

Technical Field

[0001] This disclosure relates to the field of target recognition technology, specifically to a target recognition method, a point cloud data processing method, a vehicle, and a medium. Background Technology

[0002] 3D object detection is a core technology of assisted driving perception systems. It aims to identify and locate objects in three-dimensional space from sensor data, outputting their 3D bounding boxes and attributes. This accurately reconstructs the spatial relationships of objects in the real world, providing crucial information for downstream decision-making and planning. Existing 3D object detection methods struggle to identify small targets and obtain effective feature representations of small or distant targets, affecting the final accuracy of 3D object detection. Summary of the Invention

[0003] The purpose of this disclosure is to provide a target recognition method, a point cloud data processing method, a vehicle, and a medium to solve the aforementioned technical problems.

[0004] To achieve the above objectives, in a first aspect, this disclosure provides a target recognition method, comprising: Based on the environmental perception data acquired by the vehicle's data acquisition device, objects in the environmental perception data are determined; the objects correspond to first point cloud data, which is obtained based on the environmental perception data. Target recognition is performed based on the second point cloud data; the second point cloud data is generated by supplementing and / or thinning the point cloud based on the density of the point cloud points in the first point cloud data.

[0005] Optionally, the object includes a first object and a second object, the first relative point cloud quantity is greater than the second relative point cloud quantity, the first relative point cloud quantity is the change in the number of point cloud points in the second point cloud data of the first object relative to the number of point cloud points in the first point cloud data of the first object, and the second relative point cloud quantity is the change in the number of point cloud points in the second point cloud data of the second object relative to the number of point cloud points in the first point cloud data of the second object.

[0006] Optionally, the object includes a first object and a second object, wherein the number of point cloud points in the first point cloud data of the first object is less than the number of point cloud points in the first point cloud data of the second object, the number of point cloud points in the second point cloud data of the first object is greater than the number of point cloud points in the first point cloud data of the first object, and the number of point cloud points in the second point cloud data of the second object is less than the number of point cloud points in the first point cloud data of the second object.

[0007] Optionally, the volume of the first object is smaller than the volume of the second object, and / or the distance between the first object and the vehicle is greater than the distance between the second object and the vehicle.

[0008] Optionally, the first object and the second object in the environmental perception data are obtained by local sparsity adaptive sampling based on the first point cloud data, wherein the objects corresponding to the sparse regions of the point cloud obtained by adaptive sampling are classified as the first object, and the objects corresponding to the dense regions of the point cloud obtained by adaptive sampling are classified as the second object.

[0009] Optionally, the process of obtaining the first object and the second object by performing local sparsity adaptive sampling based on the first point cloud data includes: Determine the maximum distance between any target point in the first point cloud data and the nearest multiple other point cloud points; Calculate the density of point cloud points within the maximum distance range from the target point cloud points; If the density is less than the set value, the first point cloud data forming region corresponding to the target point cloud point is a sparse point cloud region, and the object corresponding to the sparse point cloud region is classified as the first object. If the density is greater than or equal to the set value, then the first point cloud data forming region corresponding to the target point cloud point is a point cloud point dense region, and the object corresponding to the point cloud point dense region is classified as a second object.

[0010] Optionally, the environmental perception data includes radar point cloud data and image data, and the process of obtaining the first point cloud data based on the environmental perception data includes: Semantic information is extracted from image data acquired by vehicle image acquisition equipment; The first point cloud data is obtained by combining the radar point cloud data acquired by the vehicle's radar equipment with the semantic information.

[0011] Optionally, the semantic information corresponding to the image data acquired by the vehicle-based image acquisition device is extracted, including: A semantic segmentation network is used to perform pixel semantic segmentation on the image data to obtain the semantic category label of each pixel in the image data; Based on the semantic category label of each pixel, semantic information corresponding to each pixel in the image data is generated.

[0012] Optionally, the radar point cloud data acquired by the vehicle-based radar device and the semantic information are combined to obtain first point cloud data, including: The radar equipment of the vehicle and the image acquisition equipment are time-stamped to obtain radar point cloud data and image data at the same time. Obtain the relative position between the radar device and the image acquisition device; Based on the relative position, the semantic information is mapped point by point to the corresponding radar point cloud data according to the corresponding pixel to obtain the first point cloud data.

[0013] Optionally, the second point cloud data is generated by supplementing the point cloud data with the density of point cloud points in the first point cloud data, including: If the density of point cloud points in the first point cloud data is less than a set value, then based on the category constraints of the semantic information corresponding to the first point cloud data, the range corresponding to the object is fitted by combining the preset value of the category constraints. Supplementary point cloud data is generated by sampling within the specified range; The supplementary point cloud data is added to the point cloud data of the object to generate the second point cloud data.

[0014] Optionally, the second point cloud data is generated by performing point cloud sparsity processing based on the density of point cloud points in the first point cloud data, including: If the density of point cloud points in the first point cloud data is greater than or equal to a set value, then the curvature value between adjacent point cloud points in the first point cloud data is calculated, and the point cloud points with curvature values ​​greater than the curvature threshold are retained to generate the second point cloud data.

[0015] Optionally, before target recognition, the second point cloud data is further optimized, and the optimization process includes at least one of the following: The second point cloud data is denoised to generate denoised second point cloud data. The second point cloud data is smoothed to generate smoothed second point cloud data. Perform semantic consistency verification on the second point cloud data to generate semantically consistent second point cloud data.

[0016] Secondly, the present invention provides a point cloud data processing method, the method comprising: Acquire environmental perception data, determine the objects in the environmental perception data, wherein the objects are obtained by distinguishing and processing based on the density of point cloud points in the first point cloud data in the environmental perception data; The object also corresponds to the second point cloud data. If the sparsity of the first point cloud data of the object is greater than a set value, then the number of point cloud points in the second point cloud data of the object is greater than the number of point cloud points in the first point cloud data of the object; if the sparsity of the first point cloud data of the object is less than or equal to the set value, then the number of point cloud points in the second point cloud data of the object is less than the number of point cloud points in the first point cloud data of the object.

[0017] Thirdly, the present invention provides a vehicle comprising: a memory having a computer program stored thereon; and a processor for executing the computer program in the memory to implement the steps of the method described in the first aspect.

[0018] Fourthly, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the method described in the first aspect.

[0019] The above technical solution utilizes the local density of point cloud points in the point cloud data to perform point cloud supplementation and optimization on sparse point cloud points in the first point cloud data, or to perform point cloud sparsity optimization on dense point cloud points, thereby generating second point cloud data. The generated second point cloud data contains rich geometric information, providing highly reliable data input for subsequent 3D object detection networks, thus contributing to improved 3D object detection accuracy.

[0020] Other features and advantages of this disclosure will be described in detail in the following detailed description section. Attached Figure Description

[0021] The accompanying drawings are provided to further illustrate the present disclosure and form part of the specification. They are used together with the following detailed description to explain the present disclosure, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart illustrating a target recognition method provided in an embodiment of this disclosure.

[0022] Figure 2 This is a flowchart illustrating a point cloud data processing method provided in an embodiment of the present disclosure.

[0023] Figure 3 This is a schematic diagram of the structure of a target recognition device provided in an embodiment of the present disclosure. Detailed Implementation

[0024] The specific embodiments of this disclosure will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit this disclosure.

[0025] Most existing 3D target detection methods rely on point cloud data acquired by LiDAR or image data acquired by cameras for target detection. However, the inventors' research revealed that existing methods for detecting distant or small targets suffer from high false negative rates due to sparse point cloud echoes and blurred geometric features. Directly using image data is susceptible to the effects of lighting, occlusion, and sparse pixels at distances, leading to large depth estimation errors. This introduces errors in subsequent 3D target detection in scenes with sparse point clouds.

[0026] To address this, this disclosure provides a target recognition method. This method enhances the representation of geometric features for distant or small targets by supplementing sparse areas of the first point cloud data in the environmental perception data according to the density of the point cloud points. Furthermore, it sparses the point cloud points in dense areas, for example, retaining only the first point cloud data that can characterize key geometric features such as object edges and corners, to reduce interference from redundant data. The optimized second point cloud data possesses geometric integrity, providing highly reliable data input for subsequent 3D target detection networks.

[0027] Figure 1 This is a schematic flowchart illustrating a target recognition method provided in an embodiment of this disclosure. Figure 1 As shown, the method may include the following steps: S101: Based on the environmental perception data acquired by the vehicle's data acquisition device, determine the objects in the environmental perception data; the objects correspond to the first point cloud data, which is obtained from the environmental perception data.

[0028] In one embodiment of this disclosure, the vehicle's data acquisition device may include various sensors such as LiDAR and image acquisition equipment. Environmental perception data is acquired through the vehicle's acquisition device; for example, raw point cloud data can be acquired via LiDAR, or image data can be acquired via a depth camera. After acquiring the environmental perception data, a geometry-based clustering algorithm can be used to group the discrete points in the raw point cloud data into different regions, with each region corresponding to an object. A correspondence is established between this object and first point cloud data. The first point cloud data can be raw point cloud data or point cloud data obtained through multimodal fusion with image data. For example, a Euclidean distance clustering method can be used to group closely spaced point cloud points into the same object, and its raw point cloud data can be used as the first point cloud data.

[0029] S102: Target recognition is performed based on the second point cloud data; the second point cloud data is generated by supplementing and / or thinning the point cloud based on the density of the point cloud points in the first point cloud data.

[0030] In one embodiment of this disclosure, the second point cloud data is obtained by optimizing the density of point cloud points in the first point cloud data. Specifically, the density of each point cloud point can be obtained by performing a global density analysis on the first point cloud data. If the density is lower than a preset threshold, it is considered sparse; otherwise, it is considered dense. When the point cloud points in the first point cloud data are sparse in certain areas, new point cloud points are generated in those areas to supplement the first point cloud data, thereby increasing the point cloud density in those areas. The new point cloud points can be generated by random sampling within the boundaries of the geometric region. When the point cloud points in the first point cloud data are dense in certain areas, some point cloud points in those areas are removed to reduce the point cloud density in those areas while preserving key geometric features. For example, this can be achieved by uniform downsampling, removing some point cloud points at fixed intervals or proportions.

[0031] By initially adjusting the number of points in the first point cloud data, a second point cloud data is generated, providing relatively uniform and easier-to-process point cloud data for subsequent target recognition. Based on the second point cloud data, target recognition is performed to obtain various targets in the vehicle's surrounding environment, as well as the targets' position, orientation, size, and other attributes.

[0032] By supplementing or sparsening the first point cloud data, a second point cloud data more suitable for target recognition is generated. This effectively solves the problem of insufficient feature representation caused by sparse or uneven point cloud data when traditional 3D target detection methods identify small or distant targets, thereby improving the recognition accuracy of various targets in the driver assistance system.

[0033] As an optional implementation, the object includes a first object and a second object, wherein the first relative point cloud quantity is greater than the second relative point cloud quantity. The first relative point cloud quantity is the change in the number of point cloud points in the second point cloud data of the first object relative to the number of point cloud points in the first point cloud data of the first object, and the second relative point cloud quantity is the change in the number of point cloud points in the second point cloud data of the second object relative to the number of point cloud points in the first point cloud data of the second object.

[0034] In one embodiment of this disclosure, objects can be divided into a first object and a second object based on their corresponding relative point cloud count. The relative point cloud count refers to the change in the number of point clouds in the second point cloud data of an object relative to the number of point clouds in the first point cloud data after the first point cloud data of the object has undergone point cloud supplementation and / or point cloud sparsification processing.

[0035] For the first object, the difference between the number of point clouds in its second point cloud data and the number of point clouds in its first point cloud data is the first relative point cloud quantity. For the second object, the change in the number of point clouds in its second point cloud data compared to the number of point clouds in its first point cloud data is the second relative point cloud quantity. When the first relative point cloud quantity is greater than the second relative point cloud quantity, that is, during point cloud processing, the change in the number of point clouds in the first object is greater than the change in the number of point clouds in the second object. The first object is a smaller or more distant target object relative to the second object, meaning that the point cloud of the first object is optimized to a higher degree to enhance its feature representation capability.

[0036] By comparing these two relative point cloud quantities, the degree of change in data volume after point cloud processing for different objects can be quantified. Differentiated processing of point cloud data based on the different characteristics of objects allows for the division of objects into primary and secondary objects, enabling targeted point cloud data supplementation or sparsification for different objects. For example, for primary objects that are farther away or smaller, more comprehensive point cloud supplementation can be performed to provide richer geometric and semantic information, thereby improving their recognition accuracy; while for secondary objects that are closer or larger, appropriate point cloud sparsification can be performed to reduce data volume and computational load, improving processing efficiency. This effectively addresses the limitations of uniform processing and improves the overall performance of target recognition.

[0037] As an optional implementation, the object includes a first object and a second object, wherein the number of point cloud points in the first point cloud data of the first object is less than the number of point cloud points in the first point cloud data of the second object, the number of point cloud points in the second point cloud data of the first object is greater than the number of point cloud points in the first point cloud data of the first object, and the number of point cloud points in the second point cloud data of the second object is less than the number of point cloud points in the first point cloud data of the second object.

[0038] In one embodiment of this disclosure, when the number of point cloud points in the second point cloud data of a first object is greater than the number of point cloud points in the first point cloud data of the first object, the point cloud points in the first point cloud data of the first object are sparse, and the number of point cloud points in the first point cloud data of the first object is optimized to supplement the optimization. This allows us to determine whether the first object is a small target or a distant target. Conversely, when the number of point cloud points in the second point cloud data of a second object is less than the number of point cloud points in the first point cloud data of the second object, the first point cloud data of the second object is dense, and the number of point cloud points in the first point cloud data of the second object is optimized to sparse the optimization. This allows us to determine whether the second object is a large target or a close-range target in the vehicle environment. Based on the changes in the number of point cloud points before and after optimization, different target objects in the vehicle environment can be classified.

[0039] Specifically, in applications such as assisted driving, the types of targets in the environment are diverse, such as pedestrians, vehicles, and traffic signs. These targets vary significantly in size, distance, and material, resulting in different densities and information contents in their raw point cloud data. By classifying these targets into primary and secondary objects, a basis for subsequent target recognition can be provided.

[0040] In another embodiment of this disclosure, the number of point cloud points in the first point cloud data of the first object is less than the number of point cloud points in the first point cloud data of the second object. The first object typically corresponds to a target whose point cloud data is relatively sparse during initial acquisition. This target includes, but is not limited to, distant or small targets, such as distant pedestrians or bicycles, nearby stone blocks, or specific area targets that are difficult for the acquisition device to fully cover. In contrast to the first object, the second object corresponds to an area where the initial point cloud data is relatively dense, such as nearby or large vehicles or obstacles.

[0041] This distinction based on the initial point cloud point count lays the foundation for subsequent differentiated processing. The number of point clouds in the second point cloud data for the first object is greater than the number of point clouds in the first point cloud data for the first object. This means that for the sparse point cloud identified as the first object, point cloud supplementation optimization is performed to increase the density of point clouds in sparse areas, thereby compensating for the insufficient information in the original first point cloud data. This provides richer geometric features for target recognition, improving the accuracy of identifying small, distant, or difficult-to-detect objects. Conversely, the number of point clouds in the second point cloud data for the second object is less than the number of point clouds in the first point cloud data for the second object. This means that for the dense point cloud identified as the second object, point cloud sparsity optimization is performed. Point cloud sparsity removes redundant information from the first point cloud data, reducing data volume and computational complexity without losing key features, thus improving processing efficiency. This is particularly suitable for large, close-range, or feature-rich objects. This differentiation and optimization based on the initial point cloud density characteristics of the object allows for the optimization of computational resources while maintaining recognition accuracy.

[0042] As an optional implementation, the volume of the first object is smaller than the volume of the second object, and / or the distance between the first object and the vehicle is greater than the distance between the second object and the vehicle.

[0043] In one embodiment of this disclosure, when the number of point cloud points in the first point cloud data of a first object is less than the number of point cloud points in the first point cloud data of a second object, and the number of point cloud points in the second point cloud data of the first object is greater than the number of point cloud points in the first point cloud data of the first object, and the number of point cloud points in the second point cloud data of the second object is less than the number of point cloud points in the first point cloud data of the second object, the corresponding volume of the first object is smaller than the volume of the second object, and / or the distance between the first object and the vehicle is greater than the distance between the second object and the vehicle. In this way, objects can be classified according to their actual physical size or distance from the vehicle, thereby providing a more targeted basis for subsequent point cloud processing.

[0044] As an optional implementation, the first object and the second object in the environmental perception data are obtained by local sparsity adaptive sampling based on the first point cloud data. Specifically, the objects corresponding to the sparse regions of the point cloud obtained by adaptive sampling are classified as the first object, and the objects corresponding to the dense regions of the point cloud obtained by adaptive sampling are classified as the second object.

[0045] In one embodiment of this disclosure, local density adaptive sampling is a sampling method that dynamically adjusts the sampling based on the distribution density of point cloud data in local regions. The sparsity is determined by analyzing the point cloud point distribution density in each local region of the first point cloud data. For example, a neighborhood search-based method can be used to calculate the number of point cloud points within a preset neighborhood for each point cloud point, or to calculate the average distance from a point cloud point to its nearest neighbor, thereby characterizing the local sparsity. This adaptive sampling method avoids the limitations of using fixed thresholds or preset rules for object classification, making the classification results closer to the actual distribution characteristics of the point cloud data.

[0046] Objects corresponding to sparse regions of the point cloud obtained through adaptive sampling are classified as first objects. Sparse regions typically correspond to objects with limited point cloud data and large distances between points, such as distant or small targets, or partially occluded targets. Identifying these objects as first objects allows for targeted point cloud supplementation and optimization to improve the completeness and recognizability of their geometric features. Simultaneously, objects corresponding to dense regions of the point cloud obtained through adaptive sampling are classified as second objects. Dense regions typically correspond to objects with abundant point cloud data and small distances between points, such as close-range, large targets, or targets with rich detail. Identifying these objects as second objects allows for subsequent point cloud sparsity optimization to effectively reduce data redundancy without losing key features, thereby improving the efficiency of subsequent target recognition.

[0047] As an optional implementation, the process of obtaining a first object and a second object by performing local sparsity adaptive sampling based on the first point cloud data includes: determining the maximum distance between any target point cloud point in the first point cloud data and the nearest plurality of other point cloud points; calculating the density of point cloud points within the maximum distance range from the target point cloud point; if the density is less than a set value, the first point cloud data forming region corresponding to the target point cloud point is a sparse point cloud region, and the object corresponding to the sparse point cloud region is classified as the first object; if the density is greater than or equal to the set value, the first point cloud data forming region corresponding to the target point cloud point is a dense point cloud region, and the object corresponding to the dense point cloud region is classified as the second object.

[0048] In one embodiment of this disclosure, the maximum distance between the target point cloud point and its nearest other point cloud points is specifically determined. For example, the K-Nearest Neighbors (KNN) algorithm can be used to find the K nearest neighbors for each target point cloud point, calculate the distance from the target point cloud point to its K nearest neighbors, and select the maximum distance as the local neighborhood radius of the target point cloud point, i.e., the maximum distance. Through an adaptive neighborhood determination method, the neighborhood size can be dynamically adjusted according to the different local density of the point cloud, thereby more accurately reflecting local features.

[0049] After determining the maximum distance range for each target point cloud point, the density of point cloud points within the maximum distance range is calculated, and this density is defined as the density of the target point cloud point. Specifically, the number of point cloud points contained within a spherical region centered on the target point cloud point and with the maximum distance as its radius can be calculated. This number can be divided by the volume of the spherical region to obtain the density of the target point cloud point. This density visually reflects the density of the area surrounding the target point cloud point.

[0050] Based on the calculated density, the objects corresponding to the first point cloud data are classified. Specifically, the regions where the first point cloud data forms, corresponding to target point cloud points with a density less than a set value, are designated as sparse point cloud regions, and the objects corresponding to these sparse regions are classified as first objects. First objects typically correspond to small or distant target objects around the vehicle. Simultaneously, the regions where the first point cloud data forms, corresponding to target point cloud points with a density greater than or equal to a set value, are designated as dense point cloud regions, and the objects corresponding to these dense regions are classified as second objects. These points are usually located in dense regions of the point cloud and may contain redundancy, requiring point cloud sparsity optimization. The set value can be empirically set or determined through statistical analysis based on the actual application scenario and the characteristics of the point cloud data. By adaptively determining the local density of point cloud points in each first point cloud data set, the limitations of calculating density using a fixed radius or a fixed number of neighbors are avoided. This allows for more accurate identification of sparse and dense regions in the first point cloud data, providing an optimization basis for subsequent point cloud supplementation and sparsity optimization. This makes subsequent optimization more targeted and helps improve the uniformity of the point cloud data.

[0051] In one embodiment of this disclosure, after the density is calculated, it is normalized, and the normalized density value is between 0 and 1. Based on the comparison between the normalized density and a set value, the object corresponding to the first point cloud data formation area is divided into a first object or a second object. Using normalized density facilitates the comparison and division of the first point cloud data.

[0052] As an optional implementation, the environmental perception data includes radar point cloud data and image data. The process of obtaining the first point cloud data based on the environmental perception data includes: extracting semantic information corresponding to the image data acquired by the vehicle's image acquisition device; and obtaining the first point cloud data based on the radar point cloud data and semantic information acquired by the vehicle's radar device.

[0053] In one embodiment of this disclosure, radar point cloud data is acquired using a vehicle's LiDAR. This radar point cloud data represents a set of three-dimensional spatial coordinates of the surrounding object surfaces. Image data is acquired using an image acquisition device on the vehicle, such as a depth camera. This image data represents the surrounding two-dimensional pixel information. Semantic information describes the object category or attribute represented by each pixel or region in the image data. For example, pixels in the image can be labeled as "vehicle," "pedestrian," or "road." Semantic information can be extracted using manual annotation or image processing algorithms. By establishing a spatial correspondence between the pixels corresponding to the semantic information in the image data and the point cloud points in the radar point cloud data, the semantic information corresponding to a pixel in the image is mapped to the point cloud points where the pixel's two-dimensional mapping position coincides, generating first point cloud data. Each point cloud point in the first point cloud data not only has three-dimensional spatial coordinates but also corresponding semantic information, providing a more accurate data foundation for subsequent object recognition and processing.

[0054] As an optional implementation, the semantic information corresponding to the image data acquired by the vehicle's image acquisition device is extracted, including: performing pixel semantic segmentation on the image data using a semantic segmentation network to obtain the semantic category label of each pixel in the image data; and generating semantic information corresponding to each pixel in the image data based on the semantic category label of each pixel.

[0055] In one embodiment of this disclosure, a semantic segmentation network is used to perform pixel-level semantic segmentation on image data to classify each pixel in the image and identify its semantic category, such as road, vehicle, pedestrian, building, tree, etc., thereby achieving an understanding of the image data. The semantic segmentation network is a pre-trained semantic segmentation network capable of performing semantic segmentation on images within a scene. After processing the image data, the semantic segmentation network outputs a semantic image, where each pixel corresponds to a semantic category label. After obtaining the semantic category label for each pixel, semantic information corresponding to each pixel is generated based on these semantic category labels. The semantic information can be a direct representation of the semantic category label, such as using encoding to represent different categories, like 0 representing road, 1 representing vehicle, etc., or it can be converted into a feature vector. Using a semantic segmentation network to perform pixel-level semantic segmentation on image data can obtain the semantic category label for each pixel in the image, avoiding the tediousness and inefficiency of manual annotation. Based on this, generating corresponding semantic information based on the semantic category label of each pixel ensures the accuracy of the semantic information. This method provides a data foundation for subsequently associating semantic information with radar point cloud data, which is beneficial to the accuracy of the first point cloud data generation, thereby laying the foundation for subsequent point cloud optimization and enhancement, and improving the usability and application value of the second point cloud data.

[0056] As an optional implementation, the first point cloud data is obtained based on the radar point cloud data and semantic information acquired by the vehicle's radar equipment, including: synchronizing the radar equipment and image acquisition equipment of the vehicle with timestamps to obtain radar point cloud data and image data at the same time; obtaining the relative position between the radar equipment and the image acquisition equipment; and mapping the semantic information pixel by pixel onto the corresponding radar point cloud data based on the relative position to obtain the first point cloud data.

[0057] In one embodiment of this disclosure, radar point cloud data is acquired by multiple sensors on the vehicle, and image data is acquired by an image acquisition device on the vehicle. These multiple sensors may include, but are not limited to, lidar and millimeter-wave radar, etc., and acquire three-dimensional geometric information of the vehicle's surrounding environment to form radar point cloud data. The image acquisition device can be an onboard camera used to acquire two-dimensional image information of the vehicle's surrounding environment. To ensure temporal consistency of data acquired by different sensors, the timestamps of multiple sensors and the image acquisition device are synchronized to obtain radar point cloud data and image data at the same time. Specifically, a unified hardware clock signal can be used to synchronize and trigger multiple sensors and the image acquisition device, or a software algorithm can be used to calibrate and align the timestamps of the acquired radar point cloud data and image data.

[0058] Next, it is necessary to obtain the relative positions between multiple sensors and image acquisition devices. This relative position can be obtained through the sensor calibration process. For example, joint calibration between LiDAR and a camera can be achieved by collecting calibration data and using feature point matching and optimization algorithms to solve for the transformation matrix between multiple sensors and image acquisition devices. Accurate relative position information is fundamental to achieving spatial alignment of data from different sensors, ensuring that the semantic information extracted from the image can be accurately mapped to the corresponding point cloud points in the radar point cloud data.

[0059] Finally, based on relative position, semantic information is mapped pixel-by-pixel onto the corresponding radar point cloud data to generate the first point cloud data. The mapping process includes coordinate transformation and projection. Using parameters from the image acquisition device, each point in the radar point cloud data is projected onto the image data to obtain its corresponding pixel coordinates. Then, the semantic information corresponding to these pixel coordinates in the image data is obtained and associated with the point. This generates the first point cloud data, where each point contains not only three-dimensional coordinates but also semantic information. The first point cloud data provides semantic constraints for subsequent point cloud supplementation optimization and sparsity optimization, making the optimization of the point cloud data more accurate.

[0060] As an optional implementation, the second point cloud data is generated by supplementing the point cloud data with the density of the point cloud points in the first point cloud data, including: if the density of the point cloud points in the first point cloud data is less than a set value, then based on the category constraints of the semantic information corresponding to the first point cloud data, and combined with the preset value of the category constraints, the range corresponding to the object is fitted to obtain the range; sampling within the range to generate supplementary point cloud data; adding the supplementary point cloud data to the point cloud data of the object to generate the second point cloud data.

[0061] In one embodiment of this disclosure, the category constraint of semantic information utilizes the semantic category corresponding to the semantic information associated with each point in the first point cloud data to provide an optimization basis for subsequent first point cloud data. For example, the semantic information includes roads, buildings, plants, etc., where road surfaces are usually flat and continuous, buildings are usually regular planes or curved surfaces, and plants are irregular shapes, etc. A minimum 3D target range is fitted based on the target category and preset values, and the center point coordinates, length, width, height, yaw angle, and other data of the target range are calculated. Then, supplementary point cloud data is generated by uniformly sampling the surface of the target range. For example, for the road category, preset values ​​may include flatness and slope range; for the wall category, preset values ​​may include verticality and flatness, etc. Using these preset values, combined with the semantic category information of existing point points in the first point cloud data, the reasonable spatial area that the point points of that category should occupy is determined. Specifically, existing geometric fitting algorithms can be used. Sampling can be one or more of uniform sampling, random sampling, density-based sampling, or geometric feature-based sampling. For example, new point cloud points can be uniformly generated on the fitted plane or curved surface to fill sparse regions; or interpolation sampling can be performed based on the distribution of surrounding points in areas where gaps are detected. By increasing the density of point cloud points and filling gaps, the point cloud data becomes geometrically complete. Finally, the supplementary point cloud data is added to the first point cloud data to generate the second point cloud data. This process ensures the preservation of the semantic information of the original first point cloud data, while compensating for the sparsity of the first point cloud data by supplementing the point cloud points.

[0062] As an optional implementation, the second point cloud data is generated after point cloud sparsity processing based on the density of point cloud points in the first point cloud data, including: if the density of point cloud points in the first point cloud data is greater than or equal to a set value, then the curvature value between adjacent point cloud points in the first point cloud data is calculated, and point cloud points with curvature values ​​greater than the curvature threshold are retained to generate the second point cloud data.

[0063] In one embodiment of this disclosure, curvature value is a geometric attribute data describing the degree of curvature of the point cloud data surface, used to reflect changes in the local geometry of the point cloud data. For example, the curvature of a plane is close to zero, while the curvature value is larger at edges, corners, or sharp features. When the density of point cloud points in the first point cloud data is greater than or equal to a set value, the region formed by the first point cloud data is a dense region, and point cloud sparsity processing is performed on the first point cloud data. First, the curvature value between point cloud points in the first point cloud data is calculated. This can be done using a normal vector-based method, that is, by estimating the local normal vector of each point cloud point in the first point cloud data and analyzing the changes in the normal vector within the neighborhood to calculate the curvature. After calculating the curvature value between adjacent first point cloud data, sparsity optimization processing is performed by retaining the first point cloud data with curvature values ​​greater than the curvature threshold.

[0064] In one embodiment of this disclosure, a method based on fitted surfaces can also be used. A quadratic surface is fitted within the local neighborhood of each point in the first point cloud data, and then the curvature of that point is calculated based on the parameters of the fitted surface. Calculating the curvature value provides quantified geometric feature information for subsequent point cloud sparse optimization. After calculating the curvature values ​​between adjacent first point cloud data, sparse optimization is performed by retaining first point cloud data with curvature values ​​greater than a curvature threshold. The curvature threshold is a preset parameter, or it can be set according to the actual application scenario, the characteristics of the point cloud data, or determined by analyzing the curvature distribution of the point cloud data. By retaining first point cloud data with curvature values ​​exceeding the preset curvature threshold, sparse optimization of the first point cloud data is achieved, resulting in optimized second point cloud data. This reduces the amount of point cloud data while preserving key geometric features and avoids interference from redundant data.

[0065] In another embodiment of this disclosure, a surface fitting method is used to calculate the curvature value and a preset curvature threshold. Local surface fitting is performed on each point in the first point cloud data and its corresponding K nearest neighbors to calculate its curvature value. Then, the mean and standard deviation of the curvature of each point and its K nearest neighbors are calculated, and the sum of the mean and standard deviation is set as the curvature threshold for that point. Finally, the curvature value is compared with the curvature threshold; if the curvature value is greater than the threshold, the point is retained; otherwise, it is discarded.

[0066] As an optional implementation, before target recognition, the second point cloud data is further optimized. The optimization process includes at least one of the following: denoising the second point cloud data to generate denoised second point cloud data; smoothing the second point cloud data to generate smoothed second point cloud data; and performing semantic consistency verification on the second point cloud data to generate semantically consistent second point cloud data.

[0067] In one embodiment of this disclosure, to further improve the quality of the second point cloud data, optimization processing is performed on the second point cloud data. The optimization processing includes at least one of denoising processing, smoothing processing, and semantic consistency verification processing. The denoising processing can employ a statistical outlier removal method, which calculates the average distance between each point and its neighboring points and removes outliers whose average distance exceeds a set threshold. Alternatively, filtering-based methods, such as median filtering or bilateral filtering, can be used to smooth noise by considering the point's position and normal information. Denoising processing can improve the accuracy of the second point cloud data.

[0068] Smoothing can be achieved through methods such as local surface fitting or moving averages. Specifically, moving least squares can be used to fit a local surface within the neighborhood of each point cloud point and project that point onto the fitted surface, thus achieving local smoothing. Alternatively, Laplacian smoothing can be used, iteratively moving each point cloud point to the average position of its neighborhood points to achieve a smoothing effect. Smoothing can make the distribution of the second point cloud data more uniform, avoid abrupt density changes, and make the second point cloud data more naturally smooth.

[0069] Semantic consistency verification is performed on the second point cloud data to ensure that the semantic information of each point in the second point cloud data is consistent with the semantic information of its surrounding environment or the object to which it belongs. Semantic consistency verification can be implemented through various strategies. Specifically, a neighborhood voting mechanism can be used, which involves statistically analyzing the semantic categories of the point clouds surrounding a given point and correcting the semantic category of that point cloud to the category with the highest frequency in the neighborhood. Alternatively, a check based on preset semantic rules can be used; for example, stipulating that a "vehicle" point cloud cannot appear above a "sky" point cloud. Through semantic consistency verification, the generated second point cloud data is ensured to conform to the physical constraints of the semantic labels, avoiding logical errors, improving the semantic accuracy of the second point cloud data, and providing a solid foundation for subsequent object detection.

[0070] This disclosure provides a point cloud data processing method, which includes: acquiring environmental perception data, determining objects in the environmental perception data, wherein the objects are obtained by distinguishing and processing based on the density of point cloud points in a first point cloud data in the environmental perception data. The objects also correspond to a second point cloud data. If the sparsity of the first point cloud data of the object is greater than a set value, then the number of point cloud points in the second point cloud data of the object is greater than the number of point cloud points in the first point cloud data of the object; if the sparsity of the first point cloud data of the object is less than or equal to a set value, then the number of point cloud points in the second point cloud data of the object is less than the number of point cloud points in the first point cloud data of the object.

[0071] In one embodiment of this disclosure, the environmental perception data includes radar point cloud data and image data. First point cloud data is obtained based on the radar point cloud data, or semantic information from the image data can be combined with the radar point cloud data to obtain the first point cloud data. The number of point clouds in the first point cloud data is optimized based on the density of the object's first point cloud data. When the sparsity of the object's first point cloud data is greater than a set value, indicating an insufficient number of point clouds in that area, point cloud point supplementation optimization is performed to increase the number of point clouds, resulting in a greater number of point clouds in the object's second point cloud data than in the object's first point cloud data, thus strengthening feature representation. When the sparsity of the first point cloud data is less than or equal to a set value, indicating a dense point cloud in that area, point cloud sparsity optimization is performed to reduce the number of point clouds, meaning the number of point clouds in the object's second point cloud data is less than the number of point clouds in the object's first point cloud data, thus eliminating redundancy.

[0072] In one embodiment of this disclosure, the sparsity of the first point cloud data can be achieved based on local sparsity adaptive sampling. Specifically, the maximum distance between the target point cloud point and its neighboring point cloud points in the first point cloud data is calculated, and the point cloud point density within this distance range is statistically analyzed. If the density is less than a set value, it is determined to be a sparse point cloud region, and the first point cloud data is supplemented to generate second point cloud data; if the density is greater than or equal to the set value, it is determined to be a dense point cloud region, and the first point cloud data is sparsified to generate second point cloud data. Based on this, the point cloud supplementation operation can combine the category constraints of the semantic information corresponding to the image data to fit and generate supplementary point cloud data within the object range; the point cloud sparsity operation can be based on curvature value filtering, retaining key point cloud points with curvature greater than a threshold. Through the above technical solution, the distribution characteristics of the point cloud data can be dynamically optimized, supplementing effective point cloud points for sparse regions of small targets or distant targets, while reducing redundant point cloud points in dense regions, thereby improving the accuracy and efficiency of target recognition and providing more reliable environmental understanding data for assisted driving perception systems. In one embodiment of this disclosure, the object may include a first object and a second object, wherein the first relative point cloud quantity is greater than the second relative point cloud quantity, the first relative point cloud quantity is the change in the number of point cloud points in the second point cloud data of the first object relative to the number of point cloud points in the first point cloud data of the first object, and the second relative point cloud quantity is the change in the number of point cloud points in the second point cloud data of the second object relative to the number of point cloud points in the first point cloud data of the second object.

[0073] In another embodiment of this disclosure, the object may include a first object and a second object. The number of point cloud points in the first point cloud data of the first object is less than the number of point cloud points in the first point cloud data of the second object, the number of point cloud points in the second point cloud data of the first object is greater than the number of point cloud points in the first point cloud data of the first object, and the number of point cloud points in the second point cloud data of the second object is less than the number of point cloud points in the first point cloud data of the second object. In another embodiment of this disclosure, the object may include a first object and a second object, obtained by performing local sparsity adaptive sampling based on the first point cloud data. The specific steps include: determining the maximum distance between any target point cloud point in the first point cloud data and the nearest plurality of other point cloud points; calculating the density of point cloud points within the maximum distance range from the target point cloud point; if the density is less than a set value, the first point cloud data forming region corresponding to the target point cloud point is a sparse point cloud region, and the object corresponding to the sparse point cloud region is classified as the first object; if the density is greater than or equal to the set value, the first point cloud data forming region corresponding to the target point cloud point is a dense point cloud region, and the object corresponding to the dense point cloud region is classified as the second object.

[0074] In another embodiment of this disclosure, the environmental perception data includes radar point cloud data and image data, and the first point cloud data can be obtained based on the environmental perception data. First, a semantic segmentation network is used to perform pixel semantic segmentation on the image data to obtain the semantic category label for each pixel in the image data; semantic information corresponding to each pixel in the image data is generated based on the semantic category label of each pixel. Next, the vehicle's radar device and image acquisition device are time-stamped to obtain radar point cloud data and image data at the same time, and the relative position between the radar device and the image acquisition device is obtained. Finally, based on the relative position, the semantic information is mapped pixel by pixel onto the corresponding radar point cloud data to obtain the first point cloud data.

[0075] Figure 2 This is a flowchart illustrating a point cloud data processing method provided in an embodiment of this disclosure. Figure 2 As shown, the method includes: S201: Acquire the collected radar point cloud data and image data.

[0076] S202: Extract the semantic information corresponding to the image data.

[0077] S203: Associate semantic information with radar point cloud data to generate the first point cloud data.

[0078] S204: Based on the density of the first point cloud data, the region corresponding to the first point cloud data is divided into a first object and a second object.

[0079] S205: Perform point cloud supplementation optimization on the first point cloud data of the first object to obtain the second point cloud data of the first object, and perform point cloud sparsity optimization on the first point cloud data of the second object to obtain the second point cloud data of the second object; S206: Optimize the second point cloud data of the first and second objects.

[0080] As an optional implementation, radar point cloud data is semantically correlated with image data, imbuing the first point cloud data with semantic information. Then, based on the density of the first point cloud data, objects corresponding to sparse point cloud regions are classified as first objects, and objects corresponding to dense point cloud regions are classified as second objects. Different optimization processes are applied to the first and second objects. Specifically, point cloud supplementation optimization is performed on the first point cloud data of the first object based on semantic constraints within the first point cloud data. Sparse optimization is performed on the point cloud data of the second object. This generates second point cloud data with a more uniform density distribution and more complete feature information. The optimized second point cloud data improves the point cloud sparsity problem for small or distant targets, while reducing point cloud redundancy for large or near-distance targets in dense regions, providing higher-quality input for subsequent 3D target detection and contributing to improved accuracy.

[0081] As an optional implementation, the change in the number of point cloud points in the second point cloud data of the first object relative to the number of point cloud points in the first point cloud data of the first object is called the first relative point cloud number, and the change in the number of point cloud points in the second point cloud data of the second object relative to the number of point cloud points in the first point cloud data of the second object is called the second relative point cloud number. The first relative point cloud number is greater than the second relative point cloud number.

[0082] As an optional implementation, the number of point cloud points in the first point cloud data of the first object is less than the number of point cloud points in the first point cloud data of the second object, the number of point cloud points in the second point cloud data of the first object is greater than the number of point cloud points in the first point cloud data of the first object, and the number of point cloud points in the second point cloud data of the second object is less than the number of point cloud points in the first point cloud data of the second object.

[0083] As an optional implementation, the volume of the first object is smaller than the volume of the second object, and / or the distance between the first object and the vehicle is greater than the distance between the second object and the vehicle.

[0084] As an optional implementation, extracting semantic information corresponding to the image data in step S202 specifically includes: performing pixel semantic segmentation on the image data using a semantic segmentation network to obtain the semantic category label of each pixel in the image data; and generating semantic information corresponding to each pixel in the image data based on the semantic category label of each pixel.

[0085] As an optional implementation, in step S203, semantic information is associated with radar point cloud data to generate first point cloud data. Specifically, this includes: synchronizing the radar equipment and image acquisition equipment of the vehicle with timestamps to obtain radar point cloud data and image data at the same time; obtaining the relative position between the radar equipment and the image acquisition equipment; and based on the relative position, mapping the semantic information pixel by pixel onto the corresponding radar point cloud data to obtain the first point cloud data.

[0086] As an optional implementation, in step S204, the region corresponding to the first point cloud data is divided into a first object and a second object based on the density of the first point cloud data. Specifically, this includes: determining the maximum distance between any target point cloud point in the first point cloud data and the nearest plurality of other point cloud points; calculating the density of point cloud points within the maximum distance range from the target point cloud point. If the density is less than a set value, the region of the first point cloud data corresponding to the target point cloud point is a sparse region of point cloud points, and the object corresponding to the sparse region of point cloud points is classified as the first object. If the density is greater than or equal to the set value, the region of the first point cloud data corresponding to the target point cloud point is a dense region of point cloud points, and the object corresponding to the dense region of point cloud points is classified as the second object.

[0087] As an optional implementation, in step S205, the first point cloud data of the first object is supplemented and optimized to obtain the second point cloud data of the first object. Specifically, the range corresponding to the object can be obtained by fitting the category constraint based on the semantic information corresponding to the first point cloud data and the preset value of the category constraint; the supplementary point cloud data is generated by sampling within the range; and the supplementary point cloud data is added to the point cloud data of the object to generate the second point cloud data.

[0088] As an optional implementation, in step S205, point cloud sparsity optimization is performed on the first point cloud data of the second object to obtain the second point cloud data of the second object. Specifically, the curvature value between adjacent point cloud points in the first point cloud data can be calculated, and point cloud points with curvature values ​​greater than the curvature threshold can be retained to generate the second point cloud data.

[0089] As an optional implementation, in step S206, the second point cloud data of the first object and the second object are optimized, specifically including fusing the second point cloud data of the first object and the second object. After fusion, at least one of the following methods is selected to optimize the fused second point cloud data: denoising the second point cloud data to generate denoised second point cloud data; or smoothing the second point cloud data to generate smoothed second point cloud data; or performing semantic consistency verification on the second point cloud data to generate semantically consistent second point cloud data.

[0090] Figure 3 This is a schematic diagram of the structure of a target recognition device provided in an embodiment of this disclosure. Figure 3 As shown, the device 300 includes: The object determination module 301 is used to determine objects in the environmental perception data based on the environmental perception data acquired by the vehicle's acquisition device; the objects correspond to the first point cloud data, which is obtained from the environmental perception data.

[0091] The target recognition module 302 is used to perform target recognition based on the second point cloud data; the second point cloud data is generated by supplementing and / or thinning the point cloud according to the density of the point cloud points in the first point cloud data.

[0092] In one embodiment of this disclosure, the object may include a first object and a second object, wherein the first relative point cloud quantity is greater than the second relative point cloud quantity, the first relative point cloud quantity is the change in the number of point cloud points in the second point cloud data of the first object relative to the number of point cloud points in the first point cloud data of the first object, and the second relative point cloud quantity is the change in the number of point cloud points in the second point cloud data of the second object relative to the number of point cloud points in the first point cloud data of the second object.

[0093] In one embodiment of this disclosure, the object may include a first object and a second object, wherein the number of point cloud points in the first point cloud data of the first object is less than the number of point cloud points in the first point cloud data of the second object, the number of point cloud points in the second point cloud data of the first object is greater than the number of point cloud points in the first point cloud data of the first object, and the number of point cloud points in the second point cloud data of the second object is less than the number of point cloud points in the first point cloud data of the second object.

[0094] In one embodiment of this disclosure, the volume of the first object is smaller than the volume of the second object, and / or the distance between the first object and the vehicle is greater than the distance between the second object and the vehicle.

[0095] In one embodiment of this disclosure, the object determination module 301 includes an adaptive sampling module, a first object determination submodule, and a second object determination submodule. The adaptive sampling module is used to perform local sparsity adaptive sampling on the first point cloud data. The first object determination submodule is used to classify the objects corresponding to the sparse regions of the point cloud obtained by adaptive sampling as first objects. The second determination submodule is used to classify the objects corresponding to the dense regions of the point cloud obtained by adaptive sampling as second objects.

[0096] In one embodiment of this disclosure, the adaptive sampling module is further configured to: determine the maximum distance between any target point cloud point in the first point cloud data and a plurality of other nearest point cloud points; calculate the density of point cloud points within the maximum distance range from the target point. If the density is less than a set value, the first point cloud data forming region corresponding to the target point cloud point is a sparse point cloud region, and the first object determination submodule classifies the objects corresponding to the sparse point cloud region as first objects. If the density is greater than or equal to a set value, the first point cloud data forming region corresponding to the target point cloud point is a dense point cloud region, and the second object determination submodule classifies the objects corresponding to the dense point cloud region as second objects.

[0097] In one embodiment of this disclosure, the device 300 further includes a point cloud acquisition module. The environmental perception data includes radar point cloud data and image data. The point cloud acquisition module is used to obtain first point cloud data based on the environmental perception data. The point cloud acquisition module specifically includes: The semantic acquisition submodule is used to extract semantic information from image data acquired by the vehicle's image acquisition equipment. The fusion submodule is used to obtain the first point cloud data by combining radar point cloud data and semantic information acquired by the vehicle-based radar equipment.

[0098] In one embodiment of this disclosure, the semantic acquisition submodule is further configured to: perform pixel semantic segmentation on the image data using a semantic segmentation network to obtain the semantic category label of each pixel in the image data; and generate semantic information corresponding to each pixel in the image data based on the semantic category label of each pixel.

[0099] In one embodiment of this disclosure, the fusion submodule is further configured to: timestamp and synchronize the radar equipment and image acquisition equipment of the vehicle to obtain radar point cloud data and image data at the same time; obtain the relative position between the radar equipment and the image acquisition equipment; and based on the relative position, map the semantic information to the corresponding radar point cloud data pixel by pixel to obtain the first point cloud data.

[0100] In one embodiment of this disclosure, the device 300 further includes a point cloud supplementation module, which is used to supplement the point cloud data according to the density of point cloud points in the first point cloud data to generate second point cloud data. Specifically, it includes: if the density of point cloud points in the first point cloud data is less than a set value, then based on the category constraints of the semantic information corresponding to the first point cloud data, and combined with the preset value of the category constraints, to fit and obtain the range corresponding to the object; sampling within the range to generate supplementary point cloud data; and adding the supplementary point cloud data to the point cloud data of the object to generate second point cloud data.

[0101] In one embodiment of this disclosure, the device 300 further includes a point cloud sparsity module, which is used to generate second point cloud data after performing point cloud sparsity processing based on the density of point cloud points in the first point cloud data. Specifically, it includes: if the density of point cloud points in the first point cloud data is greater than or equal to a set value, calculating the curvature value between adjacent point cloud points in the first point cloud data, retaining point cloud points with curvature values ​​greater than a curvature threshold, and generating second point cloud data.

[0102] In one embodiment of this disclosure, the apparatus 300 further includes an optimization module for optimizing the second point cloud data before target recognition. The optimization module further includes at least one of the following sub-modules: The noise reduction submodule is used to denoise the second point cloud data and generate the denoised second point cloud data. The smoothing submodule is used to smooth the second point cloud data and generate smoothed second point cloud data. The consistency verification submodule is used to perform semantic consistency verification on the second point cloud data and generate semantically consistent second point cloud data.

[0103] The present invention also provides a vehicle including a memory and a processor, wherein the memory stores a computer program and the processor executes the computer program in the memory to implement the steps of the target recognition method or point cloud data processing method provided in this disclosure.

[0104] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the target recognition method or point cloud data processing method provided in this disclosure. The preferred embodiments of this disclosure have been described in detail above with reference to the accompanying drawings. However, this disclosure is not limited to the specific details of the above embodiments. Within the scope of the technical concept of this disclosure, various simple modifications can be made to the technical solutions of this disclosure, and these simple modifications all fall within the protection scope of this disclosure.

[0105] It should also be noted that the various specific technical features described in the above embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, this disclosure will not describe the various possible combinations separately.

[0106] Furthermore, various different embodiments of this disclosure can be combined in any way, as long as they do not violate the spirit of this disclosure, they should also be regarded as the content disclosed in this disclosure.

Claims

1. A target recognition method, characterized in that, The method includes: Based on the environmental perception data acquired by the vehicle's data acquisition device, objects in the environmental perception data are determined; the objects correspond to first point cloud data, which is obtained based on the environmental perception data. Target recognition is performed based on the second point cloud data; the second point cloud data is generated by supplementing and / or thinning the point cloud based on the density of the point cloud points in the first point cloud data.

2. The target recognition method according to claim 1, characterized in that, The objects include a first object and a second object. The first relative point cloud quantity is greater than the second relative point cloud quantity. The first relative point cloud quantity is the change in the number of point cloud points in the second point cloud data of the first object relative to the number of point cloud points in the first point cloud data of the first object. The second relative point cloud quantity is the change in the number of point cloud points in the second point cloud data of the second object relative to the number of point cloud points in the first point cloud data of the second object.

3. The target recognition method according to claim 1 or 2, characterized in that, The objects include a first object and a second object. The number of point cloud points in the first point cloud data of the first object is less than the number of point cloud points in the first point cloud data of the second object. The number of point cloud points in the second point cloud data of the first object is greater than the number of point cloud points in the first point cloud data of the first object. The number of point cloud points in the second point cloud data of the second object is less than the number of point cloud points in the first point cloud data of the second object.

4. The target recognition method according to claim 3, characterized in that, The volume of the first object is smaller than the volume of the second object, and / or the distance between the first object and the vehicle is greater than the distance between the second object and the vehicle.

5. The target recognition method according to any one of claims 2-4, characterized in that, The first and second objects in the environmental perception data are obtained by local sparsity adaptive sampling based on the first point cloud data. Specifically, objects corresponding to sparse regions of the point cloud obtained by adaptive sampling are classified as the first objects, and objects corresponding to dense regions of the point cloud obtained by adaptive sampling are classified as the second objects.

6. The target recognition method according to claim 5, characterized in that, The process of obtaining the first object and the second object by performing local sparseness adaptive sampling based on the first point cloud data includes: Determine the maximum distance between any target point in the first point cloud data and the nearest multiple other point cloud points; Calculate the density of point cloud points within the maximum distance range from the target point cloud points; If the density is less than the set value, the first point cloud data forming region corresponding to the target point cloud point is a sparse point cloud region, and the object corresponding to the sparse point cloud region is classified as the first object. If the density is greater than or equal to the set value, then the first point cloud data forming region corresponding to the target point cloud point is a point cloud point dense region, and the object corresponding to the point cloud point dense region is classified as a second object.

7. The target recognition method according to claim 1, characterized in that, The environmental perception data includes radar point cloud data and image data. The process of obtaining the first point cloud data based on the environmental perception data includes: Semantic information is extracted from image data acquired by vehicle image acquisition equipment; The first point cloud data is obtained by combining the radar point cloud data acquired by the vehicle's radar equipment with the semantic information.

8. The target recognition method according to claim 7, characterized in that, The semantic information corresponding to the image data acquired by the vehicle-based image acquisition device is extracted, including: A semantic segmentation network is used to perform pixel semantic segmentation on the image data to obtain the semantic category label of each pixel in the image data; Based on the semantic category label of each pixel, semantic information corresponding to each pixel in the image data is generated.

9. The target recognition method according to claim 7, characterized in that, The radar point cloud data acquired by the vehicle-based radar equipment and the semantic information are combined to obtain first point cloud data, including: The radar equipment of the vehicle and the image acquisition equipment are time-stamped to obtain radar point cloud data and image data at the same time. Obtain the relative position between the radar device and the image acquisition device; Based on the relative position, the semantic information is mapped point by point to the corresponding radar point cloud data according to the corresponding pixel to obtain the first point cloud data.

10. The target recognition method according to claim 1 or 6, characterized in that, The second point cloud data is generated by supplementing the point cloud data with the density of point cloud points in the first point cloud data, including: If the density of point cloud points in the first point cloud data is less than a set value, then based on the category constraints of the semantic information corresponding to the first point cloud data, the range corresponding to the object is fitted by combining the preset value of the category constraints. Supplementary point cloud data is generated by sampling within the specified range; The supplementary point cloud data is added to the point cloud data of the object to generate the second point cloud data.

11. The target recognition method according to claim 1 or 6, characterized in that, The second point cloud data is generated by performing point cloud sparsity processing based on the density of point cloud points in the first point cloud data, including: If the density of point cloud points in the first point cloud data is greater than or equal to a set value, then the curvature value between adjacent point cloud points in the first point cloud data is calculated, and the point cloud points with curvature values ​​greater than the curvature threshold are retained to generate the second point cloud data.

12. The target recognition method according to claim 1, characterized in that, Before target recognition, the second point cloud data is further optimized, and the optimization process includes at least one of the following: The second point cloud data is denoised to generate denoised second point cloud data. The second point cloud data is smoothed to generate smoothed second point cloud data. Perform semantic consistency verification on the second point cloud data to generate semantically consistent second point cloud data.

13. A point cloud data processing method, characterized in that, The method includes: Acquire environmental perception data, determine the objects in the environmental perception data, wherein the objects are obtained by distinguishing and processing based on the density of point cloud points in the first point cloud data in the environmental perception data; The object also corresponds to the second point cloud data. If the sparsity of the first point cloud data of the object is greater than a set value, then the number of point cloud points in the second point cloud data of the object is greater than the number of point cloud points in the first point cloud data of the object; if the sparsity of the first point cloud data of the object is less than or equal to the set value, then the number of point cloud points in the second point cloud data of the object is less than the number of point cloud points in the first point cloud data of the object.

14. A vehicle, characterized in that, include: A memory on which computer programs are stored; A processor for executing the computer program in the memory to implement the steps of the method according to any one of claims 1-13.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program performs the steps of the method described in any one of claims 1-13.

Images