Target three-dimensional data detection method, device, equipment and medium

Abstract

The present application discloses a method, device, equipment and medium for detecting three-dimensional target data, which relates to the field of vehicle-mounted interconnection technology, including: obtaining three-dimensional data of the detection target, obtaining a target height map and a target pseudo image according to the three-dimensional data, passing the target height map and the target pseudo image through a preset feature extraction network to obtain a first feature and a second feature, passing the first feature and the second feature through a preset aggregation network to obtain a fused feature, and performing detection according to the fused feature to obtain a target detection result. By processing three-dimensional data to generate a height map and a pseudo image, extracting features and fusing them, high-precision 3D target detection is finally achieved, which improves the accuracy of 3D target detection and thus improves the environmental perception capability of the autonomous driving system.

Description

Technical Field

[0001] The present invention relates to the field of vehicle-mounted interconnection technology, and in particular to a target three-dimensional data detection method, device, equipment and medium. Background Art

[0002] With the rapid development of autonomous driving technology, 3D object detection plays a vital role in intelligent driving systems. 3D object detection technology can identify and locate the position and posture of objects such as vehicles, pedestrians, and bicycles in three-dimensional space in real time, providing necessary environmental perception information for autonomous driving vehicles. This technology is of great significance for improving the safety, reliability, and adaptability of autonomous driving systems. With the advancement of sensor technology, such as the widespread use of laser radar (LiDAR) and cameras, it has become possible to obtain high-precision three-dimensional data, which provides a rich data source for 3D object detection.

[0003] At present, there are many algorithms and technologies in the field of 3D object detection. Among them, the PointPillars algorithm is a classic 3D object detection method, which converts point cloud data into a columnar structure (Pillars) and uses a 2D convolutional neural network for feature extraction and object detection. The PointPillars algorithm converts the original point cloud data into a pseudo image through downsampling, interpolation and normalization, and then extracts features through a series of convolutional layers and pooling layers, and finally classifies and locates the target through the detection head.

[0004] Although the existing technology has made some progress, there are still obvious shortcomings. The PointPillars algorithm mainly relies on 2D convolutional neural networks, which may limit its ability to capture complex spatial structures in point cloud data. In real-time applications, especially in high-speed autonomous driving vehicles, the real-time requirements of the algorithm are very high, and the PointPillars algorithm may be insufficient in processing speed. Therefore, there is an urgent need for a more accurate three-dimensional data detection method to improve the accuracy and efficiency of 3D target detection. Summary of the invention

[0005] The main purpose of this application is to provide a method, device, equipment and medium for detecting three-dimensional data of a target, aiming to solve the technical problem of how to improve the accuracy of 3D target detection.

[0006] To achieve the above objectives, the present application proposes a method for detecting target three-dimensional data, the method comprising:

[0007] Acquire three-dimensional data of the detection target;

[0008] Obtaining a target height map and a target pseudo image according to the three-dimensional data;

[0009] Passing the target height map and the target pseudo image through a preset feature extraction network to obtain a first feature and a second feature;

[0010] Passing the first feature and the second feature through a preset aggregation network to obtain a fused feature;

[0011] Detection is performed based on the fused features to obtain a target detection result.

[0012] In one embodiment, the step of obtaining a target height map and a target pseudo image according to the three-dimensional data includes:

[0013] Performing filtering processing on the three-dimensional data to obtain processed data;

[0014] Converting the processed data into two-dimensional coordinates to obtain converted data;

[0015] Performing density estimation on the converted data to obtain an initial height map;

[0016] Downsampling and difference processing are performed on the initial height map to obtain a processed height map;

[0017] The processed height map is normalized to obtain a target height map.

[0018] In one embodiment, the step of obtaining a target height map and a target pseudo image according to the three-dimensional data further includes:

[0019] Down-sampling is performed according to the three-dimensional data to obtain sampled data;

[0020] Dividing the sampled data and performing feature extraction to obtain a feature vector;

[0021] Encoding and combining the feature vectors to obtain an initial pseudo image;

[0022] The initial pseudo image is normalized to obtain a target pseudo image.

[0023] In one embodiment, the step of obtaining the first feature and the second feature by passing the target height map and the target pseudo image through a preset feature extraction network includes:

[0024] Passing the target height map through a first preset feature extraction network to obtain a first feature;

[0025] The target pseudo image is passed through a second preset feature extraction network to obtain a second feature, wherein the first preset feature extraction network includes an input layer, a pooling layer, a convolution layer, a batch normalization layer and an output layer, and the second preset feature extraction network includes an input layer, a pooling layer, a convolution layer, a batch normalization layer and an output layer.

[0026] In one embodiment, the step of passing the target height map through a first preset feature extraction network to obtain a first feature includes:

[0027] Inputting the target height map into a first preset feature extraction network to obtain an encoded feature vector;

[0028] Passing the encoded feature vector through a convolutional layer to obtain spatial features;

[0029] Passing the spatial features through a pooling layer and a batch normalization layer to obtain feature vectors of different layers;

[0030] The feature vectors of the different layers are passed through the output layer to obtain the first feature.

[0031] In one embodiment, the step of obtaining a fused feature by subjecting the first feature and the second feature to a preset aggregation network includes:

[0032] Passing the first feature and the second feature through a preset aggregation network to obtain feature vectors of multiple scales;

[0033] Passing the plurality of feature vectors through a convolutional layer to obtain feature vectors of the same scale;

[0034] The feature vectors of the same scale are concatenated to obtain fused features.

[0035] In one embodiment, the step of performing detection according to the fused features to obtain a target detection result includes:

[0036] Inputting the fused features into a detection neural network to obtain predicted category probability and specific location parameters;

[0037] Obtaining the position, size and rotation angle of the target according to the predicted category probability and the specific position parameters;

[0038] When the position, size and rotation angle of the target meet the confidence requirements, the category, position, size, rotation angle and confidence score of the target are obtained;

[0039] The category, position, size, rotation angle and confidence score of the target are taken as the target detection result.

[0040] In addition, to achieve the above-mentioned purpose, the present application also proposes a target three-dimensional data detection device, the target three-dimensional data detection device comprising:

[0041] An acquisition module, used to acquire three-dimensional data of the detection target;

[0042] A conversion module, used for obtaining a target height map and a target pseudo image according to the three-dimensional data;

[0043] A feature extraction module, used for obtaining a first feature and a second feature by passing the target height map and the target pseudo image through a preset feature extraction network;

[0044] A fusion module, used for obtaining a fused feature by combining the first feature and the second feature through a preset aggregation network;

[0045] The result module is used to perform detection according to the fused features to obtain target detection results.

[0046] In addition, to achieve the above-mentioned purpose, the present application also proposes a medium, which is a computer-readable medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the target three-dimensional data detection method described above are implemented.

[0047] In addition, to achieve the above-mentioned purpose, the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps of the target three-dimensional data detection method as described above are implemented.

[0048] This application obtains the three-dimensional data of the detection target, obtains the target height map and the target pseudo image according to the three-dimensional data, passes the target height map and the target pseudo image through a preset feature extraction network to obtain the first feature and the second feature, passes the first feature and the second feature through a preset aggregation network to obtain a fused feature, performs detection according to the fused feature, and obtains the target detection result. By processing the three-dimensional data to generate a height map and a pseudo image, extracting features and fusing them, high-precision 3D target detection is finally achieved, the accuracy of 3D target detection is improved, and thus the environmental perception ability of the autonomous driving system is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0050] Figure 1 This is a flow chart of the first embodiment of the target three-dimensional data detection method of the present application;

[0051] Figure 2 This is a flow chart of converting point cloud to height map of the first embodiment of the target three-dimensional data detection method of this application;

[0052] Figure 3This is a feature extraction network structure diagram of the first embodiment of the target three-dimensional data detection method of this application;

[0053] Figure 4 This is a feature aggregation network structure diagram of the first embodiment of the target three-dimensional data detection method of this application;

[0054] Figure 5 A schematic diagram of a flow chart of a second embodiment of the target three-dimensional data detection method of the present application;

[0055] Figure 6 This is a schematic diagram of the module structure of the target three-dimensional data detection device according to an embodiment of the present application;

[0056] Figure 7 Schematic diagram of the device structure of the hardware operating environment involved in the target three-dimensional data detection method in the embodiment of the present application.

[0057] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0058] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0059] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0060] The main solution of the embodiment of the present application is: by acquiring three-dimensional data of the detection target, obtaining a target height map and a target pseudo image according to the three-dimensional data, passing the target height map and the target pseudo image through a preset feature extraction network to obtain a first feature and a second feature, passing the first feature and the second feature through a preset aggregation network to obtain a fused feature, performing detection according to the fused feature, and obtaining a target detection result.

[0061] Based on this, the present application embodiment provides a method for detecting target three-dimensional data. Figure 1 , Figure 1 This is a flow chart of the first embodiment of the target three-dimensional data detection method of the present application.

[0062] In this embodiment, the target three-dimensional data detection method includes steps S10 to S50:

[0063] Step S10, acquiring three-dimensional data of the detection target.

[0064] It should be noted that the above three-dimensional data is usually obtained through on-board sensors such as laser radar (LiDAR) or stereo cameras. LiDAR emits lasers and receives reflected light signals to measure the distance between the laser and the object, generating dense point cloud data. These point cloud data contain the precise three-dimensional coordinates (X, Y, Z) of each point, as well as reflection intensity information. Stereo cameras simulate the binocular vision principle of the human eye, use two cameras to capture images from different angles, and calculate the depth information of objects in the scene through image matching and triangulation. The acquired three-dimensional data needs to be preprocessed, including filtering and denoising, data cleaning, coordinate conversion and other steps to ensure the quality and consistency of the data. The preprocessed data is used for subsequent height map generation and pseudo-image conversion.

[0065] Step S20, obtaining a target height map and a target pseudo image according to the three-dimensional data.

[0066] It should be noted that the target height map is created by mapping the point cloud data into a two-dimensional space, where the height value of each pixel represents the point cloud height at that location. This process first involves the spatial division of the point cloud data, such as Figure 2 The point cloud to height map flow chart shown in the figure usually determines the position of each point cloud point on the two-dimensional plane according to the ground coordinates (X, Y), and then, for each two-dimensional position, selects the corresponding point cloud height value (Z coordinate) to fill the height map. This process requires maximum pooling operations and interpolation processing to ensure the continuity and smoothness of the height map. The height map can effectively capture the terrain information of the scene, which is crucial for understanding the context of the scene and the relative position of the target.

[0067] The target pseudo image is to convert the point cloud data into a format similar to a traditional image, where the intensity value of each pixel represents the point cloud density or intensity at that location. The point cloud is converted into a target pseudo image using the PointPillars algorithm. Specifically, the point cloud space is divided into a set of subspaces along the x and y axes, each space is called a pillar, and then each point in the pillar is extracted through the PointNet network to obtain a feature vector. Finally, each feature is assigned to the image tensor to generate a pseudo image. The process of generating a pseudo image includes downsampling the point cloud data to reduce the amount of data and improve processing efficiency. Then, the pseudo image is generated by calculating the point cloud density or intensity value in the coverage area of ​​each pixel. This process also involves normalization to ensure that the intensity value of the pseudo image is within a certain range and is suitable for subsequent deep learning model processing. The pseudo image retains the spatial distribution information of the point cloud data and is converted into a 2D format, so that mature 2D convolutional neural networks can be used for feature extraction.

[0068] Further, step S20 also includes: downsampling according to the three-dimensional data to obtain sampled data, dividing the sampled data and performing feature extraction to obtain a feature vector. Specifically, downsampling is the first step in processing high-density point cloud data, and the purpose is to reduce the amount of data, reduce the computational complexity, and retain important structural information. Downsampling can be achieved by a variety of methods, such as random sampling, far-point sampling, or grid sampling. Random sampling is simple and direct, but may lose important information; far-point sampling gives priority to retaining points with farther spatial distribution, which helps to maintain the global structure of the point cloud; grid sampling divides the space into grids and retains only one point in each grid, which can effectively reduce the amount of data. Feature extraction is then performed to generate a feature vector describing each point or region. These features may include local geometric properties of the point (such as curvature, normal), color information (if available), and local statistical information of the point cloud (such as density, average height). The purpose of feature extraction is to capture useful information in point cloud data and provide a basis for subsequent image generation and target detection. Furthermore, the feature vectors are encoded and aggregated to obtain an initial pseudo image, and the initial pseudo image is normalized to obtain a target pseudo image. Specifically, the extracted feature vectors are encoded so as to convert them into a form that can be represented as image pixels. The encoding process involves mapping the feature vectors to a vector space of a fixed size, the dimension of which usually corresponds to the number of channels of the pseudo image. The encoded feature vectors are aggregated to form an initial pseudo image. This process requires interpolation or padding of the feature vectors to ensure that they can be correctly mapped to the two-dimensional image space. At the same time, since the initial pseudo image contains feature values ​​of different ranges and scales, normalization is required to make these feature values ​​suitable for processing by the deep learning model. Normalization usually involves scaling the feature values ​​to a uniform range, such as [0, 1] or [-1, 1]. Normalization helps to improve the training efficiency and convergence speed of the model, and also helps to prevent the model from experiencing gradient disappearance or explosion problems during training.

[0069] Step S30, the target height map and the target pseudo image are passed through a preset feature extraction network to obtain a first feature and a second feature.

[0070] It is important to understand that the preset feature extraction network is usually composed of an input layer, a pooling layer, a convolution layer, a batch normalization layer, and an output layer, which can effectively capture local features in the image and extract more complex features by stacking multiple layers. The convolution layer slides over the image through a filter (or convolution kernel) to extract features of local areas, such as edges, corners, etc. The pooling layer is used to reduce the spatial dimension of the features while maintaining important feature information and enhancing the model's robustness to image scaling and deformation.

[0071] Furthermore, the target height map is passed through a first preset feature extraction network to obtain a first feature, and the target pseudo image is passed through a second preset feature extraction network to obtain a second feature. It should be understood that two feature extraction networks with the same structure are used to process the target height map and the target pseudo image, that is, the first preset feature extraction network and the second preset feature extraction network have the same structure, but the processing objects are different. Figure 3 The feature extraction network structure diagram shown in the figure shows a typical convolutional neural network (CNN) structure for processing image data and extracting features. The input layer receives the target height map or target pseudo image as input, and the image is preprocessed to a size and format suitable for network input. The input image first passes through a convolution layer, which uses multiple filters (convolution kernels) to extract local features of the image, such as edges and corners. The outputs of these filters are processed by activation functions (such as ReLU) to increase nonlinearity, and then the output of the first convolution layer passes through another convolution layer again to further extract more complex features. The number of filters in this layer may increase to capture richer feature representations. After the convolution, it usually enters the pooling layer to reduce the spatial dimension of the feature map, reduce the number of parameters and the amount of calculation, while maintaining the important information of the feature, and then through the iteration of feature extraction, repeatedly passes through the "convolution-pooling" structure, so that the size of the feature map is halved after each iteration, while the number of channels increases. This structure helps the network to gradually reduce the spatial dimension of the feature map while capturing deeper features, and then outputs the processed features, that is, the first feature corresponding to the target height map or the second feature corresponding to the target pseudo image.

[0072] Furthermore, taking the target height map as an example, the target height map is input into the first preset feature extraction network to obtain the encoded feature vector. Specifically, the target height map is first encoded through the input layer to map each pixel in the height map to a high-dimensional feature space. The purpose of the encoding layer is to convert the height information into a feature vector that can capture spatial changes and patterns. The encoded feature vector is then passed through the convolution layer to obtain spatial features. Specifically, the encoded feature vector is passed through multiple convolution layers to further extract spatial features. The convolution layer slides on the feature map through a filter to capture local spatial features such as edges, textures, and shapes. The spatial features are passed through the pooling layer and the batch normalization layer to obtain feature vectors of different layers. The output of the convolution layer passes through the pooling layer to reduce the spatial dimension of the feature, reduce the amount of calculation, and maintain important feature information. The pooling operation usually uses maximum pooling, which can extract the maximum value in the feature map and enhance the robustness of the model to changes in the target position. The batch normalization layer is used to normalize the output of the convolution layer, accelerate the convergence of the network, and improve the stability of the model. Feature vectors of different levels are extracted at different layers. The feature vectors of each layer contain different levels of spatial information extracted from the input height map. These feature vectors can be viewed as multi-scale representations of the target height map, which helps capture the global and local features of the target. Finally, the feature vectors of different layers are passed through the output layer to obtain the first feature. Specifically, the feature vectors of different layers are integrated into the first feature through the output layer. The output layer may include a fully connected layer or a specific feature fusion layer, which is used to integrate feature information at different levels to form a comprehensive feature representation, namely the first feature.

[0073] Step S40, the first feature and the second feature are passed through a preset aggregation network to obtain a fused feature.

[0074] It should be noted that the height map features and pseudo image features are subjected to different convolution and pooling operations to gradually extract feature representations at different levels, and finally the fused features are obtained through the preset aggregation network for subsequent target detection tasks.

[0075] Furthermore, the first feature and the second feature are passed through a preset aggregation network to obtain feature vectors of multiple scales. The feature vectors of the same scale are concatenated to obtain fused features. Specifically, Figure 4As shown in the feature aggregation network structure diagram, both the height map feature and the pseudo image feature first undergo a series of convolution and pooling operations. For the height map feature, the initial feature dimension is C×H / 2×W / 2. After two convolution and pooling operations, the feature dimension becomes 4C×H / 8×W / 8. Similarly, the pseudo image feature is also processed similarly. After two convolution and pooling operations from C×H / 2×W / 2, the feature dimension also becomes 4C×H / 8×W / 8. Such processing helps to extract more abstract and discriminative features, while reducing the spatial dimension of the feature map and reducing the computational burden of subsequent processing. After feature extraction, the height map feature and the pseudo image feature are fused through a preset aggregation network. The design of the aggregation network is very critical, and it needs to be able to effectively integrate feature information from two different sources. The preset aggregation network contains multiple layers of deconvolution (upsampling) layers, which upsample the low-resolution feature map back to high resolution to align with the resolution of the original input. Such a design helps to retain more spatial information when fusing features, thereby improving the accuracy of object detection.

[0076] In the fusion process, the first feature and the second feature are first fused by concatenation or weighted summation, and then upsampled by the deconvolution layer to restore the spatial resolution close to the input feature map. Such a fusion strategy can not only integrate the complementary information of the two features, but also restore more spatial details through upsampling, providing richer feature representation for subsequent target detection.

[0077] Finally, the fused features are processed through the output layer to obtain fused features for target detection. These features contain all the information of the height map and pseudo image features, and can more comprehensively describe the shape, position and category of the target.

[0078] Step S50, performing detection based on the fused features to obtain target detection results.

[0079] It should be noted that the PointPillars algorithm detection head combined with the detection neural network performs target detection based on the fused features. The detection head contains the classification head (Cls_head), regression head (Reg_head) and direction head (Dir_head), which are used to predict the category, position and direction of the target respectively.

[0080] Further, step S50 also includes: inputting the fused features into the detection neural network to obtain the predicted category probability and specific location parameters, and obtaining the position, size and rotation angle of the target according to the predicted category probability and specific location parameters. Specifically, the fused features after preprocessing and feature extraction are input into the detection neural network, which is usually composed of multiple modules, including but not limited to the classification head (Cls_head), the regression head (Reg_head) and the direction head (Dir_head). The classification head is responsible for predicting the probability of which category each candidate area belongs to. For example, in autonomous driving applications, it may be necessary to identify different traffic participants such as vehicles, pedestrians, and bicycles. The regression head is used to estimate the specific location parameters of the target, that is, the coordinates of the bounding box. This step is the key to determining the exact position of the target in the image or three-dimensional space. It is specially designed to predict the direction angle of the target, which is particularly important for three-dimensional target detection, because knowing only the position and size is not enough to fully define the state of a three-dimensional target. Through the joint action of these three parts, the model can not only identify what the target is, but also accurately locate where it is, and even understand its direction.

[0081] Furthermore, when the position, size and rotation angle of the target meet the confidence requirements, the category, position, size, rotation angle and confidence score of the target are obtained. The category, position, size, rotation angle and confidence score of the target are used as the target detection result. Specifically, when the fused features are processed by the detection neural network, the category probability, position parameters, size and rotation angle of each potential target are output. These outputs need to be further screened and processed to finally determine which are the true detection results. Specifically, check whether the position, size and rotation angle of each potential target meet the preset confidence requirements. The confidence requirements usually refer to one or more thresholds used to filter out inaccurate or unreliable detection results.

[0082] Once a potential target meets the confidence requirement, the existence of the target is confirmed and its detailed detection information is recorded, including: category, position, size, rotation angle and confidence score, where the category indicates which category the detected object belongs to, such as vehicle, pedestrian or bicycle. Position refers to the precise coordinates of the target in three-dimensional space, usually expressed by the (x, y, z) coordinates of the center point. Size refers to the length, width and height of the target, which helps the system better understand the physical form of the target. The rotation angle indicates the rotation of the target relative to the reference coordinate system, usually expressed as the rotation angle around each axis. The confidence score reflects the system's confidence in the detection result, usually a value between 0 and 1, and the larger the value, the higher the confidence. The final target detection result is formed by combining these information. In practical application scenarios such as autonomous driving, these detection results are crucial for vehicle path planning, obstacle avoidance decisions and safe driving. For example, the autonomous driving system can slow down or change lanes in advance based on the detected position and direction of movement of pedestrians; and if a stationary obstacle is detected in front, braking measures can be taken in time.

[0083] This comprehensive understanding is crucial for many practical application scenarios. For example, in autonomous driving, the vehicle must be able to accurately judge the position and movement direction of other vehicles and pedestrians around it in order to make safe driving decisions.

[0084] This embodiment provides a method for detecting target three-dimensional data, by acquiring three-dimensional data of the detection target, obtaining a target height map and a target pseudo image based on the three-dimensional data, passing the target height map and the target pseudo image through a preset feature extraction network to obtain a first feature and a second feature, passing the first feature and the second feature through a preset aggregation network to obtain a fused feature, performing detection based on the fused feature, and obtaining a target detection result. By processing three-dimensional data to generate a height map and a pseudo image, extracting features and fusing them, high-precision 3D target detection is finally achieved, the accuracy of 3D target detection is improved, and thus the environmental perception capability of the autonomous driving system is improved.

[0085] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can refer to the above introduction, and will not be repeated later. Figure 5 The target three-dimensional data detection method step S20 also includes steps S201 to S205:

[0086] Step S201, filtering the three-dimensional data to obtain processed data.

[0087] It should be noted that the original point cloud data is filtered to remove noise points and outliers to ensure the cleanliness and accuracy of the data. Common filtering methods include statistical filtering, voxel filtering, Gaussian filtering and bilateral filtering. Statistical filtering is to statistically analyze the distance between each point in the point cloud and its neighboring points, and remove those points that are too far away, which are usually considered to be noise points. Voxel filtering is to divide the three-dimensional space into small voxels, and then select a representative point (such as the mean or median) in each voxel to reduce the amount of data and smooth the surface. Gaussian filtering is to achieve a smoothing effect by applying a Gaussian kernel function in three-dimensional space and performing a weighted average on each point and its neighboring points. Bilateral filtering combines the weights of spatial distance and color (or depth) differences to smooth noise and maintain edges.

[0088] Step S202, converting the processed data into two-dimensional coordinates to obtain converted data.

[0089] It is important to understand that projecting the filtered point cloud data onto a two-dimensional plane usually involves converting the (x, y, z) coordinates of the point cloud data into (x, y) plane coordinates. This process can be achieved through a simple projection transformation, that is, mapping the (x, y, z) coordinates of each point to the (x, y) plane, retaining the height information of the z axis as an additional attribute.

[0090] Step S203, performing density estimation on the converted data to obtain an initial height map.

[0091] It is important to understand that on a two-dimensional plane, the z-axis height value of each point is estimated to generate an initial height map. Density estimation can be done by the histogram method or kernel density estimation. The histogram method divides the two-dimensional plane into grids, counts the number of points in each grid, and records the z-axis height values ​​of these points. Kernel density estimation uses a kernel function to smooth the z-axis height value of each point to generate a continuous height map.

[0092] Step S204, downsampling and difference processing are performed on the initial height map to obtain a processed height map.

[0093] It should be understood that in order to reduce the amount of data and improve processing efficiency, the initial height map is downsampled, and downsampling can be achieved by uniform downsampling or random downsampling. Uniform downsampling refers to dividing the height map into uniform sub-areas, each sub-area takes a representative value (such as the average or maximum value). Random downsampling is to randomly select the height values ​​of some grids. The downsampled height map is interpolated to restore the lost detail information. Interpolation methods include nearest neighbor interpolation, bilinear interpolation, and bicubic interpolation. The nearest neighbor interpolation uses the height values ​​of the nearest neighbor points to fill in the missing points. Bilinear interpolation uses the height values ​​of the four nearest neighbor points for linear interpolation. Bicubic interpolation uses the height values ​​of the sixteen nearest neighbor points for cubic interpolation.

[0094] Step S205, normalizing the processed height map to obtain a target height map.

[0095] It is important to understand that the processed height map is normalized so that its value range is between [0, 1]. Normalization can improve the stability and consistency of subsequent processing, and then obtain the final target height map.

[0096] This embodiment performs filtering processing on three-dimensional data to obtain processed data; converts the processed data into two-dimensional coordinates to obtain converted data; performs density estimation on the converted data to obtain an initial height map; performs downsampling and difference processing on the initial height map to obtain a processed height map; and performs normalization processing on the processed height map to obtain a target height map. By filtering, projecting, density estimating, downsampling, interpolating and normalizing three-dimensional point cloud data, a high-quality target height map is generated, which improves the cleanliness and accuracy of the data and provides a reliable foundation for subsequent feature extraction and target detection tasks.

[0097] This application also provides a target three-dimensional data detection device, please refer to Figure 6 , the device comprises:

[0098] The acquisition module 10 is used to acquire three-dimensional data of the detection target.

[0099] The conversion module 20 is used to obtain a target height map and a target pseudo image according to the three-dimensional data.

[0100] The feature extraction module 30 is used to obtain a first feature and a second feature by passing the target height map and the target pseudo image through a preset feature extraction network.

[0101] The fusion module 40 is used to obtain a fused feature by combining the first feature and the second feature through a preset aggregation network.

[0102] The result module 50 is used to perform detection according to the fused features to obtain the target detection result.

[0103] The target three-dimensional data detection device provided by the present application adopts the target three-dimensional data detection method in the above embodiment, which can solve the technical problem of how to improve the accuracy of 3D target detection. Compared with the prior art, the beneficial effects of the target three-dimensional data detection device provided by the present application are the same as the beneficial effects of the target three-dimensional data detection method provided by the above embodiment, and the other technical features in the target three-dimensional data detection device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0104] In one embodiment, the conversion module 20 is also used to filter the three-dimensional data to obtain processed data; convert the processed data into two-dimensional coordinates to obtain converted data; perform density estimation on the converted data to obtain an initial height map; downsample and difference the initial height map to obtain a processed height map; and normalize the processed height map to obtain a target height map.

[0105] In one embodiment, the conversion module 20 is further used to downsample the three-dimensional data to obtain sampled data; divide the sampled data and perform feature extraction to obtain feature vectors; encode and aggregate the feature vectors to obtain an initial pseudo image; and normalize the initial pseudo image to obtain a target pseudo image.

[0106] In one embodiment, the feature extraction module 30 is also used to pass the target height map through a first preset feature extraction network to obtain a first feature; pass the target pseudo image through a second preset feature extraction network to obtain a second feature, the first preset feature extraction network includes an input layer, a pooling layer, a convolution layer, a batch normalization layer and an output layer, and the second preset feature extraction network includes an input layer, a pooling layer, a convolution layer, a batch normalization layer and an output layer.

[0107] In one embodiment, the feature extraction module 30 is also used to input the target height map into a first preset feature extraction network to obtain an encoded feature vector; pass the encoded feature vector through a convolution layer to obtain a spatial feature; pass the spatial feature through a pooling layer and a batch normalization layer to obtain feature vectors of different layers; pass the feature vectors of different layers through an output layer to obtain a first feature.

[0108] In one embodiment, the fusion module 40 is further used to pass the first feature and the second feature through a preset aggregation network to obtain feature vectors of multiple scales; pass the multiple feature vectors through a convolution layer to obtain feature vectors of the same scale; and splice the feature vectors of the same scale to obtain a fused feature.

[0109] In one embodiment, the result module 50 is also used to input the fused features into the detection neural network to obtain the predicted category probability and the specific position parameters; according to the predicted category probability and the specific position parameters, the position, size and rotation angle of the target are obtained; when the position, size and rotation angle of the target meet the confidence requirements, the category, position, size, rotation angle and confidence score of the target are obtained; the category, position, size, rotation angle and confidence score of the target are used as the target detection result.

[0110] The present application provides a target three-dimensional data detection device, which includes: at least one processor; and a memory that is communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the target three-dimensional data detection method in the above-mentioned embodiment one.

[0111] Reference below Figure 7 , which shows a schematic diagram of the structure of a target three-dimensional data detection device suitable for implementing the embodiment of the present application. The target three-dimensional data detection device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 7 The target three-dimensional data detection device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0112] like Figure 7As shown, the target three-dimensional data detection device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the target three-dimensional data detection device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the target three-dimensional data detection device to communicate with other devices wirelessly or wired to exchange data. Although the target three-dimensional data detection device with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.

[0113] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0114] The target three-dimensional data detection device provided by the present application adopts the target three-dimensional data detection method in the above embodiment, which can solve the technical problem of how to improve the accuracy of 3D target detection. Compared with the prior art, the beneficial effects of the target three-dimensional data detection device provided by the present application are the same as the beneficial effects of the target three-dimensional data detection method provided by the above embodiment, and the other technical features in the target three-dimensional data detection device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0115] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0116] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0117] The present application provides a computer-readable medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the target three-dimensional data detection method in the above-mentioned embodiment.

[0118] The computer-readable medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0119] The computer-readable medium may be included in the target three-dimensional data detection device; or may exist independently without being assembled into the target three-dimensional data detection device.

[0120] The computer-readable medium carries one or more programs. When the one or more programs are executed by the target three-dimensional data detection device, the target three-dimensional data detection device can be written in one or more programming languages ​​or a combination thereof to write computer program codes for performing the operations of the present application. The programming languages ​​include object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed completely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0121] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0122] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0123] The readable medium provided in this application is a computer-readable medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned target three-dimensional data detection method, and can solve the technical problem of how to improve the accuracy of 3D target detection. Compared with the prior art, the beneficial effects of the computer-readable medium provided in this application are the same as the beneficial effects of the target three-dimensional data detection method provided in the above-mentioned embodiment, and will not be repeated here.

[0124] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned target three-dimensional data detection method when executed by a processor.

[0125] The computer program product provided by this application can solve the technical problem of how to improve the accuracy of 3D target detection. Compared with the prior art, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the target three-dimensional data detection method provided by the above embodiment, which will not be repeated here.

[0126] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A method for detecting target three-dimensional data, characterized in that: The method comprises: Acquire three-dimensional data of the detection target; Obtaining a target height map and a target pseudo image according to the three-dimensional data; Passing the target height map and the target pseudo image through a preset feature extraction network to obtain a first feature and a second feature; Passing the first feature and the second feature through a preset aggregation network to obtain a fused feature; Detection is performed based on the fused features to obtain a target detection result.

2. The method according to claim 1, characterized in that The step of obtaining a target height map and a target pseudo image according to the three-dimensional data comprises: Performing filtering processing on the three-dimensional data to obtain processed data; Converting the processed data into two-dimensional coordinates to obtain converted data; Performing density estimation on the converted data to obtain an initial height map; Downsampling and difference processing are performed on the initial height map to obtain a processed height map; The processed height map is normalized to obtain a target height map.

3. The method according to claim 1, characterized in that The step of obtaining a target height map and a target pseudo image according to the three-dimensional data further includes: Down-sampling is performed according to the three-dimensional data to obtain sampled data; Dividing the sampled data and performing feature extraction to obtain a feature vector; Encoding and combining the feature vectors to obtain an initial pseudo image; The initial pseudo image is normalized to obtain a target pseudo image.

4. The method according to claim 1, characterized in that The step of obtaining the first feature and the second feature by passing the target height map and the target pseudo image through a preset feature extraction network comprises: Passing the target height map through a first preset feature extraction network to obtain a first feature; The target pseudo image is passed through a second preset feature extraction network to obtain a second feature, wherein the first preset feature extraction network includes an input layer, a pooling layer, a convolution layer, a batch normalization layer and an output layer, and the second preset feature extraction network includes an input layer, a pooling layer, a convolution layer, a batch normalization layer and an output layer.

5. The method according to claim 1, characterized in that The step of obtaining a first feature by passing the target height map through a first preset feature extraction network comprises: Inputting the target height map into a first preset feature extraction network to obtain an encoded feature vector; Passing the encoded feature vector through a convolutional layer to obtain spatial features; Passing the spatial features through a pooling layer and a batch normalization layer to obtain feature vectors of different layers; The feature vectors of the different layers are passed through the output layer to obtain the first feature.

6. The method according to claim 5, characterized in that The step of obtaining a fused feature by applying the first feature and the second feature to a preset aggregation network includes: Passing the first feature and the second feature through a preset aggregation network to obtain feature vectors of multiple scales; Passing the plurality of feature vectors through a convolutional layer to obtain feature vectors of the same scale; The feature vectors of the same scale are concatenated to obtain fused features.

7. The method according to claim 1, characterized in that The step of performing detection according to the fused features to obtain target detection results includes: Inputting the fused features into a detection neural network to obtain predicted category probability and specific location parameters; Obtaining the position, size and rotation angle of the target according to the predicted category probability and the specific position parameters; When the position, size and rotation angle of the target meet the confidence requirements, the category, position, size, rotation angle and confidence score of the target are obtained; The category, position, size, rotation angle and confidence score of the target are taken as the target detection result.

8. A target three-dimensional data detection device, characterized in that: The device comprises: An acquisition module, used to acquire three-dimensional data of the detection target; A conversion module, used for obtaining a target height map and a target pseudo image according to the three-dimensional data; A feature extraction module, used for obtaining a first feature and a second feature by passing the target height map and the target pseudo image through a preset feature extraction network; A fusion module, used for obtaining a fused feature by combining the first feature and the second feature through a preset aggregation network; The result module is used to perform detection according to the fused features to obtain target detection results.

9. A target three-dimensional data detection device, characterized in that: The device comprises: a memory, a processor, and a target three-dimensional data detection program stored in the memory and executable on the processor, wherein the target three-dimensional data detection program is configured to implement the steps of the target three-dimensional data detection method as described in any one of claims 1 to 7.

10. A medium, characterized in that The medium stores a target three-dimensional data detection program, and when the target three-dimensional data detection program is executed by the processor, the steps of the target three-dimensional data detection method according to any one of claims 1 to 7 are implemented.

Images