3D target detection method and device, equipment and storage medium

Abstract

The invention discloses a 3D target detection method, apparatus and device, and a storage medium. The 3D target detection method comprises the steps of obtaining an initial point cloud of a target object; based on the initial point cloud, performing feature extraction through a preset target detection model to obtain initial point cloud features; based on the initial point cloud, shielding completion processing is carried out through a preset shielding completion model, point cloud features after shielding completion are obtained, and the shielding completion model is used for extracting and completing shielding features of the target object; and performing feature fusion on the initial point cloud feature and the shielded and completed point cloud feature to obtain a target fusion feature, and detecting the target fusion feature to obtain a 3D detection frame of the target object. According to the invention, the shielding features of the target object are extracted and complemented through the pre-trained shielding complementation model, so that the collected point cloud features of the perception target are dense, and the accuracy of 3D target detection is improved.

Description

Technical Field

[0001] The present application relates to the field of automobile autonomous driving technology, and in particular to a 3D target detection method, device, equipment and storage medium. Background Art

[0002] At present, with the development of autonomous driving technology, people are paying more and more attention to the perception system of autonomous driving vehicles. Vehicle target perception is an important part of the autonomous driving system, among which 2D perception is far from enough for high-level autonomous driving, so there is a need for 3D perception in the autonomous driving system.

[0003] In related technologies, autonomous vehicles usually use radar and image fusion to achieve 3D perception of the vehicle. However, when the vehicle is driving, the perceived target is often blocked, resulting in an incomplete target shape. This method ignores the occlusion factor of the target image when the vehicle is driving, making the point cloud features collected by the perceived target sparse, resulting in low accuracy of 3D target detection. Summary of the invention

[0004] The main purpose of the present application is to provide a 3D target detection method, device, equipment and storage medium, aiming to solve the technical problem of low accuracy of 3D target detection in the prior art.

[0005] To achieve the above objectives, the present application provides a 3D object detection method, the 3D object detection method comprising:

[0006] Obtain the initial point cloud of the target object;

[0007] Based on the initial point cloud, feature extraction is performed through a preset target detection model to obtain initial point cloud features;

[0008] Based on the initial point cloud, occlusion completion processing is performed through a preset occlusion completion model to obtain point cloud features after occlusion completion, wherein the occlusion completion model is used to extract and complete the occlusion features of the target object;

[0009] The initial point cloud features and the point cloud features after occlusion completion are fused to obtain target fusion features, and the target fusion features are detected to obtain a 3D detection frame of the target object.

[0010] Optionally, before the step of acquiring an initial point cloud of the target object, the method includes:

[0011] Get point cloud training samples;

[0012] The point cloud training samples are supplemented with missing parts of the target to obtain complete point cloud labels;

[0013] Based on the point cloud training samples and the complete point cloud labels, the preset model to be trained is iteratively trained to obtain an occlusion completion model that meets the accuracy requirements.

[0014] Optionally, the step of supplementing the missing part of the point cloud training sample to obtain a complete point cloud label includes:

[0015] Extracting the initial object point cloud of each object in the point cloud training sample;

[0016] Performing symmetric processing on the initial object point cloud to obtain a corresponding target object point cloud;

[0017] Selecting a preset number of similar point clouds with the highest similarity to the initial object point cloud from the target object point cloud;

[0018] Based on the similar point clouds, the initial object point clouds of each point cloud training sample are supplemented with target missing parts to obtain complete point cloud labels.

[0019] Optionally, the step of performing symmetric processing on the initial object point cloud to obtain a corresponding target object point cloud includes:

[0020] Performing rasterization processing on the initial object point cloud to obtain a rasterized initial object point cloud;

[0021] The central axis of the rasterized initial object point cloud is determined, and the rasterized initial object point cloud is subjected to axisymmetric processing according to the central axis to obtain a target object point cloud.

[0022] Optionally, the step of selecting a preset number of similar point clouds having the highest similarity to the initial object point cloud from the target object point cloud comprises:

[0023] Rasterizing the initial object point cloud and each of the target object point clouds respectively to obtain a rasterized initial object point cloud and a rasterized target object point cloud;

[0024] Determine a first vector of the rasterized initial object point cloud and a second vector of each of the rasterized target object point clouds, and perform product calculations on the first vector and each of the second vectors to obtain a vector product result;

[0025] The target object point cloud corresponding to the vector product result of the preset number of largest scores is determined as the similar point cloud.

[0026] Optionally, before the step of fusing the target object image features and the target point cloud features, the method includes:

[0027] Acquire an initial object image of the target object;

[0028] Extracting target object image features of the initial object image;

[0029] The step of fusing the initial point cloud features with the occlusion-completing point cloud features to obtain target fused features comprises:

[0030] The initial point cloud features, the point cloud features after occlusion completion and the target object image features are fused to obtain target fusion features.

[0031] Optionally, the step of extracting target object image features from the initial object image includes:

[0032] Inputting the initial object image into a preset image feature extraction network to obtain initial object image features;

[0033] Extracting a partial image of a preset position of the initial object image, and inputting the partial image into a preset image feature extraction network to obtain enhanced object image features;

[0034] The initial object image feature and the enhanced object image feature are concatenated to obtain the target object image feature.

[0035] Optionally, before the step of fusing the initial point cloud features, the point cloud features after occlusion completion, and the target object image features to obtain target fused features, the method includes:

[0036] Inputting the initial point cloud and the initial object image into a preset depth completion model, and completing the initial point cloud and the initial object image based on the depth completion model to obtain a densified point cloud;

[0037] Performing sparse convolution on the densified point cloud and the initial point cloud to obtain dense point cloud features;

[0038] The step of fusing the initial point cloud features, the point cloud features after occlusion completion, and the target object image features to obtain target fusion features comprises:

[0039] The initial point cloud features, the point cloud features after occlusion completion, the target object image features and the dense point cloud features are fused to obtain target fusion features.

[0040] The present application also provides a 3D object detection device, the 3D object detection device comprising:

[0041] An acquisition module is used to obtain the initial point cloud of the target object;

[0042] An extraction module, used to extract features based on the initial point cloud through a preset target detection model to obtain initial point cloud features;

[0043] A completion module, configured to perform occlusion completion processing based on the initial point cloud through a preset occlusion completion model to obtain point cloud features after occlusion completion, wherein the occlusion completion model is used to extract and complete the occlusion features of the target object;

[0044] The first fusion module is used to perform feature fusion on the initial point cloud features and the point cloud features after occlusion completion to obtain target fusion features, and detect the target fusion features to obtain a 3D detection frame of the target object.

[0045] And / or, the 3D target detection device further includes: a sample acquisition module, used to acquire point cloud training samples; a missing complement module, used to complement the missing parts of the point cloud training samples to obtain a complete point cloud label; a training module, used to iteratively train a preset model to be trained based on the point cloud training samples and the complete point cloud label to obtain an occlusion completion model that meets the accuracy condition;

[0046] And / or, the missing supplement module includes: a point cloud extraction module, used to extract the initial object point cloud of each object in the point cloud training sample; a symmetry module, used to perform symmetry processing on the initial object point cloud to obtain the corresponding target object point cloud; a selection module, used to select a preset number of similar point clouds with the highest similarity to the initial object point cloud from the target object point cloud; a similar point cloud supplement module, used to supplement the target missing part of the initial object point cloud of each point cloud training sample based on the similar point cloud to obtain a complete point cloud label;

[0047] And / or, the symmetry module includes: a rasterization module, used for performing rasterization processing on the initial object point cloud to obtain a rasterized initial object point cloud; an axisymmetric module, used for determining a central axis of the rasterized initial object point cloud, and performing axisymmetric processing on the rasterized initial object point cloud according to the central axis to obtain a target object point cloud;

[0048] And / or, the selection module includes: a rasterization processing module, used to perform rasterization processing on the initial object point cloud and each of the target object point clouds, respectively, to obtain a rasterized initial object point cloud and a rasterized target object point cloud; a vector calculation module, used to determine a first vector of the rasterized initial object point cloud and a second vector of each of the rasterized target object point clouds, and perform product calculation on the first vector and each of the second vectors, respectively, to obtain a vector product result; a similar point cloud determination module, used to determine the target object point clouds corresponding to the vector product results with the largest preset scores as similar point clouds;

[0049] And / or, the 3D object detection device further includes: an image acquisition module, used to acquire an initial object image of the target object; a feature extraction module, used to extract target object image features of the initial object image; a second fusion module, used to perform feature fusion on the initial point cloud features, the point cloud features after occlusion completion, and the target object image features to obtain target fusion features;

[0050] And / or, the feature extraction module includes: an initial feature extraction module, used to input the initial object image into a preset image feature extraction network to obtain initial object image features; an enhanced feature extraction module, used to extract a partial image of a preset position of the initial object image, and input the partial image into a preset image feature extraction network to obtain enhanced object image features; a splicing module, used to splice the initial object image features with the enhanced object image features to obtain target object image features;

[0051] And / or, the 3D target detection device also includes: a dense module, used to input the initial point cloud and the initial object image into a preset depth completion model, and based on the depth completion model, complete the initial point cloud and the initial object image to obtain a densified point cloud; a convolution module, used to perform sparse convolution on the densified point cloud and the initial point cloud to obtain dense point cloud features; a third fusion module, used to perform feature fusion on the initial point cloud features, the point cloud features after occlusion completion, the target object image features and the dense point cloud features to obtain target fusion features.

[0052] The present application also provides a 3D object detection device, the 3D object detection device comprising: a memory, a processor, and a program stored in the memory for implementing the 3D object detection method,

[0053] The memory is used to store a program for implementing a 3D object detection method;

[0054] The processor is used to execute a program for implementing the 3D object detection method to implement the steps of the 3D object detection method.

[0055] The present application also provides a storage medium, on which is stored a program for implementing the 3D object detection method, and the program for implementing the 3D object detection method is executed by a processor to implement the steps of the 3D object detection method.

[0056] The present application provides a 3D target detection method, device, equipment and storage medium. Compared with the related art in which the perceived target is often occluded when the vehicle is driving, resulting in incomplete target shape, sparse point cloud features collected by the perceived target, and low accuracy of 3D target detection, in the present application, an initial point cloud of the target object is obtained; based on the initial point cloud, feature extraction is performed through a preset target detection model to obtain initial point cloud features; based on the initial point cloud, occlusion completion processing is performed through a preset occlusion completion model to obtain point cloud features after occlusion completion, wherein the occlusion completion model is used to extract and complete the occlusion features of the target object; the initial point cloud features and the point cloud features after occlusion completion are feature fused to obtain target fusion features, and the target fusion features are detected to obtain a 3D detection frame of the target object. That is, in the present application, the occlusion features of the target object are extracted and completed through the pre-trained occlusion completion model, so that the collected point cloud features of the perceived target are dense, and the point cloud features after occlusion completion are fused with the initial point cloud features, so that the fused target fusion features contain more object information, thereby improving the accuracy of 3D target detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application. In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art description will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0058] Figure 1 It is a schematic diagram of the device structure of the hardware operating environment involved in the embodiment of the present application;

[0059] Figure 2 This is a flow chart of the first embodiment of the 3D object detection method of the present application;

[0060] Figure 3 This is a module diagram of the 3D object detection device of the present application;

[0061] Figure 4 A schematic diagram of a scenario in which the initial target point cloud is symmetrically processed in the 3D target detection method of this application;

[0062] Figure 5 A schematic diagram of a scene of similar point cloud computing in the 3D object detection method of this application;

[0063] Figure 6 This is a flowchart of the third embodiment of the 3D object detection method of the present application.

[0064] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0065] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0066] like Figure 1 As shown, Figure 1 It is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiment of the present application.

[0067] The terminal in the embodiment of the present application can be a PC, or it can be a smart phone, a tablet computer, an e-book reader, an MP3 (Moving Picture Experts Group Audio Layer III) player, an MP4 (Moving Picture Experts Group Audio Layer IV) player, a portable computer, or other portable terminal devices with display function.

[0068] like Figure 1 As shown, the terminal may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0069] Optionally, the terminal may also include a camera, an RF (Radio Frequency) circuit, a sensor, an audio circuit, a WiFi module, and the like. Among them, sensors include light sensors, motion sensors, and other sensors. Specifically, the light sensor may include an ambient light sensor and a proximity sensor, wherein the ambient light sensor may adjust the brightness of the display screen according to the brightness of the ambient light, and the proximity sensor may turn off the display screen and / or backlight when the mobile terminal is moved to the ear. As a type of motion sensor, the gravity acceleration sensor can detect the magnitude of acceleration in each direction (generally three axes), and can detect the magnitude and direction of gravity when stationary. It can be used for applications that identify the posture of the mobile terminal (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer, tapping), etc.; of course, the mobile terminal may also be equipped with other sensors such as gyroscopes, barometers, hygrometers, thermometers, infrared sensors, etc., which will not be repeated here.

[0070] Those skilled in the art will understand that Figure 1 The terminal structure shown in the figure does not constitute a limitation on the terminal, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0071] like Figure 1 As shown, the memory 1005 as a computer storage medium may include an operating device, a network communication module, a user interface module, and a 3D object detection program.

[0072] exist Figure 1 In the terminal shown, the network interface 1004 is mainly used to connect to the background server and communicate data with the background server; the user interface 1003 is mainly used to connect to the client (user end) and communicate data with the client; and the processor 1001 can be used to call the 3D target detection program stored in the memory 1005.

[0073] Reference Figure 2 , an embodiment of the present application provides a 3D target detection method, the 3D target detection method comprising:

[0074] Step S100, obtaining an initial point cloud of a target object;

[0075] Step S200, based on the initial point cloud, feature extraction is performed through a preset target detection model to obtain initial point cloud features;

[0076] Step S300, based on the initial point cloud, performing occlusion completion processing through a preset occlusion completion model to obtain point cloud features after occlusion completion, wherein the occlusion completion model is used to extract and complete the occlusion features of the target object;

[0077] Step S400, performing feature fusion on the initial point cloud features and the point cloud features after occlusion completion to obtain target fusion features, and detecting the target fusion features to obtain a 3D detection frame of the target object.

[0078] In this embodiment, the application scenarios are:

[0079] As an example, the background involved in this embodiment is: At present, with the development of autonomous driving technology, people pay more and more attention to the perception system of autonomous driving vehicles. Vehicle target perception is an important part of the autonomous driving system, among which 2D perception is far from enough for high-level autonomous driving, so there is a demand for 3D perception in the autonomous driving system.

[0080] As an example, the scenario of 3D target detection can be the detection of vehicles / or pedestrians / or obstacles by an autonomous driving car during driving. In the related art, radar and image fusion are usually used on autonomous driving vehicles to achieve 3D perception of the vehicle. However, the perceived target is often occluded when the vehicle is driving, resulting in an incomplete target shape. This method ignores the occlusion factor of the target image when the vehicle is driving, making the point cloud features collected by the perceived target sparse, resulting in low accuracy of 3D target detection. For this scenario, the 3D target detection method of this embodiment extracts and completes the occlusion features of the target object through a pre-trained occlusion completion model to obtain the target point cloud features, so that the collected point cloud features of the perceived target are dense, and the target point cloud features are feature-fused with the object image features, thereby improving the accuracy of 3D target detection.

[0081] As an example, the application scenarios of 3D target detection include not only the detection of vehicles / pedestrians / or obstacles during driving of the above-mentioned car, but also various 3D target detection scenarios, which are not specifically limited here.

[0082] This embodiment aims to: improve the accuracy of 3D object detection.

[0083] In this embodiment, the 3D object detection method is applied to a 3D object detection device.

[0084] The specific steps are as follows:

[0085] Step S100, obtaining an initial point cloud of a target object;

[0086] In this embodiment, the target object refers to objects such as vehicles / pedestrians / obstacles that can be detected within the range of the vehicle during driving, and the initial point cloud refers to a set of vectors in a three-dimensional coordinate system, that is, the three-dimensional coordinate data of the target object.

[0087] In this embodiment, the device obtains the initial point cloud of the target object by using a radar or a laser radar pre-installed on the moving vehicle to obtain the initial point cloud of objects that can be detected within the range during the vehicle's driving process.

[0088] Step S200, based on the initial point cloud, feature extraction is performed through a preset target detection model to obtain initial point cloud features;

[0089] In this embodiment, the device performs feature extraction based on the initial point cloud through a preset target detection model to obtain initial point cloud features, wherein the target detection model is trained through a preset 3D detection algorithm network structure, and the preferred 3D detection algorithm network is a 3D backbone network, which performs convolution and other steps on the initial point cloud to extract corresponding features and obtain initial point cloud features.

[0090] Step S300, based on the initial point cloud, performing occlusion completion processing through a preset occlusion completion model to obtain point cloud features after occlusion completion, wherein the occlusion completion model is used to extract and complete the occlusion features of the target object;

[0091] In this embodiment, the device performs occlusion completion processing based on the initial point cloud through a preset occlusion completion model to obtain target point cloud features, wherein the occlusion completion model is used to extract and complete the occlusion features of the target object, and the occlusion completion model is obtained by iteratively training a preset model to be trained based on point cloud training samples and complete point cloud labels, and the complete point cloud label is the supervision signal of the model, which is generated by additional processing based on the point cloud training samples.

[0092] In this embodiment, it should be noted that the target detection model and the occlusion completion model can be built into a server distributedly deployed inside the terminal device computer system, and the target detection model and the occlusion completion model can also be deployed in a GPU hardware parallel computing accelerator inside the terminal device computer system to improve computing efficiency.

[0093] In this embodiment, in many cases, due to the different degrees of occlusion of the target objects detected at different positions and angles during the dynamic driving of the vehicle, the point cloud features collected by the perceived target are sparse, resulting in low accuracy of 3D target detection. Therefore, the present application extracts and completes the occlusion features of the target object through a pre-trained occlusion completion model, making the collected point cloud features of the perceived target dense, thereby improving the accuracy of 3D target detection.

[0094] Step S400, performing feature fusion on the initial point cloud features and the point cloud features after occlusion completion to obtain target fusion features, and detecting the target fusion features to obtain a 3D detection frame of the target object.

[0095] In this embodiment, the device fuses the initial point cloud features and the point cloud features after occlusion completion to obtain target fusion features, and detects the target fusion features to obtain a 3D detection frame of the target object, wherein the category of the detection frame is determined by the requirements of the application, such as in the autonomous driving scene, there are categories such as motor vehicles, non-motor vehicles, and people. The scale of the target frame is defined by the following parameters (x, y, z, w, l, h, θ), where (x, y, z) represents the coordinates of the center point of the frame, and (w, l, h, θ) represents the width, length, height, and angle of the frame.

[0096] In this embodiment, after obtaining the initial point cloud features and the point cloud features after occlusion completion, the device uses a preset feature fusion module to fuse the two. Specifically, a query with learnable parameters is preset, whose physical meaning is Bev features (features from a bird's eye view or a bird's-eye view), and the two features are fused in sequence using a mutual attention mechanism. Finally, the 3D target detection frame is predicted by the detection head, that is, feature fusion is performed through the initial point cloud features and the point cloud features after occlusion completion, so as to further improve the accuracy of 3D target detection.

[0097] A 3D target detection method provided by the present application is compared with the related art in which the perceived target is often occluded when the vehicle is driving, resulting in incomplete target shape, sparse point cloud features collected by the perceived target, and low accuracy of 3D target detection. In the present application, an initial point cloud of the target object is obtained; based on the initial point cloud, feature extraction is performed through a preset target detection model to obtain initial point cloud features; based on the initial point cloud, occlusion completion processing is performed through a preset occlusion completion model to obtain point cloud features after occlusion completion, wherein the occlusion completion model is used to extract and complete the occlusion features of the target object; the initial point cloud features and the point cloud features after occlusion completion are feature fused to obtain target fusion features, and the target fusion features are detected to obtain a 3D detection frame of the target object. That is, in the present application, the occlusion features of the target object are extracted and completed through the pre-trained occlusion completion model, so that the collected point cloud features of the perceived target are dense, and the point cloud features after occlusion completion are fused with the initial point cloud features, so that the fused target fusion features contain more object information, thereby improving the accuracy of 3D target detection.

[0098] Based on the above first embodiment, the present application also provides another embodiment, the 3D object detection method includes:

[0099] Before the step S100 of acquiring the initial point cloud of the target object, the method includes the following steps A100-A300:

[0100] Step A100, obtaining point cloud training samples;

[0101] In this embodiment, the point cloud training sample is a point cloud used for model training, specifically a manually annotated point cloud, wherein each point cloud training sample contains multiple frames of point clouds. For example, the image of point cloud training sample A is collected in the time period P1 to P2, and the point cloud training sample A includes point clouds A1-A20 in the time period P1 to P2, for a total of 20 frames of point clouds.

[0102] Step A200, supplementing the missing part of the point cloud training sample to obtain a complete point cloud label;

[0103] In this embodiment, the device performs supplementary processing on the missing parts of the target on the point cloud training samples to obtain a complete point cloud label, wherein the complete point cloud label is a supervisory signal of the model and is generated by additional processing of the point cloud training samples.

[0104] Specifically, the step A200 includes the following steps A210-A240:

[0105] Step A210, extracting the initial object point cloud of the point cloud training sample;

[0106] In this embodiment, the device extracts the initial object point cloud of the point cloud training sample, wherein, with reference to Figure 4 The point cloud on the left, the initial object point cloud refers to the target object on each frame of the point cloud in each point cloud training sample. The device marks and / or calibrates it to obtain the initial object point cloud of the point cloud training sample. The marking method may be to frame it. For example, the point cloud training sample A includes point clouds A1-A2, A1 includes the target objects car B1 and pedestrian C, A2 includes the target objects car B2 and pedestrian D. The device marks the car B1, pedestrian C, object car B2, and pedestrian D on the point cloud A1-A2, where car B1 and car B2 may refer to the same car, but are collected at different times.

[0107] Step A220, performing symmetric processing on each of the initial object point clouds to obtain a corresponding target object point cloud;

[0108] In this embodiment, the device performs symmetrical processing on each of the initial object point clouds to obtain a corresponding target object point cloud, that is, symmetrical processing is performed on each initial object point cloud to obtain the same number of corresponding target object point clouds. For example, point cloud A1-A2 includes initial object point clouds car B1, pedestrian C, object car B2, and pedestrian D, then the corresponding target object point clouds generated are the symmetrical car B11, pedestrian C1, object car B21, and pedestrian D1, respectively.

[0109] Specifically, the step A220 includes the following steps A221-A222:

[0110] Step A221, performing rasterization processing on each of the initial object point clouds to obtain a rasterized initial object point cloud;

[0111] In this embodiment, the device performs rasterization processing on each of the initial object point clouds to obtain a rasterized initial object point cloud, wherein the size of the image grid can be set by itself. Specifically, the symmetric processing is to first count the maximum size of each type of target and perform rasterization, and then for the current target, count the grids with points and mark them to obtain the rasterized initial object point cloud.

[0112] Step A222, determining the central axis of the rasterized initial object point cloud, and performing axisymmetric processing on the rasterized initial object point cloud according to the central axis to obtain the target object point cloud.

[0113] In this embodiment, refer to Figure 4 The device first determines the central axis of the initial object point cloud after rasterization, and then performs axisymmetric processing on the initial object point cloud after rasterization according to the central axis to obtain the target object point cloud. Specifically, the device performs symmetrical processing on the current initial object point cloud after rasterization along the central axis, retains the points in the empty grid, and combines the original points and the symmetrical points to obtain an unobstructed new target.

[0114] Step A230, selecting a preset number of similar point clouds having the highest similarity to the initial object point cloud from the target object point cloud;

[0115] In this embodiment, the device selects a preset number of similar point clouds with the highest similarity to the initial object point cloud from the target object point cloud, wherein each of the initial object point clouds is different in different frames of images (i.e., the object changes dynamically), for example, the detected part of vehicle A in the initial object point cloud B is C, and the detected part of vehicle A in the initial object point cloud D is E, wherein C is smaller than E. Therefore, the device selects a preset number of similar point clouds with the highest similarity to the initial object point cloud from the target object point cloud, and the similar point clouds detect different parts of the initial object point cloud, so as to use the similar point clouds to supplement the missing parts (or occluded parts) of the initial object point cloud, thereby making the point cloud features collected by the perceived target dense, thereby improving the accuracy of 3D target detection.

[0116] Specifically, the step A230 includes the following steps A231-A233:

[0117] Step A231, rasterizing the initial object point cloud and each of the target object point clouds to obtain a rasterized initial object point cloud and a rasterized target object point cloud;

[0118] In this embodiment, the device rasterizes the initial object point cloud and each target object point cloud respectively to obtain a rasterized initial object point cloud and a rasterized target object point cloud, wherein the rasterization step refers to step A222 above and will not be repeated here.

[0119] Step A232, determining the first vector of the rasterized initial object point cloud and the second vector of each rasterized target object point cloud, and performing product calculations on the first vector of the rasterized initial object point cloud and the second vector of each rasterized target object point cloud to obtain a vector product result;

[0120] In this embodiment, the device determines the first vector of the rasterized initial object point cloud and the second vector of each rasterized target object point cloud. Specifically, referring to Figure 5 The device arranges the grids in the rasterized initial object point cloud and each rasterized target object point cloud into a one-dimensional vector in sequence to obtain a first vector and multiple second vectors, wherein arranging into a one-dimensional vector in sequence can specifically be marking the grids with points in the grid as 1, marking the grids without points in the grid as 0, and arranging them in sequence according to the number of grids to obtain a one-dimensional vector.

[0121] In this embodiment, the device multiplies the first vector of the rasterized initial object point cloud with the second vector of each rasterized target object point cloud to obtain a vector product result. For example, the first vector of the initial object point cloud A is A1, and the second vectors of the target object point cloud include B1, B2, B3, and B4. A1B1, A1B2, A1B3, and A1B4 are calculated respectively to obtain four vector product results [A1B1, A1B2, A1B3, and A1B4].

[0122] Step A233, determining the target object point cloud corresponding to the vector product result with the largest preset number of scores as the similar point cloud.

[0123] In this embodiment, the device determines the target object point cloud corresponding to the preset number of vector product results with the largest score as a similar point cloud. For example, the preset number is 2, and the vector product result is [A1B1, A1B2, A1B3, A1B4], where A1B1>A1B2>A1B3>A1B4, then the target object point clouds corresponding to B1 and B2 are determined as similar point clouds.

[0124] Step A240: Based on the similar point cloud, the initial object point cloud of each point cloud training sample is supplemented with the missing part of the target to obtain a complete point cloud label.

[0125] In this embodiment, the device supplements the missing parts of the target in the initial object point cloud of each point cloud training sample based on the similar point cloud to obtain a complete point cloud label. Specifically, the device uses the similar point cloud to supplement the missing parts of target A due to occlusion, and finally obtains a complete point cloud, that is, a complete point cloud label.

[0126] Step A300: Based on the point cloud training samples and the complete point cloud labels, the model to be trained is iteratively trained to obtain an occlusion completion model that meets the accuracy requirements.

[0127] In this embodiment, the device iteratively trains the model to be trained based on the point cloud training samples and the complete point cloud labels to obtain an occlusion completion model that meets the accuracy requirements, wherein the model to be trained is a preset initial model with occlusion completion processing capabilities, and only differs from the occlusion completion model in terms of accuracy.

[0128] Specifically, the step A300 includes the following steps A310-A340:

[0129] Step A310, inputting the point cloud training sample into a preset model to be trained to obtain predicted point cloud features;

[0130] In this embodiment, the device inputs the point cloud training samples into a preset model to be trained to obtain predicted point cloud features, wherein the predicted point cloud features are obtained by performing occlusion completion on the model being trained.

[0131] Step A320, performing difference calculation between the predicted point cloud feature and the complete point cloud label to obtain an error result;

[0132] In this embodiment, the device calculates the difference between the predicted point cloud features and the complete point cloud labels to obtain an error result, that is, verifies whether the results obtained by the model in training are consistent with the known results, and calculates the difference between the results to obtain an error result.

[0133] Step A330, based on the error result, determining whether the error result meets the error standard indicated by a preset error threshold range;

[0134] In this embodiment, since there is an error between the result after model training and the actual result, the error result is allowed to be within a preset error threshold range, so as to further determine whether the error result meets the error standard indicated by the preset error threshold range.

[0135] Step A340, if the error result does not meet the error standard indicated by the preset error threshold range, return to the step of inputting the fused feature sample into the preset first model to be trained to obtain the predicted user preference evaluation value, and stop training until the error result meets the error standard indicated by the preset error threshold range to obtain an occlusion completion model that meets the accuracy conditions.

[0136] In this embodiment, if the error result does not meet the error standard indicated by the preset error threshold range, it means that the error of the model in this training is too large, and the device returns to the step of inputting the point cloud training sample into the preset model to be trained to obtain the predicted point cloud features, and the training is stopped until the error result meets the error standard indicated by the preset error threshold range, so as to obtain an occlusion completion model that meets the accuracy conditions, thereby improving the accuracy of occlusion feature completion, and further making the point cloud features collected by the perceived target dense, thereby improving the accuracy of 3D target detection.

[0137] Based on the above first and second embodiments, the present application also provides another embodiment, wherein the 3D object detection method includes:

[0138] Before the step of fusing the initial point cloud features and the point cloud features after occlusion completion to obtain target fused features in step S400, the method includes the following steps B100-B300:

[0139] Step B100, obtaining an initial object image of the target object;

[0140] In this embodiment, the target object refers to objects such as vehicles and / or pedestrians and / or obstacles that can be detected within the range of the vehicle during driving, and the initial object image is an image captured by the vehicle's camera in the scene of vehicle driving.

[0141] In this embodiment, the device obtains the initial object image of the target object by using a camera pre-set on the moving vehicle.

[0142] Step B200, extracting target object image features from the initial object image;

[0143] In this embodiment, the device extracts the target object image features of the initial object image, wherein the image feature extraction method includes but is not limited to a template-based feature extraction method, an edge-based feature extraction method, a grayscale-based feature extraction method, and a spatial transformation-based feature extraction method, which are not specifically limited here.

[0144] Specifically, the step B200 includes the following steps B210-B230:

[0145] Step B210, inputting the initial object image into a preset image feature extraction network to obtain initial object image features;

[0146] In this embodiment, the device inputs the initial object image into a preset image feature extraction network to obtain the initial object image features, wherein the image feature extraction network is pre-trained using a preset image feature extraction method, and specific image feature extraction methods include but are not limited to template-based feature extraction methods, edge-based feature extraction methods, grayscale-based feature extraction methods, and spatial transformation-based feature extraction methods, which are not specifically limited here.

[0147] Step B220, extracting a partial image at a preset position of the initial object image, and inputting the partial image into a preset image feature extraction network to obtain enhanced object image features;

[0148] In this embodiment, the device extracts a partial image of a preset position of the initial object image, and inputs the partial image into a preset image feature extraction network to obtain enhanced object image features, wherein the partial image at the preset position refers to an image portion with the largest feature quantity, preferably the middle portion of the initial object image, and may also be other set image portions. The enhanced object image features obtained by feature extraction of the partial image are used to enhance the image features of the initial object image, so as to make the collected point cloud features of the target object dense.

[0149] Step B230, concatenating the initial object image feature with the enhanced object image feature to obtain the target object image feature.

[0150] In this embodiment, the device splices the initial object image features with the enhanced object image features to obtain the target object image features, so as to enhance the image features of the initial object image, and further make the collected point cloud features of the target object dense, so as to improve the accuracy of 3D target detection.

[0151] The step S400, in which the initial point cloud features and the point cloud features after occlusion completion are fused to obtain target fused features, further includes the following step B300:

[0152] Step B300, performing feature fusion on the initial point cloud features, the point cloud features after occlusion completion, and the target object image features to obtain target fusion features.

[0153] In this embodiment, after obtaining the initial point cloud features, the point cloud features after occlusion completion, and the target object image features, the device uses a preset feature fusion module to fuse the three. Specifically, a query with learnable parameters is preset, and its physical meaning is the Bev feature (features from a bird's-eye view or a bird's-eye view). The two features are fused in sequence using the mutual attention mechanism, and finally the 3D target detection frame is predicted by the detection head, that is, the feature fusion is performed through the initial point cloud features, the point cloud features after occlusion completion, and the target object image features, so as to further improve the accuracy of 3D target detection.

[0154] Based on the above-mentioned first embodiment, second embodiment and third embodiment, the present application also provides another embodiment, referring to Figure 6 , the 3D target detection method comprises:

[0155] Before the step B300 of fusing the initial point cloud features, the point cloud features after occlusion completion, and the target object image features to obtain target fused features, the method includes the following steps C100-C300:

[0156] Step C100, inputting the initial point cloud and the initial object image into a preset depth completion model, and completing the initial point cloud and the initial object image based on the depth completion model to obtain a densified point cloud;

[0157] In this embodiment, the device inputs the initial point cloud and the initial object image into a preset depth completion model, and based on the depth completion model, completes the initial point cloud and the initial object image to obtain a densified point cloud, wherein the depth completion model is a depth completion network trained based on object image samples and sparse point cloud samples as input data and dense point cloud samples as supervisory signals, and the depth completion network will further complete the features of the initial point cloud and the initial object image, so as to further make the point cloud features collected by the perceived target dense.

[0158] Step C200, performing sparse convolution on the densified point cloud and the initial point cloud to obtain dense point cloud features;

[0159] In this embodiment, the device performs sparse convolution on the densified point cloud and the initial point cloud to obtain dense point cloud features. Specifically, the device obtains dense point cloud features by subjecting the dense point cloud output by the deep completion network and the initial point cloud to voxelization and sparse convolutional networks, respectively. Voxelization is the process of converting the geometric representation of an object into a voxel representation that is closest to the object, thereby generating a volume data set that not only contains the surface information of the model, but also describes the internal properties of the model.

[0160] The step B300 of fusing the initial point cloud features, the point cloud features after occlusion completion, and the target object image features to obtain target fusion features includes the following steps C300:

[0161] Step C300 , fusing the initial point cloud features, the point cloud features after occlusion completion, the target object image features and the dense point cloud features to obtain target fused features.

[0162] In this embodiment, refer to Figure 6 , the device fuses the target object image features, the target point cloud features and the dense point cloud features to obtain target fusion features, and detects the target fusion features to obtain the 3D detection frame of the target object. Specifically, after obtaining the target object image features, the target point cloud features and the dense point cloud features, the device uses a feature fusion module to fuse the three. Specifically, a parameter-learnable query is preset, and its physical meaning is the Bev feature. The three features are fused in sequence using the mutual attention mechanism, and finally the 3D target detection frame is predicted by the detection head. Since the dense features of the image are 2D, the dense point cloud features and the dense occluded target features are all under Bev. The feature fusion module solves the feature fusion problem in different dimensions well by presetting the query and the mutual attention mechanism.

[0163] In this embodiment, the device uses the depth completion network dense point cloud, uses the image dense point cloud features, and uses the occlusion feature completion module to densely occlude the target features, which comprehensively and perfectly solves the point cloud sparsity problem and makes the point cloud features collected by the perceived target dense, thereby improving the accuracy of 3D target detection; this application also designs a feature fusion module, which uses the mutual attention mechanism to fuse multiple features to solve the fusion problem of 2D features and 3D features.

[0164] The present application also provides a 3D target detection device, referring to Figure 3 , the 3D target detection device comprises:

[0165] An acquisition module 10 is used to acquire an initial point cloud of a target object;

[0166] An extraction module 20 is used to extract features based on the initial point cloud by using a preset target detection model to obtain initial point cloud features;

[0167] A completion module 30, configured to perform occlusion completion processing based on the initial point cloud through a preset occlusion completion model to obtain point cloud features after occlusion completion, wherein the occlusion completion model is used to extract and complete the occlusion features of the target object;

[0168] The first fusion module 40 is used to perform feature fusion on the initial point cloud features and the point cloud features after occlusion completion to obtain target fusion features, and detect the target fusion features to obtain a 3D detection frame of the target object.

[0169] And / or, the 3D target detection device further includes: a sample acquisition module, used to acquire point cloud training samples; a missing complement module, used to complement the missing parts of the point cloud training samples to obtain a complete point cloud label; a training module, used to iteratively train a preset model to be trained based on the point cloud training samples and the complete point cloud label to obtain an occlusion completion model that meets the accuracy condition;

[0170] And / or, the missing supplement module includes: a point cloud extraction module, used to extract the initial object point cloud of each object in the point cloud training sample; a symmetry module, used to perform symmetry processing on the initial object point cloud to obtain the corresponding target object point cloud; a selection module, used to select a preset number of similar point clouds with the highest similarity to the initial object point cloud from the target object point cloud; a similar point cloud supplement module, used to supplement the target missing part of the initial object point cloud of each point cloud training sample based on the similar point cloud to obtain a complete point cloud label;

[0171] And / or, the symmetry module includes: a rasterization module, used for performing rasterization processing on the initial object point cloud to obtain a rasterized initial object point cloud; an axisymmetric module, used for determining a central axis of the rasterized initial object point cloud, and performing axisymmetric processing on the rasterized initial object point cloud according to the central axis to obtain a target object point cloud;

[0172] And / or, the selection module includes: a rasterization processing module, used to perform rasterization processing on the initial object point cloud and each of the target object point clouds, respectively, to obtain a rasterized initial object point cloud and a rasterized target object point cloud; a vector calculation module, used to determine a first vector of the rasterized initial object point cloud and a second vector of each of the rasterized target object point clouds, and perform product calculation on the first vector and each of the second vectors, respectively, to obtain a vector product result; a similar point cloud determination module, used to determine the target object point clouds corresponding to the vector product results with the largest preset scores as similar point clouds;

[0173] And / or, the 3D object detection device further includes: an image acquisition module, used to acquire an initial object image of the target object; a feature extraction module, used to extract target object image features of the initial object image; a second fusion module, used to perform feature fusion on the initial point cloud features, the point cloud features after occlusion completion, and the target object image features to obtain target fusion features;

[0174] And / or, the feature extraction module includes: an initial feature extraction module, used to input the initial object image into a preset image feature extraction network to obtain initial object image features; an enhanced feature extraction module, used to extract a partial image of a preset position of the initial object image, and input the partial image into a preset image feature extraction network to obtain enhanced object image features; a splicing module, used to splice the initial object image features with the enhanced object image features to obtain target object image features;

[0175] And / or, the 3D target detection device also includes: a dense module, used to input the initial point cloud and the initial object image into a preset depth completion model, and based on the depth completion model, complete the initial point cloud and the initial object image to obtain a densified point cloud; a convolution module, used to perform sparse convolution on the densified point cloud and the initial point cloud to obtain dense point cloud features; a third fusion module, used to perform feature fusion on the initial point cloud features, the point cloud features after occlusion completion, the target object image features and the dense point cloud features to obtain target fusion features.

[0176] The specific implementation of the 3D object detection device of the present application is basically the same as the above-mentioned embodiments of the 3D object detection method, and will not be repeated here.

[0177] Reference Figure 1 , Figure 1 It is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiment of the present application.

[0178] like Figure 1 As shown, the terminal may include: a processor 1001, such as a CPU, a network interface 1004, a user interface 1003, a memory 1005, and a communication bus 1002. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface). The memory 1005 may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0179] Optionally, the 3D object detection device may also include a rectangular user interface, a network interface, a camera, an RF (Radio Frequency) circuit, a sensor, an audio circuit, a WiFi module, etc. The rectangular user interface may include a display screen (Display), an input submodule such as a keyboard (Keyboard), and the optional rectangular user interface may also include a standard wired interface and a wireless interface. The network interface may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

[0180] Those skilled in the art will understand that Figure 1 The structure of the 3D object detection device shown in the figure does not constitute a limitation on the 3D object detection device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.

[0181] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, and a 3D object detection program. The operating system is a program that manages and controls the hardware and software resources of the 3D object detection device, and supports the operation of the 3D object detection program and other software and / or programs. The network communication module is used to realize the communication between the components inside the memory 1005, and the communication with other hardware and software in the 3D object detection system.

[0182] exist Figure 1 In the 3D object detection device shown, the processor 1001 is used to execute the 3D object detection program stored in the memory 1005 to implement the steps of any of the above-mentioned 3D object detection methods.

[0183] The specific implementation of the 3D object detection device of the present application is basically the same as the above-mentioned embodiments of the 3D object detection method, and will not be repeated here.

[0184] The present application also provides a storage medium, on which a program for implementing a 3D object detection method is stored, and the program for implementing a 3D object detection method is executed by a processor to implement the following 3D object detection method:

[0185] Obtain the initial point cloud of the target object;

[0186] Based on the initial point cloud, feature extraction is performed through a preset target detection model to obtain initial point cloud features;

[0187] Based on the initial point cloud, occlusion completion processing is performed through a preset occlusion completion model to obtain point cloud features after occlusion completion, wherein the occlusion completion model is used to extract and complete the occlusion features of the target object;

[0188] The initial point cloud features and the point cloud features after occlusion completion are fused to obtain target fusion features, and the target fusion features are detected to obtain a 3D detection frame of the target object.

[0189] Optionally, before the step of acquiring an initial point cloud of the target object, the method includes:

[0190] Get point cloud training samples;

[0191] The point cloud training samples are supplemented with missing parts of the target to obtain complete point cloud labels;

[0192] Based on the point cloud training samples and the complete point cloud labels, the preset model to be trained is iteratively trained to obtain an occlusion completion model that meets the accuracy requirements.

[0193] Optionally, the step of supplementing the missing part of the point cloud training sample to obtain a complete point cloud label includes:

[0194] Extracting the initial object point cloud of each object in the point cloud training sample;

[0195] Performing symmetric processing on the initial object point cloud to obtain a corresponding target object point cloud;

[0196] Selecting a preset number of similar point clouds with the highest similarity to the initial object point cloud from the target object point cloud;

[0197] Based on the similar point clouds, the initial object point clouds of each point cloud training sample are supplemented with target missing parts to obtain complete point cloud labels.

[0198] Optionally, the step of performing symmetric processing on the initial object point cloud to obtain a corresponding target object point cloud includes:

[0199] Performing rasterization processing on the initial object point cloud to obtain a rasterized initial object point cloud;

[0200] The central axis of the rasterized initial object point cloud is determined, and the rasterized initial object point cloud is subjected to axisymmetric processing according to the central axis to obtain a target object point cloud.

[0201] Optionally, the step of selecting a preset number of similar point clouds having the highest similarity to the initial object point cloud from the target object point cloud comprises:

[0202] Rasterizing the initial object point cloud and each of the target object point clouds respectively to obtain a rasterized initial object point cloud and a rasterized target object point cloud;

[0203] Determine a first vector of the rasterized initial object point cloud and a second vector of each of the rasterized target object point clouds, and perform product calculations on the first vector and each of the second vectors to obtain a vector product result;

[0204] The target object point cloud corresponding to the vector product result of the preset number of largest scores is determined as the similar point cloud.

[0205] Optionally, before the step of fusing the target object image features and the target point cloud features, the method includes:

[0206] Acquire an initial object image of the target object;

[0207] Extracting target object image features of the initial object image;

[0208] The step of fusing the initial point cloud features with the occlusion-completing point cloud features to obtain target fused features comprises:

[0209] The initial point cloud features, the point cloud features after occlusion completion and the target object image features are fused to obtain target fusion features.

[0210] Optionally, the step of extracting target object image features from the initial object image includes:

[0211] Inputting the initial object image into a preset image feature extraction network to obtain initial object image features;

[0212] Extracting a partial image of a preset position of the initial object image, and inputting the partial image into a preset image feature extraction network to obtain enhanced object image features;

[0213] The initial object image feature and the enhanced object image feature are concatenated to obtain the target object image feature.

[0214] Optionally, before the step of fusing the initial point cloud features, the point cloud features after occlusion completion, and the target object image features to obtain target fused features, the method includes:

[0215] Inputting the initial point cloud and the initial object image into a preset depth completion model, and completing the initial point cloud and the initial object image based on the depth completion model to obtain a densified point cloud;

[0216] Performing sparse convolution on the densified point cloud and the initial point cloud to obtain dense point cloud features;

[0217] The step of fusing the initial point cloud features, the point cloud features after occlusion completion, and the target object image features to obtain target fusion features comprises:

[0218] The initial point cloud features, the point cloud features after occlusion completion, the target object image features and the dense point cloud features are fused to obtain target fusion features.

[0219] The specific implementation of the storage medium of the present application is basically the same as the above-mentioned embodiments of the 3D target detection method, and will not be repeated here.

[0220] The present application also provides a computer program product, including a computer program, which implements the steps of the above-mentioned 3D object detection method when executed by a processor.

[0221] The specific implementation of the computer program product of the present application is basically the same as the above-mentioned embodiments of the 3D object detection method, and will not be repeated here.

[0222] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0223] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0224] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0225] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A 3D object detection method, characterized in that: The 3D target detection method comprises: Obtain the initial point cloud of the target object; Based on the initial point cloud, feature extraction is performed through a preset target detection model to obtain initial point cloud features; Based on the initial point cloud, occlusion completion processing is performed through a preset occlusion completion model to obtain point cloud features after occlusion completion, wherein the occlusion completion model is used to extract and complete the occlusion features of the target object; The initial point cloud features and the point cloud features after occlusion completion are fused to obtain target fusion features, and the target fusion features are detected to obtain a 3D detection frame of the target object.

2. The 3D object detection method according to claim 1, characterized in that: Before the step of obtaining an initial point cloud of the target object, the method includes: Get point cloud training samples; The point cloud training samples are supplemented with missing parts of the target to obtain complete point cloud labels; Based on the point cloud training samples and the complete point cloud labels, the preset model to be trained is iteratively trained to obtain an occlusion completion model that meets the accuracy requirements.

3. The 3D object detection method according to claim 2, characterized in that: The step of supplementing the missing part of the point cloud training sample to obtain a complete point cloud label includes: Extracting the initial object point cloud of each object in the point cloud training sample; Performing symmetric processing on the initial object point cloud to obtain a corresponding target object point cloud; Selecting a preset number of similar point clouds with the highest similarity to the initial object point cloud from the target object point cloud; Based on the similar point clouds, the initial object point clouds of each point cloud training sample are supplemented with target missing parts to obtain complete point cloud labels.

4. The 3D object detection method according to claim 3, characterized in that: The step of performing symmetric processing on the initial object point cloud to obtain a corresponding target object point cloud comprises: Performing rasterization processing on the initial object point cloud to obtain a rasterized initial object point cloud; The central axis of the rasterized initial object point cloud is determined, and the rasterized initial object point cloud is subjected to axisymmetric processing according to the central axis to obtain a target object point cloud.

5. The 3D object detection method according to claim 3, characterized in that: The step of selecting a preset number of similar point clouds having the highest similarity to the initial object point cloud from the target object point cloud comprises: Rasterizing the initial object point cloud and each of the target object point clouds respectively to obtain a rasterized initial object point cloud and a rasterized target object point cloud; Determine a first vector of the rasterized initial object point cloud and a second vector of each of the rasterized target object point clouds, and perform product calculations on the first vector and each of the second vectors to obtain a vector product result; The target object point cloud corresponding to the vector product result of the preset number of largest scores is determined as the similar point cloud.

6. The 3D object detection method according to claim 1, characterized in that: Before the step of fusing the target object image features and the target point cloud features, the method includes: Acquire an initial object image of the target object; Extracting target object image features of the initial object image; The step of fusing the initial point cloud features with the occlusion-completing point cloud features to obtain target fused features comprises: The initial point cloud features, the point cloud features after occlusion completion and the target object image features are fused to obtain target fusion features.

7. The 3D object detection method according to claim 6, characterized in that: The step of extracting target object image features of the initial object image comprises: Inputting the initial object image into a preset image feature extraction network to obtain initial object image features; Extracting a partial image of a preset position of the initial object image, and inputting the partial image into a preset image feature extraction network to obtain enhanced object image features; The initial object image feature and the enhanced object image feature are concatenated to obtain the target object image feature.

8. The 3D object detection method according to claim 6, characterized in that: Before the step of fusing the initial point cloud features, the point cloud features after occlusion completion, and the target object image features to obtain target fusion features, the method includes: Inputting the initial point cloud and the initial object image into a preset depth completion model, and completing the initial point cloud and the initial object image based on the depth completion model to obtain a densified point cloud; Performing sparse convolution on the densified point cloud and the initial point cloud to obtain dense point cloud features; The step of fusing the initial point cloud features, the point cloud features after occlusion completion, and the target object image features to obtain target fusion features comprises: The initial point cloud features, the point cloud features after occlusion completion, the target object image features and the dense point cloud features are fused to obtain target fusion features.

9. A 3D object detection device, characterized in that: The 3D target detection device comprises: An acquisition module is used to obtain the initial point cloud of the target object; An extraction module, used to extract features based on the initial point cloud through a preset target detection model to obtain initial point cloud features; A completion module, configured to perform occlusion completion processing based on the initial point cloud through a preset occlusion completion model to obtain point cloud features after occlusion completion, wherein the occlusion completion model is used to extract and complete the occlusion features of the target object; The first fusion module is used to perform feature fusion on the initial point cloud features and the point cloud features after occlusion completion to obtain target fusion features, and detect the target fusion features to obtain a 3D detection frame of the target object.

10. The 3D object detection device according to claim 9, characterized in that: The 3D object detection device also includes: A sample acquisition module is used to acquire point cloud training samples; a missing complement module is used to complement the missing parts of the point cloud training samples to obtain complete point cloud labels; a training module is used to iteratively train the preset model to be trained based on the point cloud training samples and the complete point cloud labels to obtain an occlusion completion model that meets the accuracy conditions; And / or, the missing supplement module includes: a point cloud extraction module, used to extract the initial object point cloud of each object in the point cloud training sample; a symmetry module, used to perform symmetry processing on the initial object point cloud to obtain the corresponding target object point cloud; a selection module, used to select a preset number of similar point clouds with the highest similarity to the initial object point cloud from the target object point cloud; a similar point cloud supplement module, used to supplement the target missing part of the initial object point cloud of each point cloud training sample based on the similar point cloud to obtain a complete point cloud label; And / or, the symmetry module includes: a rasterization module, used for performing rasterization processing on the initial object point cloud to obtain a rasterized initial object point cloud; an axisymmetric module, used for determining a central axis of the rasterized initial object point cloud, and performing axisymmetric processing on the rasterized initial object point cloud according to the central axis to obtain a target object point cloud; And / or, the selection module includes: a rasterization processing module, used to perform rasterization processing on the initial object point cloud and each of the target object point clouds, respectively, to obtain a rasterized initial object point cloud and a rasterized target object point cloud; a vector calculation module, used to determine a first vector of the rasterized initial object point cloud and a second vector of each of the rasterized target object point clouds, and perform product calculation on the first vector and each of the second vectors, respectively, to obtain a vector product result; a similar point cloud determination module, used to determine the target object point clouds corresponding to the vector product results with the largest preset scores as similar point clouds; And / or, the 3D object detection device further includes: an image acquisition module, used to acquire an initial object image of the target object; a feature extraction module, used to extract target object image features of the initial object image; a second fusion module, used to perform feature fusion on the initial point cloud features, the point cloud features after occlusion completion, and the target object image features to obtain target fusion features; And / or, the feature extraction module includes: an initial feature extraction module, used to input the initial object image into a preset image feature extraction network to obtain initial object image features; an enhanced feature extraction module, used to extract a partial image of a preset position of the initial object image, and input the partial image into a preset image feature extraction network to obtain enhanced object image features; a splicing module, used to splice the initial object image features with the enhanced object image features to obtain target object image features; And / or, the 3D target detection device also includes: a dense module, used to input the initial point cloud and the initial object image into a preset depth completion model, and based on the depth completion model, complete the initial point cloud and the initial object image to obtain a densified point cloud; a convolution module, used to perform sparse convolution on the densified point cloud and the initial point cloud to obtain dense point cloud features; a third fusion module, used to perform feature fusion on the initial point cloud features, the point cloud features after occlusion completion, the target object image features and the dense point cloud features to obtain target fusion features.

11. A 3D object detection device, characterized in that: The 3D object detection device comprises: a memory, a processor, and a program stored in the memory for implementing the 3D object detection method. The memory is used to store a program for implementing a 3D object detection method; The processor is used to execute a program for implementing the 3D object detection method to implement the steps of the 3D object detection method as claimed in any one of claims 1 to 8.

12. A storage medium, characterized in that: The storage medium stores a program for implementing the 3D object detection method, and the program for implementing the 3D object detection method is executed by a processor to implement the steps of the 3D object detection method as claimed in any one of claims 1 to 8.

Images