Full sparse point cloud detection model construction method, detection method, equipment and medium

Abstract

The invention provides a full sparse point cloud detection model construction method, a detection method, equipment and a medium. The method comprises the following steps: performing voxelization processing on initial point cloud data to obtain structured point cloud data; carrying out voxel feature coding on the structured point cloud data, and converting the structured point cloud data into three-dimensional voxel grid features; performing multi-scale feature extraction and feature fusion on the three-dimensional voxel grid features to obtain point cloud features; dividing positive sample features from the point cloud features according to a dynamic positive and negative sample distribution strategy; and performing iterative training according to the positive sample features to obtain a full sparse point cloud detection model. According to the full-sparse point cloud detection method, the calculation amount is not increased sharply along with the increase of the sensing range, and dynamic positive and negative sample distribution is applied to full-sparse point cloud detection, so that the point cloud detection effect is improved while the high efficiency of the full-sparse point cloud detection method is kept.

Description

Technical Field

[0001] The present invention belongs to the field of point cloud detection technology, and specifically relates to a fully sparse point cloud detection model construction method, a detection method, a device and a medium. Background Art

[0002] Point cloud detection technology, with its precise 3D environmental perception and object recognition capabilities, plays a vital role in numerous key sectors. In autonomous driving, it helps vehicles accurately perceive their surroundings and make safe and reliable driving decisions. In robotic navigation, it provides real-time environmental information, enabling robots to complete tasks efficiently. In areas such as construction engineering and drone mapping, it enables precise modeling and measurement of complex scenarios. In public safety and healthcare, it also provides strong technical support for related applications. This technology not only significantly enhances the intelligence and automation of systems in various fields, but also greatly improves work efficiency and enhances safety.

[0003] With the continuous advancement of deep learning technology, point cloud detection technology has been widely applied and developed in academia and industry. Existing point cloud detection technologies can be mainly divided into two types based on the prediction method: dense prediction and sparse prediction. Point clouds are sparse and disordered. Point cloud data is mainly distributed on the surface of the object, which leads to the problem of missing target center features. Dense prediction techniques use 3D sparse convolution and 2D convolution to diffuse the center features to obtain the center features of the point cloud. Based on the center features, the position and category of the detection box are predicted. However, center feature-based methods are more suitable for structured image data and do not take into account the sparsity and unstructured nature of point cloud data. Moreover, as the perception range increases, the proportion of foreground objects decreases, resulting in a waste of computational resources and low model efficiency. Sparse prediction is simple and efficient. It only focuses on the features of non-empty voxels in space and directly predicts the position and category of the detection box based on these non-empty voxel features. The allocation of positive and negative samples is a key step in sparse prediction schemes. Existing methods allocate positive samples by calculating center distance, which does not conform to the sparsity of point clouds. Such static allocation methods are difficult to handle the complex and diverse objects in real scenes. At the same time, the non-empty voxel features in the sparse prediction scheme have insufficient receptive field, resulting in low detection accuracy for large targets. Summary of the Invention

[0004] The purpose of the present invention is to propose a fully sparse point cloud detection model construction method, detection method, equipment and medium, which solves the problem of low target detection accuracy of fully sparse point cloud prediction.

[0005] To this end, in a first aspect, the present invention provides a method for constructing a fully sparse point cloud detection model, comprising the following steps:

[0006] Performing voxel processing on the initial point cloud data to obtain structured point cloud data;

[0007] Performing voxel feature encoding on the structured point cloud data to convert the structured point cloud data into three-dimensional voxel grid features;

[0008] Perform multi-scale feature extraction and feature fusion on 3D voxel grid features to obtain point cloud features;

[0009] According to the dynamic positive and negative sample allocation strategy, positive sample features are divided from the point cloud features;

[0010] Iterative training is performed based on the positive sample features to obtain a fully sparse point cloud detection model.

[0011] Optionally, the receiving the collected initial point cloud data and performing voxel processing on the initial point cloud data to obtain structured point cloud data includes:

[0012] According to a preset voxel size, the three-dimensional space where the initial point cloud data is located is divided into regular voxel grids, and the initial point cloud data is allocated to the voxel grids.

[0013] Optionally, performing voxel feature encoding on the structured point cloud data to convert the structured point cloud data into three-dimensional voxel grid features includes:

[0014] A plurality of point clouds within each voxel in the voxel grid are encoded into feature vectors, and the feature vectors are arranged according to spatial positions to obtain three-dimensional voxel grid features.

[0015] Optionally, performing multi-scale feature extraction and feature fusion on the three-dimensional voxel grid features to obtain point cloud features includes:

[0016] Extracting point cloud features of multiple scales from the three-dimensional voxel grid features, and recording the feature coordinates of non-empty voxels at each scale;

[0017] The point cloud features of each scale are sampled and merged step by step to generate fusion features;

[0018] The fusion features are progressively fused through convolution kernels of different sizes to obtain point cloud features.

[0019] Optionally, the step of upsampling and merging the point cloud features at each scale to generate fused features includes:

[0020] F=(F i1 ∪F i2 ∪F i3 ∪F i4 )

[0021] Among them, F is the fusion feature, F i1、F i2 、F i3 and F i4 are features extracted at different scales, and F i1 、F i2 、F i3 and F i4 The sampling multiple increases gradually.

[0022] Optionally, dividing positive sample features from point cloud features according to the dynamic positive and negative sample allocation strategy includes:

[0023] Map the position coordinates of the pre-annotated true value box to the dimension of the point cloud feature, and calculate the square distance between the non-empty voxel and the true value box in the dimension of the point cloud feature;

[0024] Sort the squared distances between all non-empty voxels and the true value box, and select several non-empty voxels with the smallest squared distances as candidate positive samples;

[0025] The voxel features of the candidate positive sample are decoded to obtain the predicted box coordinates and the predicted category, and the loss of the candidate positive sample is calculated by the first loss function based on the true value box coordinates and the true value category. The candidate positive sample with the smallest loss is used as the positive sample feature.

[0026] Optionally, performing iterative training based on positive sample features to obtain a fully sparse point cloud detection model includes:

[0027] A second loss function is established, the loss is calculated according to the positive sample features and gradient backpropagation is performed, and iterative training is performed to obtain a fully sparse point cloud detection model.

[0028] In a second aspect, a fully sparse point cloud detection method is provided, comprising the following steps:

[0029] Constructing a fully sparse point cloud detection model according to the method for constructing a fully sparse point cloud detection model;

[0030] Collect initial point cloud data through lidar;

[0031] The initial point cloud data is input into the fully sparse point cloud model, and the position and category of the target are predicted by the fully sparse point cloud model.

[0032] In a third aspect, an electronic device is provided, including a memory and a processor;

[0033] The memory stores computer-executable instructions;

[0034] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the method for constructing a fully sparse point cloud detection model or the method for detecting a fully sparse point cloud.

[0035] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and the computer program is used to enable a processor to implement the method for constructing a fully sparse point cloud detection model or a fully sparse point cloud detection method when executed.

[0036] Beneficial effects:

[0037] (1) The present disclosure provides a method for constructing a fully sparse point cloud detection model, a detection method, a device, and a medium. The fully sparse point cloud detection method does not cause a sharp increase in computational complexity as the perception range increases. Dynamic positive and negative sample allocation is applied to the fully sparse point cloud detection, thereby improving the point cloud detection effect while maintaining the high efficiency of the fully sparse point cloud detection method.

[0038] (2) The dynamic positive and negative sample allocation strategy disclosed in this disclosure screens candidate positive samples based on the squared distance between the ground truth box and non-empty voxels, ensuring that the candidate covers the potential target area; by selecting the candidate positive sample with the smallest loss as the positive sample feature. This allows the model to autonomously optimize the sample selection strategy during training, rather than relying on a fixed threshold, and dynamically determine the positive sample features, thus solving the problem of missing feature centers caused by point cloud sparsity.

[0039] (3) In this disclosure, multi-scale feature extraction and feature fusion are performed, and upsampling and merging are performed step by step to retain multi-level information from fine-grained local details to coarse-grained global structures. Feature progressive fusion is performed through convolution kernels of different sizes to gradually refine features. Large-scale convolution expands the receptive field to capture distant targets, and small-scale convolution strengthens the consistency of local features, thereby solving the problem of large target scale differences and easy loss of details in sparse point clouds.

[0040] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0042] Figure 1 A flowchart of a method for constructing a fully sparse point cloud detection model in the present disclosure;

[0043] Figure 2A network architecture diagram of a method for constructing a fully sparse point cloud detection model in the present disclosure;

[0044] Figure 3 This is a flowchart of a fully sparse point cloud detection method disclosed in the present invention;

[0045] Figure 4 This is a schematic structural diagram of an electronic device in the present disclosure.

[0046] In the figure, 100 is an electronic device, 110 is a processor, 120 is a memory, 130 is a communication component, and 140 is a bus. DETAILED DESCRIPTION

[0047] To make the objectives, technical solutions, and advantages of this application more clear, the technical solutions in this application will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0048] In the specification and claims of this application, as well as in the accompanying drawings, the terms "first," "second," "third," "fourth," and the like are used to distinguish similar objects and are not necessarily used to describe a particular order or precedence. It should be understood that such terms are interchangeable where appropriate. For example, first information could be referred to as second information, and similarly, second information could be referred to as first information without departing from the scope of this disclosure.

[0049] The word "if" as used herein may be interpreted as "when" or "when" or "in response to determining," depending on the context.

[0050] Furthermore, as used herein, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context indicates otherwise.

[0051] It should be further understood that the terms “comprises” and “includes” indicate the existence of features, steps, operations, elements, components, items, types, and / or groups, but do not preclude the existence, occurrence, or addition of one or more other features, steps, operations, elements, components, items, types, and / or groups.

[0052] The terms "or" and "and / or" as used herein are to be interpreted as inclusive, or mean any one or any combination. Thus, "A, B, or C" or "A, B, and / or C" means "any one of the following: A; B; C; A and B; A and C; B and C; A, B, and C." An exception to this definition occurs only when a combination of elements, functions, steps, or operations are inherently mutually exclusive in some manner.

[0053] Point cloud detection technology, with its precise 3D environmental perception and object recognition capabilities, plays a vital role in numerous key sectors. In autonomous driving, it helps vehicles accurately perceive their surroundings and make safe and reliable driving decisions. In robotic navigation, it provides real-time environmental information, enabling robots to complete tasks efficiently. In areas such as construction engineering and drone mapping, it enables precise modeling and measurement of complex scenarios. In public safety and healthcare, it also provides strong technical support for related applications. This technology not only significantly enhances the intelligence and automation of systems in various fields, but also greatly improves work efficiency and enhances safety.

[0054] With the continuous advancement of deep learning technology, point cloud detection technology has been widely applied and developed in academia and industry. Existing point cloud detection technologies can be mainly divided into two types based on the prediction method: dense prediction and sparse prediction. Point clouds are sparse and disordered. Point cloud data is mainly distributed on the surface of the object, which leads to the problem of missing target center features. Dense prediction techniques use 3D sparse convolution and 2D convolution to diffuse the center features to obtain the center features of the point cloud. Based on the center features, the position and category of the detection box are predicted. However, center feature-based methods are more suitable for structured image data and do not take into account the sparsity and unstructured nature of point cloud data. Moreover, as the perception range increases, the proportion of foreground objects decreases, resulting in a waste of computational resources and low model efficiency. Sparse prediction is simple and efficient. It only focuses on the features of non-empty voxels in space and directly predicts the position and category of the detection box based on these non-empty voxel features. The allocation of positive and negative samples is a key step in sparse prediction schemes. Existing methods allocate positive samples by calculating center distance, which does not conform to the sparsity of point clouds. Such static allocation methods are difficult to handle the complex and diverse objects in real scenes. At the same time, the non-empty voxel features in the sparse prediction scheme have insufficient receptive field, resulting in low detection accuracy for large targets.

[0055] To this end, in a first aspect, the present disclosure provides Figure 1 A fully sparse point cloud detection method shown in FIG. 1 includes the following steps:

[0056] S110, performing voxel processing on the initial point cloud data to obtain structured point cloud data;

[0057] According to a preset voxel size, the three-dimensional space where the initial point cloud data is located is divided into regular voxel grids, and the initial point cloud data is allocated to the voxel grids.

[0058] The initial point cloud data is marked with the true value box coordinates and true value categories, which are used to mark the position and category of the target to be detected in the point cloud, providing a basis for subsequent model training.

[0059] Voxel is the smallest unit in three-dimensional space. Voxelization is the process of dividing a continuous three-dimensional object or space into regularly arranged small cubes. These small cubes are called voxels. Each voxel can contain specific attribute information, such as density, color, material, etc.

[0060] According to a preset voxel size, the three-dimensional space where the initial point cloud data is located is divided into a regular voxel grid. The method of dividing the voxel grid can be uniform voxelization, adaptive voxelization or octree-based voxelization.

[0061] In the process of allocating the initial point cloud data to the voxel grid, the maximum number of point clouds M in each voxel is set. When the number of point clouds in a voxel exceeds M, M points are randomly selected in the voxel; when the number of point clouds in a voxel is less than M, the number of point clouds less than M is supplemented with 0 until the number of point clouds reaches M.

[0062] S120, performing voxel feature encoding on the structured point cloud data, and converting the structured point cloud data into three-dimensional voxel grid features;

[0063] A plurality of point clouds within each voxel in the voxel grid are encoded into feature vectors, and the feature vectors are arranged according to spatial positions to obtain three-dimensional voxel grid features.

[0064] The point cloud is encoded using the Voxel Feature Encoder (VFE). This module encodes the features of each point within each voxel of the voxelized 3D point cloud, converting the sparse point cloud data into a dense voxel feature tensor suitable for subsequent convolution operations. The encoding method can be either mean encoding or Pointnet encoding. This encoding converts irregular structured point cloud data into a regular 3D voxel grid feature, facilitating subsequent processing using convolutional neural networks.

[0065] S130, performing multi-scale feature extraction and feature fusion on the three-dimensional voxel grid features to obtain point cloud features;

[0066] like Figure 2 The figure shows the network architecture for multi-scale feature extraction and feature fusion of 3D voxel grid features, including:

[0067] S131, extracting point cloud features of multiple scales from the three-dimensional voxel grid features, and recording the feature coordinates of non-empty voxels at each scale;

[0068] Point cloud features at multiple scales are extracted using the Submanifold Sparse Convolution Encoder (SSCE). The SSCE efficiently handles data sparsity while preserving spatial structure information through submanifold sparse convolution.

[0069] In one embodiment, the extracted point cloud features at multiple scales are F8, F 16 、F 32 and F 64 , among which F8, F 16 、F 32 and F 64 Represents the point cloud features sampled at 8 times, 16 times, 32 times and 64 times respectively. 16 、F 32 and F 64 The non-empty voxel features under are P8, P 16 、P 32 and P 64 .

[0070] S132, upsampling and merging the point cloud features of each scale step by step to generate fusion features;

[0071] The point cloud features at multiple scales are merged step by step upwards, preserving multi-level information from fine-grained local details to coarse-grained global structures. In one embodiment, the fused features are:

[0072] F=(F∪F 16 ∪F 32 ∪F 64 )

[0073] Among them, F is the fusion feature, F8, F 16 、F 32 and F 64 Point cloud features sampled at 8x, 16x, 32x, and 64x magnification.

[0074] S133. The fused features are progressively fused through convolution kernels of different sizes to obtain point cloud features.

[0075] In one embodiment, the convolution kernels are C7, C5, and C3, representing 7×7, 5×5, and 3×3 convolution kernels, respectively. Through the progressive convolution operations of C7, C5, and C3, the model gradually optimizes features from coarse to fine: C7 extracts global context and eliminates redundant background information; C5 enhances mid-scale features and establishes a connection between the local and global scales; and C3 refines local details to improve detection accuracy. By progressively fusing features through convolution kernels of different sizes, features are gradually refined. Large-scale convolution expands the receptive field to capture distant objects, while small-scale convolution strengthens the consistency of local features, thus addressing the problem of large object scale differences and easy loss of details in sparse point clouds.

[0076] S140, dividing positive sample features from the point cloud features according to a dynamic positive and negative sample allocation strategy;

[0077] These include:

[0078] S141, mapping the position coordinates of the pre-annotated true value box to the dimension of the point cloud feature, and calculating the square distance between the non-empty voxel and the true value box in the dimension of the point cloud feature;

[0079] The true value box is the position of the target pre-annotated in the initial point cloud data, including G x and G y , mapped to the dimension of the point cloud feature to obtain the feature position coordinate G x ′ and G y ′ , the calculation formula is as follows:

[0080]

[0081] Among them, G x and G y The position coordinates of the center of the truth box, V x and V y is the voxel size, S represents the feature downsampling ratio, R x0 and R y0 Indicates the starting position of the perception range.

[0082] S142, sorting the squared distances between all non-empty voxels and the ground truth box, and selecting several non-empty voxels with the smallest squared distances as candidate positive samples;

[0083] The calculation formula of the square distance between the non-empty voxel and the true value box is as follows,

[0084] D=(G x ′ -P x ) 2 +(G y ′ -P y )2

[0085] Among them, P x and P y is the position coordinate of the non-empty voxel, D is the feature position coordinate G x ′ and G y ′ The squared distance to the position coordinates of non-empty voxels.

[0086] Sort the square distances between all non-empty voxels and the true value box, and select several non-empty voxels with the smallest square distance as candidate positive samples. The calculation formula is as follows:

[0087] C = topk(P, argsort(D))

[0088] Among them, C is the candidate positive sample, D is the square distance, and P represents the prediction results of all non-empty voxels.

[0089] S143. Decode the voxel features of the candidate positive sample to obtain predicted box coordinates and predicted categories, combine the true box coordinates and the true category, calculate the loss of the candidate positive sample through a first loss function, and use the candidate positive sample with the smallest loss as the positive sample feature.

[0090] The first loss function includes a regression loss function and a classification loss function. In one embodiment, the regression loss function selects L2 and the classification loss function selects FocalLoss. The calculation formula is as follows:

[0091] Cost = W r ×L2(C b , G b )+W c ×FocalLoss(C c , G c )

[0092] Pos=(P,argmin(Cost))

[0093] Among them, W r and W c are the weights of the regression loss function and the classification loss function, respectively, C b and C c For candidate positive sample prediction box and prediction category, G b and G c is the true value box and true value category, Cost is the loss of the candidate positive sample, and Pos is the point with the minimum loss among all losses.

[0094] The dynamic positive and negative sample allocation strategy screens candidate positive samples based on the squared distance between the ground truth box and non-empty voxels, ensuring that the candidate samples cover the potential target area. Positive samples with the lowest loss are selected as positive sample features. This allows the model to autonomously optimize the sample selection strategy during training, rather than relying on a fixed threshold. This dynamically determines positive sample features and addresses the issue of missing feature centers caused by point cloud sparsity. The positive and negative sample allocation process does not involve gradient backpropagation updates, preventing training instability.

[0095] S150. Perform iterative training based on the positive sample features to obtain a fully sparse point cloud detection model.

[0096] A second loss function is established, the loss is calculated according to the positive sample features, and gradient backpropagation is performed. After iterative training, a fully sparse point cloud detection model is obtained.

[0097] The second loss function includes a regression loss function and a classification function. In one embodiment, the regression loss function selects L2 and the classification loss function selects FocalLoss. The calculation formula is as follows:

[0098] Loss=Mean(L2(Pos b ,G b )+FocalLoss(Pos c ,G c ))

[0099] Among them, Loss is the final loss value, participating in gradient back propagation, G b and G c is the true value frame and true value category, Pos b and Pos c Predict the box and category for the positive sample.

[0100] By iteratively training the positive sample features, the accuracy of the predicted boxes and predicted categories is gradually improved, and the allocation results are dynamically adjusted accordingly. Finally, a point cloud 3D detection model is trained.

[0101] Second, as Figure 3 As shown, a fully sparse point cloud detection method is provided, comprising the following steps:

[0102] S210, constructing a fully sparse point cloud detection model according to the method for constructing a fully sparse point cloud detection model;

[0103] S220, collecting initial point cloud data through laser radar;

[0104] S230: Input the initial point cloud data into the fully sparse point cloud model, and predict the position and category of the target using the fully sparse point cloud model.

[0105] Thirdly, as Figure 4 As shown, an electronic device is provided, characterized in that it includes: a memory, a processor;

[0106] The memory stores computer-executable instructions;

[0107] The processor executes the computer-executable instructions stored in the memory, so that the processor performs the above method.

[0108] In one embodiment, the electronic device 100 includes: at least one processor 110 and a memory 120. Optionally, the electronic device 100 further includes a communication component 130. The processor 110, the memory 120, and the communication component 130 are connected via a bus 140.

[0109] During the specific implementation process, at least one processor 110 executes the computer-executable instructions stored in the memory 120, so that the at least one processor 110 performs the above method.

[0110] The specific implementation process of the processor 110 can be found in the above method embodiment. Its implementation principle and technical effects are similar and will not be repeated here in this embodiment.

[0111] In the above embodiments, it should be understood that the processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), etc. A general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in the present invention may be directly implemented by a hardware processor or implemented by a combination of hardware and software modules in the processor.

[0112] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (NVM), such as at least one disk memory.

[0113] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus. Buses can be classified into address buses, data buses, and control buses. For ease of illustration, the buses in the drawings of this application are not limited to just one bus or just one type of bus.

[0114] In a fourth aspect, the present application also provides a computer-readable storage medium, in which computer-executable instructions are stored. When a processor executes the computer-executable instructions, the above-mentioned method is implemented.

[0115] The above-mentioned readable storage medium can be implemented by any type of volatile or non-volatile memory device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium that can be accessed by a general-purpose or special-purpose computer.

[0116] An exemplary readable storage medium is coupled to a processor so that the processor can read information from the readable storage medium and write information to the readable storage medium. Of course, the readable storage medium can also be an integral part of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist in the device as discrete components.

[0117] The division of units is merely a logical functional division; actual implementations may employ alternative divisions, such as combining or integrating multiple units or components into another system, or omitting or disabling certain features. Furthermore, any direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units, either through an interface, electrical, mechanical, or other means.

[0118] Units described as separate components may or may not be physically separate, and components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0119] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0120] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling an electronic device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, and other media that can store program code.

[0121] Those skilled in the art will appreciate that all or part of the steps in the above-described method embodiments can be implemented using hardware associated with program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments. The aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0122] Finally, it should be noted that those skilled in the art will readily identify other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. The present invention is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art not disclosed herein. The present invention is not limited to the precise structure described above and illustrated in the accompanying drawings, and various modifications and variations may be made without departing from the scope thereof. The scope of the present invention is limited solely by the appended claims.

Claims

1. A method for constructing a fully sparse point cloud detection model, characterized in that: The steps include: Performing voxel processing on the initial point cloud data to obtain structured point cloud data; Performing voxel feature encoding on the structured point cloud data to convert the structured point cloud data into three-dimensional voxel grid features; Perform multi-scale feature extraction and feature fusion on 3D voxel grid features to obtain point cloud features; According to the dynamic positive and negative sample allocation strategy, positive sample features are divided from the point cloud features; Iterative training is performed based on the positive sample features to obtain a fully sparse point cloud detection model.

2. A method for constructing a fully sparse point cloud detection model according to claim 1, characterized in that: The receiving of the collected initial point cloud data and performing voxel processing on the initial point cloud data to obtain structured point cloud data includes: According to a preset voxel size, the three-dimensional space where the initial point cloud data is located is divided into regular voxel grids, and the initial point cloud data is allocated to the voxel grids.

3. The method for constructing a fully sparse point cloud detection model according to claim 2, wherein: The step of performing voxel feature encoding on the structured point cloud data and converting the structured point cloud data into three-dimensional voxel grid features comprises: A plurality of point clouds within each voxel in the voxel grid are encoded into feature vectors, and the feature vectors are arranged according to spatial positions to obtain three-dimensional voxel grid features.

4. The method for constructing a fully sparse point cloud detection model according to claim 1, wherein: The multi-scale feature extraction and feature fusion of the three-dimensional voxel grid features to obtain point cloud features include: Extracting point cloud features of multiple scales from the three-dimensional voxel grid features, and recording the feature coordinates of non-empty voxels at each scale; The point cloud features of each scale are sampled and merged step by step to generate fusion features; The fusion features are progressively fused through convolution kernels of different sizes to obtain point cloud features.

5. The method for constructing a fully sparse point cloud detection model according to claim 4, wherein: The step of upsampling and merging the point cloud features at each scale to generate fusion features includes: F=(F i1 ∪F i2 ∪F i3 ∪F i4 ) Among them, F is the fusion feature, F i1 、F i2 、F i3 and F i4 are features extracted at different scales, and F i1 、F i2 、F i3 and F i4 The sampling multiple increases gradually.

6. The method for constructing a fully sparse point cloud detection model according to claim 1, wherein: The method of dividing positive sample features from point cloud features according to the dynamic positive and negative sample allocation strategy includes: Map the position coordinates of the pre-annotated true value box to the dimension of the point cloud feature, and calculate the square distance between the non-empty voxel and the true value box in the dimension of the point cloud feature; Sort the squared distances between all non-empty voxels and the true value box, and select several non-empty voxels with the smallest squared distances as candidate positive samples; The voxel features of the candidate positive sample are decoded to obtain the predicted box coordinates and the predicted category, and the loss of the candidate positive sample is calculated by the first loss function based on the true value box coordinates and the true value category. The candidate positive sample with the smallest loss is used as the positive sample feature.

7. The method for constructing a fully sparse point cloud detection model according to claim 1, wherein: The iterative training based on the positive sample features to obtain a fully sparse point cloud detection model includes: A second loss function is established, the loss is calculated according to the positive sample features and gradient backpropagation is performed, and iterative training is performed to obtain a fully sparse point cloud detection model.

8. A fully sparse point cloud detection method, characterized in that: The steps include: Constructing a fully sparse point cloud detection model according to a fully sparse point cloud detection model construction method according to any one of claims 1 to 7; Collect initial point cloud data through lidar; The initial point cloud data is input into the fully sparse point cloud model, and the position and category of the target are predicted by the fully sparse point cloud model.

9. An electronic device, characterized in that: Including memory, processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor executes the fully sparse point cloud detection model construction method described in any one of claims 1 to 7 or the fully sparse point cloud detection method described in claim 8.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which is used to enable a processor to implement a fully sparse point cloud detection model construction method according to any one of claims 1 to 7 or a fully sparse point cloud detection method according to claim 8 when executed.

Images