Point cloud target detection method, terminal device and computer readable storage medium

Abstract

The application is suitable for the technical field of detection, and provides a point cloud target detection method and device, terminal equipment and computer readable storage medium, which comprises the following steps: dividing three-dimensional point cloud data to be processed into multiple voxels; for each voxel, generating a representation vector of the voxel according to a reference position in the voxel and a relative position of a point contained in the voxel relative to the reference position; extracting feature information of each voxel according to the representation vector of each voxel; and detecting a target object in the point cloud data according to the feature information of each voxel. Through the above method, the fine granularity of voxel feature information can be improved, and the accuracy of point cloud target detection is improved.

Description

Technical Field

[0001] This application belongs to the field of detection technology, and in particular relates to point cloud target detection methods, devices, terminal equipment and computer-readable storage media. Background Technology

[0002] Point cloud data is a collection of point data on the surface of a target object detected by scanning devices such as LiDAR. Target detection based on point cloud data is a common technique in computer vision. Existing point cloud target detection methods typically divide the point cloud data into multiple voxels, then use the average value of the points within each voxel as the feature information for that voxel, and then use this feature information for target detection. When a voxel contains a large number of points, the feature information extracted by existing methods is relatively coarse, resulting in the loss of significant feature information and thus affecting the accuracy of subsequent target detection. Summary of the Invention

[0003] This application provides a point cloud target detection method, apparatus, terminal device, and computer-readable storage medium, which can improve the fineness of voxel feature information and thus improve the accuracy of point cloud target detection.

[0004] In a first aspect, embodiments of this application provide a point cloud target detection method, including:

[0005] The 3D point cloud data to be processed is divided into multiple voxels;

[0006] For each voxel, a representation vector of the voxel is generated based on the reference position in the voxel and the relative positions of the points contained in the voxel with respect to the reference position.

[0007] Extract the feature information of each of the multiple voxels based on their respective representation vectors;

[0008] Target objects in the point cloud data are detected based on the feature information of each of the multiple voxels.

[0009] In this embodiment, when extracting feature information for each voxel, a representation vector for the voxel is generated using the reference position within the voxel and the relative positions of the points contained within the voxel relative to the reference position. Then, the feature information of the voxel is extracted based on this representation vector. This method effectively avoids losing a significant amount of feature information when a voxel contains a large number of points, improving the fineness of the feature information. Furthermore, compared to the absolute positions of points in the sampled voxel, using the relative positions of the voxel points avoids positional confusion during feature extraction, effectively improving the accuracy of feature extraction and thus enhancing the precision of point cloud target detection.

[0010] In one possible implementation of the first aspect, dividing the 3D point cloud data to be processed into multiple voxels includes:

[0011] Based on the preset voxel size and the coordinates of each point in the three-dimensional point cloud data in the three-dimensional space, calculate the voxel index corresponding to each point in the three-dimensional point cloud data, wherein the three-dimensional space is the space to which the three-dimensional point cloud data belongs;

[0012] Points with the same voxel index are grouped into the same voxel, and each voxel corresponds to a voxel index.

[0013] In one possible implementation of the first aspect, for each voxel, generating a representation vector for the voxel based on a reference position in the voxel and the relative positions of points contained in the voxel with respect to the reference position includes:

[0014] Determine whether the number of points contained in the voxel is greater than or equal to a preset number;

[0015] If the number of points contained in the voxel is greater than or equal to a preset number, then the preset number of target points are obtained from the points contained in the voxel.

[0016] Calculate the relative position of each target point with respect to the reference position;

[0017] The representation vector of the voxel is generated based on the reference position and the relative position of each target point.

[0018] In one possible implementation of the first aspect, calculating the relative position of each target point with respect to the reference position includes:

[0019] For each target point, calculate the coordinate difference between the coordinates of the target point in the three-dimensional space and the coordinates of the reference position, where the three-dimensional space is the space to which the three-dimensional point cloud data belongs;

[0020] The coordinate difference corresponding to the target point is determined as the relative position of the target point with respect to the reference position.

[0021] In one possible implementation of the first aspect, after determining whether the number of points contained in the voxel is greater than or equal to a preset number, the method further includes:

[0022] If the number of points contained in the voxel is less than the preset number, then the representation vector of the voxel is generated based on the reference position in the voxel and the relative position of each point in the voxel relative to the reference position.

[0023] The representation vector is padded so that the dimension of the padded representation vector reaches a preset dimension, wherein the preset dimension is determined by the preset number of elements.

[0024] In one possible implementation of the first aspect, for each voxel, the reference position in the voxel is the position corresponding to the center point of the voxel.

[0025] In one possible implementation of the first aspect, the step of extracting feature information of each of the plurality of voxels based on their respective representation vectors includes:

[0026] For each voxel, the representation vector of the voxel is sparsely convolved to obtain the first intermediate feature of the voxel.

[0027] The first intermediate feature of the voxel is normalized to obtain the second intermediate feature of the voxel.

[0028] The second intermediate feature of the voxel is input into a preset activation function to obtain the feature information of the voxel.

[0029] Secondly, embodiments of this application provide a point cloud target detection device, comprising:

[0030] A voxel division unit is used to divide the 3D point cloud data to be processed into multiple voxels;

[0031] A vector generation unit is used to generate a representation vector for each voxel based on a reference position in the voxel and the relative positions of points contained in the voxel with respect to the reference position.

[0032] The feature extraction unit is used to extract the feature information of each of the multiple voxels based on their respective representation vectors.

[0033] The target detection unit is used to detect target objects in the point cloud data based on the feature information of each of the plurality of voxels.

[0034] Thirdly, embodiments of this application provide a terminal device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the point cloud target detection method as described in any one of the first aspects above.

[0035] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, characterized in that the computer program, when executed by a processor, implements the point cloud target detection method as described in any one of the first aspects above.

[0036] Fifthly, embodiments of this application provide a computer program product that, when run on a terminal device, causes the terminal device to execute the point cloud target detection method described in any one of the first aspects.

[0037] It is understood that the beneficial effects of the second to fifth aspects mentioned above can be found in the relevant descriptions in the first aspect mentioned above, and will not be repeated here. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a flowchart illustrating the point cloud target detection method provided in the embodiments of this application;

[0040] Figure 2 This is a schematic diagram of the point cloud target detection device provided in an embodiment of this application;

[0041] Figure 3 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application. Detailed Implementation

[0042] In the following description, specific details such as particular system architectures and techniques are set forth for illustrative purposes and not for limitation, in order to provide a thorough understanding of the embodiments of this application. However, those skilled in the art will understand that this application may also be implemented in other embodiments without these specific details. In other instances, detailed descriptions of well-known systems, apparatuses, circuits, and methods have been omitted so as not to obscure the description of this application with unnecessary detail.

[0043] It should be understood that, when used in this application specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or a collection thereof.

[0044] As used in this application specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrases "if determined" or "if detected [the described condition or event]" may be interpreted, depending on the context, as "once determined," "in response to determination," "once detected [the described condition or event]," or "in response to detection [the described condition or event]."

[0045] Furthermore, in the description of this application and the appended claims, the terms "first," "second," "third," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0046] References to "one embodiment" or "some embodiments" in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized.

[0047] See Figure 1 This is a schematic flowchart of the point cloud target detection method provided in the embodiments of this application. It is intended as an example and not a limitation. The method may include the following steps:

[0048] S101 divides the 3D point cloud data to be processed into multiple voxels.

[0049] The three-dimensional point cloud data in this embodiment can be point cloud data obtained by scanning devices such as lidar.

[0050] A voxel is the smallest unit of 3D spatial segmentation and can be viewed as a cuboid within 3D space. In this embodiment, the size of the voxel can be preset. The size of a voxel includes length, width, and height. Dividing 3D point cloud data into multiple voxels essentially divides the 3D space to which the 3D point cloud data belongs into multiple voxels. Due to the sparsity of point cloud data, each voxel after segmentation may contain multiple points or may not contain any points.

[0051] In one embodiment, one way to implement voxel division includes:

[0052] Based on the preset voxel size and the coordinates of each point in the three-dimensional point cloud data in the three-dimensional space, the voxel index corresponding to each point in the three-dimensional point cloud data is calculated, wherein the three-dimensional space is the space to which the three-dimensional point cloud data belongs; points with the same voxel index are grouped into the same voxel, and each voxel corresponds to one voxel index.

[0053] One way to calculate the voxel index corresponding to each point is to divide the coordinates of each point in three-dimensional space by the preset voxel size to obtain the voxel index corresponding to each point.

[0054] For example, assuming the coordinates of a point in three-dimensional space are [x, y, z], and the preset voxel size is [voxel_x, voxel_y, voxel_z], then the voxel index corresponding to this point is:

[0055] [index_x,index_y,index_z]=[x,y,z] / [voxel_x,voxel_y,voxel_z].

[0056] For ease of calculation, one optional method for calculating the voxel index corresponding to each point is to divide the coordinates of each point in 3D space by a preset voxel size to obtain the voxel index corresponding to each point. This division can be either rounding down or rounding up. When rounding down, the starting index of the voxel index is 0; when rounding up, the starting index of the voxel index is 1.

[0057] Through the voxel division described above, each voxel corresponds to a voxel index, and different voxels correspond to different voxel indices. Therefore, points with the same voxel index belong to the same voxel. In this way, we can obtain the points contained in each voxel, the number of points contained in each voxel, and the voxel index corresponding to each voxel.

[0058] S102, for each voxel, generate a representation vector for the voxel based on the reference position in the voxel and the relative positions of the points contained in the voxel with respect to the reference position.

[0059] In the prior art, the method for generating the representation vector of a voxel is to calculate the mean of the positions of all points in the voxel and generate the representation vector of the voxel based on the mean.

[0060] Using larger voxel sizes can reduce the computational load required for model inference, improve inference speed, and reduce the amount of GPU memory used. The results are shown in the table below:

[0061] voxel size Number of effective voxels per point cloud Inference memory usage Reasoning time [0.05,0.05,0.1] 15172 1865MiB 34.75ms [0.1,0.1,0.1] 11108 1339MiB 26.27ms

[0062] The point cloud data was divided according to the current mainstream voxel size [0.05, 0.05, 0.1]. The number of points within each voxel in all point clouds was counted, as shown in the table below:

[0063] Number of voxel interior points voxel count percentage 1 9.5947e+7 82.77％ 2 1.6238e+7 14.31％ 3 2.9098e+6 2.56％ 4 2.5727e+6 0.23％ 5 8.2367e+4 0.07％ (5,100] 7.1087e+4 0.06％

[0064] When the voxel size is [0.1, 0.1, 0.1], the voxel statistics for all point clouds are shown in the table below:

[0065] Number of voxel interior points voxel count percentage 1 5.3667e+7 64.58％ 2 1.5906e+7 19.14％ 3 6.9616e+6 8.38％ 4 3.8191e+6 4.6％ 5 1.9711e+6 2.37％ (5,100] 7.7737e+5 0.93％

[0066] In the table above, "Percentage" represents the percentage of all voxels whose in-voxel count is X. For example, if the in-voxel count is 1, the percentage is 64.58%, meaning that voxels with 1 in-voxel count account for 64.58% of all voxels.

[0067] As the above comparison shows, the number of points within a voxel gradually increases with its size. When a voxel contains a large number of points, the existing method of calculating the average of multiple points within the voxel affects the positional information of individual points, and the generated voxel representation vector cannot accurately reflect the true position of the voxel, thus affecting the accuracy of subsequent target detection. To solve the above problem, in this embodiment, the voxel representation vector is generated using the reference position and multiple points within the voxel. This improves the fine-grainedness of the voxel's feature information.

[0068] In one embodiment, one way to generate the representation vector of a voxel is as follows: for each voxel, a representation vector of the voxel is generated based on the reference position in the voxel and the absolute position of the points contained in the voxel. Here, the absolute position refers to the coordinates of the points contained in the voxel in three-dimensional space.

[0069] If the absolute positions of multiple points within a voxel are used, positional confusion may occur during subsequent feature extraction, making it impossible for the feature extraction model to distinguish which points are real and which are reference positions within the voxel. To address this issue, this embodiment uses the reference positions within the voxel and the relative positions of multiple points to generate the voxel representation vector. This avoids positional confusion during feature extraction and effectively improves the accuracy of feature extraction.

[0070] Optionally, a voxel representation vector can be generated based on the relative position of each point in the voxel. However, when the number of points in a voxel is large, the data processing volume is high, affecting the feature extraction efficiency. To solve this problem, a preset number of points in the voxel can be used for calculation. The preset number can be set manually according to actual needs.

[0071] In one embodiment, another way to generate the representation vector of voxels is as follows:

[0072] S1021, determine whether the number of points contained in the voxel is greater than or equal to a preset number.

[0073] If the number of points contained in the voxel is greater than or equal to the preset number, execute S1022-S1024; if the number of points contained in the voxel is less than the preset number, execute S1025 to S1026.

[0074] S1022, Obtain the preset number of target points from the points contained in the voxel.

[0075] A preset number of target points can be randomly selected from voxels, or a preset number of target points can be selected from voxels according to preset rules; no specific limitation is made here. For example, when it is desirable to avoid inaccurate feature information extracted due to chance, a random selection method can be used; when it is desirable to extract a specific type of feature information, a selection method based on preset rules can be used.

[0076] S1023, calculate the relative position of each target point with respect to the reference position.

[0077] Optionally, the relative position of the target point in the voxel can be calculated as follows:

[0078] Calculate the coordinate difference between the coordinates of the target point and the coordinates of the reference position in three-dimensional space, where the three-dimensional space is the space to which the three-dimensional point cloud data belongs; determine the coordinate difference of the target point as the relative position of the target point with respect to the reference position.

[0079] S1024, Generate the representation vector of the voxel based on the reference position and the relative position corresponding to each target point.

[0080] S1025, Generate a representation vector for the voxel based on the reference position in the voxel and the relative positions of all points contained in the voxel with respect to the reference position.

[0081] S1026, The representation vector is padded so that the dimension of the padded representation vector reaches a preset dimension, wherein the preset dimension is determined by the preset number of elements.

[0082] In practical applications, each point in a point cloud includes information in four dimensions: its three-dimensional coordinates (x, y, z) and its reflection intensity. Therefore, the data dimension corresponding to a preset number of target points is 4N, where N is the preset number. Since the reference position is preset, it may not contain actual points; therefore, the reference position only includes information in three dimensions: three-dimensional coordinates. In summary, the data dimension of the voxel representation vector is 4N+3.

[0083] If a voxel does not contain any points, a representation vector of the voxel is generated based on the reference position in the voxel, and then the representation vector is filled so that the dimension of the filled representation vector reaches the preset dimension.

[0084] If a voxel contains points, but the number of points is less than a preset number, then a representation vector of the voxel is generated based on the reference position in the voxel and the relative positions of all points in the voxel relative to the reference position. The representation vector is then filled so that the dimension of the filled representation vector reaches the preset dimension.

[0085] For example, assuming the preset number is 3, the corresponding dimension of the representation vector is 15, and a certain voxel contains 1 point, then a 7-dimensional representation vector is generated based on the reference position in the voxel and the 1 point it contains. Then, the 7-dimensional representation vector is filled so that the dimension of the filled representation vector reaches 15.

[0086] Using the above method, the network model can not only perceive the spatial location of the current voxel, but also know the detailed features at the point level inside the voxel, and can distinguish between generated virtual points and real points.

[0087] Optionally, preset values ​​can be used for filling. For example, fill with 0.

[0088] Optionally, for each voxel, the reference position within the voxel is the position corresponding to the center point of the voxel. The reference position can also be the position corresponding to a vertex of the voxel. No specific limitation is made here; it can be set according to actual needs.

[0089] S103, extract the feature information of each of the multiple voxels based on their respective representation vectors.

[0090] Alternatively, a multilayer perceptron method can be used to extract voxel feature information. However, when the voxel size is small, the number of voxels contained in the three-dimensional space is large. The multilayer perceptron method needs to calculate each voxel in the three-dimensional space, resulting in a large amount of data processing and affecting the efficiency of feature extraction.

[0091] To improve the efficiency of feature extraction, in one embodiment, S103 may include:

[0092] For each voxel, a sparse convolution is performed on the voxel's representation vector to obtain the voxel's first intermediate feature; the voxel's first intermediate feature is normalized to obtain the voxel's second intermediate feature; the voxel's second intermediate feature is input into a preset activation function to obtain the voxel's feature information.

[0093] For example, the steps of sparse convolution processing may include:

[0094] The input to sparse convolution is a matrix M and a hash table H. Matrix M represents the non-empty portions of the input feature map (a feature map composed of the feature information of multiple voxels), with a size of a×n, where a represents the number of non-empty positions and n represents the feature dimension. The key values ​​in hash table H represent the coordinates of the non-empty positions in the input, and the encoded value of the key indicates which row of feature M corresponds to that coordinate. Calculation process:

[0095] 1. Iterate through the keyword values ​​in H to determine the coordinates and number of non-empty positions on the convolution output feature map. Simultaneously, determine the rule engine. Each row in the rule engine corresponds to the position on the input feature map required to calculate a non-empty position on the convolution output feature map. Obtain the keyword value corresponding to that position in the input hash table through this position and populate it into the rule engine.

[0096] 2. Perform parallel computation on each row in the rule engine to obtain the output for each non-empty output position.

[0097] By using sparse convolution, some voxels that do not contain points can be filtered out, and only voxels that do contain points are calculated, thereby greatly reducing the amount of data processing.

[0098] S104, Detect the target object in the point cloud data based on the feature information of each of the plurality of voxels.

[0099] The feature information of multiple voxels can be input into a preset detection model to obtain the detection bounding box information of the target object in the point cloud data.

[0100] In this embodiment, when extracting feature information for each voxel, a representation vector for the voxel is generated using the reference position within the voxel and the relative positions of the points contained within the voxel relative to the reference position. Then, the feature information of the voxel is extracted based on this representation vector. This method effectively avoids losing a significant amount of feature information when a voxel contains a large number of points, improving the fineness of the feature information. Furthermore, compared to the absolute positions of points in the sampled voxel, using the relative positions of the voxel points avoids positional confusion during feature extraction, effectively improving the accuracy of feature extraction and thus enhancing the precision of point cloud target detection.

[0101] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0102] Corresponding to the point cloud target detection method described in the above embodiments, Figure 2 This is a structural block diagram of the point cloud target detection device provided in the embodiments of this application. For ease of explanation, only the parts related to the embodiments of this application are shown.

[0103] Reference Figure 2 The device includes:

[0104] Voxel division unit 21 is used to divide the three-dimensional point cloud data to be processed into multiple voxels.

[0105] The vector generation unit 22 is used to generate a representation vector for each voxel based on the reference position in the voxel and the relative position of the points contained in the voxel with respect to the reference position.

[0106] The feature extraction unit 23 is used to extract the feature information of each of the multiple voxels based on their respective representation vectors.

[0107] The target detection unit 24 is used to detect target objects in the point cloud data based on the feature information of each of the plurality of voxels.

[0108] Optionally, the voxel division unit 21 is also used for:

[0109] Based on the preset voxel size and the coordinates of each point in the three-dimensional point cloud data in the three-dimensional space, the voxel index corresponding to each point in the three-dimensional point cloud data is calculated, wherein the three-dimensional space is the space to which the three-dimensional point cloud data belongs; points with the same voxel index are grouped into the same voxel, and each voxel corresponds to one voxel index.

[0110] Optionally, the vector generation unit 22 is also used for:

[0111] Determine whether the number of points contained in the voxel is greater than or equal to a preset number;

[0112] If the number of points contained in the voxel is greater than or equal to a preset number, then the preset number of target points are obtained from the points contained in the voxel; the relative position of each target point relative to the reference position is calculated; and a representation vector of the voxel is generated based on the reference position and the relative position corresponding to each target point.

[0113] If the number of points contained in the voxel is less than the preset number, then a representation vector of the voxel is generated based on the reference position in the voxel and the relative positions of all points in the voxel relative to the reference position; the representation vector is padded so that the dimension of the padded representation vector reaches the preset dimension, wherein the preset dimension is determined by the preset number.

[0114] Optionally, the vector generation unit 22 is also used for:

[0115] For each target point, calculate the coordinate difference between the coordinates of the target point in the three-dimensional space and the coordinates of the reference position, where the three-dimensional space is the space to which the three-dimensional point cloud data belongs;

[0116] The coordinate difference corresponding to the target point is determined as the relative position of the target point with respect to the reference position.

[0117] Optionally, for each voxel, the reference position in the voxel is the position corresponding to the center point of the voxel.

[0118] Optionally, the feature extraction unit 23 is also used for:

[0119] For each voxel, a sparse convolution is performed on the voxel's representation vector to obtain the voxel's first intermediate feature; the voxel's first intermediate feature is normalized to obtain the voxel's second intermediate feature; the voxel's second intermediate feature is input into a preset activation function to obtain the voxel's feature information.

[0120] It should be noted that the information interaction and execution process between the above-mentioned devices / units are based on the same concept as the method embodiments of this application. For details on their specific functions and technical effects, please refer to the method embodiments section, and they will not be repeated here.

[0121] in addition, Figure 2 The point cloud target detection device shown can be a software unit, hardware unit, or a combination of software and hardware built into an existing terminal device, or it can be integrated into the terminal device as an independent component, or it can exist as an independent terminal device.

[0122] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the above-described division of functional units and modules is merely an example. In practical applications, the above functions can be assigned to different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiments can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit. Furthermore, the specific names of the functional units and modules are only for easy differentiation and are not intended to limit the scope of protection of this application. The specific working process of the units and modules in the above system can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0123] Figure 3 This is a schematic diagram of the structure of the terminal device provided in the embodiments of this application. For example... Figure 3 As shown, the terminal device 3 in this embodiment includes: at least one processor 30 ( Figure 3 (Only one is shown in the diagram) a processor, a memory 31, and a computer program 32 stored in the memory 31 and executable on the at least one processor 30, wherein the processor 30 executes the computer program 32 to implement the steps in any of the above-described point cloud target detection method embodiments.

[0124] The terminal device may be a desktop computer, laptop, handheld computer, or cloud server, etc. This terminal device may include, but is not limited to, a processor and memory. Those skilled in the art will understand that... Figure 3 This is merely an example of terminal device 3 and does not constitute a limitation on terminal device 3. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, it may also include input / output devices, network access devices, etc.

[0125] The processor 30 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor.

[0126] In some embodiments, the memory 31 may be an internal storage unit of the terminal device 3, such as a hard disk or memory of the terminal device 3. In other embodiments, the memory 31 may be an external storage device of the terminal device 3, such as a plug-in hard disk, smart media card (SMC), secure digital card (SD), flash card, etc., equipped on the terminal device 3. Furthermore, the memory 31 may include both internal and external storage units of the terminal device 3. The memory 31 is used to store the operating system, applications, boot loader, data, and other programs, such as the program code of the computer program. The memory 31 can also be used to temporarily store data that has been output or will be output.

[0127] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps described in the various method embodiments above.

[0128] This application provides a computer program product that, when run on a terminal device, enables the terminal device to implement the steps described in the various method embodiments above.

[0129] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments of this application can be implemented by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium, and when executed by a processor, it can implement the steps of the various method embodiments described above. The computer program includes computer program code, which can be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium can include at least: any entity or device capable of carrying computer program code to a device / terminal equipment, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium. Examples include USB flash drives, portable hard drives, magnetic disks, or optical disks. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electrical carrier signals or telecommunication signals.

[0130] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail or recorded in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0131] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0132] In the embodiments provided in this application, it should be understood that the disclosed devices / terminal equipment and methods can be implemented in other ways. For example, the device / terminal equipment embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling or direct coupling or communication connection may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0133] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0134] The above-described embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be included within the protection scope of this application.

Claims

1. A point cloud target detection method, characterized in that, include: Based on the preset voxel size and the coordinates of each point in the three-dimensional point cloud data to be processed in the three-dimensional space, calculate the voxel index corresponding to each point in the three-dimensional point cloud data, wherein the three-dimensional space is the space to which the three-dimensional point cloud data belongs; Points with the same voxel index are grouped into the same voxel, and each voxel corresponds to a voxel index; For each voxel, a representation vector of the voxel is generated based on the reference position in the voxel and the relative positions of the points contained in the voxel with respect to the reference position. Extract the feature information of each of the multiple voxels based on their respective representation vectors; Detect the target object in the point cloud data based on the feature information of each of the multiple voxels; The step of extracting feature information of each of the multiple voxels based on their respective representation vectors includes: For each voxel, the representation vector of the voxel is sparsely convolved to obtain the first intermediate feature of the voxel; the first intermediate feature of the voxel is normalized to obtain the second intermediate feature of the voxel; the second intermediate feature of the voxel is input into a preset activation function to obtain the feature information of the voxel. The sparse convolution processing steps include: Obtain an input matrix and a hash table. The input matrix is ​​used to represent the representation vector of a voxel at a non-empty voxel position in the input feature map. The input feature map includes the representation vectors of multiple voxels. The hash table is used to represent the coordinates of the non-empty voxel positions in the input feature map and the correspondence between the coordinates and the rows in the input matrix. The rule engine is determined based on the hash table; each row in the rule engine includes a first position in the output feature map that is not empty, corresponding to a second position in the input feature map, and the coordinates of the second position in the hash table; For each row in the rule engine, the first intermediate feature of the voxel at each non-empty voxel position in the output feature map is calculated.

2. The point cloud target detection method as described in claim 1, characterized in that, For each voxel, generating a representation vector for the voxel based on a reference position within the voxel and the relative positions of points contained within the voxel with respect to the reference position includes: Determine whether the number of points contained in the voxel is greater than or equal to a preset number; If the number of points contained in the voxel is greater than or equal to a preset number, then the preset number of target points are obtained from the points contained in the voxel. Calculate the relative position of each target point with respect to the reference position; The representation vector of the voxel is generated based on the reference position and the relative position of each target point.

3. The point cloud target detection method as described in claim 2, characterized in that, The calculation of the relative position of each target point with respect to the reference position includes: For each target point, calculate the coordinate difference between the coordinates of the target point in the three-dimensional space and the coordinates of the reference position, where the three-dimensional space is the space to which the three-dimensional point cloud data belongs; The coordinate difference corresponding to the target point is determined as the relative position of the target point with respect to the reference position.

4. The point cloud target detection method as described in claim 2, characterized in that, After determining whether the number of points contained in the voxel is greater than or equal to a preset number, the method further includes: If the number of points contained in the voxel is less than the preset number, then the representation vector of the voxel is generated based on the reference position in the voxel and the relative position of each point in the voxel relative to the reference position. The representation vector is padded so that the dimension of the padded representation vector reaches a preset dimension, wherein the preset dimension is determined by the preset number of elements.

5. The point cloud target detection method according to any one of claims 1 to 4, characterized in that, For each voxel, the reference position in the voxel is the position corresponding to the center point of the voxel.

6. A point cloud target detection device, characterized in that, include: A voxel partitioning unit is used to calculate the voxel index corresponding to each point in the three-dimensional point cloud data according to the preset voxel size and the coordinates of each point in the three-dimensional point cloud data to be processed in the three-dimensional space, wherein the three-dimensional space is the space to which the three-dimensional point cloud data belongs; points with the same voxel index are partitioned into the same voxel, and each voxel corresponds to one voxel index; A vector generation unit is used to generate a representation vector for each voxel based on a reference position in the voxel and the relative positions of points contained in the voxel with respect to the reference position. The feature extraction unit is used to extract the feature information of each of the multiple voxels based on their respective representation vectors; The target detection unit is used to detect target objects in the point cloud data based on the feature information of each of the plurality of voxels; The feature extraction unit is further configured to: for each voxel, perform sparse convolution processing on the voxel's representation vector to obtain the voxel's first intermediate feature; normalize the voxel's first intermediate feature to obtain the voxel's second intermediate feature; and input the voxel's second intermediate feature into a preset activation function to obtain the voxel's feature information. The feature extraction unit is further used for: Obtain an input matrix and a hash table. The input matrix is ​​used to represent the representation vector of a voxel at a non-empty voxel position in the input feature map. The input feature map includes the representation vectors of multiple voxels. The hash table is used to represent the coordinates of the non-empty voxel positions in the input feature map and the correspondence between the coordinates and the rows in the input matrix. The rule engine is determined based on the hash table; each row in the rule engine includes a first position in the output feature map that is not empty, corresponding to a second position in the input feature map, and the coordinates of the second position in the hash table; For each row in the rule engine, the first intermediate feature of the voxel at each non-empty voxel position in the output feature map is calculated.

7. A terminal device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 5.

Images