Point cloud detection method, electronic equipment and computer readable storage medium

Abstract

The invention discloses a point cloud detection method, electronic equipment and a computer readable storage medium. The method comprises the following steps: acquiring a first point cloud data feature at a current moment; obtaining an initial query feature group; wherein the initial query feature group comprises at least one initial query feature; fusing context information in the first point cloud data features into the initial query feature group by using a cross attention mechanism to obtain a first fused query feature group; performing self-attention processing on the first fusion query feature group to obtain a target query feature group corresponding to the current moment; and acquiring a target query feature group at the previous moment as a historical query feature group, and performing target detection by using the target query feature group and the historical query feature group to obtain a target detection result at the current moment. In this way, the target detection speed can be increased.

Description

Technical Field

[0001] This application relates to the technical field of point cloud detection, and particularly to a point cloud detection method, an electronic device, and a computer-readable storage medium. Background Art

[0002] LiDAR is one of the important sensors on autonomous vehicles, and accurate three-dimensional spatial point cloud data can be directly obtained through LiDAR.

[0003] Currently, according to the different numbers of point cloud inputs, point cloud detection algorithms can be divided into single-frame point cloud detection algorithms and temporal point cloud detection algorithms. Generally, the temporal point cloud model uses BEV features for fusion, and BEV features are usually dense, occupying a large amount of memory, which greatly wastes memory resources, further limiting the number of fused frames and affecting the target detection speed. Summary of the Invention

[0004] The main technical problem to be solved by this application is to provide a point cloud detection method, an electronic device, and a computer-readable storage medium, which can improve the speed of target detection.

[0005] To solve the above technical problem, a technical solution adopted by this application is: providing a point cloud detection method, the method includes: obtaining the first point cloud data feature at the current moment; and, obtaining an initial query feature group; wherein, the initial query feature group includes at least one initial query feature; using a cross-attention mechanism to fuse the context information in the first point cloud data feature into the initial query feature group to obtain a first fused query feature group; performing self-attention processing on the first fused query feature group to obtain a target query feature group corresponding to the current moment; obtaining the target query feature group at the previous moment as a historical query feature group, and using the target query feature group and the historical query feature group to perform target detection to obtain the target detection result at the current moment.

[0006] Wherein, using a cross-attention mechanism to fuse the context information in the first point cloud data feature into the initial query feature group to obtain a first fused query feature group includes: taking the initial query feature group as the first query, the first point cloud data feature as the first key and the first value; performing cross-attention processing using the first query, the first key and the first value to obtain a first fused query feature group; performing self-attention processing on the first fused query feature group to obtain a target query feature group corresponding to the current moment includes: obtaining a second query, a second key and a second value using the first fused query feature group; performing self-attention processing using the second query, the second key and the second value to obtain a target query feature group corresponding to the current moment.

[0007] Among them, before using the target query feature group and the historical query feature group to perform target detection and obtain the target detection result at the current moment, the point cloud detection method further includes: for each current query feature in the target query feature group, encoding the position information of the current query feature to obtain the position encoding vector of the current query feature; adding the current query feature and the corresponding position encoding vector to obtain the updated current query feature, where the updated target query feature group is used for subsequent target detection.

[0008] Among them, before using the target query feature group and the historical query feature group to perform target detection and obtain the target detection result at the current moment, the point cloud detection method further includes: aligning the historical query feature group using the pose information at the current moment, where the pose information is the pose of the target device, and the target device is provided with a detection device for collecting three-dimensional point cloud data corresponding to the first point cloud data feature; and / or, using the target query feature group and the historical query feature group to perform target detection and obtain the target detection result at the current moment, including: using the cross-attention mechanism to fuse the historical information in the historical query feature group into the target query feature group to obtain the second fused query feature group; performing self-attention processing on the second fused query feature group as the updated target query feature group; inputting the updated target query feature group into the detection head for detection to obtain the target detection result at the current moment.

[0009] Among them, aligning the historical query feature group using the pose information at the current moment includes: calculating the transformation parameter from the previous moment to the current moment using the motion matrix of the target device at the current moment and the motion matrix at the previous moment; multiplying the transformation parameter by the historical query feature group to obtain the aligned historical query feature group.

[0010] Among them, using the cross-attention mechanism to fuse the historical information in the historical query feature group into the target query feature group to obtain the second fused query feature group includes: using the target query feature group as the third query, the historical query feature group as the third key and the third value; performing cross-attention processing using the third query, the third key and the third value to obtain the second fused query feature group; performing self-attention processing on the second fused query feature group as the updated target query feature group includes: using the second fused query feature group to obtain the fourth query, the fourth key and the fourth value; performing self-attention processing using the fourth query, the fourth key and the fourth value to obtain the updated target query feature group.

[0011] Among them, obtaining the first point cloud data feature at the current moment includes: extracting features from the three-dimensional point cloud data at the current moment to obtain the second point cloud data feature; where the second point cloud data feature is a three-dimensional space feature; converting the second point cloud data feature to two-dimensional pseudo-image data to obtain the first point cloud data feature in the two-dimensional space.

[0012] Among them, feature extraction is performed on the three-dimensional point cloud data at the current moment to obtain the second point cloud data feature, including: dividing the three-dimensional point cloud data into a first number of spatial blocks according to spatial positions, and obtaining the original point features of each point data in each spatial block; wherein, the number of point data in each spatial block included in the three-dimensional point cloud data is not greater than a second number; respectively performing dimension elevation processing on the original point features of each point data to obtain the dimension-elevated point features of each point data; wherein, the dimension elevation processing includes adding enhanced point features and / or performing feature transformation; for each spatial block, performing pooling processing on the dimension-elevated point features of each point data in the spatial block to obtain the block feature of the spatial block; wherein, the second point cloud data feature includes the block features of each spatial block.

[0013] Among them, the original point feature includes the three-dimensional coordinates and reflection intensity corresponding to the point data; respectively performing dimension elevation processing on the original point features of each point data to obtain the dimension-elevated point features of each point data, including: for each spatial block, obtaining the central tendency representation value of the three-dimensional coordinates corresponding to all point data in the spatial block, and for each point data in the spatial block, using the deviation between the three-dimensional coordinates corresponding to the point data and the central tendency representation value of the three-dimensional coordinates as the enhanced point feature; and / or, for each point data in the spatial block, using the deviations between the X coordinate and Y coordinate corresponding to the point data and the X coordinate and Y coordinate of the center point of the spatial block respectively as the enhanced point feature; combining the original point feature and the enhanced point feature of the point data to obtain the combined point feature of the point data; using a conversion network to perform feature transformation on the combined point feature to obtain the dimension-elevated point feature of the point data; wherein, the conversion network includes a two-dimensional convolutional layer, a batch normalization layer, and an activation function layer.

[0014] To solve the above technical problems, another technical solution adopted by this application is: to provide an electronic device, which includes a memory and a processor, and the memory stores program instructions that can be executed to implement the above-mentioned point cloud detection method.

[0015] To solve the above technical problems, another technical solution adopted by this application is: to provide a computer-readable storage medium, which is used to store program instructions that can be executed to implement the above-mentioned point cloud detection method.

[0016] In the above technical solution, since the feature dimension of the target query feature included in the target query feature group at the current moment is smaller than that of the dense BEV feature; therefore, the target query feature occupies less memory than the dense BEV feature, significantly reducing the memory occupancy and being able to improve the speed of target detection. In addition, since target detection is performed after fusing the historical feature of the previous moment and the current feature of the current moment, the accuracy of target detection can be improved. Description of the Drawings

[0017] Figure 1 It is a schematic flowchart of an embodiment of the point cloud detection method provided by this application;

[0018] Figure 2 is Figure 1 a schematic flowchart of an embodiment of step S11 shown;

[0019] Figure 3 is Figure 2 a schematic flowchart of an embodiment of step S21 shown;

[0020] Figure 4 It is a schematic structural diagram of an embodiment of the point cloud detection device provided by this application;

[0021] Figure 5 It is a schematic structural diagram of an embodiment of the electronic device provided by this application;

[0022] Figure 6 It is a schematic structural diagram of an embodiment of the computer-readable storage medium provided by this application. Specific embodiments

[0023] The following will combine the description of the accompanying drawings of the specification to elaborate on the solutions of the embodiments of this application in detail.

[0024] In the following description, specific details such as specific system structures, interfaces, and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand this application.

[0025] The term "and / or" in this article is merely a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. In addition, the character " / " in this article generally represents an "or" relationship between the associated objects before and after. In addition, "multiple" in this article means two or more than two. In addition, the term "at least one" in this article represents any one of multiple or any combination of at least two of multiple. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set composed of A, B, and C.

[0026] Please refer to Figure 1 , Figure 1 is a schematic flowchart of an embodiment of the point cloud detection method provided by this application. It should be noted that if there are substantially the same results, this embodiment is not limited to Figure 1 the process sequence shown. As Figure 1 shown, this embodiment includes:

[0027] Step S11: Obtain an initial query feature group.

[0028] In this embodiment, an initial query feature group is obtained; wherein, the initial query feature group includes at least one initial query feature.

[0029] For example, a group of initial query features (query features) is initialized as the query in the attention module.

[0030] Step S12: Obtain the first point cloud data feature at the current moment.

[0031] In this embodiment, the first point cloud data feature at the current moment is obtained. It should be noted that the first point cloud data feature at the current moment is the BEV (Bird's Eye View) feature of the point cloud data. The BEV feature of the point cloud data is usually dense, which means that in the BEV representation, each pixel or grid cell contains information about that location. The BEV feature of the point cloud data is very useful in the fields of autonomous driving and robotics because it provides a concise and comprehensive representation of the environment.

[0032] Step S13: Use the cross-attention mechanism to fuse the context information in the first point cloud data feature into the initial query feature group to obtain the first fused query feature group.

[0033] In this embodiment, the cross-attention mechanism is used to fuse the context information in the first point cloud data feature into the initial query feature group to obtain the first fused query feature group.

[0034] For example, taking the initialization of a group of query features as the query in the attention module and the first point cloud data feature as the BEV feature as an example: Specifically:

[0035] Step 1: Use the initial query feature group as the first query, the first point cloud data feature as the first key and the first value. Specifically, a group of initialized query features is used as the query (the first query) in the attention module, and the BEV feature is used as the key (the first key) and the value (the first value).

[0036] Step 2: Perform cross-attention processing using the first query, the first key and the first value to obtain the first fused query feature group. Specifically, cross-attention is used between the query feature and the BEV feature to gather the context information in the BEV feature into the query feature. The specific formula is as follows:

[0037]

[0038] where, F q1 represents the first fused query feature, and the first fused query features corresponding to each initial query feature are combined to obtain the first fused query feature group; Q 1Represents the first query; K 1 Represents the first key; V 1 Represents the first value; d_k 1 Represents Q 1 The dimension of

[0039] Step S14: Perform self-attention processing on the first fused query feature group to obtain the target query feature group corresponding to the current moment.

[0040] In this embodiment, self-attention processing is performed on the first fused query feature group to obtain the target query feature group corresponding to the current moment. Since the feature dimension of the target query features included in the target query feature group at the current moment is smaller than that of the dense BEV features; therefore, the target query features occupy less memory than the dense BEV features, significantly reducing the memory occupancy and improving the speed of subsequent object detection. In addition, since the target query features occupy less memory, there is extra memory available for establishing a feature buffer, and the features of historical moments can be stored in the feature buffer. When it is necessary to combine the features of historical moments for object detection subsequently, the features of historical moments cached in the feature buffer can be directly utilized, avoiding the repeated inference process and improving the inference speed.

[0041] In one embodiment, performing self-attention processing on the first fused query feature group to obtain the target query feature group corresponding to the current moment specifically includes the following content:

[0042] Step 1: Use the first fused query feature group to obtain a second query, a second key, and a second value.

[0043] Step 2: Perform self-attention processing using the second query, the second key, and the second value to obtain the target query feature group corresponding to the current moment, in order to model the mutual relationship between different first fused query features using the self-attention mechanism among the first fused query features. Among them, the specific formula is as follows:

[0044]

[0045] Among them, F q2 Represents the current query feature corresponding to the initial query feature, and the current query features corresponding to each initial query feature are combined to obtain the target query feature group; Q 2 Represents the second query; K 2 Represents the second key; V 2 Represents the second value; d_k 2 Represents Q 2 The dimension of. It should be noted that in the self-attention processing, Q 2 , K 2 , V 2 Are all query features.

[0046] In one embodiment, before performing object detection using the target query feature group to obtain the object detection result at the current moment, the following steps are further included:

[0047] Step 1: For each current query feature in the target query feature group, encode the position information of the current query feature to obtain the position encoding vector of the current query feature.

[0048] Specifically, use an MLP (Multi-Layer Perceptron) to encode the position information Q_P of the current query feature into a d-dimensional position encoding vector. The specific formula is as follows:

[0049] F q_p = MLP(Q_P)

[0050] Where F q_p represents the position encoding vector of the current query feature; Q_P represents the position information of the current query feature.

[0051] Step 2: Add the current query feature and the corresponding position encoding vector to obtain the updated current query feature. Among them, the updated target query feature group includes both the position of the query feature and the context information of the BEV feature for subsequent object detection. The specific formula is as follows:

[0052] F q3 = F q2 + F q_p

[0053] Where F q3 represents the updated current query feature; F q_p represents the position encoding vector of the current query feature; F q2 represents the current query feature.

[0054] Step S15: Obtain the target query feature group at the previous moment as the historical query feature group, and use the target query feature group and the historical query feature group to perform object detection to obtain the object detection result at the current moment.

[0055] In this embodiment, obtaining the target query feature group at the previous moment as the historical query feature group and using the target query feature group and the historical query feature group to perform object detection to obtain the object detection result at the current moment. Since the feature dimension of the target query feature included in the target query feature group at the current moment is smaller than that of the dense BEV feature; therefore, the target query feature occupies less memory than the dense BEV feature, significantly reducing the memory occupancy and improving the speed of object detection. In addition, since object detection is performed after fusing the historical features at the previous moment and the current features at the current moment, the accuracy of object detection can be improved.

[0056] In one embodiment, before performing object detection using the target query feature group and the historical query feature group to obtain the object detection result at the current moment, the historical query feature group is aligned using the pose information at the current moment, where the pose information is the pose of the target device, and the target device is provided with a detection device for collecting three-dimensional point cloud data corresponding to the first point cloud data features. That is, before performing object detection using the target query feature group and the historical query feature group to obtain the object detection result at the current moment, a hard alignment is performed on the historical query feature group and the target query feature group using the pose information at the current moment.

[0057] At this time, performing the object detection using the target query feature group and the historical query feature group to obtain the object detection result at the current moment specifically includes: using the cross-attention mechanism to fuse the historical information in the historical query feature group into the target query feature group to obtain a second fused query feature group; performing self-attention processing on the second fused query feature group as the updated target query feature group; and inputting the updated target query feature group into the detection head for detection to obtain the object detection result at the current moment.

[0058] In a specific embodiment, aligning the historical query feature group using the pose information at the current moment specifically includes: calculating the transformation parameter from the previous moment to the current moment using the motion matrix of the target device at the current moment and the motion matrix at the previous moment; and multiplying the transformation parameter by the historical query feature group to obtain the aligned historical query feature group.

[0059] Specifically, calculating the transformation parameter from the previous moment to the current moment using the motion matrix of the target device at the current moment and the motion matrix at the previous moment. Among them, the transformation parameter from the previous moment to the current moment is as follows:

[0060] T relat =(T cur *T pre ) -1

[0061] Among them, T cur represents the motion matrix of the target device at the current moment; T pre represents the motion matrix of the target device at the previous moment; T relat represents the transformation parameter from the previous moment to the current moment.

[0062] Then, multiplying the transformation parameter by the historical query feature group to obtain the aligned historical query feature group. Among them, the specific formula is as follows:

[0063] F q_h2c =T relat *F q_h

[0064] Among them, F q_h2c represents the historical query feature group after alignment; F q_h represents the historical query feature group; T relat represents the transformation parameter from the previous moment to the current moment.

[0065] In a specific embodiment, the cross-attention mechanism is used to fuse the historical information in the historical query feature group into the target query feature group to obtain the second fused query feature group, which specifically includes the following content:

[0066] Step 1: Use the target query feature group as the third query, the historical query feature group as the third key, and the third value.

[0067] Step 2: Perform cross-attention processing using the third query, the third key, and the third value to obtain the second fused query feature group. Specifically, use the cross-attention mechanism to gather the target query features at the current moment to the feature information at the previous moment. Among them, the specific formula is as follows:

[0068]

[0069] Among them, F q4 represents the second fused query feature, and each combination of the second fused query features forms the second fused query feature group; Q 3 represents the third query; K 3 represents the third key; V 3 represents the third value; d_k 3 represents the dimension of Q 3 .

[0070] In a specific embodiment, self-attention processing is performed on the second fused query feature group as the updated target query feature group, which specifically includes the following content:

[0071] Step 1: Use the second fused query feature group to obtain the fourth query, the fourth key, and the fourth value.

[0072] Step 2: Perform self-attention processing using the fourth query, the fourth key, and the fourth value to obtain the updated target query feature group. Specifically, use the self-attention mechanism to establish connections between the historical query features. Among them, the specific formula is as follows:

[0073]

[0074] Among them, F q5 represents the updated target query feature group; Q 4 represents the fourth query; K 4 represents the fourth key; V 4 represents the fourth value; d_k4 Denote Q 3 as the dimension of. It should be noted that in self-attention processing, Q 4 , K 4 , V 4 are all query features.

[0075] That is to say, information interaction is performed on the historical query feature group and the target query feature group. Through this implicit learning method, the historical query feature group at the previous moment and the target query feature group at the current moment are further aligned, so that the target query feature group at the current moment can more fully and accurately utilize historical information to judge the target at the current moment, greatly improving the stability of target detection and improving the problems of target category jump, size and heading angle jump.

[0076] Please refer to Figure 2 , Figure 2 which is Figure 1 a schematic flowchart of an embodiment of step S11 shown. It should be noted that if there are substantially the same results, this embodiment is not limited to Figure 2 the flowchart order shown. As Figure 2 shown, this embodiment includes:

[0077] Step S21: Extract features from the three-dimensional point cloud data at the current moment to obtain the second point cloud data feature.

[0078] In this embodiment, features are extracted from the three-dimensional point cloud data at the current moment to obtain the second point cloud data feature; wherein, the second point cloud data feature is a three-dimensional space feature.

[0079] In one embodiment, sensors such as lidar, camera, and radar can be used to obtain the three-dimensional point cloud data at the current moment. For example, in the field of autonomous driving, lidar is an important sensor on autonomous vehicles, and the three-dimensional spatial point cloud data at the current moment can be accurately obtained through the lidar on autonomous vehicles.

[0080] In one embodiment, as Figure 3 shown, Figure 3 which is Figure 2 a schematic flowchart of an embodiment of step S21 shown, features are extracted from the three-dimensional point cloud data at the current moment to obtain the second point cloud data feature, which specifically includes the following sub-steps:

[0081] Step S31: Divide the three-dimensional point cloud data into a first number of spatial blocks according to spatial positions, and obtain the original point features of each point data in each spatial block.

[0082] In this embodiment, the three-dimensional point cloud data is divided into a first number of spatial blocks according to spatial positions, and the original point features of each point data in each spatial block are obtained; wherein, the number of point data in each spatial block included in the three-dimensional point cloud data is not greater than a second number.

[0083] Specifically, the three-dimensional point cloud data is divided into spatial blocks pillar with a length of h, a width of w, and a height of h according to spatial positions, and the number of point data in each spatial block pillar included in the three-dimensional point cloud data is not greater than the second number.

[0084] Among them, the representation of the three-dimensional point cloud data is as follows:

[0085] S = {(x, y, z)|x min ≤ x ≤ x max , y min ≤ y ≤ y max , z min ≤ z ≤ z max}

[0086] Among them, S represents the three-dimensional point cloud data; (x, y, z) represents the three-dimensional coordinates of each point data in the three-dimensional point cloud data; x min , y min , z min respectively represent the minimum values in the x direction, y direction, and z direction; x max , y max , z max respectively represent the maximum values in the x direction, y direction, and z direction.

[0087] The grid sizes of each spatial block pillar are specifically as follows:

[0088]

[0089] Among them, represents the size of the spatial block pillar in the x direction; represents the size of the spatial block pillar in the y direction; D = z max represents the size of the spatial block pillar in the z direction.

[0090] The three-dimensional point cloud data is represented as (P, the second number, 3), where P represents the number of spatial block pillars, and 3 represents the coordinate dimension of the point data in the three-dimensional point cloud data.

[0091] In one embodiment, the size of the first number is not limited and can be specifically set according to actual usage needs.

[0092] In one embodiment, the size of the second number is not limited and can be specifically set according to actual usage needs; for example, the second number is 48.

[0093] In one embodiment, the original point features of each point data in each spatial block include the three-dimensional coordinates and reflection intensity corresponding to the point data. Specifically, the original point features of each point data in each spatial block are specifically represented as: (x i , y i , z i , r i ), where (x i , y i , z i ) represents the three-dimensional coordinates of the point data, and r i represents the reflection intensity of the point data.

[0094] Step S32: Perform dimensionality elevation processing on the original point features of each point data respectively to obtain the dimensionality-elevated point features of each point data.

[0095] In this embodiment, perform dimensionality elevation processing on the original point features of each point data respectively to obtain the dimensionality-elevated point features of each point data; wherein, the dimensionality elevation processing includes adding enhanced point features and / or performing feature transformation.

[0096] In one embodiment, the dimensionality elevation processing includes adding enhanced point features and performing feature transformation; at this time, perform dimensionality elevation processing on the original point features of each point data respectively to obtain the dimensionality-elevated point features of each point data, specifically: for each spatial block, obtain the central tendency characterization value of the three-dimensional coordinate set corresponding to all point data in the spatial block, and for each point data in the spatial block, use the deviation between the three-dimensional coordinates corresponding to the point data and the central tendency characterization value of the three-dimensional coordinates as the enhanced point feature; and / or, for each point data in the spatial block, use the deviation between the X coordinate and Y coordinate corresponding to the point data and the X coordinate and Y coordinate of the center point of the spatial block respectively as the enhanced point feature; combine the original point feature and the enhanced point feature of the point data to obtain the combined point feature of the point data; use a conversion network to perform feature transformation on the combined point feature to obtain the dimensionality-elevated point feature of the point data; wherein, the conversion network includes a two-dimensional convolutional layer, a batch normalization layer, and an activation function layer.

[0097] Among them, the central tendency characterization value of the three-dimensional coordinates corresponding to all point data can be the average value, median value, etc. of the three-dimensional coordinates corresponding to all point data, which is not specifically limited herein. In addition, the dimension of the dimensionality-elevated point feature of the point data obtained by performing feature transformation on the combined point feature using the conversion network is not specifically limited; for example, the dimension of the dimensionality-elevated point feature is 64, etc.

[0098] Specifically, for each spatial block, obtain the average value of the three-dimensional coordinates corresponding to all point data in the spatial block. For each point data in the spatial block, use the deviation between the three-dimensional coordinates corresponding to the point data and the central tendency representation value of the three-dimensional coordinates as the enhanced point feature; and use the deviations between the X coordinate and Y coordinate corresponding to the point data and the X coordinate and Y coordinate of the center point of the spatial block respectively as the enhanced point feature. The following is a detailed description based on this.

[0099] Step 1: For each spatial block, calculate the average value of the three-dimensional coordinates corresponding to all point data in the spatial block, denoted as (x v , y v , z v ).

[0100] Step 2: For each point data in each spatial block, calculate the deviation (difference) between the point data and the average value of the three-dimensional coordinates corresponding to the spatial block to which it belongs, denoted as (x c , y c , z c ), as an enhanced point feature of the point data; and for each point data in each spatial block, calculate the deviations (differences) between the X coordinate and Y coordinate corresponding to the point data and the X coordinate and Y coordinate of the center point of the spatial block to which it belongs respectively, denoted as x p , y p respectively, as another enhanced point feature of the point data.

[0101] Step 3: Combine the original point feature and the enhanced point feature of the point data to obtain the combined point feature of the point data. The combined point feature of the point data is denoted as (x i , y i , z i , r i , x c , y c , z c , x p , y p ).

[0102] Step 5: Use a two-dimensional convolutional layer (Con2d) + batch normalization layer (BN) + activation function layer (Relu) to perform feature transformation on the combined point feature of the point data to obtain the upsampled point feature of the point data.

[0103] Step S33: For each spatial block, perform pooling processing on the upsampled point features of the point data in the spatial block to obtain the block feature of the spatial block.

[0104] In this embodiment, for each spatial block, the dimensionality - raised point features of the point data in the spatial block are pooled to obtain the block feature of the spatial block; wherein, the second point - cloud data feature includes the block features of each spatial block. That is to say, each spatial block is encoded as a feature vector with a fixed dimension.

[0105] Step S22: Convert the second point - cloud data feature into two - dimensional pseudo - image data to obtain the first point - cloud data feature in the two - dimensional space.

[0106] In this embodiment, the second point - cloud data feature is converted into two - dimensional pseudo - image data to obtain the first point - cloud data feature in the two - dimensional space. That is to say, the second point - cloud data feature is converted into a two - dimensional BEV feature, thereby simplifying the complexity of the problem and improving the calculation efficiency.

[0107] Please refer to Figure 4 , Figure 4 FIG. is a schematic structural diagram of an embodiment of a point - cloud detection device provided by the present application. The point - cloud detection device 40 includes a first acquisition module 41, a second acquisition module 42, a fusion module 43, a processing module 44, and a detection module 45; the first acquisition module 41 is used to acquire the first point - cloud data feature at the current moment; and the second acquisition module 42 is used to acquire an initial query feature group; wherein, the initial query feature group includes at least one initial query feature; the fusion module 43 is used to fuse the context information in the first point - cloud data feature into the initial query feature group by using the cross - attention mechanism to obtain the first fused query feature group; the processing module 44 is used to perform self - attention processing on the first fused query feature group to obtain the target query feature group corresponding to the current moment; the detection module 45 is used to acquire the target query feature group at the previous moment as the historical query feature group, and perform target detection by using the target query feature group and the historical query feature group to obtain the target detection result at the current moment.

[0108] Among them, the fusion module 43 is used to fuse the context information in the first point - cloud data feature into the initial query feature group by using the cross - attention mechanism to obtain the first fused query feature group, including: taking the initial query feature group as the first query, the first point - cloud data feature as the first key and the first value; performing cross - attention processing by using the first query, the first key and the first value to obtain the first fused query feature group; the processing module 44 is used to perform self - attention processing on the first fused query feature group to obtain the target query feature group corresponding to the current moment, including: obtaining the second query, the second key and the second value by using the first fused query feature group; performing self - attention processing by using the second query, the second key and the second value to obtain the target query feature group corresponding to the current moment.

[0109] Among them, before the processing module 44 uses the target query feature group and the historical query feature group to perform target detection and obtain the target detection result at the current moment, it further includes: for each current query feature in the target query feature group, encoding the position information of the current query feature to obtain the position encoding vector of the current query feature; adding the current query feature and the corresponding position encoding vector to obtain the updated current query feature, where the updated target query feature group is used for subsequent target detection.

[0110] Among them, before the processing module 44 uses the target query feature group and the historical query feature group to perform target detection and obtain the target detection result at the current moment, it further includes: aligning the historical query feature group by using the pose information at the current moment, where the pose information is the pose of the target device, and the target device is provided with a detection device for collecting three-dimensional point cloud data corresponding to the first point cloud data feature; and / or, the detection module 45 is used to perform target detection by using the target query feature group and the historical query feature group to obtain the target detection result at the current moment, including: using the cross-attention mechanism to fuse the historical information in the historical query feature group into the target query feature group to obtain the second fused query feature group; performing self-attention processing on the second fused query feature group as the updated target query feature group; inputting the updated target query feature group into the detection head for detection to obtain the target detection result at the current moment.

[0111] Among them, the processing module 44 is used to align the historical query feature group by using the pose information at the current moment, including: calculating the transformation parameter from the previous moment to the current moment by using the motion matrix of the target device at the current moment and the motion matrix at the previous moment; multiplying the transformation parameter by the historical query feature group to obtain the aligned historical query feature group.

[0112] Among them, the processing module 44 is used to fuse the historical information in the historical query feature group into the target query feature group by using the cross-attention mechanism to obtain the second fused query feature group, including: using the target query feature group as the third query, the historical query feature group as the third key and the third value; performing cross-attention processing by using the third query, the third key and the third value to obtain the second fused query feature group; the processing module 44 is used to perform self-attention processing on the second fused query feature group as the updated target query feature group, including: using the second fused query feature group to obtain the fourth query, the fourth key and the fourth value; performing self-attention processing by using the fourth query, the fourth key and the fourth value to obtain the updated target query feature group.

[0113] Among them, the first acquisition module 41 is configured to acquire the first point cloud data feature at the current moment, including: performing feature extraction on the three-dimensional point cloud data at the current moment to obtain the second point cloud data feature; wherein, the second point cloud data feature is a three-dimensional space feature; converting the second point cloud data feature into two-dimensional pseudo-image data to obtain the first point cloud data feature in the two-dimensional space.

[0114] Among them, the first acquisition module 41 is configured to perform feature extraction on the three-dimensional point cloud data at the current moment to obtain the second point cloud data feature, including: dividing the three-dimensional point cloud data into a first number of spatial blocks according to spatial positions, and acquiring the original point features of each point data in each spatial block; wherein, the number of point data in each spatial block in the three-dimensional point cloud data is not greater than a second number; respectively performing dimensionality increase processing on the original point features of each point data to obtain the dimensionality-increased point features of each point data; wherein, the dimensionality increase processing includes adding enhanced point features and / or performing feature conversion; for each spatial block, performing pooling processing on the dimensionality-increased point features of each point data in the spatial block to obtain the block feature of the spatial block; wherein, the second point cloud data feature includes the block features of each spatial block.

[0115] Among them, the above-mentioned original point features include the three-dimensional coordinates and reflection intensity corresponding to the point data; the first acquisition module 41 is configured to respectively perform dimensionality increase processing on the original point features of each point data to obtain the dimensionality-increased point features of each point data, including: for each spatial block, acquiring the central tendency representation value of the three-dimensional coordinates corresponding to all point data in the spatial block, and for each point data in the spatial block, using the deviation between the three-dimensional coordinates corresponding to the point data and the central tendency representation value of the three-dimensional coordinates as the enhanced point feature; and / or, for each point data in the spatial block, using the deviation between the X coordinate and Y coordinate corresponding to the point data and the X coordinate and Y coordinate of the center point of the spatial block as the enhanced point feature; combining the original point feature and the enhanced point feature of the point data to obtain the combined point feature of the point data; using a conversion network to perform feature conversion on the combined point feature to obtain the dimensionality-increased point feature of the point data; wherein, the conversion network includes a two-dimensional convolutional layer, a batch normalization layer, and an activation function layer.

[0116] Please refer to Figure 5 , Figure 5 FIG. is a schematic structural diagram of an embodiment of an electronic device provided in the present application. The electronic device 50 includes a memory 51 and a processor 52 that are coupled to each other. The processor 52 is configured to execute program instructions stored in the memory 51 to implement the steps of any of the above-mentioned point cloud detection method embodiments. In a specific implementation scenario, the electronic device 50 may include, but is not limited to: a microcomputer, a server. In addition, the electronic device 50 may also include mobile devices such as a laptop computer, a tablet computer, etc., which are not limited herein.

[0117] Specifically, the processor 52 is used to control itself and the memory 51 to implement the steps of any of the above point cloud detection method embodiments. The processor 52 can also be referred to as a CPU (Central Processing Unit). The processor 52 may be an integrated circuit chip with signal processing capabilities. The processor 52 can also be a general-purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. Additionally, the processor 52 can be implemented jointly by integrated circuit chips.

[0118] Please refer to Figure 6 , Figure 6 which is a schematic structural diagram of an embodiment of the computer-readable storage medium provided by the present application. The computer-readable storage medium 60 of the embodiment of the present application stores program instructions 61, and when the program instructions 61 are executed, the methods provided by any embodiment of the point cloud detection method of the present application and any non-conflicting combination are implemented. Among them, the program instructions 61 can form a program file and be stored in the above computer-readable storage medium 60 in the form of a software product, so that a computer device (which can be a personal computer, a server, or a network device, etc.) can execute all or part of the steps of the methods of various embodiments of the present application. The aforementioned computer-readable storage medium 60 includes: various media that can store program codes such as USB flash drives, mobile hard disks, read-only memories (ROM, Read-Only Memory), random access memories (RAM, Random Access Memory), magnetic disks, or optical discs, or terminal devices such as computers, servers, mobile phones, and tablets.

[0119] If the technical solution of this application involves personal information, before the product applying the technical solution of this application processes personal information, it has clearly informed the personal information processing rules and obtained the individual's independent consent. If the technical solution of this application involves sensitive personal information, before the product applying the technical solution of this application processes sensitive personal information, it has obtained the individual's separate consent and at the same time meets the requirements of "express consent". For example, at personal information collection devices such as cameras, clear and prominent signs are set to inform that the personal information collection scope has been entered and personal information will be collected. If an individual voluntarily enters the collection scope, it is regarded as consenting to the collection of their personal information; or on the device for personal information processing, when the personal information processing rules are informed by obvious signs / information, personal authorization is obtained through pop-up messages or by asking the individual to upload their personal information by themselves, etc.; among them, the personal information processing rules may include information such as the personal information processor, the purpose of personal information processing, the processing method, and the types of personal information processed.

[0120] The above are only the implementation manners of this application, and do not limit the patent scope of this application accordingly. Any equivalent structure or equivalent process transformation made by using the content of the specification and drawings of this application, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of this application.

Claims

1. A point cloud detection method, characterized in that: include: Get the first point cloud data feature at the current moment; And, obtaining an initial query feature group; wherein the initial query feature group includes at least one initial query feature; Using a cross attention mechanism, the context information in the first point cloud data feature is fused into the initial query feature group to obtain a first fused query feature group; Performing self-attention processing on the first fused query feature group to obtain a target query feature group corresponding to the current moment; The target query feature group at the previous moment is obtained as the historical query feature group, and the target query feature group and the historical query feature group are used to perform target detection to obtain the target detection result at the current moment.

2. The method according to claim 1, characterized in that The cross attention mechanism is used to fuse the context information in the first point cloud data feature into the initial query feature group to obtain a first fused query feature group, including: Using the initial query feature set as a first query and the first point cloud data feature as a first key and a first value; performing cross attention processing using the first query, the first key, and the first value to obtain the first fused query feature group; The performing self-attention processing on the first fusion query feature group to obtain the target query feature group corresponding to the current moment includes: Obtaining a second query, a second key, and a second value using the first fused query feature group; The second query, the second key and the second value are used to perform self-attention processing to obtain a target query feature group corresponding to the current moment.

3. The method according to claim 1, characterized in that Before performing target detection using the target query feature group and the historical query feature group to obtain the target detection result at the current moment, the method further includes: For each current query feature in the target query feature group, encoding the position information of the current query feature to obtain a position encoding vector of the current query feature; The current query feature and the corresponding position encoding vector are added to obtain an updated current query feature, wherein the updated target query feature group is used for subsequent target detection.

4. The method according to claim 1, characterized in that: Before performing target detection using the target query feature group and the historical query feature group to obtain the target detection result at the current moment, the method further includes: Aligning the historical query feature group using the posture information at the current moment, wherein the posture information is the posture of a target device, and the target device is provided with a detection device for collecting three-dimensional point cloud data corresponding to the first point cloud data feature; And / or, performing target detection using the target query feature group and the historical query feature group to obtain the target detection result at the current moment includes: By using a cross attention mechanism, the historical information in the historical query feature group is fused into the target query feature group to obtain a second fused query feature group; Performing self-attention processing on the second fused query feature group to obtain the updated target query feature group; The updated target query feature group is input into the detection head for detection to obtain the target detection result at the current moment.

5. The method according to claim 4, characterized in that The aligning the historical query feature group by using the posture information at the current moment includes: Calculate the transformation parameters from the previous moment to the current moment by using the motion matrix of the target device at the current moment and the motion matrix at the previous moment; The historical query feature group is multiplied by the transformation parameter to obtain the aligned historical query feature group.

6. The method according to claim 4, characterized in that The cross attention mechanism is used to fuse the historical information in the historical query feature group into the target query feature group to obtain a second fused query feature group, including: Using the target query feature group as a third query and the historical query feature group as a third key and a third value; performing cross attention processing using the third query, the third key, and the third value to obtain the second fused query feature group; The performing self-attention processing on the second fused query feature group as the updated target query feature group includes: Obtaining a fourth query, a fourth key, and a fourth value using the second fused query feature group; The fourth query, the fourth key and the fourth value are used to perform self-attention processing to obtain the updated target query feature group.

7. The method according to claim 1, characterized in that The obtaining of the first point cloud data feature at the current moment includes: Performing feature extraction on the three-dimensional point cloud data at the current moment to obtain a second point cloud data feature; wherein the second point cloud data feature is a three-dimensional space feature; The second point cloud data features are converted into two-dimensional pseudo image data to obtain the first point cloud data features in two-dimensional space.

8. The method according to claim 7, characterized in that The extracting features of the three-dimensional point cloud data at the current moment to obtain second point cloud data features includes: Dividing the three-dimensional point cloud data into a first number of spatial blocks according to spatial positions, and obtaining original point features of each point data in each of the spatial blocks; wherein the number of point data in the three-dimensional point cloud data contained in each of the spatial blocks is not greater than a second number; Performing dimension-upgrading processing on the original point features of each of the point data to obtain dimension-upgraded point features of each of the point data; wherein the dimension-upgrading processing includes adding enhanced point features and / or performing feature conversion; For each of the spatial blocks, the dimensionally increased point features of each of the point data in the spatial block are pooled to obtain the block features of the spatial block; wherein the second point cloud data features include the block features of each of the spatial blocks.

9. The method according to claim 8, characterized in that The original point features include the three-dimensional coordinates and reflection intensity corresponding to the point data; The step of performing dimension-upgrading processing on the original point features of each point data to obtain dimension-upgraded point features of each point data comprises: For each of the spatial blocks, a three-dimensional coordinate central tendency characterization value corresponding to all the point data in the spatial block is obtained, and for each of the point data in the spatial block, a deviation between the three-dimensional coordinate corresponding to the point data and the three-dimensional coordinate central tendency characterization value is used as an enhanced point feature; and / or, for each of the point data in the spatial block, taking the deviations between the X coordinate and the Y coordinate corresponding to the point data and the X coordinate and the Y coordinate of the center point of the spatial block as enhanced point features; Combining the original point feature of the point data and the enhanced point feature to obtain a combined point feature of the point data; The combined point features are transformed by using a transformation network to obtain dimension-increased point features of the point data; wherein the transformation network includes a two-dimensional convolutional layer, a batch normalization layer, and an activation function layer.

10. An electronic device, characterized in that: The electronic device includes a memory and a processor, the memory stores program instructions, and the program instructions can be executed to implement the point cloud detection method according to any one of claims 1-9.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store program instructions, and the program instructions can be executed to implement the point cloud detection method according to any one of claims 1 to 9.

Images