Object detection method based on 3D point cloud and deformable multi-head attention network

Abstract

The present invention proposes a three-dimensional point cloud target detection method based on a deformable multi-head attention network, aiming to provide an efficient, accurate and reliable solution for robot target detection. First, the three-dimensional point cloud data of the target is acquired through a sensor, and the point cloud is expanded using a surface description method; then, a scale point cloud is generated using an improved coccygeal network, and information at different scales is integrated through a feature pyramid network to improve the accuracy and efficiency of target detection; finally, a deformable multi-head attention network is used to identify the target position of the original point cloud data, and the network includes an encoder module and a decoder module. The present invention can achieve high-performance target detection in industrial scenarios and tasks.

Description

Technical Field

[0001] The present invention belongs to the technical field of three-dimensional target detection, and in particular relates to a feature extraction and target detection method for industrial robots in an industrial environment. Background Art

[0002] The upgrade of the manufacturing industry is essentially the upgrade of core technology, production methods, and application scenarios, all of which are inseparable from the supply and empowerment of advanced technologies, especially industrial vision including smart sensors. Machine vision is the eye of intelligent manufacturing in the era of Industry 4.0. Machine vision technology promotes the transformation of traditional production to automation and intelligence. In the industrial production process, many production processes are manually operated, which has problems such as low efficiency and high cost. Industrial robots have the characteristics of high work efficiency and work quality, high repeatability and positioning accuracy, and it is a development trend for industrial robots to replace manual labor. The integrated application of machine vision and robots will meet the needs of the transformation and development of the intelligent manufacturing industry to the greatest extent, and greatly improve product quality and the degree of automation of production lines.

[0003] 3D Object Detection is one of the important technologies in environmental perception systems. Deep Learning, especially Convolutional Neural Networks (CNN), makes accurate object detection possible, and promotes 3D object detection to play an important role in autonomous driving, robotics and other fields. Its performance has also been significantly improved with the rapid development of deep learning.

[0004] With the popularization of depth sensors and 3D laser scanners, 3D target detection based on point clouds has received widespread attention. The current mainstream 3D data representation methods mainly include depth maps, triangular meshes, voxels, and point clouds. Among them, point clouds are the simplest 3D data representation method, with the advantages of simple acquisition, easy storage, strong visibility, and fine structure description. In addition, point clouds can be easily converted to other data formats such as depth maps and voxels. They have become the most basic data format in research fields such as 3D reconstruction, 3D target detection, and Simultaneous Localization and Mapping (SLAM). Summary of the invention

[0005] The purpose of this invention is to address the shortcomings of existing methods and propose a target detection method based on three-dimensional point cloud and deformable multi-head attention network.

[0006] In the first aspect, a target detection method based on three-dimensional point cloud and deformable multi-head attention network comprises the following steps:

[0007] Step S1: Acquire the three-dimensional point cloud data of the target through the sensor N represents the number of points x, and F represents the number of features of each point x; wherein the three-dimensional point cloud data includes multiple features, and the features include target color and target position;

[0008] Step S2: Use the surface description method to expand the above point cloud to obtain the first scale point cloud

[0009] Step S3, using a deformable multi-head attention network to identify the target position of the original point cloud data X0;

[0010] The deformable multi-head attention network includes an enhanced coccygeal network embedded with self-attention, an encoder module, and a decoder module.

[0011] In a second aspect, the present invention provides a target detection system, comprising:

[0012] Data acquisition module, used to obtain the three-dimensional point cloud data X0 of the target;

[0013] The data extension module uses the surface description method to expand the above point cloud to obtain the first scale point cloud X1;

[0014] The target location recognition module uses a deformable multi-head attention network to recognize the target location of the original point cloud data X0.

[0015] In a third aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to execute the method described.

[0016] In a fourth aspect, the present invention provides a computing device, comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the described method is implemented.

[0017] The beneficial effects of the present invention are:

[0018] The present invention first performs a first scale expansion on the multi-information point cloud, which can enhance the spatial resolution of the point cloud data, so that the target object can be more accurately identified and located during the point cloud data processing process; at the same time, the original data is processed using a multi-layer perceptron embedded with self-attention. The multi-layer perceptron embedded with self-attention is a neural network structure based on the self-attention mechanism, which can effectively capture the long-distance dependencies and local features in the input point cloud. Then the two scale point clouds are fused using a feature pyramid. The feature pyramid is a multi-scale processing technology that can effectively integrate information at different scales, thereby realizing the detection and recognition of target objects at different point cloud feature scales. During the fusion process, a multi-scale convolutional neural network will be used to process multi-scale point cloud information and perform feature fusion on multi-scale point clouds. This will help retain important features at each scale and improve the accuracy of target detection. Finally, the features are applied to multi-scale target detection. Specifically, the scale features in the feature pyramid will be input into the Transformer encoder to capture global context information. Then, the Transformer decoder will generate predicted target bounding boxes and categories. In summary, by fusing two scale point clouds using feature pyramids, combining multi-scale point cloud generation, multi-scale point cloud feature pyramid fusion, multi-scale target detection framework, multi-scale loss function and other technologies, the present invention aims to provide an efficient, accurate and reliable solution for robot target detection.

[0019] In summary, the present invention proposes a target detection method based on three-dimensional point cloud and deformable multi-head attention network, which uses multi-information point cloud input, feature extraction based on backbone network and object refinement prediction method based on multi-head attention network in its implementation process. The vector input of multi-information point cloud uses geometric feature embedding method to expand the features of point cloud set and increase the local information abstraction ability of point cloud. The backbone network takes the position information of point cloud as input, uses neural network to extract features, and combines feature description optimization and feature point sampling screening to perform prediction from point cloud to the target to be detected. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a diagram of the architecture of the deformable multi-head attention network of the present invention;

[0021] Figure 2 Flow chart of the method of the present invention. DETAILED DESCRIPTION

[0022] The three-dimensional object detection method provided by the present invention will be described in detail below.

[0023] The three-dimensional target detection method provided by the present invention is specifically used in the robot scene of Hangzhou Xio Elevator Company. Its main goal is to perform three-dimensional target detection on the robotic arm in the assembly line, so that three-dimensional calibration and position detection can be performed based on the detection frame.

[0024] The present invention uses the 3D point cloud sensor of Hangzhou Xio Elevator Company to process the continuously captured 3D point cloud and continuously output the frame of the target detection. Here, one frame of the captured image is taken as an example to process the 3D point cloud of a single frame.

[0025] A target detection method based on 3D point cloud and deformable multi-head attention network, such as Figure 2 The steps include:

[0026] Step S1: Acquire the three-dimensional point cloud data of the target through the sensor N represents the number of points x, and F represents the number of features of each point x; wherein the three-dimensional point cloud data includes multiple features, and the features include target color and target position;

[0027] Step S2: Use the surface description method to expand the above point cloud to obtain the first scale point cloud Specifically:

[0028] 2-1 Find the neighbors of all point clouds:

[0029] For each point x i ∈X0, find its K nearest neighbors, denoted as neighbors i ; These neighbor sets are represented as

[0030] 2-2 Calculate the edges of all point clouds:

[0031] Point x i All neighbors of are sorted in clockwise direction to get edges i ; These edge sets are represented as

[0032] 2-3 Calculate the neighbor vector pairs of all point clouds:

[0033] For each point x i All neighbors of , calculate the vector pairs between adjacent neighbors i ; These vector sets are represented as

[0034] 2-4 Calculate the centroids of all neighbor vector pairs:

[0035] Calculate the centroids of each pair of neighbor vectors i ; These centroid sets are represented as

[0036] 2-5 Calculate normals;

[0037] Calculate the cross product of each pair of neighbor vectors and normalize them to get normals i ; These normal sets are represented as

[0038] 2-6 Calculate the position of each local surface and construct a feature vector;

[0039] Calculate the position of each local surface using the normal and the centroid i , these position sets are represented as

[0040] The centroid, normal and position are then connected to form a feature vector features i , these feature vector sets are represented as

[0041] 2-7 Input the feature vector features into the multi-layer perceptron (MLP) to obtain the output features F represents the feature dimension; at the same time, for each point x i Features of mlp features Perform the maximum pooling operation to generate the final feature representation of point cloud X0 The final feature representation is X features Connect with X0 to get the expanded first scale point cloud

[0042] Step S3: Use Figure 1 The deformable multi-head attention network recognizes the target position of the original point cloud data X0; specifically:

[0043] 3-1 Using an enhanced coccyx network embedded with self-attention for feature extraction; the enhanced coccyx network embedded with self-attention includes a sampling layer, a multi-layer perceptron embedded with self-attention, a feature pyramid network, and a multi-scale convolutional neural network;

[0044] The sampling layer screens the original point cloud data X0 by using a point neighbor sampling method; the point neighbor sampling method adopts k-nearest neighbor point sampling (KPS); specifically, the nearest k neighbors of each point in the point cloud X0 are sampled to obtain k candidate objects X′0={x1,…,x k};

[0045] The multi-layer perceptron embedded with self-attention is used to extract features from the data filtered by the sampling layer to obtain a scale point cloud X2 after feature embedding, each layer of which includes a hidden layer, a layer attention module, and a fully connected layer cascaded in sequence;

[0046] The hidden layer applies multiple linear transformations (i.e., fully connected layers) and activation functions (such as ReLU) to process the output X′0 of the sampling layer, which will produce a new feature matrix Where H is the feature dimension of the hidden layer. Assuming that the multilayer perceptron has L layers, the feature matrix H generation process is expressed as follows:

[0047] H (l) =σ(W (l) *H (l-1 )+b (l) ), l = 1, 2, ..., L (1)

[0048] Where L is the number of linear transformation-activation function combinations contained in the hidden layer, and W (l) and b (l) is the weight matrix and bias vector of the lth hidden layer, σ(·) is the ReLU activation function;

[0049] The layer attention module is specifically:

[0050] Use the self-attention mechanism to calculate the attention weights between each pair of points in the input point cloud; for each point x i ∈X′0 and its k nearest neighbors, the element α in the attention weight is calculated using the following formula ij ;

[0051]

[0052] in, represents the query matrix (using point cloud X′0), K ij Represents point x i With neighbors x j The relationship between k represents the dimension of the attention head, and softmax(·) represents the normalized exponential function, which mainly scales the elements in the matrix to the interval [0, 1];

[0053] Each element α in the attention weight matrix ij Aggregation, represented by

[0054] The attention weight matrix A is applied to the hidden layer feature matrix H to calculate the attention weighted feature M; it is expressed as follows:

[0055]

[0056] The fully connected layer uses linear transformation and activation function (such as ReLU) to generate the output feature matrix Where O is the output feature dimension; it is defined as follows:

[0057] Y=σ(W out *M+b out ) (4)

[0058] where σ(·) is the ReLU activation function, W out and b out is the weight matrix and bias vector of the fully connected layer; for the obtained Y, it is defined as the new scale point cloud set

[0059] The feature pyramid network is a feature pyramid based on a convolutional neural network, and its main function is to perform feature fusion on the multi-scale point cloud set {X1, X2} obtained in steps S2 and S3 to generate a single-scale feature F. The process is defined as follows:

[0060] F′1=Conv 1×1 (X1)

[0061] F′2=Conv 1×1 (UpSample(X2))

[0062] F fused =F′1+F′2

[0063] F=Conv 3×3 (F fused ) (5)

[0064] Among them, Conv 1×1 (·) represents a convolutional layer with a convolution kernel size of 1×1. 3×3 (·) represents a convolutional layer with a convolution kernel size of 3×3, and UpSample(·) represents an upsampling network using nearest neighbor interpolation, which mainly interpolates the smaller matrix X2 and scales it to the size of X1;

[0065] The multi-scale convolutional neural network mainly convolves a single-scale feature F and enriches the feature into a feature F′. The process is defined as follows:

[0066] ΔP=Conv offset (F)

[0067] F sampled =F+ΔP

[0068] F′Conv deformable(F sampled , W cd ) (6)

[0069] Among them, Conv offset (·) is a convolutional neural network using maximum pooling to learn the offset on feature F, Conv deformable (·) is a variable convolutional layer, W cd is the convolution kernel of the variable convolution layer;

[0070] 3-2 Use the encoder module to process the multi-scale point cloud features F′ output by the enhanced coccyx network embedded with self-attention to obtain the feature matrix

[0071] The encoder module includes a position encoder and a Transformer attention-based encoder;

[0072] The position encoder is a module that encodes the position information of each feature point in the feature F′ into the feature representation. First, the encoding matrix PE is calculated by the following formula matrix ;

[0073] PE matrix =PositionEncoding(F′) (7)

[0074] Where PositionEncoding(·) is the position encoding function;

[0075] Secondly, the encoding matrix PE matrix Add it back to the feature matrix F′ to get the feature matrix F containing position information PE ′;

[0076] The Transformer attention-based encoder is used to transform the feature matrix F PE ′ is encoded to generate a matrix Z0 containing feature encoding; it includes a linear layer and a Transformer encoder;

[0077] The linear layer takes as input the feature matrix F eE ′ Linear projection, resulting in a matrix Z0 of dimension [N, D], where D is the hidden layer dimension of the encoder;

[0078] Z0=Linear(F PE ′) (8)

[0079] Among them, Linear(·) represents the linear layer (using the fully connected layer), and Z0 is the feature matrix after linear projection;

[0080] The Transformer encoder typically includes multiple encoder layers, each layer including a multi-head self-attention module and a feed-forward neural network module;

[0081] Assume that the encoder contains L T Layer; for each layer l of the encoder T , do the following;

[0082]

[0083]

[0084] Among them, MultiHeadSelfAttention(·) represents the multi-head self-attention function, and LayerNorm(·) represents the layer normalization operation;

[0085] For a feedforward neural network, the main operation is as follows;

[0086]

[0087]

[0088] Among them, FeedForward(·) represents the feedforward neural network module;

[0089] Applying these operations to all encoder layers, we finally get the output of the last layer of the encoder, that is, the feature matrix of the final output It is the feature matrix F PE 'The result after Transformer encoding;

[0090] 3-3 Use the decoder module to finally decode the input point cloud X0 query, concatenate the target prediction results, and finally output the bounding box and category of the target task;

[0091] The decoder module includes a Transformer attention-based decoder;

[0092] The multi-layer cascaded Transformer attention-based decoder; specifically:

[0093] First, initialize the point cloud X0, X in The dimension is [N×F], which is defined as the initial input query matrix Q0;

[0094] Similar to the encoder, the Transformer decoder contains multiple decoder layers, each of which contains a multi-head self-attention module, a multi-head cross-attention module, and a feed-forward neural network module; a total of L T decoder layers;

[0095] For each layer l′ of the decoder T , first execute the multi-head self-attention module, the specific operation is defined as follows;

[0096]

[0097]

[0098] Among them, MultiHeadSelfAttention(·) represents the multi-head self-attention function, and LayerNorm(·) represents the layer normalization operation;

[0099] Next, the multi-head cross-attention module is executed. The specific operation is defined as follows:

[0100]

[0101]

[0102] Among them, MultiHeadCrossAttention(·) represents the multi-head cross attention module, which receives the output features from the encoder As key and value;

[0103] Finally, the feedforward neural network module is executed, and the specific operation is defined as follows;

[0104]

[0105]

[0106] Among them, FeedForward(·) represents the feedforward neural network module; these operations are applied to all decoder layers, and finally the output feature matrix of the decoder is obtained Its dimension is [T, D];

[0107] The target prediction result uses two parallel linear layers; one linear layer outputs the prediction result of the target task, and the other linear layer outputs the category prediction and bounding box prediction;

[0108] The target task prediction output ClassPrediction is defined as follows;

[0109]

[0110] Among them, LinearClass(·) represents the linear layer output by the target task;

[0111] The bounding box and category prediction output BoxPrediction are defined as follows;

[0112] BoxPrediction=LinearBox(M L ) (15)

[0113] Among them, LinearBox(·) represents the linear layer of target bounding box and category prediction;

[0114] The loss function of the deformable multi-head attention network Contains encoder loss Decoder loss The definition is as follows;

[0115]

[0116] For the above encoder and decoder losses, the total loss includes L loss stages; a single stage l loss The loss function consists of a single-stage loss of the encoder and the loss of a single-stage decoder The decoder loses and the encoder loss is the average of the losses of all decoder stages and is defined as follows;

[0117]

[0118] The first loss The encoder loss of the encoder stage It consists of four parts, namely binary focal loss Objectness prediction with smooth L1 loss Closed center offset prediction with cross entropy loss Size classification and smooth L1 loss Closed size offset prediction; single-stage loss of the encoder The definitions are as follows;

[0119]

[0120] Among them, γ1, γ2, γ3, and γ4 represent adjustable hyperparameters;

[0121] The first loss The loss of the decoder stage It contains five parts, namely binary focus loss Objectness prediction with cross entropy loss Box classification with smooth L1 loss Closed center offset prediction with cross entropy loss Size classification and smooth L1 loss Offset prediction; Single-stage loss of the decoder The definitions are as follows;

[0122]

[0123] Among them, β1, β2, β3, β4, and β5 represent adjustable hyperparameters.

Claims

1. A target detection method based on 3D point cloud and deformable multi-head attention network, characterized in that The steps include: Step S1: Acquire the three-dimensional point cloud data of the target through the sensor N represents the number of points x, and F represents the number of features of each point x; wherein the three-dimensional point cloud data includes multiple features, and the features include target color and target position; Step S2: Use the surface description method to expand the above point cloud to obtain the first scale point cloud Specifically: 2-1 Find the neighbors of all point clouds: For each point x i ∈X0, find its K nearest neighbors, denoted as neighbors i ; These neighbor sets are represented as 2-2 Calculate the edges of all point clouds: Point x i All neighbors of are sorted in clockwise direction to get edges i ; These edge sets are represented as 2-3 Calculate the neighbor vector pairs of all point clouds: For each point x i All neighbors of , calculate the vector pairs between adjacent neighbors i ; These vector sets are represented as 2-4 Calculate the centroids of all neighbor vector pairs: Calculate the centroids of each pair of neighbor vectors i ; These centroid sets are represented as 2-5 Calculate normals; Calculate the cross product of each pair of neighbor vectors and normalize them to get normals i ; The set of these normals is represented as 2-6 Calculate the position of each local surface and construct a feature vector; Calculate the position of each local surface using the normal and the centroid i , these position sets are represented as The centroid, normal and position are then connected to form a feature vector features i , these feature vector sets are represented as 2-7 Input the feature vector features into the multi-layer perceptron MLP to obtain the output features F represents the feature dimension; at the same time, for each point x i Features of mlp features Perform the maximum pooling operation to generate the final feature representation of point cloud X0 The final feature representation is X features Connect with X0 to get the expanded first scale point cloud Step S3: Use a deformable multi-head attention network to identify the target position of the original point cloud data X0; specifically: 3-1 Using an enhanced coccyx network embedded with self-attention for feature extraction; the enhanced coccyx network embedded with self-attention includes a sampling layer, a multi-layer perceptron embedded with self-attention, a feature pyramid network, and a multi-scale convolutional neural network; The multi-layer perceptron embedded with self-attention is used to extract features from the data filtered by the sampling layer to obtain a scale point cloud X2 after feature embedding, each layer of which includes a hidden layer, a layer attention module, and a fully connected layer cascaded in sequence; The feature pyramid network is a feature pyramid based on a convolutional neural network, which is used to perform feature fusion on the multi-scale point cloud set X1 expanded in step S2 and the multi-scale point cloud set X2 output by the multi-layer perceptron embedded with self-attention, and generate a single-scale feature F; The multi-scale convolutional neural network convolves a single-scale feature F and enriches the feature into a feature F'; 3-2 Use the encoder module to process the multi-scale point cloud features F' output by the enhanced coccyx network embedded with self-attention to obtain the feature matrix 3-3 Use the decoder module to finally decode the input point cloud X0 query, concatenate the target prediction results, and finally output the bounding box and category of the target task.

2. The method according to claim 1, characterized in that In step 3-1, the hidden layer applies multiple linear transformations and activation functions to process the output X'0 of the sampling layer, which will produce a new feature matrix Where H is the feature dimension of the hidden layer; assuming that the multilayer perceptron has L layers, the feature matrix H generation process is expressed as follows; H (l) =σ(W (l) *H (l-1 )+b (l) ),l=1,2,...,L (1) Where L is the number of linear transformation-activation function combinations contained in the hidden layer, and W (l) and b (l) is the weight matrix and bias vector of the lth hidden layer, σ(·) is the ReLU activation function; The layer attention module is specifically: Use the self-attention mechanism to calculate the attention weights between each pair of points in the input point cloud; for each point x i ∈X'0 and its k nearest neighbors, the element α in the attention weight is calculated using the following formula ij ; in, represents the query matrix, K ij Represents point x i With neighbors x j The relationship between k represents the dimension of the attention head, and softmax(·) represents the normalized exponential function, which scales the elements in the matrix to the interval [0,1]; Each element α in the attention weight matrix ij Aggregation, represented by The attention weight matrix A is applied to the hidden layer feature matrix H to calculate the attention weighted feature m; it is expressed as follows: The fully connected layer uses linear transformation and activation function to generate the output feature matrix Where o is the output feature dimension; it is defined as follows: Y=σ(W out *M+b out ) (4) where σ(·) is the ReLU activation function, W out and b out is the weight matrix and bias vector of the fully connected layer; For the obtained Y, it is defined as a new scale point cloud set 3. The method according to claim 2, characterized in that The feature pyramid network described in step 3-1 is specifically: F′1=Conv 1×1 (X1) F′2=Conv 1×1 (UpSample(X2)) F fused =F′1+F′2 F=Conv 3×3 (F fused ) (5) Among them, Conv 1×1 (·) represents a convolutional layer with a convolution kernel size of 1×1. 3×3 (·) represents a convolutional layer with a convolution kernel size of 3×3, and UpSample(·) represents an upsampling network using nearest neighbor interpolation, which interpolates the smaller matrix X2 and scales it to the size of X1.

4. The method according to claim 3, characterized in that The feature pyramid network described in step 3-1 is specifically: ΔP=Conv offset (F) F sampled =F+ΔP F′=Conv deformable (F sampled ,W cd ) (6) Among them, Conv offset (·) is a convolutional neural network using maximum pooling to learn the offset on feature F, Conv deformable (·) is a variable convolutional layer, W cd is the convolution kernel of the variable convolution layer.

5. The method according to claim 4, characterized in that The encoder module described in step 3-2 includes a position encoder and a Transformer attention-based encoder; The position encoder is a module that encodes the position information of each feature point in the feature F′ into the feature representation. First, the encoding matrix PE is calculated by the following formula matrix ; PE matrix =PositionEncoding(F′) (7) Where PositionEncoding(·) is the position encoding function; Secondly, the encoding matrix PE matrix Add it back to the feature matrix F′ to get the feature matrix F containing position information PE ′; The Transformer attention-based encoder is used to transform the feature matrix F PE ′ is encoded to generate a matrix Z0 containing feature encoding; it includes a linear layer and a Transformer encoder; The linear layer takes as input the feature matrix F PE ′ Linear projection, resulting in a matrix Z0 of dimension [N, D], where D is the hidden layer dimension of the encoder; Z0=Linear(F PE ′) (8) Among them, Linear(·) represents the linear layer, and Z0 is the feature matrix after linear projection; The Transformer encoder typically includes multiple encoder layers, each layer including a multi-head self-attention module and a feedforward neural network module; Assume that the encoder contains L T Layer; for each layer l of the encoder T , do the following; Among them, MultiHeadSelfAttention(·) represents the multi-head self-attention function, and LayerNorm(·) represents the layer normalization operation; For a feedforward neural network, the operation is as follows: Among them, FeedForward(·) represents the feedforward neural network module; Applying these operations to all encoder layers finally results in the output of the last layer of the encoder, i.e. the feature matrix of the final output It is the feature matrix F PE 'The result after Transformer encoding.

6. The method according to claim 5, characterized in that The decoder module described in step 3-2 includes a Transformer attention-based decoder; The Transformer attention-based decoder; Specifically: First, initialize the point cloud X0, X in The dimension is [N×F], which is defined as the initial input query matrix Q0; The Transformer decoder contains multiple decoder layers, each of which contains a multi-head self-attention module, a multi-head cross-attention module, and a feedforward neural network module; a total of L T decoder layers; For each layer l' of the decoder T , first execute the multi-head self-attention module, the specific operation is defined as follows; Among them, MultiHeadSelfAttention(·) represents the multi-head self-attention function, and LayerNorm(·) represents the layer normalization operation; Next, execute the multi-head cross-attention module. The specific operation is defined as follows: Among them, MultiHeadCrossAttention(·) represents the multi-head cross attention module, which receives the output features from the encoder as keys and values; Finally, the feedforward neural network module is executed. The specific operation is defined as follows: Among them, FeedForward(·) represents the feedforward neural network module; these operations are applied to all decoder layers, and finally the output feature matrix of the decoder is obtained Its dimension is [T,D]; The target prediction result uses two parallel linear layers; one linear layer outputs the prediction result of the target task, and the other linear layer outputs the category prediction and bounding box prediction; The target task prediction output ClassPrediction is defined as follows: ClassPrediction=LinearClass(Q LT ) (14) Among them, LinearClass(·) represents the linear layer output by the target task; The bounding box and category prediction output BoxPrediction is defined as follows: BoxPrediction=LinearBox(M L ) (15) Among them, LinearBox(·) represents the linear layer of object bounding box and category prediction.

7. The method according to claim 1, characterized in that The loss function of the deformable multi-head attention network Contains encoder loss Decoder loss The definition is as follows; For the above encoder and decoder losses, the total loss includes L loss stages; a single stage l loss The loss function includes a single-stage loss of the encoder and the loss of a single-stage decoder The decoder loses and the encoder loss is the average of the losses of all decoder stages and is defined as follows; No.1 loss The encoder loss of the encoder stage It consists of four parts, namely binary focal loss Objectness prediction with smooth L1 loss Closed center offset prediction with cross entropy loss Size classification and smooth L1 loss Size offset prediction turned off; The single-stage loss of the encoder The definitions are as follows; Among them, γ1, γ2, γ3, and γ4 represent adjustable hyperparameters; The first loss The loss of the decoder stage It contains five parts, namely binary focus loss Objectness prediction with cross entropy loss Box classification with smooth L1 loss Closed center offset prediction with cross entropy loss Size classification and smooth L1 loss Size offset prediction turned off; The single-stage loss of the decoder The definitions are as follows; Among them, β1, β2, β3, β4, and β5 represent adjustable hyperparameters.

8. A target detection system implementing the method according to any one of claims 1 to 7, characterized in that include: Data acquisition module, used to obtain the three-dimensional point cloud data X0 of the target; The data extension module uses the surface description method to expand the above point cloud to obtain the first scale point cloud X1; The target location recognition module uses a deformable multi-head attention network to recognize the target location of the original point cloud data X0.

9. A computing device, comprising a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, the method according to any one of claims 1 to 7 is implemented.

Images