Target detection methods, devices, electronic equipment and storage media

Abstract

This application discloses a target detection method, apparatus, electronic device, and storage medium, comprising: acquiring point cloud data; executing an iterative process: dividing the point cloud data into multiple point cloud windows through a constructed window mechanism, the window mechanism determining the point cloud window based on key points and their neighboring points in the point cloud data; within each point cloud window, determining the high-dimensional features of each point cloud based on a local attention mechanism; in the iterative process of determining the high-dimensional features of each point cloud based on the local attention mechanism, when the iteration process is the first iteration process, the corresponding input feature is the original point cloud feature of each point cloud; when the iteration process is other iteration processes, the corresponding input feature is the high-dimensional feature of each point cloud determined in the previous iteration process; if a preset iteration stop condition is met, the iteration stops, and target detection is performed based on the high-dimensional features of each point cloud in the point cloud data at the time of iteration stop. This application aims to improve the accuracy of target detection.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a target detection method, apparatus, electronic device, and storage medium. Background Technology

[0002] Perception is the foundation and core of autonomous driving systems. 2D perception is far from sufficient for high-level autonomous driving, and accurate 3D perception is particularly important.

[0003] Current 3D object detection methods often employ 3D sparse convolution or multilayer perceptrons to extract features from point cloud data. Both of these feature extraction processes utilize convolution and downsampling operations. However, for autonomous driving scenarios, the features corresponding to non-motorized vehicles and pedestrians in the feature map obtained after downsampling only occupy a small portion in the BEV (bird's-eye view), resulting in low detection accuracy for targets (non-motorized vehicles and pedestrians). Summary of the Invention

[0004] In view of this, this application provides a target detection method, apparatus, electronic device and storage medium, aiming to improve the target detection accuracy and solve the technical problem of low target detection accuracy in existing 3D target detection methods.

[0005] This application provides a target detection method, including the following steps:

[0006] Acquire point cloud data;

[0007] The following iterative process is executed: the point cloud data is divided into multiple point cloud windows through the constructed window mechanism. The window mechanism is based on the key points and their neighboring points in the point cloud data to determine the point cloud windows; within each point cloud window, the high-dimensional features of each point cloud are determined based on the local attention mechanism.

[0008] In the iterative process of determining the high-dimensional features of each point cloud based on the local attention mechanism, when the iterative process is the first iterative process, the corresponding input feature is the original point cloud feature of each point cloud. When the iterative process is any iterative process other than the first iterative process, the corresponding input feature is the high-dimensional feature of each point cloud determined by the previous iterative process.

[0009] If the preset iteration stop condition is met, the iteration stops, and target detection is performed based on the high-dimensional features of each point cloud in the point cloud data at the time of iteration stop.

[0010] In one possible implementation of this application, the step of dividing the point cloud data into multiple point cloud windows through a constructed window mechanism includes:

[0011] The point cloud data is sampled using the FPS algorithm to obtain N key points;

[0012] The point cloud data is divided into N windows by finding and grouping the neighboring points around each key point using the K-nearest neighbor algorithm.

[0013] In one possible implementation of this application, the number of point clouds in the point cloud window obtained by the j-th iteration is different from the number of point clouds in the point cloud window obtained by the (j-1)-th iteration, where j is a positive integer greater than or equal to 2.

[0014] In one possible implementation of this application, the step of determining the high-dimensional features of each point cloud based on a local attention mechanism within each point cloud window includes:

[0015] Within each point cloud window, a multilayer perceptron is used to perform feature mapping on the input features of each point cloud in the point cloud window to obtain the corresponding query matrix, key matrix, and value matrix.

[0016] Based on the query matrix, the key matrix, and the value matrix, the high-dimensional features of each point cloud are determined.

[0017] In one possible implementation of this application, the step of determining the high-dimensional features of each point cloud based on the query matrix, the key matrix, and the value matrix includes:

[0018] Multiply the query matrix by the key matrix to obtain the mutual attention matrix;

[0019] Based on the attention coefficients in the mutual attention matrix, the value matrix is ​​weighted to determine the high-dimensional features of each point cloud.

[0020] In one possible implementation of this application, the step of performing target detection based on the high-dimensional features of each point cloud in the point cloud data at the time of stopping iteration includes:

[0021] When the iteration stops, the high-dimensional features of each point cloud in the point cloud data are projected onto the bird's-eye view.

[0022] The bird's-eye view is used to extract features from the convolutional neural network to generate a 3D detection box;

[0023] The target detection result is obtained by predicting the category and scale of the 3D detection box using a convolutional neural network.

[0024] In one possible implementation of this application, the step of extracting features from the bird's-eye view using a convolutional neural network to generate a 3D detection box is as follows:

[0025] The features of the input bird's-eye view image are sequentially convolved to obtain bird's-eye view image features at multiple scales;

[0026] Deconvolution operations are performed on the bird's-eye view image features at multiple scales to obtain multiple bird's-eye view image features at the same scale.

[0027] Multiple bird's-eye view image features of the same scale are concatenated to obtain concatenated bird's-eye view image features, and 3D detection boxes are generated.

[0028] This application also provides a target detection device, the target detection device comprising:

[0029] The data acquisition module is used to acquire point cloud data;

[0030] The window partitioning module is used to execute the following iterative process: the point cloud data is partitioned through the constructed window mechanism to obtain multiple point cloud windows. The window mechanism is based on the key points in the point cloud data and their neighboring points to determine the point cloud windows.

[0031] The feature determination module is used to perform the following iterative process: within each point cloud window, based on a local attention mechanism, determine the high-dimensional features of each point cloud;

[0032] In the iterative process of determining the high-dimensional features of each point cloud based on the local attention mechanism, when the iterative process is the first iterative process, the corresponding input feature is the original point cloud feature of each point cloud. When the iterative process is any iterative process other than the first iterative process, the corresponding input feature is the high-dimensional feature of each point cloud determined by the previous iterative process.

[0033] The target detection module is used to stop iteration if a preset iteration stop condition is met, and to perform target detection based on the high-dimensional features of each point cloud in the point cloud data at the time of stopping iteration.

[0034] In one possible implementation of this application, the window partitioning module is specifically used for:

[0035] The point cloud data is sampled using the FPS algorithm to obtain N key points;

[0036] The point cloud data is divided into N windows by finding and grouping the neighboring points around each key point using the K-nearest neighbor algorithm.

[0037] And / or the number of point clouds in the point cloud window obtained by the window partitioning module when executing the j-th iteration process is different from the number of point clouds in the point cloud window obtained when executing the (j-1)-th iteration process, where j is a positive integer greater than or equal to 2;

[0038] And / or the feature determination module, including:

[0039] The feature mapping unit is used to perform feature mapping on the input features of each point cloud in each point cloud window using a multilayer perceptron to obtain the corresponding query matrix, key matrix and value matrix.

[0040] The feature determination unit is used to determine the high-dimensional features of each point cloud based on the query matrix, the key matrix, and the value matrix.

[0041] And / or the feature determination unit, specifically used for:

[0042] Multiply the query matrix by the key matrix to obtain the mutual attention matrix;

[0043] Based on the attention coefficients in the mutual attention matrix, the value matrix is ​​weighted to determine the high-dimensional features of each point cloud.

[0044] And / or the target detection module, comprising:

[0045] The projection unit is used to project the high-dimensional features of each point cloud in the point cloud data onto the bird's-eye view when the iteration stops.

[0046] The detection box generation unit is used to extract features from the bird's-eye view through a convolutional neural network and generate 3D detection boxes;

[0047] The detection result acquisition unit is used to predict the category and scale of the 3D detection box through a convolutional neural network to obtain the target detection result;

[0048] And / or the detection box generation unit, specifically used for:

[0049] The features of the input bird's-eye view image are sequentially convolved to obtain bird's-eye view image features at multiple scales;

[0050] Deconvolution operations are performed on the bird's-eye view image features at multiple scales to obtain multiple bird's-eye view image features at the same scale.

[0051] Multiple bird's-eye view image features of the same scale are concatenated to obtain concatenated bird's-eye view image features, and 3D detection boxes are generated.

[0052] This application also provides an electronic device, the device comprising: a memory, a processor, and a target detection program stored in the memory and executable on the processor, the target detection program being configured to implement the steps of the target detection method as described above.

[0053] This application also provides a storage medium storing a target detection program, which, when executed by a processor, implements the steps of the target detection method described above.

[0054] This application provides a target detection method, apparatus, electronic device, and storage medium. Compared to existing 3D target detection methods that use convolution and downsampling operations during feature extraction, resulting in features corresponding to non-motorized vehicles and pedestrians occupying only a small portion of the extracted feature map from a BEV perspective and leading to low target (non-motorized vehicle and pedestrian) detection accuracy, this application acquires point cloud data and executes the following iterative process: The point cloud data is divided into multiple point cloud windows using a constructed window mechanism. This window mechanism determines the point cloud based on key points and their neighboring points. Within each point cloud window, high-dimensional features of each point cloud are determined based on a local attention mechanism. In the iterative process of determining the high-dimensional features of each point cloud based on the local attention mechanism, when the iteration process is the first iteration, the corresponding input feature is the original point cloud feature of each point cloud. When the iteration process is any iteration other than the first iteration, the corresponding input feature is the high-dimensional feature of each point cloud determined in the previous iteration. If a preset iteration stop condition is met, the iteration stops, and target detection is performed based on the high-dimensional features of each point cloud in the point cloud data at the time of iteration stop. Therefore, in this application, feature extraction of point cloud data is no longer performed through convolution and downsampling operations. Instead, an iterative process is executed: first, the point cloud data is divided into multiple point cloud windows using a constructed window mechanism; then, feature extraction is performed on the point cloud data within each point cloud window based on a local attention mechanism; finally, target detection is performed based on the high-dimensional features of each point cloud obtained when the iteration stops. Without downsampling, the feature of the point cloud data is guaranteed to be preserved to the greatest extent possible during the extraction process, thereby improving the detection accuracy of 3D targets (non-motorized vehicles and pedestrians). Attached Figure Description

[0055] Figure 1 This is a flowchart illustrating the first embodiment of the target detection method of this application;

[0056] Figure 2 This is a schematic diagram illustrating the window division for this application.

[0057] Figure 3 This is a schematic diagram illustrating the process of determining the high-dimensional features of each point cloud based on the local attention mechanism in this application;

[0058] Figure 4 A schematic diagram of the network structure for generating 3D detection boxes for this application;

[0059] Figure 5This is a schematic diagram of the electronic device structure of the hardware operating environment involved in the embodiments of this application.

[0060] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0061] It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0062] This application provides a target detection method, such as... Figure 1 As shown, in the first embodiment of the target detection method of this application, the method includes the following steps:

[0063] Step S10: Obtain point cloud data;

[0064] Step S20: Execute the following iterative process: Divide the point cloud data into multiple point cloud windows using the constructed window mechanism. The window mechanism is based on key points and their neighboring points in the point cloud data to determine the point cloud windows. Within each point cloud window, determine the high-dimensional features of each point cloud based on a local attention mechanism.

[0065] In the iterative process of determining the high-dimensional features of each point cloud based on the local attention mechanism, when the iterative process is the first iterative process, the corresponding input feature is the original point cloud feature of each point cloud. When the iterative process is any iterative process other than the first iterative process, the corresponding input feature is the high-dimensional feature of each point cloud determined by the previous iterative process.

[0066] Step S30: If the preset iteration stop condition is met, the iteration is stopped, and target detection is performed based on the high-dimensional features of each point cloud in the point cloud data at the time of iteration stop.

[0067] This embodiment aims to improve the accuracy of target detection.

[0068] Specifically, in this application, feature extraction of point cloud data is no longer performed through convolution and downsampling operations. Instead, an iterative process is executed: first, the point cloud data is divided into multiple point cloud windows using a constructed window mechanism; then, feature extraction is performed on the point cloud data within each point cloud window based on a local attention mechanism; finally, target detection is performed based on the high-dimensional features of each point cloud obtained when the iteration stops. Without downsampling, the feature of the point cloud data is ensured to the greatest extent possible without being lost during the extraction process, thereby improving the detection accuracy of targets (non-motorized vehicles and pedestrians).

[0069] Specifically, in this application, the point cloud data is first sampled using the FPS algorithm to obtain N key points. Then, the K-nearest neighbor algorithm is used to find and group the neighboring points around each key point, thereby dividing the point cloud data into N windows. This allows the determination of the high-dimensional features of each point cloud based on the local attention mechanism to be performed within the window, reducing the computational complexity and thus improving the efficiency of target detection.

[0070] Specifically, in this application, the number of point clouds in the point cloud window obtained by the j-th iteration in the iterative process is different from the number of point clouds in the point cloud window obtained by the (j-1)-th iteration. By changing the size of the point cloud window in each iteration, the connectivity between the point cloud window and its surrounding point cloud windows is increased, thereby ensuring the connectivity between local point cloud data.

[0071] Specifically, in this application, a multilayer perceptron is used to perform feature mapping on the input features of each point cloud in the point cloud window to obtain the corresponding query matrix, key matrix and value matrix. Then, the query matrix and key matrix are multiplied to obtain the mutual attention matrix. Finally, based on the attention coefficients in the mutual attention matrix, the value matrix is ​​weighted to determine the high-dimensional features of each point cloud, thereby enhancing the high-dimensional features of each point cloud and improving the detection accuracy of targets (non-motorized vehicles and pedestrians).

[0072] Specifically, in this application, when the iteration stops, the high-dimensional features of each point cloud in the point cloud data are projected onto the bird's-eye view. Then, a convolutional neural network is used to extract features from the bird's-eye view to generate a 3D detection box. This 3D detection box can identify targets (non-motorized vehicles, pedestrians) in the bird's-eye view. The convolutional neural network is then used to predict the category and scale of the targets identified by the 3D detection box to obtain the target detection result. While detecting the category of the target, the spatial location, size, orientation and other three-dimensional information of the target can be estimated.

[0073] The specific steps are as follows:

[0074] Step S10: Obtain point cloud data.

[0075] It should be noted that the target detection method provided in this application embodiment can be applied to scenarios including a target detection device and a LiDAR connected together. The target detection device can be located in a server. The server can be a single server or a server cluster consisting of multiple servers. Specifically, the target detection device can be a processor within the server. The LiDAR is used to collect point cloud data, and then the target detection device obtains the collected point cloud data from the LiDAR.

[0076] As an example, a scenario involving a connected target detection device and a LiDAR can be used for autonomous driving. In this scenario, the LiDAR is mounted on the vehicle, and the target detection device can be located in the vehicle's controller.

[0077] Step S20: Execute the following iterative process: Divide the point cloud data into multiple point cloud windows using the constructed window mechanism. The window mechanism is based on key points and their neighboring points in the point cloud data to determine the point cloud windows. Within each point cloud window, determine the high-dimensional features of each point cloud based on a local attention mechanism.

[0078] In the iterative process of determining the high-dimensional features of each point cloud based on the local attention mechanism, when the iterative process is the first iteration, the corresponding input feature is the original point cloud feature of each point cloud. When the iterative process is any iteration other than the first iteration, the corresponding input feature is the high-dimensional feature of each point cloud determined by the previous iteration.

[0079] As an example, in each iteration, the step of dividing the point cloud data into multiple point cloud windows using the constructed window mechanism includes:

[0080] Step A1: Sample the point cloud data using the FPS algorithm to obtain N key points;

[0081] Step A2: Use the K-nearest neighbor algorithm to find and group the neighboring points around each key point, and divide the point cloud data into N windows.

[0082] Specifically, the point cloud data is first sampled using the FPS algorithm to obtain N keypoints. Then, the K-nearest neighbor algorithm is used to find M-1 neighboring points around each keypoint, grouping each keypoint and its M-1 neighboring points into a window. In other words, the point cloud data is divided into N windows, with M point clouds in each window. Figure 2 As shown, Figure 2 The dashed circles in the diagram represent a point cloud window. The usage of the FPS and K-nearest neighbor algorithms can be found in relevant technical documents and will not be elaborated upon here.

[0083] As an example, in the iterative process, the number of point clouds in the point cloud window obtained by the j-th iteration is different from the number of point clouds in the point cloud window obtained by the (j-1)-th iteration, where j is a positive integer greater than or equal to 2.

[0084] As an example, a pre-set data pool is used, where all values ​​are distinct. During each iteration, a value is selected from this data pool to determine the number of point cloud windows obtained after partitioning the point cloud data in the current iteration. It's important to note that the value selected from the data pool in each iteration is a value that was not selected in previous iterations. The values ​​in the data pool are either unrelated, exhibiting either an arithmetic or geometric progression.

[0085] As an example, such as Figure 3 As shown, in each iteration process, the step of determining the high-dimensional features of each point cloud based on a local attention mechanism within each point cloud window includes:

[0086] Step B1: Within each point cloud window, use a multilayer perceptron to perform feature mapping on the input features of each point cloud in the point cloud window to obtain the corresponding query matrix, key matrix, and value matrix.

[0087] In the first iteration of the iteration process, the input feature for each point cloud in the point cloud window is the original point cloud feature of each point cloud. The original point cloud feature includes the X coordinate, Y coordinate, Z coordinate, and reflectance of each point cloud.

[0088] Wherein, when the iteration process is any iteration process other than the first iteration process, the input feature of each point cloud in the point cloud window is the high-dimensional feature of each point cloud determined by the previous iteration process.

[0089] As an example, in step B1, when the multilayer perceptron performs feature mapping on the input features of each point cloud in the point cloud window to obtain the corresponding query matrix, key matrix, and value matrix, the connection weight coefficient matrix corresponding to the query matrix, the connection weight coefficient matrix corresponding to the key matrix, and the connection weight coefficient matrix corresponding to the value matrix in each intermediate layer of the multilayer perceptron are all different from each other; and the connection weight coefficient matrices corresponding to the query matrix, the connection weight coefficient matrix corresponding to the key matrix, and the connection weight coefficient matrix corresponding to the value matrix in the multilayer perceptron are also different from each other between intermediate layers.

[0090] As an example, the query matrix Q, key matrix K, and value matrix V in step B1 can be represented as Q, K, V ∈ R r ×c Where r represents the number of points in the point cloud data, and c represents the dimension. c can be either 128 or 256.

[0091] Step B2: Based on the query matrix, the key matrix, and the value matrix, determine the high-dimensional features of each point cloud.

[0092] As an example, the step of determining the high-dimensional features of each point cloud based on the query matrix, the key matrix, and the value matrix includes:

[0093] Step B21: Multiply the query matrix by the key matrix to obtain the mutual attention matrix.

[0094] Multiplying the query matrix with the key matrix involves multiplying the transpose of the query matrix with the transpose of the key matrix to obtain the mutual attention matrix.

[0095] Step B22: Based on the attention coefficients in the mutual attention matrix, perform weighted calculations on the value matrix to determine the high-dimensional features of each point cloud.

[0096] As an example, the high-dimensional features of each point cloud are determined by weighting the value matrix based on the attention coefficients in the mutual attention matrix, using the following calculation formula:

[0097] F = softmax(A)·V;

[0098] c

[0099] Where F is the high-dimensional feature corresponding to the point cloud window, including the high-dimensional feature of each point cloud in the point cloud window; A is the attention matrix; V is the value matrix; c is the dimension (in this embodiment, it can be 128 or 256); softmax is the normalization exponential function in the field of deep learning.

[0100] Step S30: If the preset iteration stop condition is met, the iteration is stopped, and target detection is performed based on the high-dimensional features of each point cloud in the point cloud data at the time of iteration stop.

[0101] As an example, the preset iteration stopping condition is: iteration stops when the number of iterations meets a threshold. This threshold can be set according to the actual application and is not specifically limited in this embodiment.

[0102] As an example, the step of performing object detection based on the high-dimensional features of each point cloud in the point cloud data at the time of stopping iteration includes:

[0103] Step C1: When the iteration stops, project the high-dimensional features of each point cloud in the point cloud data onto the bird's-eye view.

[0104] Specifically, based on the X and Y coordinates of each point cloud, the position coordinates (X, Y) of the high-dimensional features of each point cloud in the bird's-eye view are determined. The high-dimensional features of each point cloud are then placed in the bird's-eye view according to the aforementioned position coordinates, thus completing the operation of projecting the high-dimensional features of each point cloud in the point cloud data onto the bird's-eye view.

[0105] Step C2: Extract features from the bird's-eye view using a convolutional neural network to generate a 3D detection box.

[0106] As an example, the step of extracting features from the bird's-eye view using a convolutional neural network to generate a 3D detection box includes:

[0107] Step C21: Perform convolution operations on the input bird's-eye view image features sequentially to obtain bird's-eye view image features at multiple scales;

[0108] Step C22: Perform deconvolution operations on the bird's-eye view image features at multiple scales to obtain multiple bird's-eye view image features at the same scale;

[0109] Step C23: Concatenate the features of multiple bird's-eye view images of the same scale to obtain concatenated bird's-eye view image features and generate 3D detection boxes.

[0110] Specifically, such as Figure 4 As shown, the convolutional neural network performs three 3×3 convolution operations (Conv) on the input bird's-eye view image features (represented as (C, W, H)) in sequence, with a convolution stride of 2 for each operation, to obtain bird's-eye view image features at three different scales, represented as (C, W / 2, H / 2), (2C, W / 4, H / 4), and (4C, W / 8, H / 8), respectively. Then, it performs deconvolution operations (deconv) on the bird's-eye view image features at the above three scales to restore them to the same scale. Finally, the three bird's-eye view image features at the same scale are concatenated (concat) to obtain concatenated bird's-eye view image features and generate 3D detection boxes.

[0111] Step C3: Predict the category and scale of the 3D detection box using a convolutional neural network to obtain the target detection result.

[0112] Specifically, 1x1 convolutions can be used to extract features from 3D detection boxes to predict the category and scale of the 3D detection boxes, thereby obtaining the target detection results.

[0113] This application provides a target detection method, apparatus, electronic device, and storage medium. Compared to existing 3D target detection methods that use convolution and downsampling operations during feature extraction, resulting in features corresponding to non-motorized vehicles and pedestrians occupying only a small portion of the extracted feature map from a BEV perspective and leading to low target (non-motorized vehicle and pedestrian) detection accuracy, this application acquires point cloud data and executes the following iterative process: The point cloud data is divided into multiple point cloud windows using a constructed window mechanism. This window mechanism determines the point cloud based on key points and their neighboring points. Within each point cloud window, high-dimensional features of each point cloud are determined based on a local attention mechanism. In the iterative process of determining the high-dimensional features of each point cloud based on the local attention mechanism, when the iteration process is the first iteration, the corresponding input feature is the original point cloud feature of each point cloud. When the iteration process is any iteration other than the first iteration, the corresponding input feature is the high-dimensional feature of each point cloud determined in the previous iteration. If a preset iteration stop condition is met, the iteration stops, and target detection is performed based on the high-dimensional features of each point cloud in the point cloud data at the time of iteration stop. Therefore, in this application, feature extraction of point cloud data is no longer performed through convolution and downsampling operations. Instead, an iterative process is executed: first, the point cloud data is divided into multiple point cloud windows using a constructed window mechanism; then, feature extraction is performed on the point cloud data within each point cloud window based on a local attention mechanism; finally, target detection is performed based on the high-dimensional features of each point cloud obtained when the iteration stops. Without downsampling, the feature of the point cloud data is guaranteed to be preserved to the greatest extent possible during the extraction process, thereby improving the detection accuracy of targets (non-motorized vehicles and pedestrians).

[0114] Reference Figure 5 , Figure 5 This is a schematic diagram of the electronic device structure of the hardware operating environment involved in the embodiments of this application.

[0115] like Figure 5As shown, the electronic device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0116] Those skilled in the art will understand that Figure 5 The structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0117] like Figure 5 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a data storage module, a network communication module, a user interface module, and a target detection program.

[0118] exist Figure 5 In the electronic device shown, the network interface 1004 is mainly used for data communication with other devices; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the electronic device of this application can be set in the electronic device, and the electronic device calls the target detection program stored in the memory 1005 through the processor 1001 to implement the steps of the target detection method described above.

[0119] The specific implementation of the electronic device in this application is basically the same as the embodiments of the target detection method described above, and will not be repeated here.

[0120] This application also provides a target detection device, including:

[0121] The data acquisition module is used to acquire point cloud data;

[0122] The window partitioning module is used to execute the following iterative process: the point cloud data is partitioned through the constructed window mechanism to obtain multiple point cloud windows. The window mechanism is based on the key points in the point cloud data and their neighboring points to determine the point cloud windows.

[0123] The feature determination module is used to perform the following iterative process: within each point cloud window, based on a local attention mechanism, determine the high-dimensional features of each point cloud;

[0124] In the iterative process of determining the high-dimensional features of each point cloud based on the local attention mechanism, when the iterative process is the first iterative process, the corresponding input feature is the original point cloud feature of each point cloud. When the iterative process is any iterative process other than the first iterative process, the corresponding input feature is the high-dimensional feature of each point cloud determined by the previous iterative process.

[0125] The target detection module is used to stop iteration if a preset iteration stop condition is met, and to perform target detection based on the high-dimensional features of each point cloud in the point cloud data at the time of stopping iteration.

[0126] In one possible implementation of this application, the window partitioning module is specifically used for:

[0127] The point cloud data is sampled using the FPS algorithm to obtain N key points;

[0128] The point cloud data is divided into N windows by finding and grouping the neighboring points around each key point using the K-nearest neighbor algorithm.

[0129] And / or the number of point clouds in the point cloud window obtained by the window partitioning module when executing the j-th iteration process is different from the number of point clouds in the point cloud window obtained when executing the (j-1)-th iteration process, where j is a positive integer greater than or equal to 2;

[0130] And / or the feature determination module, including:

[0131] The feature mapping unit is used to perform feature mapping on the input features of each point cloud in each point cloud window using a multilayer perceptron to obtain the corresponding query matrix, key matrix and value matrix.

[0132] The feature determination unit is used to determine the high-dimensional features of each point cloud based on the query matrix, the key matrix, and the value matrix.

[0133] And / or the feature determination unit, specifically used for:

[0134] Multiply the query matrix by the key matrix to obtain the mutual attention matrix;

[0135] Based on the attention coefficients in the mutual attention matrix, the value matrix is ​​weighted to determine the high-dimensional features of each point cloud.

[0136] And / or the target detection module, comprising:

[0137] The projection unit is used to project the high-dimensional features of each point cloud in the point cloud data onto the bird's-eye view when the iteration stops.

[0138] The detection box generation unit is used to extract features from the bird's-eye view through a convolutional neural network and generate 3D detection boxes;

[0139] The detection result acquisition unit is used to predict the category and scale of the 3D detection box through a convolutional neural network to obtain the target detection result;

[0140] And / or the detection box generation unit, specifically used for:

[0141] The features of the input bird's-eye view image are sequentially convolved to obtain bird's-eye view image features at multiple scales;

[0142] Deconvolution operations are performed on the bird's-eye view image features at multiple scales to obtain multiple bird's-eye view image features at the same scale.

[0143] Multiple bird's-eye view image features of the same scale are concatenated to obtain concatenated bird's-eye view image features, and 3D detection boxes are generated.

[0144] The specific implementation of the target detection device in this application is basically the same as the various embodiments of the target detection method described above, and will not be repeated here.

[0145] This application provides a storage medium that stores one or more programs, which can be executed by one or more processors to implement the steps of the target detection method described in any of the above claims.

[0146] The specific implementation of the storage medium in this application is basically the same as the embodiments of the target detection method described above, and will not be repeated here.

[0147] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the target detection method described above.

[0148] The specific implementation of the computer program product of this application is basically the same as the various embodiments of the target detection method described above, and will not be repeated here.

[0149] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0150] The sequence numbers of the embodiments in this application are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0151] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0152] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A target detection method, characterized in that, The target detection method includes the following steps: Acquire point cloud data; The following iterative process is executed: the point cloud data is divided into multiple point cloud windows through the constructed window mechanism. The window mechanism is based on the key points and their neighboring points in the point cloud data to determine the point cloud windows; within each point cloud window, the high-dimensional features of each point cloud are determined based on the local attention mechanism. In the iterative process of determining the high-dimensional features of each point cloud based on the local attention mechanism, when the iterative process is the first iteration, the corresponding input feature is the original point cloud feature of each point cloud. When the iterative process is any iteration other than the first iteration, the corresponding input feature is the high-dimensional feature of each point cloud determined by the previous iteration. In the iterative process, the number of point clouds in the point cloud window obtained by the j-th iteration is different from the number of point clouds in the point cloud window obtained by the (j-1)-th iteration, where j is a positive integer greater than or equal to 2. In each iteration, the number of point cloud windows obtained after dividing the point cloud data is selected from a preset data pool. The values ​​in the data pool are all different, and the values ​​selected in the current iteration are values ​​that have not been selected in previous iterations. If the preset iteration stop condition is met, the iteration stops, and target detection is performed based on the high-dimensional features of each point cloud in the point cloud data at the time of iteration stop.

2. The target detection method as described in claim 1, characterized in that, The step of dividing point cloud data into multiple point cloud windows through the constructed window mechanism includes: The point cloud data is sampled using the FPS algorithm to obtain N key points, where N is an integer greater than one. The point cloud data is divided into N windows by finding and grouping the neighboring points around each key point using the K-nearest neighbor algorithm.

3. The target detection method as described in claim 1, characterized in that, The step of determining the high-dimensional features of each point cloud based on a local attention mechanism within each point cloud window includes: Within each point cloud window, a multilayer perceptron is used to perform feature mapping on the input features of each point cloud in the point cloud window to obtain the corresponding query matrix, key matrix, and value matrix. Based on the query matrix, the key matrix, and the value matrix, the high-dimensional features of each point cloud are determined.

4. The target detection method as described in claim 3, characterized in that, The step of determining the high-dimensional features of each point cloud based on the query matrix, the key matrix, and the value matrix includes: Multiply the query matrix by the key matrix to obtain the mutual attention matrix; Based on the attention coefficients in the mutual attention matrix, the value matrix is ​​weighted to determine the high-dimensional features of each point cloud.

5. The target detection method as described in claim 1, characterized in that, The step of performing target detection based on the high-dimensional features of each point cloud in the point cloud data at the time of stopping iteration includes: When the iteration stops, the high-dimensional features of each point cloud in the point cloud data are projected onto the bird's-eye view. Feature extraction from the bird's-eye view is performed using a convolutional neural network to generate 3D detection boxes; The target detection result is obtained by predicting the category and scale of the 3D detection box using a convolutional neural network.

6. The target detection method as described in claim 5, characterized in that, The step of extracting features from the bird's-eye view using a convolutional neural network to generate a 3D detection box: The features of the input bird's-eye view image are sequentially convolved to obtain bird's-eye view image features at multiple scales; Deconvolution operations are performed on the bird's-eye view image features at multiple scales to obtain multiple bird's-eye view image features at the same scale. Multiple bird's-eye view image features of the same scale are concatenated to obtain concatenated bird's-eye view image features, and 3D detection boxes are generated.

7. A target detection device, characterized in that, The target detection device includes: The data acquisition module is used to acquire point cloud data; The window partitioning module is used to execute the following iterative process: the point cloud data is partitioned through the constructed window mechanism to obtain multiple point cloud windows. The window mechanism is based on the key points in the point cloud data and their neighboring points to determine the point cloud windows. The feature determination module is used to perform the following iterative process: within each point cloud window, based on a local attention mechanism, determine the high-dimensional features of each point cloud; In the iterative process of determining the high-dimensional features of each point cloud based on the local attention mechanism, when the iterative process is the first iteration, the corresponding input feature is the original point cloud feature of each point cloud. When the iterative process is any iteration other than the first iteration, the corresponding input feature is the high-dimensional feature of each point cloud determined by the previous iteration. In the iterative process, the number of point clouds in the point cloud window obtained by the j-th iteration is different from the number of point clouds in the point cloud window obtained by the (j-1)-th iteration, where j is a positive integer greater than or equal to 2. In each iteration, the number of point cloud windows obtained after dividing the point cloud data is selected from a preset data pool. The values ​​in the data pool are all different, and the values ​​selected in the current iteration are values ​​that have not been selected in previous iterations. The target detection module is used to stop iteration if a preset iteration stop condition is met, and to perform target detection based on the high-dimensional features of each point cloud in the point cloud data at the time of stopping iteration.

8. The target detection device as described in claim 7, characterized in that, The window partitioning module is specifically used for: The point cloud data is sampled using the FPS algorithm to obtain N key points, where N is an integer greater than one. The point cloud data is divided into N windows by finding and grouping the neighboring points around each key point using the K-nearest neighbor algorithm. And / or the feature determination module, including: The feature mapping unit is used to perform feature mapping on the input features of each point cloud in each point cloud window using a multilayer perceptron to obtain the corresponding query matrix, key matrix and value matrix. The feature determination unit is used to determine the high-dimensional features of each point cloud based on the query matrix, the key matrix, and the value matrix. And / or the feature determination unit, specifically used for: Multiply the query matrix by the key matrix to obtain the mutual attention matrix; Based on the attention coefficients in the mutual attention matrix, the value matrix is ​​weighted to determine the high-dimensional features of each point cloud. And / or the target detection module, comprising: The projection unit is used to project the high-dimensional features of each point cloud in the point cloud data onto the bird's-eye view when the iteration stops. The detection box generation unit is used to extract features from the bird's-eye view through a convolutional neural network and generate 3D detection boxes; The detection result acquisition unit is used to predict the category and scale of the 3D detection box through a convolutional neural network to obtain the target detection result; And / or the detection box generation unit, specifically used for: The features of the input bird's-eye view image are sequentially convolved to obtain bird's-eye view image features at multiple scales; Deconvolution operations are performed on the bird's-eye view image features at multiple scales to obtain multiple bird's-eye view image features at the same scale. Multiple bird's-eye view image features of the same scale are concatenated to obtain concatenated bird's-eye view image features, and 3D detection boxes are generated.

9. An electronic device, characterized in that, The device includes: a memory, a processor, and a target detection program stored in the memory and executable on the processor, the target detection program being configured to implement the steps of the target detection method as described in any one of claims 1 to 6.

10. A storage medium, characterized in that, The storage medium stores a target detection program, which, when executed by a processor, implements the steps of the target detection method as described in any one of claims 1 to 6.

Images