4D millimeter wave radar target detection method and device based on global relation perception

Abstract

The invention provides a 4D millimeter wave radar target detection method and device based on global relation perception. The method comprises the steps of obtaining original point cloud data collected by a 4D millimeter wave radar on a vehicle; performing pillar feature coding on the original point cloud data to obtain a feature map of each voxel; processing the feature maps of all voxels by using a global relationship sensing network to obtain a feature map with global information, and restoring the feature map with the global information into enhanced point cloud data; and processing the enhanced point cloud data by using a detection head to obtain a 3D target detection result. According to the invention, the accuracy and precision of 3D target detection of the point cloud data are improved, and especially, the detection effect of small targets such as pedestrians and riding people is remarkably improved.

Description

Technical Field

[0001] The present application relates to the field of autonomous driving technology, and in particular to a 4D millimeter-wave radar target detection method and device based on global relationship perception. Background Art

[0002] In order to solve the problem of insufficient detection capability when using 4D millimeter-wave radar, deep learning methods are considered to directly process point cloud data. However, millimeter-wave radar point cloud is four-dimensional data, and it has sparsity and noise problems. It is difficult for traditional three-dimensional target detection methods to ensure the accuracy of target detection. The commonly used method is to divide the point cloud data into multiple voxels and extract information from each voxel. However, this method does not pay attention to global feature information, reduces the efficiency of long-distance information capture, and loses some target detection accuracy. Summary of the invention

[0003] In view of this, the present application provides a 4D millimeter-wave radar target detection method and device based on global relationship perception to solve the above technical problems.

[0004] In a first aspect, an embodiment of the present application provides a 4D millimeter wave radar target detection method based on global relationship perception, including:

[0005] Obtain point cloud data collected by the 4D millimeter wave radar on the vehicle;

[0006] Performing pillar feature encoding on the point cloud data to obtain a feature map of each voxel;

[0007] The feature maps of all voxels are processed using a global relationship perception network to obtain feature maps with global relationships, and the feature maps with global information are restored to enhanced point cloud data;

[0008] The enhanced point cloud data is processed using a detection head to obtain a 3D target detection result.

[0009] In a possible implementation, pillar feature encoding is performed on the point cloud data to obtain a feature map of each voxel, including:

[0010] The coordinates of each point in the point cloud are corrected using the divergence angle of each point to obtain the corrected point cloud data;

[0011] The corrected point cloud data is feature encoded in a pillar manner to obtain the first tensor (D, P, N) of each voxel; wherein P is the number of voxels, N is the maximum number of points stored in the voxel, and D is the attribute of each point in the voxel;

[0012] Perform convolution operation on the first tensor (D, P, N) of each voxel to generate the second tensor (C, P, N), where C is the number of channels; perform maximum pooling operation on the number of channels C to obtain the third tensor (C, P);

[0013] Convert P in the third tensor (C, P) to (W, H) to obtain a feature map of size (C, W, H), where W is the width of the feature map and H is the height of the feature map.

[0014] In a possible implementation, the coordinates of each point in the point cloud are corrected using the divergence angle of each point to obtain corrected point cloud data; including:

[0015] Calculate the divergence angle of each point in the point cloud data and determine whether the divergence angles of all points conform to the normal distribution; if so, use the average value of the divergence angles of all points θ d Represent the divergence angles for all points:

[0016]

[0017] Otherwise, use the mean value θ d The divergence angles of all points are expressed as the median value in the Gaussian normal distribution statistical model;

[0018] Correct the X and Z coordinate values ​​of each point in the point cloud data:

[0019]

[0020] Among them, θ i ,x i and z i is the original divergence angle, X coordinate and Z coordinate of the i-th point, x i,t and z i,t are the X and Z coordinates of the i-th point after adjustment.

[0021] In a possible implementation, the global relationship perception network includes: a maximum pooling layer, three parallel channel cross transformers, an attention weight module and a fusion module; the three parallel channel cross transformers include: a first channel cross transformer, a second channel cross transformer and a third channel cross transformer;

[0022] The feature maps of all voxels are processed using the global relationship perception network to obtain feature maps with global information, including:

[0023] The feature map of each voxel is processed using the maximum pooling layer to obtain the original feature map of size W×H;

[0024] The original feature map of size W×H is processed by the first channel cross transformer to obtain the first attention feature map of size W×1;

[0025] The second channel cross transformer is used to process the transpose of the original feature map of size W×H to obtain a second attention feature map of size 1×W;

[0026] The original feature map of size W×H is processed by the third channel cross transformer to obtain the third attention feature map of size W×1;

[0027] The first attention feature map and the second attention feature map are fused using the attention weight module to obtain an attention weight feature map of size W×W;

[0028] The fusion module is used to fuse the third attention feature map and the attention weight feature map to obtain a feature map with global information.

[0029] In one possible implementation, the first channel cross transformer includes: a self-attention mechanism operation unit, a residual unit, a fully connected layer, and an average pooling layer;

[0030] The original feature map of size W×H is processed by the first channel cross transformer to obtain a first attention feature map of size W×1; including:

[0031] The original feature map of W×H is processed by the self-attention mechanism operation unit to obtain features with global context information: query vector Q, key vector K and value vector V;

[0032] The residual unit is used to perform residual connection on the features with global context information and the original feature map of size W×H to obtain the first intermediate feature map of size W×H;

[0033] The first intermediate feature map of W×H and the original feature map are processed by a fully connected layer to obtain a second intermediate feature map of size W×H;

[0034] The second intermediate feature map of size W×H is processed using an average pooling layer to obtain the first attention feature map of size W×1.

[0035] In one possible implementation, the attention weight module includes: a first multiplication unit and a normalization unit;

[0036] The first attention feature map and the second attention feature map are fused using the attention weight module to obtain an attention weight feature map, including:

[0037] Using the first multiplication unit to perform a dot multiplication operation on the first attention feature map and the second attention feature map, a third intermediate feature map of size W×W is obtained;

[0038] The third intermediate feature map of size W×W is processed by a normalization unit to obtain an attention weight feature map of size W×W.

[0039] In a possible implementation, the fusion module includes a parallel first branch and a second branch and an addition unit; the first branch includes: a second multiplication unit and a subtraction unit, and the second branch includes: a processing unit and a third multiplication unit;

[0040] The third attention feature map and the attention weight feature map are fused by using a fusion module to obtain a feature map with global information; including:

[0041] Use the second multiplication unit to perform a dot multiplication operation on the third attention feature map and the attention weight feature map to obtain a fourth intermediate feature map of size W×1;

[0042] The fourth intermediate feature map of size W×1 and the first attention feature map are processed by a subtraction unit to obtain a fifth intermediate feature map of size W×1;

[0043] Using the processing unit, processing any row of the third attention feature map of size W×1 and the third intermediate feature map of size W×W and their transposition to obtain a sixth intermediate feature map of size W×W;

[0044] Use the third multiplication unit to multiply the sixth intermediate feature map of size W×W and the first attention feature map of size W×1 to obtain a seventh intermediate feature map of size W×1;

[0045] The fifth intermediate feature map having a size of W×1 and the seventh intermediate feature map having a size of W×1 are added by using an adding unit to obtain a feature map having a size of W×1 and having global information.

[0046] In a second aspect, an embodiment of the present application provides a 4D millimeter wave radar target detection device based on global relationship perception, comprising:

[0047] An acquisition unit, used to acquire point cloud data collected by the 4D millimeter wave radar on the vehicle;

[0048] An encoding unit, used for performing pillar feature encoding on the point cloud data to obtain a feature map of each voxel;

[0049] A processing unit, used to process the feature maps of all voxels using a global relationship perception network to obtain a feature map with global information, and restore the feature map with global information to enhanced point cloud data;

[0050] The detection unit is used to process the enhanced point cloud data using a detection head to obtain a 3D target detection result.

[0051] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method of the embodiment of the present application when executing the computer program.

[0052] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method of the embodiment of the present application is implemented.

[0053] This application improves the accuracy and precision of 3D target detection of point cloud data, especially the detection effect of small targets such as pedestrians and cyclists is significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the specific implementation methods of the present application or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0055] Figure 1 A flowchart of a 4D millimeter-wave radar target detection method based on global relationship perception provided in an embodiment of the present application;

[0056] Figure 2 A schematic diagram of the structure of a global relationship perception network provided in an embodiment of the present application;

[0057] Figure 3 A functional structure diagram of a 4D millimeter wave radar target detection device based on global relationship perception provided in an embodiment of the present application;

[0058] Figure 4 A functional structure diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0060] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0061] First, the design concept of the embodiments of the present application is briefly introduced.

[0062] 4D millimeter-wave radar is a sensor that acquires dynamic information about the surrounding environment by emitting millimeter-wave signals and measuring their reflections. In recent years, 4D millimeter-wave radar has been widely used in autonomous driving and intelligent transportation systems. Compared with traditional radar, it contains point cloud information in four dimensions, including speed, azimuth, pitch angle, and distance, thus providing a more comprehensive environmental perception. However, 4D millimeter-wave radar also has its limitations; due to the low resolution of millimeter-wave signals, the generated point cloud data is relatively sparse and noisy, making it difficult to perform high-precision target detection in complex environments. In addition, millimeter-wave radar has a low reflectivity to certain materials, which may lead to reduced accuracy in target detection.

[0063] In order to solve the problem of insufficient detection capability when using 4D millimeter-wave radar, deep learning methods are considered to directly process point cloud data. However, millimeter-wave radar point cloud is four-dimensional data, and it has sparsity and noise problems. It is difficult for traditional three-dimensional target detection methods to ensure the accuracy of target detection. The commonly used method is to divide the point cloud data into multiple voxels and extract information from each voxel. However, this method does not pay attention to global feature information, reduces the efficiency of long-distance information capture, and loses some target detection accuracy.

[0064] In order to solve the above technical problems, the present application provides a 4D millimeter-wave radar target detection method based on global relationship perception, including: collecting laser radar point cloud data (Point Cloud, PC) of the autonomous driving vehicle, and completing calibration and alignment; processing the point cloud data through the fast coding target detection framework pointpillars, thereby converting the millimeter-wave radar data from the sparse PC representation to a vertical columnar point cloud representation, and each point cloud in the converted column corresponds to a part of the points in the space, including the four-dimensional information of the point cloud such as speed, azimuth, pitch angle and distance. Convert the point cloud in each column into a feature map; use the maximum pooling method to compress the feature map so that it can extract the features that best represent the point cloud information. Based on the idea of ​​self-attention mechanism, a channel cross-transformer (CCT) is designed. The input is the compressed feature map, and the output is the feature map after cross-channel information fusion. The feature map after cross-channel information fusion is spliced, channel converted and fused to output a feature map with global information. The feature map is then sent to the deconvolution layer to enlarge it to its original size. According to the spatial coordinate information corresponding to each pixel, it is restored to the representation of spatial point cloud. The enhanced point cloud is sent to the point cloud 3D target detection framework to obtain the target detection result.

[0065] The advantages of this application are:

[0066] 1. This application can effectively improve the amount of information in the point cloud through global features. This process is a way of point cloud feature enhancement. Using the enhanced point cloud for 3D target detection can verify the effectiveness of this method.

[0067] 2. This application uses a global relationship perception network to effectively integrate global information, improve the accuracy of the 3D target detection algorithm and the target detection precision, especially for small targets such as pedestrians and cyclists.

[0068] After introducing the application scenarios and design concepts of the embodiments of the present application, the technical solutions provided by the embodiments of the present application are described below.

[0069] like Figure 1 As shown, the present application implements an overall flow chart of a 4D millimeter wave radar target detection method based on global relationship perception, including the following steps:

[0070] Step 101: Obtain point cloud data collected by a 4D millimeter wave radar on the vehicle;

[0071] Step 102: Encoding the point cloud data by pillar features to obtain a feature map of each voxel;

[0072] Step 103: Process the feature maps of all voxels using a global relationship perception network to obtain a feature map with global information, and restore the feature map with global information to enhanced point cloud data;

[0073] Step 104: Process the enhanced point cloud data using a detection head to obtain a 3D target detection result.

[0074] In some embodiments, the specific implementation process of step 102 includes:

[0075] First, calculate the divergence angle of each point in the point cloud data to determine whether the divergence angles of all points conform to the normal distribution; if so, use the average value θ of the divergence angles of all points d Represent the divergence angles for all points:

[0076]

[0077] Otherwise, use the mean value θ d The divergence angles of all points are expressed as the median value in the Gaussian normal distribution statistical model;

[0078] Correct the X and Z coordinate values ​​of each point in the point cloud data:

[0079]

[0080] Among them, θ i ,x i and z i is the original divergence angle, X coordinate and Z coordinate of the i-th point, x i,t and z i,t are the X and Z coordinates of the i-th point after adjustment.

[0081] The corrected point cloud data is feature encoded in a pillar manner to obtain the first tensor (D, P, N) of each voxel; wherein P is the number of voxels, N is the maximum number of points stored in the voxel, and D is the attribute of each point in the voxel;

[0082] Perform convolution operation on the first tensor (D, P, N) of each voxel to generate the second tensor (C, P, N), where C is the number of channels; perform maximum pooling operation on the number of channels C to obtain the third tensor (C, P);

[0083] Convert P in the third tensor (C, P) to (W, H) to obtain a feature map of size (C, W, H), where W is the width of the feature map and H is the height of the feature map.

[0084] In some embodiments, Figure 2As shown, the global relationship perception network includes: a maximum pooling layer, three parallel channel cross transformers (CCT), an attention weight module and a fusion module; the three parallel channel cross transformers include: a first channel cross transformer, a second channel cross transformer and a third channel cross transformer;

[0085] The feature maps of all voxels are processed using the global relationship perception network to obtain feature maps with global information, including:

[0086] The feature map of each voxel is processed using the maximum pooling layer to obtain the original feature map of size W×H;

[0087] The original feature map of size W×H is processed by the first channel cross transformer to obtain the first attention feature map of size W×1;

[0088] The second channel cross transformer is used to process the transpose of the original feature map of size W×H to obtain a second attention feature map of size 1×W;

[0089] The original feature map of size W×H is processed by the third channel cross transformer to obtain the third attention feature map of size W×1;

[0090] The first attention feature map and the second attention feature map are fused using the attention weight module to obtain an attention weight feature map of size W×W;

[0091] The fusion module is used to fuse the third attention feature map and the attention weight feature map to obtain a feature map with global information.

[0092] In some embodiments, the first channel cross transformer includes: a self-attention mechanism operation unit, a residual unit, a fully connected layer, and an average pooling layer;

[0093] The original feature map of size W×H is processed by the first channel cross transformer to obtain a first attention feature map of size W×1; including:

[0094] The original feature map of W×H is processed by the self-attention mechanism operation unit to obtain features with global context information: query vector Q, key vector K and value vector V;

[0095] The residual unit is used to perform residual connection on the features with global context information and the original feature map of size W×H to obtain the first intermediate feature map of size W×H;

[0096] The first intermediate feature map of W×H and the original feature map are processed by a fully connected layer to obtain a second intermediate feature map of size W×H;

[0097] The second intermediate feature map of size W×H is processed using an average pooling layer to obtain the first attention feature map of size W×1.

[0098] In some embodiments, the attention weight module includes: a first multiplication unit and a normalization unit;

[0099] The first attention feature map and the second attention feature map are fused using the attention weight module to obtain an attention weight feature map, including:

[0100] Using the first multiplication unit to perform a dot multiplication operation on the first attention feature map and the second attention feature map, a third intermediate feature map of size W×W is obtained;

[0101] The third intermediate feature map of size W×W is processed by a normalization unit to obtain an attention weight feature map of size W×W.

[0102] In some embodiments, the fusion module includes a parallel first branch and a second branch and an addition unit; the first branch includes: a second multiplication unit and a subtraction unit, and the second branch includes: a processing unit and a third multiplication unit;

[0103] The third attention feature map and the attention weight feature map are fused by using a fusion module to obtain a feature map with global information; including:

[0104] The third attention feature map and the attention weight feature map are multiplied by the second multiplication unit to obtain a fourth intermediate feature map of size W×1;

[0105] The fourth intermediate feature map of size W×1 and the first attention feature map are processed by a subtraction unit to obtain a fifth intermediate feature map of size W×1;

[0106] Using processing units ( Figure 2 GA in ) processes any row of the third attention feature map of size W×1 and the third intermediate feature map of size W×W and their transposition to obtain a sixth intermediate feature map of size W×W;

[0107] Use the third multiplication unit to multiply the sixth intermediate feature map of size W×W and the first attention feature map of size W×1 to obtain a seventh intermediate feature map of size W×1;

[0108] The fifth intermediate feature map having a size of W×1 and the seventh intermediate feature map having a size of W×1 are added by using an adding unit to obtain a feature map having a size of W×1 and having global information.

[0109] Based on the same inventive idea, the embodiment of the present application provides a 4D millimeter wave radar target detection device based on global relationship perception, see Figure 3 As shown, the 4D millimeter wave radar target detection device 200 based on global relationship perception provided in the embodiment of the present application at least includes:

[0110] An acquisition unit 201 is used to acquire point cloud data collected by a 4D millimeter wave radar on the vehicle;

[0111] The encoding unit 202 is used to perform pillar feature encoding on the point cloud data to obtain a feature map of each voxel;

[0112] A processing unit 203 is used to process the feature maps of all voxels using a global relationship perception network to obtain a feature map with global information, and restore the feature map with global information to enhanced point cloud data;

[0113] The detection unit 204 is used to process the enhanced point cloud data using a detection head to obtain a 3D target detection result.

[0114] It should be noted that the principle of solving the technical problem of the 4D millimeter-wave radar target detection device 200 based on global relationship perception provided in the embodiment of the present application is similar to the method provided in the embodiment of the present application. Therefore, the implementation of the 4D millimeter-wave radar target detection device 200 based on global relationship perception provided in the embodiment of the present application can refer to the implementation of the method provided in the embodiment of the present application, and the repeated parts will not be repeated.

[0115] Based on the above embodiments, the present application also provides an electronic device, referring to Figure 4 As shown, the electronic device 300 provided in the embodiment of the present application includes at least: a processor 301, a memory 302, and a computer program stored in the memory 302 and executable on the processor 301. When the processor 301 executes the computer program, the 4D millimeter-wave radar target detection method based on global relationship perception provided in the embodiment of the present application is implemented.

[0116] The electronic device 300 provided in the embodiment of the present application may further include a bus 303 connecting different components (including the processor 301 and the memory 302). The bus 303 represents one or more of several types of bus structures, including a memory bus, a peripheral bus, a local bus, and the like.

[0117] The memory 302 may include a readable medium in the form of a volatile memory, such as a random access memory (RAM) 3021 and / or a cache memory 3022 , and may further include a read-only memory (ROM) 3023 .

[0118] The memory 302 may also include a program tool 3025 having a set (at least one) of program modules 3024, the program modules 3024 including but not limited to: an operating subsystem, one or more applications, other program modules and program data, each of which or some combination may include the implementation of a network environment.

[0119] The electronic device 300 may also communicate with one or more external devices 304 (e.g., keyboards, remote controls, etc.), and may also communicate with one or more devices that enable a user to interact with the electronic device 300 (e.g., mobile phones, computers, etc.), and / or communicate with any device that enables the electronic device 300 to communicate with one or more other electronic devices 300 (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 305. Furthermore, the electronic device 300 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 306. Figure 4 As shown, the network adapter 306 communicates with other modules of the electronic device 300 via the bus 303. It should be understood that although Figure 4 Not shown, other hardware and / or software modules may be used in conjunction with the electronic device 300, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, disk arrays (Redundant Arrays of Independent Disks, RAID) subsystems, tape drives, and data backup storage subsystems.

[0120] It should be noted that Figure 4 The electronic device 300 shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0121] The embodiment of the present application also provides a computer-readable storage medium, which stores computer instructions, and when the computer instructions are executed by the processor, the 4D millimeter-wave radar target detection method based on global relationship perception provided in the embodiment of the present application is implemented. Specifically, the executable program can be built-in or installed in the electronic device 300, so that the electronic device 300 can implement the 4D millimeter-wave radar target detection method based on global relationship perception provided in the embodiment of the present application by executing the built-in or installed executable program.

[0122] The 4D millimeter-wave radar target detection method based on global relationship perception provided in an embodiment of the present application can also be implemented as a program product, which includes a program code. When the program product can be run on an electronic device 300, the program code is used to enable the electronic device 300 to execute the 4D millimeter-wave radar target detection method based on global relationship perception provided in an embodiment of the present application.

[0123] The program product provided in the embodiments of the present application may adopt any combination of one or more readable media, wherein the readable medium may be a readable signal medium or a readable storage medium, and the readable storage medium may be but is not limited to an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination of the above. Specifically, more specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, RAM, ROM, Erasable Programmable Read Only Memory (EPROM), optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0124] The program product provided in the embodiment of the present application may adopt a CD-ROM and include program code, and may also be run on a computing device. However, the program product provided in the embodiment of the present application is not limited thereto. In the embodiment of the present application, the readable storage medium may be any tangible medium containing or storing a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device.

[0125] It should be noted that, although several units or subunits of the device are mentioned in the above detailed description, this division is merely exemplary and not mandatory. In fact, according to the embodiments of the present application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided into multiple units to be embodied.

[0126] In addition, although the operations of the method of the present application are described in a specific order in the drawings, this does not require or imply that the operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application is described in detail with reference to the embodiments, a person skilled in the art should understand that any modification or equivalent replacement of the technical solution of the present application does not depart from the spirit and scope of the technical solution of the present application and should be included in the scope of the claims of the present application.

Claims

1. A 4D millimeter wave radar target detection method based on global relationship perception, characterized in that: include: Obtain the original point cloud data collected by the 4D millimeter wave radar on the vehicle; Performing pillar feature encoding on the original point cloud data to obtain a feature map of each voxel; The feature maps of all voxels are processed using a global relationship perception network to obtain feature maps with global information, and the feature maps with global information are restored to enhanced point cloud data; The enhanced point cloud data is processed using a detection head to obtain a 3D target detection result.

2. The method according to claim 1, characterized in that The original point cloud data is encoded with pillar features to obtain a feature map of each voxel, including: The coordinates of each point are corrected using the divergence angle of each point in the original point cloud data to obtain the corrected point cloud data; The corrected point cloud data is feature encoded in a pillar manner to obtain the first tensor (D, P, N) of each voxel; wherein P is the number of voxels, N is the maximum number of points stored in the voxel, and D is the attribute of each point in the voxel; Perform convolution operation on the first tensor (D, P, N) of each voxel to generate the second tensor (C, P, N), where C is the number of channels; perform maximum pooling operation on the number of channels C to obtain the third tensor (C, P); Convert P in the third tensor (C, P) to (W, H) to obtain a feature map of size (C, W, H), where W is the width of the feature map and H is the height of the feature map.

3. The method according to claim 2, characterized in that The coordinates of each point in the point cloud are corrected using the divergence angle of each point to obtain the corrected point cloud data; including: Calculate the divergence angle of each point in the point cloud data and determine whether the divergence angles of all points conform to the normal distribution; if so, use the average value of the divergence angles of all points θ d Represent the divergence angles for all points: Otherwise, use the mean value θ d The divergence angles of all points are expressed as the median value in the Gaussian normal distribution statistical model; Correct the X and Z coordinate values ​​of each point in the point cloud data: Among them, θ i ,x i and z i is the original divergence angle, X coordinate and Z coordinate of the i-th point, x i,t and z i,t are the X and Z coordinates of the i-th point after adjustment.

4. The method according to claim 1, characterized in that: The global relationship perception network includes: a maximum pooling layer, three parallel channel cross transformers, an attention weight module and a fusion module; the three parallel channel cross transformers include: a first channel cross transformer, a second channel cross transformer and a third channel cross transformer; The feature maps of all voxels are processed using the global relationship perception network to obtain feature maps with global information, including: The feature map of each voxel is processed using the maximum pooling layer to obtain the original feature map of size W×H; The original feature map of size W×H is processed by the first channel cross transformer to obtain the first attention feature map of size W×1; The second channel cross transformer is used to process the transpose of the original feature map of size W×H to obtain a second attention feature map of size 1×W; The original feature map of size W×H is processed by the third channel cross transformer to obtain the third attention feature map of size W×1; The first attention feature map and the second attention feature map are fused using the attention weight module to obtain an attention weight feature map of size W×W; The fusion module is used to fuse the third attention feature map and the attention weight feature map to obtain a feature map with global information.

5. The method according to claim 4, characterized in that The first channel cross transformer includes: a self-attention mechanism operation unit, a residual unit, a fully connected layer and an average pooling layer; The original feature map of size W×H is processed by the first channel cross transformer to obtain a first attention feature map of size W×1; including: The original feature map of W×H is processed by the self-attention mechanism operation unit to obtain features with global context information: query vector Q, key vector K and value vector V; The residual unit is used to perform residual connection on the features with global context information and the original feature map of size W×H to obtain the first intermediate feature map of size W×H; The first intermediate feature map of W×H and the original feature map are processed by a fully connected layer to obtain a second intermediate feature map of size W×H; The second intermediate feature map of size W×H is processed using an average pooling layer to obtain the first attention feature map of size W×1.

6. The method according to claim 5, characterized in that The attention weight module includes: a first multiplication unit and a normalization unit; The first attention feature map and the second attention feature map are fused using the attention weight module to obtain an attention weight feature map, including: Using the first multiplication unit to perform a dot multiplication operation on the first attention feature map and the second attention feature map, a third intermediate feature map of size W×W is obtained; The third intermediate feature map of size W×W is processed by a normalization unit to obtain an attention weight feature map of size W×W.

7. The method according to claim 6, characterized in that The fusion module includes a first branch and a second branch in parallel and an adding unit; The first branch includes: a second multiplication unit and a subtraction unit, and the second branch includes: a processing unit and a third multiplication unit; The third attention feature map and the attention weight feature map are fused by using a fusion module to obtain a feature map with global information; including: The third attention feature map and the attention weight feature map are multiplied by the second multiplication unit to obtain a fourth intermediate feature map of size W×1; The fourth intermediate feature map of size W×1 and the first attention feature map are processed by a subtraction unit to obtain a fifth intermediate feature map of size W×1; Using the processing unit, processing any row of the third attention feature map of size W×1 and the third intermediate feature map of size W×W and their transposition to obtain a sixth intermediate feature map of size W×W; Use the third multiplication unit to multiply the sixth intermediate feature map of size W×W and the first attention feature map of size W×1 to obtain a seventh intermediate feature map of size W×1; The fifth intermediate feature map having a size of W×1 and the seventh intermediate feature map having a size of W×1 are added by using an adding unit to obtain a feature map having a size of W×1 and having global information.

8. A 4D millimeter wave radar target detection device based on global relationship perception, characterized in that: include: An acquisition unit, used to acquire point cloud data collected by the 4D millimeter wave radar on the vehicle; An encoding unit, used for performing pillar feature encoding on the point cloud data to obtain a feature map of each voxel; A processing unit, used to process the feature maps of all voxels using a global relationship perception network to obtain a feature map with global information, and restore the feature map with global information to enhanced point cloud data; The detection unit is used to process the enhanced point cloud data using a detection head to obtain a 3D target detection result.

9. An electronic device, characterized in that: include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the method according to any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images