4d millimeter wave radar target detection method and device based on global relationship perception

Abstract

This application provides a 4D millimeter-wave radar target detection method and apparatus based on global relation awareness. The method includes: acquiring raw point cloud data collected by a vehicle-mounted 4D millimeter-wave radar; performing pillar feature encoding on the raw point cloud data to obtain feature maps of each voxel; processing the feature maps of all voxels using a global relation awareness network to obtain feature maps with global information; restoring the feature maps with global information to enhanced point cloud data; and processing the enhanced point cloud data using a detection head to obtain 3D target detection results. This application improves the accuracy and precision of 3D target detection using point cloud data, especially significantly enhancing the detection effect of small targets such as pedestrians and cyclists.

Description

Technical Field

[0001] This application relates to the field of autonomous driving technology, and in particular to a 4D millimeter-wave radar target detection method and apparatus based on global relational awareness. Background Technology

[0002] To address the insufficient detection capabilities of 4D millimeter-wave radar, deep learning methods were considered for direct processing of point cloud data. However, millimeter-wave radar point clouds are four-dimensional data, and they suffer from sparsity and noise issues, making it difficult for traditional three-dimensional target detection methods to guarantee accuracy. Currently, a common approach is to divide the point cloud data into multiple voxels and extract information from each voxel. However, this method neglects global feature information, reducing the efficiency of long-range information acquisition and sacrificing some target detection accuracy. Summary of the Invention

[0003] In view of this, this application provides a 4D millimeter-wave radar target detection method and apparatus based on global relationship awareness to solve the above-mentioned technical problems.

[0004] In a first aspect, embodiments of this application provide a 4D millimeter-wave radar target detection method based on global relation awareness, including:

[0005] Point cloud data acquired from the vehicle's 4D millimeter-wave radar;

[0006] The point cloud data is subjected to pillar feature encoding to obtain the feature map of each voxel;

[0007] The feature maps of all voxels are processed using a global relational awareness network to obtain feature maps with global relations. The feature maps with global information are then restored to enhanced point cloud data.

[0008] The enhanced point cloud data is processed using a detection head to obtain 3D target detection results.

[0009] In one possible implementation, the point cloud data is subjected to pillar feature encoding to obtain feature maps of each voxel, including:

[0010] The coordinates of each point are corrected by using the divergence angle of each point in the point cloud to obtain the corrected point cloud data;

[0011] The corrected point cloud data is subjected to pillar-based feature encoding to obtain the first tensor (D, P, N) of each voxel; where P is the number of voxels, N is the maximum number of points stored in the voxel, and D is the attribute of each point in the voxel.

[0012] Convolution is performed on the first tensor (D,P,N) of each voxel to generate the second tensor (C,P,N), where C is the number of channels; max pooling is performed on the number of channels C to obtain the third tensor (C,P).

[0013] Transform P in the third tensor (C,P) into (W,H) to obtain a feature map of size (C,W,H), where W is the width of the feature map and H is the height of the feature map.

[0014] In one possible implementation, the coordinates of each point are corrected using the divergence angle of each point in the point cloud to obtain corrected point cloud data; including:

[0015] Calculate the divergence angle of each point in the point cloud data, and determine whether the divergence angles of all points conform to a normal distribution; if so, use the average value θ of the divergence angles of all points. d Represents the divergence angle at all points:

[0016]

[0017] Otherwise, use the average value θ d The median of the Gaussian normal distribution statistical model is used to represent the divergence angle of all points.

[0018] Correct the X and Z coordinate values ​​of each point in the point cloud data:

[0019]

[0020] Where, θ i ,x i and z i Let x be the original divergence angle of the i-th point, x coordinate and z coordinate, x i,t and z i,t These are the adjusted X and Z coordinates of the i-th point.

[0021] In one possible implementation, the global relation-aware network includes: a max-pooling layer, three parallel channel cross-transformers, an attention weight module, and a fusion module; the three parallel channel cross-transformers include: a first channel cross-transformer, a second channel cross-transformer, and a third channel cross-transformer.

[0022] A global relation-aware network is used to process the feature maps of all voxels to obtain feature maps with global information, including:

[0023] The feature maps of each voxel are processed using a max pooling layer to obtain an original feature map of size W×H;

[0024] The original feature map of size W×H is processed using the first channel cross-converter to obtain a first attention feature map of size W×1;

[0025] The transpose of the original feature map of size W×H is processed by the second channel cross converter to obtain a second attention feature map of size 1×W.

[0026] The original feature map of size W×H is processed using the third-channel cross-converter to obtain a third attention feature map of size W×1;

[0027] The first attention feature map and the second attention feature map are fused using the attention weight module to obtain an attention weight feature map of size W×W;

[0028] The third attention feature map and the attention weight feature map are fused using the fusion module to obtain a feature map with global information.

[0029] In one possible implementation, the first channel cross converter includes: a self-attention mechanism operation unit, a residual unit, a fully connected layer, and an average pooling layer;

[0030] The original feature map of size W×H is processed using the first-channel cross-transformer to obtain a first attention feature map of size W×1; including:

[0031] The original feature map W×H is processed using a self-attention mechanism operation unit to obtain features with global context information: query vector Q, key vector K, and value vector V;

[0032] The residual unit is used to perform residual connection between the features with global context information and the original feature map of size W×H to obtain the first intermediate feature map of size W×H;

[0033] The first intermediate feature map of size W×H and the original feature map are processed by a fully connected layer to obtain a second intermediate feature map of size W×H;

[0034] The second intermediate feature map of size W×H is processed using an average pooling layer to obtain the first attention feature map of size W×1.

[0035] In one possible implementation, the attention weight module includes: a first multiplication unit and a normalization unit;

[0036] The attention weighting module is used to fuse the first attention feature map and the second attention feature map to obtain the attention weighting feature map, including:

[0037] The first attention feature map and the second attention feature map are multiplied by the first multiplication unit to obtain a third intermediate feature map of size W×W.

[0038] The third intermediate feature map of size W×W is processed using a normalization unit to obtain an attention weight feature map of size W×W.

[0039] In one possible implementation, the fusion module includes a first branch and a second branch in parallel, as well as an addition unit; the first branch includes a second multiplication unit and a subtraction unit, and the second branch includes a processing unit and a third multiplication unit;

[0040] The fusion module fuses the third attention feature map and the attention weight feature map to obtain a feature map with global information; including:

[0041] The second multiplication unit is used to perform a dot product operation on the third attention feature map and the attention weight feature map to obtain a fourth intermediate feature map of size W×1;

[0042] The fourth intermediate feature map and the first attention feature map of size W×1 are processed by the subtraction unit to obtain the fifth intermediate feature map of size W×1.

[0043] The processing unit processes any row of the third attention feature map of size W×1 and the third intermediate feature map of size W×W, as well as their transpose, to obtain the sixth intermediate feature map of size W×W.

[0044] The third multiplication unit is used to multiply the sixth intermediate feature map of size W×W and the first attention feature map of size W×1 to obtain the seventh intermediate feature map of size W×1.

[0045] The addition unit is used to add the fifth intermediate feature map of size W×1 and the seventh intermediate feature map of size W×1 to obtain a feature map of size W×1 with global information.

[0046] Secondly, embodiments of this application provide a 4D millimeter-wave radar target detection device based on global relationship awareness, comprising:

[0047] The acquisition unit is used to acquire point cloud data collected by the on-board 4D millimeter-wave radar;

[0048] The encoding unit is used to encode the pillar features of the point cloud data to obtain the feature map of each voxel;

[0049] The processing unit is used to process the feature maps of all voxels using a global relational awareness network to obtain feature maps with global information, and then restore the feature maps with global information to the enhanced point cloud data.

[0050] The detection unit is used to process the enhanced point cloud data using a detection head to obtain 3D target detection results.

[0051] Thirdly, embodiments of this application provide an electronic device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method of embodiments of this application.

[0052] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the methods of embodiments of this application.

[0053] This application improves the accuracy and precision of 3D target detection using point cloud data, especially significantly enhancing the detection performance of small targets such as pedestrians and cyclists. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the specific embodiments of this application or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0055] Figure 1 A flowchart of a 4D millimeter-wave radar target detection method based on global relation awareness provided in an embodiment of this application;

[0056] Figure 2 This is a schematic diagram of the structure of a global relation-aware network provided in an embodiment of this application;

[0057] Figure 3 A functional structure diagram of a 4D millimeter-wave radar target detection device based on global relationship awareness provided in an embodiment of this application;

[0058] Figure 4 A functional structure diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0059] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0060] Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely to illustrate selected embodiments of the application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0061] First, a brief introduction to the design concept of the embodiments of this application will be given.

[0062] 4D millimeter-wave radar is a sensor that acquires dynamic information about the surrounding environment by emitting millimeter-wave signals and measuring their reflection. In recent years, 4D millimeter-wave radar has been widely used in autonomous driving and intelligent transportation systems. Compared with traditional radar, it contains point cloud information in four dimensions: velocity, azimuth, elevation, and range, thus providing more comprehensive environmental perception. However, 4D millimeter-wave radar also has its limitations; due to the low resolution of millimeter-wave signals, the generated point cloud data is relatively sparse and contains noise, making it difficult to perform high-precision target detection in complex environments. Furthermore, millimeter-wave radar has low reflectivity to certain materials, which may lead to a decrease in the accuracy of target detection.

[0063] To address the insufficient detection capabilities of 4D millimeter-wave radar, deep learning methods were considered for direct processing of point cloud data. However, millimeter-wave radar point clouds are four-dimensional data, and they suffer from sparsity and noise issues, making it difficult for traditional three-dimensional target detection methods to guarantee accuracy. Currently, a common approach is to divide the point cloud data into multiple voxels and extract information from each voxel. However, this method neglects global feature information, reducing the efficiency of long-range information acquisition and sacrificing some target detection accuracy.

[0064] To address the aforementioned technical issues, this application provides a 4D millimeter-wave radar target detection method based on global relation awareness, comprising: acquiring LiDAR point cloud data (PC) of an autonomous vehicle, which has already been calibrated and aligned; processing the point cloud data using the fast encoding target detection framework PointPillars, thereby transforming the millimeter-wave radar data from a sparse PC representation into a vertically columnar point cloud representation, where each column of the transformed point cloud corresponds to a portion of points in space, containing four-dimensional information such as velocity, azimuth, pitch, and range; converting the point cloud within each column into a feature map; and compressing the feature map using a max-pooling method to extract the features that best represent the point cloud information. A Channel Cross-Transformer (CCT) was designed based on the self-attention mechanism. The input is a compressed feature map, and the output is a feature map after cross-channel information fusion. The feature map after cross-channel information fusion is concatenated, channel transformed, and fused to output a feature map with global information. Then, it is fed into a deconvolution layer to enlarge the feature map to its original size. Based on the spatial coordinate information corresponding to each pixel, it is restored to the spatial point cloud representation. The enhanced point cloud is fed into the point cloud 3D object detection framework to obtain the object detection result.

[0065] The advantages of this application are:

[0066] 1. This application can effectively enhance the information content of point clouds through global features. This process is a way of enhancing point cloud features. The effectiveness of this method can be verified by using the enhanced point cloud for 3D object detection.

[0067] 2. This application uses a global relation-aware network to effectively integrate global information, improving the accuracy and precision of the 3D target detection algorithm, especially for small targets such as pedestrians and cyclists, where the accuracy is significantly improved.

[0068] After introducing the application scenarios and design concepts of the embodiments of this application, the technical solutions provided by the embodiments of this application will be described below.

[0069] like Figure 1 As shown, this application provides an overall flowchart of a 4D millimeter-wave radar target detection method based on global relationship awareness, including the following steps:

[0070] Step 101: Acquire point cloud data collected by the onboard 4D millimeter-wave radar;

[0071] Step 102: Perform pillar feature encoding on the point cloud data to obtain feature maps of each voxel;

[0072] Step 103: Use a global relational awareness network to process the feature maps of all voxels to obtain feature maps with global information, and restore the feature maps with global information to the enhanced point cloud data.

[0073] Step 104: Process the enhanced point cloud data using the detection head to obtain 3D target detection results.

[0074] In some embodiments, the specific implementation process of step 102 includes:

[0075] First, calculate the divergence angle of each point in the point cloud data, and determine whether the divergence angles of all points conform to a normal distribution; if so, use the average value θ of the divergence angles of all points. d Represents the divergence angle at all points:

[0076]

[0077] Otherwise, use the average value θ d The median of the Gaussian normal distribution statistical model is used to represent the divergence angle of all points.

[0078] Correct the X and Z coordinate values ​​of each point in the point cloud data:

[0079]

[0080] Where, θ i ,x i and z i Let x be the original divergence angle of the i-th point, x coordinate and z coordinate, x i,t and z i,t These are the adjusted X and Z coordinates of the i-th point.

[0081] The corrected point cloud data is subjected to pillar-based feature encoding to obtain the first tensor (D, P, N) of each voxel; where P is the number of voxels, N is the maximum number of points stored in the voxel, and D is the attribute of each point in the voxel.

[0082] Convolution is performed on the first tensor (D,P,N) of each voxel to generate the second tensor (C,P,N), where C is the number of channels; max pooling is performed on the number of channels C to obtain the third tensor (C,P).

[0083] Transform P in the third tensor (C,P) into (W,H) to obtain a feature map of size (C,W,H), where W is the width of the feature map and H is the height of the feature map.

[0084] In some embodiments, such as Figure 2As shown, the global relation-aware network includes: a max pooling layer, three parallel channel cross transformers (CCTs), an attention weight module, and a fusion module; the three parallel channel cross transformers include: a first channel cross transformer, a second channel cross transformer, and a third channel cross transformer;

[0085] A global relation-aware network is used to process the feature maps of all voxels to obtain feature maps with global information, including:

[0086] The feature maps of each voxel are processed using a max pooling layer to obtain an original feature map of size W×H;

[0087] The original feature map of size W×H is processed using the first channel cross-converter to obtain a first attention feature map of size W×1;

[0088] The transpose of the original feature map of size W×H is processed by the second channel cross converter to obtain a second attention feature map of size 1×W.

[0089] The original feature map of size W×H is processed using the third-channel cross-converter to obtain a third attention feature map of size W×1;

[0090] The first attention feature map and the second attention feature map are fused using the attention weight module to obtain an attention weight feature map of size W×W;

[0091] The third attention feature map and the attention weight feature map are fused using the fusion module to obtain a feature map with global information.

[0092] In some embodiments, the first channel cross converter includes: a self-attention mechanism operation unit, a residual unit, a fully connected layer, and an average pooling layer;

[0093] The original feature map of size W×H is processed using the first-channel cross-transformer to obtain a first attention feature map of size W×1; including:

[0094] The original feature map W×H is processed using a self-attention mechanism operation unit to obtain features with global context information: query vector Q, key vector K, and value vector V;

[0095] The residual unit is used to perform residual connection between the features with global context information and the original feature map of size W×H to obtain the first intermediate feature map of size W×H;

[0096] The first intermediate feature map of size W×H and the original feature map are processed by a fully connected layer to obtain a second intermediate feature map of size W×H;

[0097] The second intermediate feature map of size W×H is processed using an average pooling layer to obtain the first attention feature map of size W×1.

[0098] In some embodiments, the attention weighting module includes: a first multiplication unit and a normalization unit;

[0099] The attention weighting module is used to fuse the first attention feature map and the second attention feature map to obtain the attention weighting feature map, including:

[0100] The first attention feature map and the second attention feature map are multiplied by the first multiplication unit to obtain a third intermediate feature map of size W×W.

[0101] The third intermediate feature map of size W×W is processed using a normalization unit to obtain an attention weight feature map of size W×W.

[0102] In some embodiments, the fusion module includes a parallel first branch and a second branch, as well as an addition unit; the first branch includes a second multiplication unit and a subtraction unit, and the second branch includes a processing unit and a third multiplication unit;

[0103] The fusion module fuses the third attention feature map and the attention weight feature map to obtain a feature map with global information; including:

[0104] The second multiplication unit is used to multiply the third attention feature map and the attention weight feature map to obtain a fourth intermediate feature map of size W×1;

[0105] The fourth intermediate feature map and the first attention feature map of size W×1 are processed by the subtraction unit to obtain the fifth intermediate feature map of size W×1.

[0106] Using processing unit ( Figure 2 The GA in the process processes any row of the third attention feature map of size W×1 and the third intermediate feature map of size W×W, as well as their transpose, to obtain the sixth intermediate feature map of size W×W.

[0107] The third multiplication unit is used to multiply the sixth intermediate feature map of size W×W and the first attention feature map of size W×1 to obtain the seventh intermediate feature map of size W×1.

[0108] The addition unit is used to add the fifth intermediate feature map of size W×1 and the seventh intermediate feature map of size W×1 to obtain a feature map of size W×1 with global information.

[0109] Based on the same inventive concept, this application provides a 4D millimeter-wave radar target detection device based on global relationship awareness, see below. Figure 3 As shown, the 4D millimeter-wave radar target detection device 200 based on global relationship awareness provided in this application embodiment includes at least:

[0110] Acquisition unit 201 is used to acquire point cloud data collected by the 4D millimeter-wave radar on the vehicle;

[0111] Encoding unit 202 is used to encode the pillar features of the point cloud data to obtain feature maps of each voxel;

[0112] Processing unit 203 is used to process the feature maps of all voxels using a global relational awareness network to obtain feature maps with global information, and restore the feature maps with global information to enhanced point cloud data.

[0113] The detection unit 204 is used to process the enhanced point cloud data using the detection head to obtain 3D target detection results.

[0114] It should be noted that the principle of the 4D millimeter-wave radar target detection device 200 based on global relation awareness provided in this application embodiment to solve the technical problem is similar to the method provided in this application embodiment. Therefore, the implementation of the 4D millimeter-wave radar target detection device 200 based on global relation awareness provided in this application embodiment can refer to the implementation of the method provided in this application embodiment, and the repeated parts will not be described again.

[0115] Based on the above embodiments, this application also provides an electronic device, see below. Figure 4 As shown, the electronic device 300 provided in this application embodiment includes at least: a processor 301, a memory 302, and a computer program stored in the memory 302 and executable on the processor 301. When the processor 301 executes the computer program, it implements the 4D millimeter-wave radar target detection method based on global relation awareness provided in this application embodiment.

[0116] The electronic device 300 provided in this application embodiment may further include a bus 303 connecting different components (including processor 301 and memory 302). The bus 303 represents one or more types of bus structures, including memory bus, peripheral bus, local area bus, etc.

[0117] The memory 302 may include a readable medium in the form of volatile memory, such as random access memory (RAM) 3021 and / or cache memory 3022, and may further include read-only memory (ROM) 3023.

[0118] The memory 302 may also include a program tool 3025 having a set (at least one) of program modules 3024, including but not limited to: an operating subsystem, one or more application programs, other program modules, and program data, each or some combination of these examples may include an implementation of a network environment.

[0119] Electronic device 300 can also communicate with one or more external devices 304 (e.g., keyboard, remote control, etc.), and with one or more devices that enable a user to interact with electronic device 300 (e.g., mobile phone, computer, etc.), and / or with any device that enables electronic device 300 to communicate with one or more other electronic devices 300 (e.g., router, modem, etc.). This communication can be performed through input / output (I / O) interface 305. Furthermore, electronic device 300 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) through network adapter 306. Figure 4 As shown, network adapter 306 communicates with other modules of electronic device 300 via bus 303. It should be understood that, although... Figure 4 As not shown, other hardware and / or software modules may be used in conjunction with electronic device 300, including but not limited to: microcode, device drivers, redundant processors, external disk drive arrays, Redundant Arrays of Independent Disks (RAID) subsystems, tape drives, and data backup storage subsystems.

[0120] It should be noted that, Figure 4 The electronic device 300 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0121] This application also provides a computer-readable storage medium storing computer instructions. When executed by a processor, these instructions implement the 4D millimeter-wave radar target detection method based on global relation awareness provided in this application. Specifically, the executable program can be built into or installed in an electronic device 300, so that the electronic device 300 can implement the 4D millimeter-wave radar target detection method based on global relation awareness provided in this application by executing the built-in or installed executable program.

[0122] The 4D millimeter-wave radar target detection method based on global relation awareness provided in this application embodiment can also be implemented as a program product. The program product includes program code. When the program product can run on the electronic device 300, the program code is used to make the electronic device 300 execute the 4D millimeter-wave radar target detection method based on global relation awareness provided in this application embodiment.

[0123] The program product provided in this application embodiment can be any combination of one or more readable media, wherein the readable media can be a readable signal medium or a readable storage medium, and the readable storage medium can be, but is not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination thereof. Specifically, more specific examples of readable storage media (a non-exhaustive list) include: electrical connections with one or more wires, portable disks, hard disks, RAM, ROM, erasable programmable read-only memory (EPROM), optical fibers, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0124] The program product provided in this application embodiment can be a CD-ROM and include program code, and can also run on a computing device. However, the program product provided in this application embodiment is not limited thereto. In this application embodiment, the readable storage medium can be any tangible medium that contains or stores a program, which can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0125] It should be noted that although several units or sub-units of the device have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this application, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units.

[0126] Furthermore, although the operations of the method of this application are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application and are not intended to limit it. Although this application has been described in detail with reference to the embodiments, those skilled in the art should understand that modifications or equivalent substitutions to the technical solutions of this application do not depart from the spirit and scope of the technical solutions of this application, and should all be covered within the scope of the claims of this application.

Claims

1. A 4D millimeter-wave radar target detection method based on global relation awareness, characterized in that, include: Raw point cloud data acquired from the vehicle's 4D millimeter-wave radar; The original point cloud data is subjected to pillar feature encoding to obtain feature maps of each voxel; The feature maps of all voxels are processed using a global relational awareness network to obtain feature maps with global information, and the feature maps with global information are then restored to enhanced point cloud data. The enhanced point cloud data is processed using a detection head to obtain 3D target detection results; The global relation-aware network includes: a max pooling layer, three parallel channel cross-transformers, an attention weight module, and a fusion module; the three parallel channel cross-transformers include: a first channel cross-transformer, a second channel cross-transformer, and a third channel cross-transformer; A global relation-aware network is used to process the feature maps of all voxels to obtain feature maps with global information, including: The feature maps of each voxel are processed using a max pooling layer to obtain a value of [size missing]. The original feature map; Using the first-channel cross converter to convert the size to The original feature map is processed to obtain a size of First attention feature map; Using the second-channel cross converter to convert the size to The original feature map is transposed and processed to obtain a size of The second attention feature map; Using a third-channel cross converter to convert a value of The original feature map is processed to obtain a size of The third attention feature map; The first and second attention feature maps are fused using the attention weighting module to obtain a result of size [value missing]. Attention weight feature map; The third attention feature map and the attention weight feature map are fused using the fusion module to obtain a feature map with global information; The attention weighting module includes: a first multiplication unit and a normalization unit; The attention weighting module fuses the first attention feature map and the second attention feature map to obtain the attention weighting feature map, including: Using the first multiplication unit, a dot product operation is performed on the first attention feature map and the second attention feature map to obtain a value of... The third intermediate feature map; Using normalized units for sizes The third intermediate feature map is processed to obtain a size of Attention weight feature map; The fusion module includes a parallel first branch and a second branch, as well as an addition unit; the first branch includes a second multiplication unit and a subtraction unit, and the second branch includes a processing unit and a third multiplication unit; The fusion module fuses the third attention feature map and the attention weight feature map to obtain a feature map with global information; including: The second multiplication unit performs multiplication on the third attention feature map and the attention weight feature map to obtain a value of... The fourth intermediate feature map; Using the subtraction unit to calculate the size of The fourth intermediate feature map and the first attention feature map are processed to obtain a size of The fifth intermediate feature map; Using the processing unit to process a size of The third attention feature map and its size are Process any row of the third intermediate feature map and its transpose to obtain a size of The sixth intermediate feature map; Using the third multiplication unit for a size of The sixth intermediate feature map and its size are Multiply the first attention feature map to obtain a value of... The seventh intermediate feature map; Using the addition unit for a size of The fifth intermediate feature map and its size are The seventh intermediate feature map is added to obtain a value of [value]. A feature map containing global information.

2. The method according to claim 1, characterized in that, The original point cloud data is subjected to pillar feature encoding to obtain feature maps of each voxel, including: The coordinates of each point are corrected by using the divergence angle of each point in the original point cloud data to obtain the corrected point cloud data. The corrected point cloud data is subjected to pillar-based feature encoding to obtain the first tensor (D, P, N) of each voxel; where P is the number of voxels, N is the maximum number of points stored in the voxel, and D is the attribute of each point in the voxel. Convolution is performed on the first tensor (D,P,N) of each voxel to generate the second tensor (C,P,N), where C is the number of channels; max pooling is performed on the number of channels C to obtain the third tensor (C,P); Transform P in the third tensor (C,P) into (W,H) to obtain a feature map of size (C,W,H), where W is the width of the feature map and H is the height of the feature map.

3. The method according to claim 2, characterized in that, The coordinates of each point are corrected using the divergence angle of each point in the point cloud, resulting in corrected point cloud data; including: Calculate the divergence angle of each point in the point cloud data, and determine whether the divergence angles of all points conform to a normal distribution; if so, use the average value of the divergence angles of all points. Represents the divergence angle at all points: Otherwise, use the average value. The median is used in the Gaussian normal distribution statistical model to represent the divergence angle of all points; Correct the X and Z coordinate values ​​of each point in the point cloud data: in, , and Let the original divergence angle, X coordinate, and Z coordinate of the i-th point be denoted as . and These are the adjusted X and Z coordinates of the i-th point.

4. The method according to claim 1, characterized in that, The first channel cross converter includes: a self-attention mechanism operation unit, a residual unit, a fully connected layer, and an average pooling layer; Using the first-channel cross converter to convert the size to The original feature map is processed to obtain a size of The first attention feature map includes: Operators using self-attention mechanisms The original feature map is processed to obtain features with global context information: query vector Q, key vector K, and value vector V; Using residual units for features with global context information and sizes of Residual connections are performed on the original feature maps to obtain a result of size [missing information]. The first intermediate feature map; Using fully connected layers The first intermediate feature map and the original feature map are processed to obtain a size of The second intermediate feature map; Using average pooling layers for sizes The second intermediate feature map is processed to obtain a size of [missing value]. The first attention feature map.

5. A 4D millimeter-wave radar target detection device based on global relationship awareness, characterized in that, include: The acquisition unit is used to acquire point cloud data collected by the on-board 4D millimeter-wave radar; The encoding unit is used to encode the pillar features of the point cloud data to obtain the feature map of each voxel; The processing unit is used to process the feature maps of all voxels using a global relational awareness network to obtain feature maps with global information, and then restore the feature maps with global information to the enhanced point cloud data. The detection unit is used to process the enhanced point cloud data using a detection head to obtain 3D target detection results; The global relation-aware network includes: a max pooling layer, three parallel channel cross-transformers, an attention weight module, and a fusion module; the three parallel channel cross-transformers include: a first channel cross-transformer, a second channel cross-transformer, and a third channel cross-transformer; A global relation-aware network is used to process the feature maps of all voxels to obtain feature maps with global information, including: The feature maps of each voxel are processed using a max pooling layer to obtain a value of [size missing]. The original feature map; Using the first-channel cross converter to convert the size to The original feature map is processed to obtain a size of First attention feature map; Using the second-channel cross converter to convert the size to The original feature map is transposed and processed to obtain a size of The second attention feature map; Using a third-channel cross converter to convert a value of The original feature map is processed to obtain a size of The third attention feature map; The first and second attention feature maps are fused using the attention weighting module to obtain a result of size [value missing]. Attention weight feature map; The third attention feature map and the attention weight feature map are fused using the fusion module to obtain a feature map with global information; The attention weighting module includes: a first multiplication unit and a normalization unit; The attention weighting module fuses the first attention feature map and the second attention feature map to obtain the attention weighting feature map, including: Using the first multiplication unit, a dot product operation is performed on the first attention feature map and the second attention feature map to obtain a value of... The third intermediate feature map; Using normalized units for sizes The third intermediate feature map is processed to obtain a size of Attention weight feature map; The fusion module includes a parallel first branch and a second branch, as well as an addition unit; the first branch includes a second multiplication unit and a subtraction unit, and the second branch includes a processing unit and a third multiplication unit; The fusion module fuses the third attention feature map and the attention weight feature map to obtain a feature map with global information; including: The second multiplication unit performs multiplication on the third attention feature map and the attention weight feature map to obtain a value of... The fourth intermediate feature map; Using the subtraction unit to calculate the size of The fourth intermediate feature map and the first attention feature map are processed to obtain a size of The fifth intermediate feature map; Using the processing unit to process a size of The third attention feature map and its size are Process any row of the third intermediate feature map and its transpose to obtain a size of The sixth intermediate feature map; Using the third multiplication unit for a size of The sixth intermediate feature map and its size are Multiply the first attention feature map to obtain a value of... The seventh intermediate feature map; Using the addition unit for a size of The fifth intermediate feature map and its size are The seventh intermediate feature map is added to obtain a value of [value]. A feature map containing global information.

6. An electronic device, characterized in that, include: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the computer program, implements the method as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the method as described in any one of claims 1-4.

Images