4D millimeter wave radar point cloud enhancement and target detection method based on bayesian fusion

Abstract

The application provides a 4D millimeter wave radar point cloud enhancement and target detection method based on Bayesian fusion, comprising the following steps: step one: projecting the 4D millimeter wave radar point cloud to an image plane and matching the image plane with an image instance segmentation result to obtain an image semantic category and an image segmentation confidence; step two: constructing a conditional probability density model based on the physical characteristics of the radar points; weighting and fusing the image segmentation confidence and the category probability density based on a Bayesian probability framework to obtain a fusion confidence; step three: using the fusion confidence and the semantic category as augmented features to splice the features of the 4D millimeter wave radar point cloud to generate a semantic enhanced point cloud; and using a fusion confidence guided sampling method to sample the semantic enhanced point cloud and inputting the semantic enhanced point cloud into a detection network for training and reasoning. The application solves the technical problems of one-way dependence, shallow geometric matching and insufficient utilization of radar physical characteristics in the existing image and radar point cloud fusion technology.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of automatic driving environment perception, and in particular to a 4D millimeter wave radar point cloud enhancement and target detection method based on Bayesian fusion. BACKGROUND

[0002] Currently, the environment perception technology of automatic driving and intelligent transportation systems is developing towards multi-sensor fusion to cope with complex scenarios and adverse weather challenges. Cameras can provide high-resolution texture and color information, which is the key sensor for realizing fine target classification and segmentation, but its performance is easily affected by light and weather. Laser radar can generate dense three-dimensional geometric point cloud, but it is costly and its performance decreases in fog, rain and smoke. Traditional millimeter wave radar has the advantages of accurate speed measurement and all-weather operation, but its point cloud is sparse and lacks height information, limiting its perception dimension. In recent years, the 4D millimeter wave radar has emerged, which can provide dense point cloud containing distance, azimuth angle, elevation angle and Doppler velocity, greatly improving the target contour resolution and spatial information recognition, providing rich and reliable information support for building a high-performance fusion perception system.

[0003] In the fusion scheme of image and radar point cloud, a typical existing technology is a geometric matching method based on image segmentation and radar point cloud projection. The basic process of this technology is: first, extract the two-dimensional semantic mask of the target through the image segmentation network; second, use the extrinsic matrix to project the 4D millimeter wave radar point cloud to the image plane in one direction; finally, determine only according to the spatial geometric position relationship, that is, directly determine whether the projected point is contained in the image mask area. If the inclusion relationship is satisfied, the class label perceived by the camera is forcibly assigned to the corresponding radar point.

[0004] However, the core drawback of the above-mentioned existing technology is its one-way and shallow fusion architecture. On the one hand, this technology highly depends on camera perception results and lacks robustness to deal with image segmentation failures. Since it adopts a "camera-first, radar-second" fusion mode, any false positives or false negatives in image segmentation will be propagated to the radar modality through the projection process, resulting in radar point cloud being assigned with incorrect semantic labels. On the other hand, the information utilization rate of the 4D millimeter wave radar is low, and the fusion process only stays at the spatial geometric matching level, failing to effectively extract the deep semantic value of features such as radial velocity and radar cross-sectional area, resulting in the modality advantage of 4D radar in multi-dimensional physical perception not being effectively released. SUMMARY

[0005] The present application provides a 4D millimeter wave radar point cloud enhancement and target detection method based on Bayesian fusion, which solves the technical problems of one-way dependence, shallow geometric matching and insufficient utilization of radar physical features in existing image and radar point cloud fusion technology.

[0006] To solve the above technical problems, the technical scheme of the present application is as follows:

[0007] The present application provides a 4D millimeter wave radar point cloud enhancement and target detection method based on Bayesian fusion, comprising the following steps:

[0008] Step one: project the 4D millimeter wave radar point cloud to the image plane and match it with the image instance segmentation result to obtain the image semantic category and image segmentation confidence corresponding to each radar point;

[0009] Step two: based on the physical characteristics of the radar points in the 4D millimeter wave radar point cloud, construct a class conditional probability density model for different semantic categories to obtain the probability density of the corresponding semantic category physical characteristics; map the probability density of the physical characteristics to the logarithmic space and perform standardization processing to obtain the standardized log-likelihood value; in the logarithmic domain, the image segmentation confidence is used as a priori term and is weighted and fused with the standardized log-likelihood value to obtain the fusion confidence of each radar point belonging to each semantic category; the physical characteristics include absolute radial velocity and radar cross-sectional area;

[0010] Step three: encode the fusion confidence and the corresponding semantic category as augmented features, and perform feature splicing on the 4D millimeter wave radar point cloud to generate a semantic enhanced point cloud; use a key point sampling method based on fusion confidence guidance to sample the semantic enhanced point cloud, and input the sampled point cloud into a three-dimensional target detection network for training and reasoning to realize target detection.

[0011] Further, the step one specifically comprises:

[0012] The radar point coordinates are converted to the camera coordinate system using the extrinsic matrix of the radar coordinate system to the camera coordinate system;

[0013] The radar point coordinates in the camera coordinate system are projected to the image pixel coordinate system using the camera intrinsic matrix to obtain the projected pixel coordinates of the radar points on the image;

[0014] Based on the binary mask of each semantic category in the image obtained by the instance segmentation network, it is determined whether the projected pixel coordinates fall within the binary mask area of each semantic category, if so, the image semantic category corresponding to the radar point is determined, and the segmentation confidence corresponding to the category is taken as the image segmentation confidence.

[0015] Further, in step two, the construction of the class conditional probability density model for different semantic categories specifically comprises:

[0016] For each semantic category, based on the statistical distribution of the absolute radial velocity and radar cross-section of radar points belonging to that category in the training data, a Gaussian distribution model is used to parameterize and fit the class probability density of the absolute radial velocity and radar cross-section respectively.

[0017] Furthermore, in step two, within the logarithmic domain, the image segmentation confidence is used as a priori and weighted with the standardized log-likelihood value to obtain the fused confidence of each radar point belonging to each semantic category, specifically including:

[0018] For radar points, they belong to semantic categories. Fusion confidence Its logarithmic form satisfy:

[0019] ;

[0020] in, Segment the image with confidence; The velocity feature score is calculated and standardized based on the class conditional probability of absolute radial velocity; The radar cross-section feature score is calculated and standardized based on the class conditional probability of radar cross-section. and The fusion weights are obtained through training and satisfy the following conditions: .

[0021] Furthermore, the velocity feature score The calculation formula is:

[0022]

[0023] in, The absolute radial velocity calculated based on the Gaussian distribution model. Class-conditional probability density; Indicates that the training set belongs to category The mean of the samples in the logarithmic probability space. This represents the corresponding standard deviation.

[0024] Furthermore, the radar cross-section feature score The calculation formula is:

[0025]

[0026] in, Radar cross-section calculated based on Gaussian distribution model Class-conditional probability density; Indicates that the training set belongs to category The mean of the samples in the logarithmic probability space. denotes the corresponding standard deviation.

[0027] Further, in step three, the fusion confidence guided key point sampling method is an improved farthest point sampling method, and a sampling weighted distance measurement formula of the improved farthest point sampling method is as follows:

[0028]

[0029] wherein, is the minimum Euclidean distance from the current point to be investigated to the key point set; is a preset semantic guidance weight coefficient; is a normalized fusion confidence score, and the value range is 0~1; is a comprehensive weighted distance for sampling, and in each iteration, the point with the maximum value is selected as a new key point.

[0030] Further, the three-dimensional target detection network adopts PV-RCNN as a detection framework, and the improved farthest point sampling method is used to improve the key point sampling strategy in the network.

[0031] Compared with the prior art, the present application at least includes the following beneficial effects:

[0032] The present application breaks the one-way dependence mode of the prior art that the camera is the main mode and the radar is the auxiliary mode by constructing a unified Bayesian probability fusion framework to deeply fuse the semantic confidence of image segmentation and the physical features (absolute radial velocity, radar cross section) of 4D millimeter wave radar. By establishing a statistical distribution model (such as Gaussian distribution) of radar physical features for different categories of targets and weighting and fusing the model as a likelihood term with the image prior confidence, the present application realizes the probabilistic cross verification and error correction of the image segmentation result. This means that when the image segmentation is misjudged or the confidence is reduced due to reasons such as light and shielding, the physical evidence of the radar mode can provide effective correction through the likelihood term, thereby significantly improving the overall robustness of the multi-sensor fusion system in complex scenes.

[0033] The present application fully utilizes the multi-dimensional physical information unique to 4D millimeter wave radar. Traditional methods only use the spatial position of point cloud for geometric matching, while the present application deeply excavates the statistical value of class discrimination of radial velocity and radar cross section. By parameterizing fitting, logarithmic transformation and standardization of these features, a standardized feature score is generated, which can be directly compared and fused with the image confidence in a unified dimension. This extraction and utilization of the deep value of radar physical features fully releases the modality advantage of 4D millimeter wave radar, realizes the leap from shallow data alignment to deep feature and decision fusion, and effectively breaks through the bottleneck of low utilization rate of non-geometric information of radar in the prior art.​

[0034] The application innovatively applies the fusion generated 'fusion confidence' to the key point cloud sampling link of the downstream three-dimensional target detection network. The proposed 'fusion confidence guided farthest point sampling method' introduces the fused multi-modal confidence as a weight into the sampling distance measurement criterion, so that the sampling process can preferentially retain high-confidence foreground target points while maintaining spatial coverage, solving the problem of easy loss of long-distance sparse small targets. Experimental data show that after applying this method to the PV-RCNN framework on the VoD dataset, the average precision (mAP) of detection is improved from 44.637% and 68.657% to 54.027% and 76.213% in the entire annotation region and the region of interest, respectively, effectively verifying the significant effect of the method on improving target detection, especially difficult target detection accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0035] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the drawings needed in the specific embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0036] Figure 1 The 4D millimeter wave radar point cloud enhancement and target detection method based on Bayesian fusion provided for the present embodiment is shown in the flowchart;

[0037] Figure 2 The farthest point sampling method based on fusion confidence provided for the present embodiment is shown in the flowchart;

[0038] Figure 3 The test evaluation results of the method based on the VoD dataset provided for the present embodiment are shown in the table;

[0039] Figure 4 The visualization results of the method based on the VoD dataset provided for the present embodiment are shown in the figure. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0041] The embodiment provides a 4D millimeter wave radar point cloud enhancement and target detection method based on Bayesian fusion, please refer to Figure 1 The method comprises three core steps:

[0042] Firstly, the 4D millimeter wave radar point cloud is projected to an image plane through coordinate transformation (utilizing an external parameter matrix and an internal parameter matrix), and is matched with an image semantic mask and a confidence degree obtained through an instance segmentation network, so that each radar point is endowed with a preliminary image semantic category and a confidence degree.

[0043] Secondly, statistical probability models (conditional probabilities of categories) are constructed for different categories (such as cars and pedestrians) based on physical characteristics (absolute radial velocity and radar cross section) of the radar points, and in a Bayesian framework, the model is taken as a 'likelihood', and is weighted and fused with the image confidence degree (as a 'prior') in the previous step, so that a final 'fusion confidence degree' of each radar point belonging to each category is calculated.

[0044] Finally, the confidence degrees and categories are encoded as augmented features, and are spliced with the 4D millimeter wave radar point cloud to form a'semantic enhanced point cloud'. Before three-dimensional target detection is performed on the point cloud, an improved sampling method, that is, a farthest point sampling guided by the fusion confidence degree, is used to select key points, and then the key points are input into a PV-RCNN or the like three-dimensional detection network to complete training and reasoning.

[0045] The method above constructs a complete, probability inference-based deep fusion process, and unifies image semantics and radar physical characteristics at a decision level, instead of simple geometric matching or feature splicing. It solves the problems of excessive dependence on a single sensor, poor robustness and insufficient utilization of deep radar information in the prior art, realizes cross verification between sensors through probability fusion, and thus improves the overall precision and reliability of target detection in a complex scene.

[0046] The present application is implemented by the following contents:

[0047] The overall flowchart of the present application is shown in the following figure: Figure 1 The overall flowchart of the present application is shown in the following figure:

[0048] Step 1: Through coordinate mapping and mask screening, fusion and matching of radar point cloud and image segmentation data are realized

[0049] Consider a frame of 4D millimeter wave radar point cloud, which is a set composed of multiple radar points. We use to represent any radar point in it, wherein represents a coordinate vector of the radar point in a radar coordinate system, represents a radar cross section, represents a relative radial velocity, Indicates absolute radial velocity. This represents the fusion timestamp of the point cloud. (This is used to merge radar points.) Matching with the image mask requires transforming it from the radar coordinate system to the camera coordinate system. To satisfy matrix multiplication operations, the extrinsic parameter matrix obtained from the calibration of the radar to the camera is used. And the coordinate vector of the radar point in the radar coordinate system. Represented as a homogeneous coordinate vector, i.e. (superscript) (This indicates transpose). Finally, the coordinate vector of the radar point in the camera coordinate system. , can be represented as

[0050] (1)

[0051] Obtain the coordinate vector of the radar point in the camera coordinate system. Then, through the camera intrinsic parameter matrix Projecting onto the pixel coordinate system. This process involves two steps: first, calculating the homogeneous pixel coordinate vector of the normalized camera coordinates. It can be represented as

[0052] (2)

[0053] Final radar point pixel coordinates on the image It can be represented as

[0054] (3)

[0055] (4)

[0056] Finally, the input image is processed by a pre-trained deep learning instance segmentation network to obtain pixel-level prediction results. The instance segmentation network is configured to identify multiple preset target categories (cars, pedestrians, and cyclists), and output a binary mask corresponding to each target category. and split confidence .

[0057] The projected pixel coordinates calculated based on the aforementioned steps Determine whether the projected pixel coordinates fall within the binary mask region of each semantic category: if satisfied... Then the radar point is determined. Category And the corresponding image segmentation confidence score This serves as a reliable feature for image segmentation of the radar point.

[0058] Step two: build class prior model and perform Bayesian probability fusion

[0059] For each 4D millimeter wave radar point projected into the segmentation mask region, the present application builds a class conditional probability density model based on its physical observation values, including absolute radial velocity and radar cross section . By statistical analysis of the physical quantities of radar points of different classes, statistical distribution models of each class in physical characteristics are established. In the present application, it is observed that and The distribution shows unimodal characteristics, so Gaussian distribution model is used for parameterized fitting.

[0060] Based on the above analysis, the class conditional probability density function in absolute radial velocity is , which can be expressed as

[0061] (5)

[0062] where and are the statistical mean and standard deviation of the absolute radial velocity of all radar points belonging to class in the training data, which are obtained by offline statistics. is the standard mathematical symbol of the exponential function.

[0063] Similarly, the class conditional probability density function in radar cross section is , which can be expressed as

[0064] (6)

[0065] where , are the statistical mean and standard deviation of the radar cross section of all radar points belonging to class in the training data.

[0066] In order to solve the problem that the image segmentation confidence and the probability density of radar physical characteristics cannot be directly weighted, the present application uses a logarithmic normalization method to map the probability density of physical characteristics to a unified metric space. Specifically, first take the logarithm of the conditional probability of the radar point's absolute radial velocity and radar cross section to get the logarithmic likelihood value of the absolute radial velocity and the logarithmic likelihood value of the radar cross section, which can be expressed as

[0067] (7)

[0068] (8)

[0069] Subsequently, the distribution parameters are pre-calculated and the score standard score transformation is performed to eliminate the dimensional difference. The normalized speed feature score , can be expressed as

[0070] (9)

[0071] wherein represents the mean value of the samples belonging to the class in the training set in the logarithmic probability space, represents the corresponding standard deviation.

[0072] Similarly, the normalized radar cross section feature score , can be expressed as

[0073] (10)

[0074] wherein represents the mean value of the samples belonging to the class in the training set in the logarithmic probability space, represents the corresponding standard deviation.

[0075] The image segmentation confidence obtained in step one is weighted and fused with the normalized physical feature score to obtain the logarithmic form of the final fusion confidence score , which can be expressed as

[0076] (11)

[0077] wherein is the class confidence output by the image segmentation network in step one, and the logarithmic form is taken as the prior term of fusion. and are the fusion weights of the absolute radial velocity and the radar cross section, respectively, which are obtained by training and satisfy the normalization condition

[0078] (12)

[0079] Finally, the logarithmic fusion result is taken as an exponential operation to obtain the final fusion confidence score , which can be expressed as

[0080] (13)

[0081] The fusion confidence score represents an estimated value of the radar point projected on the instance after integrating the radar physical features .

[0082] Step three: generating a fusion enhanced point cloud and performing three-dimensional network training and target detection

[0083] The fusion confidence score calculated in step two, representing the multi-sensor fusion result and the semantic category code are spliced as augmented features with the 4D millimeter wave radar point cloud to construct an enhanced fusion point cloud. Specifically, for any one point in the enhanced point cloud , it can be represented as

[0084] (14)

[0085] The enhanced point cloud contains geometric, motion and deep semantic multi-dimensional information. It is fed as input data to a three-dimensional target detection network for feature extraction and target regression.

[0086] In the specific implementation of three-dimensional detection network training and reasoning, the present application selects PV-RCNN as the detection framework. In order to further improve the detection performance, especially to improve the feature extraction capability for sparse small targets, the present application improves the key point sampling strategy in the three-dimensional detection network and proposes a farthest point sampling method guided by fusion confidence. This method not only considers the spatial distance of the point when sampling, but also introduces the fusion confidence as a weight, so that the sampled key points can not only maintain uniform spatial distribution, but also tend to be distributed in areas with clear semantic information and high reliability. As shown in the accompanying Figure 2 , the specific process is as follows:

[0087] 1. Algorithm input

[0088] The input of the algorithm includes: a set of point cloud coordinates after fusion enhancement , wherein each point contains three-dimensional spatial coordinates and other physical attributes; the fusion confidence corresponding to each point calculated by the foregoing steps; and a preset number of sampling key points .

[0089] 2. Initialization

[0090] First, initialize an empty key point set , a distance array for recording the minimum distance of each point in the point cloud to the set ​(infinitely large initially), and an array of whether a landmark point has been visited. Then traverse the input point cloud, select the point with the highest fusion confidence score as the first key point, add it to the set , and update the distance array .

[0091] 3. Weighted iterative sampling

[0092] In each subsequent round of sampling, calculate the minimum Euclidean distance of the remaining points to the current key point set , the present application constructs the following weighted distance measurement formula, denoted as

[0093] (15)

[0094] wherein, is the normalized fusion confidence score (value range from 0 ~ 1), is the semantic guidance weight coefficient, used to adjust the influence degree of confidence on sampling, is the final comprehensive index used for point selection, which replaces the single geometric distance evaluation index.

[0095] According to the above formula, the radar points with high confidence, i.e. , will be calculated to have a larger weighted distance , so as to be preferentially selected under the farthest point sampling mechanism.

[0096] 4. Output

[0097] Repeat the above calculation, select the largest radar point to join the set , until the number of sampling points reaches the preset value . The final generated key point set takes into account the spatial coverage and target semantic information, which will be used as the input for feature interpolation and region of interest extraction in the subsequent PV-RCNN.

[0098] II. Test and evaluation of the method proposed by the invention

[0099] The method proposed by the invention is tested as follows:

[0100] All test data are based on the VoD (View-of-Delft, VoD) dataset, which has a total of 8693 frame scenes, and the dataset is divided into training, validation and test sets in the proportions of 59%, 15% and 26%, and the labeled targets include pedestrians, cyclists, cars, etc. During all training processes, three classes of cars (Car), pedestrians (Pedestrian) and cyclists (Cyclist) are trained and validated. Under the VoD dataset, the mean average precision (mAP) is selected as the evaluation index, and the performance of various existing target detection algorithms in the entire annotation area and the area of interest is tested. The entire annotation area refers to a 50m fan-shaped area in front of the camera, and the area of interest refers to a specific area in the driving space near the front of the vehicle in the camera coordinate system, i.e. .

[0101] The test uses the PyTorch framework, and the training and evaluation are carried out in a hardware environment equipped with a 4-card 4090 GPU and an Intel i7-12700K CPU. The network is trained for 80 epochs using the Adam optimizer, and the small batch size is set to 4. The initial learning rate is 0.001, and the learning rate is dynamically adjusted through the OneCycle strategy. The number of key point sampling points is set to 512.

[0102] The Figure 3 The evaluation results of the present application on the VoD dataset are shown, and are compared with other existing target detection algorithms. The results show that the present application has obvious advantages based on the benchmark model PV-RCNN, and the mAP value is increased from 44.637% and 68.657% of the benchmark model to 54.027% and 76.213%, respectively. Figure 4 The visual detection results of the present application on the VoD dataset are shown, and it can be seen from the figure that in a complex road scene, the present application can accurately identify the targets in the scene.

[0103] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or make equivalent substitutions for part or all of the technical features; and these modifications or substitutions do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for enhancing 4D millimeter-wave radar point clouds and detecting targets based on Bayesian fusion, characterized in that, Includes the following steps: Step 1: Project the 4D millimeter-wave radar point cloud onto the image plane and match it with the image instance segmentation results to obtain the image semantic category and image segmentation confidence of each radar point; Step 2: Based on the physical features of radar points in the 4D millimeter-wave radar point cloud, construct class-conditional probability density models for different semantic categories to obtain the probability density of physical features for the corresponding semantic categories; map the probability density of the physical features to the logarithmic space and perform standardization to obtain the standardized log-likelihood value; in the logarithmic domain, use the image segmentation confidence as a prior term and perform weighted fusion with the standardized log-likelihood value to obtain the fusion confidence of each radar point belonging to each semantic category; the physical features include absolute radial velocity and radar cross-section. Step 3: The fusion confidence score and the corresponding semantic category encoding are used as augmenting features and concatenated with the 4D millimeter-wave radar point cloud to generate a semantically enhanced point cloud; the semantically enhanced point cloud is sampled using a key point sampling method guided by fusion confidence score, and the sampled point cloud is input into a 3D target detection network for training and inference to achieve target detection.

2. The 4D millimeter-wave radar point cloud enhancement and target detection method based on Bayesian fusion according to claim 1, characterized in that, Step one specifically includes: The radar point coordinates are transformed to the camera coordinate system using the extrinsic parameter matrix from the radar coordinate system to the camera coordinate system. By using the camera intrinsic parameter matrix, the radar point coordinates in the camera coordinate system are projected onto the image pixel coordinate system to obtain the projected pixel coordinates of the radar point on the image. Based on the binary mask of each semantic category in the image obtained by the instance segmentation network, it is determined whether the projected pixel coordinates fall within the binary mask region of each semantic category. If so, the image semantic category corresponding to the radar point is determined, and the segmentation confidence corresponding to the category is used as the image segmentation confidence.

3. The 4D millimeter-wave radar point cloud enhancement and target detection method based on Bayesian fusion according to claim 1, characterized in that, Step two, specifically, involves constructing class-conditional probability density models for different semantic categories, including: For each semantic category, based on the statistical distribution of the absolute radial velocity and radar cross-section of radar points belonging to that category in the training data, a Gaussian distribution model is used to parameterize and fit the class probability density of the absolute radial velocity and radar cross-section respectively.

4. The 4D millimeter-wave radar point cloud enhancement and target detection method based on Bayesian fusion according to claim 3, characterized in that, In step two, within the logarithmic domain, the image segmentation confidence is used as a priori and weighted with the standardized log-likelihood value to obtain the fused confidence of each radar point belonging to each semantic category. Specifically, this includes: For radar points, they belong to semantic categories. Fusion confidence Its logarithmic form satisfy: ； in, Segment the image with confidence; The velocity feature score is calculated and standardized based on the class conditional probability of absolute radial velocity; The radar cross-section feature score is calculated and standardized based on the class conditional probability of radar cross-section. and The fusion weights are obtained through training and satisfy the following conditions: .

5. The 4D millimeter-wave radar point cloud enhancement and target detection method based on Bayesian fusion according to claim 4, characterized in that, The velocity feature score The calculation formula is: ； in, The absolute radial velocity calculated based on the Gaussian distribution model Class-conditional probability density; Indicates that the training set belongs to category The mean of the samples in the logarithmic probability space. This represents the corresponding standard deviation.

6. The 4D millimeter-wave radar point cloud enhancement and target detection method based on Bayesian fusion according to claim 4, characterized in that, Radar cross-sectional feature score The calculation formula is: ； in, , Radar cross-section calculated based on Gaussian distribution model Class-conditional probability density; Indicates that the training set belongs to category The mean of the samples in the logarithmic probability space. This represents the corresponding standard deviation.

7. The 4D millimeter-wave radar point cloud enhancement and target detection method based on Bayesian fusion according to any one of claims 1-6, characterized in that, In step three, the keypoint sampling method guided by fusion confidence is an improved farthest point sampling method, and its sampling weighted distance metric formula is as follows: ； in, It is the minimum Euclidean distance from the current point to be examined to the existing set of key points; These are preset semantic guidance weight coefficients; This is the normalized fusion confidence score, with a value ranging from 0 to 1; For the comprehensive weighted distance used for sampling, in each iteration, select The largest point is selected as the new key point.

8. The 4D millimeter-wave radar point cloud enhancement and target detection method based on Bayesian fusion according to claim 7, characterized in that, The three-dimensional target detection network uses PV-RCNN as the detection framework and improves the key point sampling strategy in the network using the improved farthest point sampling method.

Images