Target positioning method and system based on radar photoelectric fusion technology

Abstract

The invention relates to the technical field of target positioning, and particularly discloses a target positioning method and system based on a radar photoelectric fusion technology, and the method comprises the steps: collecting the radar data and image data of a target; the radar data and the photoelectric image data are processed, micro-Doppler features are extracted from the radar data, and PCA dimension reduction processing is performed on the photoelectric image data; time synchronization is realized, the photoelectric sensor is calibrated, and a space conversion model of the radar and the photoelectric sensor is established to realize space registration; carrying out data layer fusion, feature layer fusion and decision-making layer fusion; based on the fused data, adopting an EKF algorithm to carry out target positioning, and adopting an MHT algorithm to carry out multi-target tracking management; and constructing a performance evaluation index system, and optimizing system parameters according to a performance evaluation result by adopting a self-adaptive parameter adjustment algorithm. The accurate positioning tracking algorithm and the performance optimization mechanism guarantee the stable and efficient operation of the system.

Description

Technical Field

[0001] The present invention relates to the field of target positioning technology, and in particular to a target positioning method and system based on radar optoelectronic fusion technology. Background Art

[0002] In the field of target positioning, radar technology, by emitting electromagnetic waves and receiving echoes, can obtain target distance, speed, and angle information, enabling long-range detection and operation in adverse weather conditions. However, radar resolution is limited, making it difficult to provide accurate and detailed information for small targets or scenarios requiring fine feature recognition. Photoelectric sensors, such as cameras and infrared detectors, can capture high-resolution images, clearly displaying features such as target shape, color, and texture, facilitating target identification and classification. However, photoelectric sensors are significantly constrained by the environment, and their performance can be severely degraded or even fail in adverse conditions such as strong light, smoke, and dust.

[0003] Currently, most target positioning systems rely solely on radar or optoelectronic sensors, failing to integrate the strengths of both. In scenarios such as military reconnaissance, intelligent transportation, and security monitoring, single-sensor positioning systems frequently experience misjudgments and missed detections when faced with complex environments and similar targets. For example, during military operations, radar struggles to accurately identify target types in inclement weather, while optoelectronic equipment is often obstructed by the weather. In intelligent transportation, optoelectronic sensors are prone to misidentifying vehicles in strong direct sunlight, while radar struggles to accurately locate small vehicles. These issues severely impact positioning accuracy and reliability, making them incapable of meeting practical application requirements. Summary of the Invention

[0004] The present invention aims to provide a target positioning method and system based on radar optoelectronic fusion technology to solve the problem of poor accuracy and reliability of existing single sensor target positioning in complex environments.

[0005] The purpose of the present invention can be achieved through the following technical solutions:

[0006] A target positioning method based on radar optoelectronic fusion technology includes the following steps:

[0007] S1: Integrates radar sensors and multiple optoelectronic sensors to collect radar data and image data of the target;

[0008] S2: Perform wavelet denoising on the radar data, perform median filtering on the photoelectric image data, extract micro-Doppler features from the radar data, and perform PCA dimensionality reduction on the photoelectric image data;

[0009] S3: GPS timing and hardware clock synchronization technology are used to achieve time synchronization, Zhang's calibration method is used to calibrate the photoelectric sensor, and a spatial conversion model of the radar and photoelectric sensor is established to achieve spatial registration;

[0010] S4: pixel-level fusion method combined with wavelet transform is used for data layer fusion, SVM algorithm is used for feature layer fusion, and DS evidence theory is used for decision layer fusion;

[0011] S5: Based on the fused data, the EKF algorithm is used for target positioning, and the MHT algorithm is used for multi-target tracking management;

[0012] S6: Build a performance evaluation index system and use an adaptive parameter adjustment algorithm to optimize system parameters based on the performance evaluation results.

[0013] As a further solution of the present invention: in said S2, the following steps are specifically included:

[0014] Assume that the original radar signal is x(n), and the wavelet coefficient d of the jth layer is obtained through wavelet decomposition. j (n) and scale factor c j (n), the wavelet coefficients are processed according to the threshold rule λ, and the processed wavelet coefficients are The calculation formula is as follows:

[0015]

[0016] Then the denoised radar signal is reconstructed by inverse wavelet transform

[0017] Assume that the neighborhood of a pixel point (i, j) in the image data is N(i, j), and the set of pixel values ​​in the neighborhood is {p k}, k=1,2,...,m, then the value of the pixel after filtering is for:

[0018]

[0019] As a further solution of the present invention: in said S2, it further includes:

[0020] Assume that the radar echo signal is s(t). After time-frequency analysis, the time-frequency distribution S(t,f) is obtained. The micro-Doppler feature can be obtained by feature extraction from S(t,f).

[0021] Let the image data matrix be X, and its mean is The covariance matrix is ​​C:

[0022]

[0023] Where N is the number of samples, x i is the i-th sample;

[0024] Perform eigenvalue decomposition on the covariance matrix C to obtain the eigenvalue λ iand the corresponding eigenvector e i , select the eigenvectors corresponding to the first k largest eigenvalues ​​to form the projection matrix P, and project the original image data into the new low-dimensional space to obtain the principal component feature Y:

[0025]

[0026] T represents the transpose operator.

[0027] As a further solution of the present invention: in said S3, the following steps are specifically included:

[0028] Assume that the radar acquisition time is t r , the photoelectric acquisition time is t o , the time synchronization error Δt satisfies:

[0029] |Δt|=|t r -t o |≤∈where∈ is the maximum allowed time error;

[0030] Assume that the coordinates of the point in the radar coordinate system are (x r ,y r ,z r ), the point coordinates in the photoelectric coordinate system are (x o ,y o ,z o ), the conversion relationship is:

[0031]

[0032] Where R represents the rotation matrix and T represents the translation vector.

[0033] As a further solution of the present invention: in said S4, the following steps are specifically included:

[0034] Using the fusion algorithm based on wavelet transform, the radar range image I r and photoelectric image I o Decompose into sub-bands of different frequencies, and let the low-frequency sub-band after the j-th layer wavelet decomposition be and The high frequency subband is and

[0035] According to the energy criterion, the sub-band coefficients are selected for fusion, and the fused low-frequency sub-band and high frequency sub-band The calculation formula is as follows:

[0036]

[0037]

[0038] Where E() represents the energy of the subband;

[0039] The fused image is reconstructed by inverse wavelet transform

[0040] As a further solution of the present invention: in said S4, it further includes:

[0041] Assume that the training sample set is where x i is the eigenvector, y i ∈{-1, 1} is the category label, and the following optimization problem is solved:

[0042]

[0043] Where ω is the weight vector, b is the bias, ξ i is the slack variable, C is the penalty factor, is the feature mapping function. By solving the optimization problem, we can get the optimal ω and b, thus achieving feature selection and fusion.

[0044] As a further solution of the present invention: in said S4, it further includes:

[0045] Use radar data and optoelectronic data to locate and identify targets respectively, and obtain their respective decision results D r and D o ;

[0046] DS evidence theory is used for decision-making layer fusion, and the basic probability distribution function of radar decision results is assumed to be m r , the basic probability distribution function of the optoelectronic decision result is m o , the fused basic probability distribution function m can be obtained by D empster Combination rule calculation:

[0047]

[0048] Where K = ∑ B∩C＝φ m r (B)m o (C) is the conflict coefficient, and φ represents the empty set.

[0049] A target positioning system based on radar optoelectronic fusion technology, comprising:

[0050] Data acquisition module: Integrates radar sensors and multiple photoelectric sensors to collect radar data and image data of the target;

[0051] Data preprocessing module: performs wavelet denoising on radar data, median filtering on photoelectric image data, extracts micro-Doppler features from radar data, and performs PCA dimensionality reduction on photoelectric image data;

[0052] Time and space calibration module: GPS timing and hardware clock synchronization technology are used to achieve time synchronization, Zhang calibration method is used to calibrate the photoelectric sensor, and a spatial conversion model of radar and photoelectric sensor is established to achieve spatial registration;

[0053] Fusion processing module: pixel-level fusion method combined with wavelet transform is used for data layer fusion, SVM algorithm is used for feature layer fusion, and DS evidence theory is used for decision layer fusion;

[0054] Target positioning and tracking module: Based on the fused data, the EKF algorithm is used for target positioning, and the MHT algorithm is used for multi-target tracking management;

[0055] Performance evaluation and optimization module: Build a performance evaluation index system and use an adaptive parameter adjustment algorithm to optimize system parameters based on performance evaluation results.

[0056] The present invention's beneficial effects: By deeply integrating radar and optoelectronic technologies and combining multiple advanced algorithms, the system significantly improves the accuracy and reliability of target positioning in complex environments. Multi-source data acquisition and preprocessing ensure data quality, while spatiotemporal calibration and registration achieve data temporal and spatial unification. A multi-level fusion strategy leverages the strengths of both technologies, while precise positioning and tracking algorithms and performance optimization mechanisms ensure stable and efficient system operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The present invention will be further described below with reference to the accompanying drawings.

[0058] Figure 1 It is a flow chart of a target positioning method based on radar optoelectronic fusion technology of the present invention. DETAILED DESCRIPTION

[0059] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0060] See also Figure 1 As shown, the present invention is a target positioning method based on radar optoelectronic fusion technology, comprising the following steps:

[0061] Multi-source data collection and preprocessing:

[0062] Data acquisition: Radar sensors are used to obtain radar data such as the target's distance r, speed v, angle θ, etc. At the same time, photoelectric sensors are used to collect target image data, covering visible light images and infrared images, to ensure that comprehensive target information is obtained.

[0063] Data cleaning:

[0064] Radar data cleaning: For radar data, wavelet denoising algorithm is used. Let the original radar signal be x(n), and the wavelet coefficient d of the jth layer is obtained through wavelet decomposition. j (n) and scale factor c j (n), the wavelet coefficients are processed according to the threshold rule λ, and the processed wavelet coefficients are The calculation formula is as follows:

[0065]

[0066] Then the denoised radar signal is reconstructed by inverse wavelet transform

[0067] Photoelectric image data cleaning: Use the median filter algorithm to replace the central pixel value with the median of the neighborhood pixels to remove salt and pepper noise, etc. Suppose the neighborhood of a pixel point (i, j) in the image is N(i, j), and the pixel value set in the neighborhood is {p k}, k=1, 2, ..., m, then the value of the pixel after filtering for:

[0068]

[0069] Feature extraction:

[0070] Radar data feature extraction: Extract micro-Doppler features from radar data to analyze the micro-motion characteristics of the target, such as the target's rotation, vibration, etc. Let the radar echo signal be s(t). After time-frequency analysis (such as short-time Fourier transform), the time-frequency distribution S(t,f) is obtained. The micro-Doppler feature can be obtained by feature extraction of S(t,f), for example, extracting the peak frequency f of the time-frequency distribution. peak and other features.

[0071] Photoelectric image data feature extraction: The principal component analysis (PCA) algorithm is used to reduce the dimension of the photoelectric image data and extract the principal component features. Let the image data matrix be X, and its mean is The covariance matrix is ​​C:

[0072]

[0073] Where N is the number of samples, x iis the i-th sample. Perform eigenvalue decomposition on the covariance matrix C and obtain the eigenvalue λ i and the corresponding eigenvector e i , select the eigenvectors corresponding to the first k largest eigenvalues ​​to form the projection matrix P, and project the original image data into the new low-dimensional space to obtain the principal component feature Y:

[0074] T represents the transpose operator.

[0075] Spatiotemporal calibration and registration:

[0076] Time synchronization: Build a high-precision time synchronization system, use the precise timing function of the global positioning system (GPS), and combine it with hardware clock synchronization technology to ensure the time consistency of radar and photoelectric sensor data collection, with the error controlled at the microsecond level to avoid positioning deviation caused by time difference. Assume that the radar collection time is t r , the photoelectric acquisition time is t o , the time synchronization error Δt satisfies:

[0077] |Δt|=|t r -t o |≤∈;

[0078] Where ∈ is the maximum allowed time error, which is generally controlled at the microsecond level.

[0079] Spatial registration: Based on the calibration plate or known feature points, the photoelectric sensor is calibrated using Zhang's calibration method to obtain its internal and external parameters. By establishing a spatial transformation model between the radar and the photoelectric sensor, the rotation matrix R and the translation vector T are used to realize the transformation between the radar coordinate system and the photoelectric coordinate system. Assume that the coordinates of the point in the radar coordinate system are (x r ,y r ,z r ), the point coordinates in the photoelectric coordinate system are (x o ,y o ,z o ), the conversion relationship is:

[0080]

[0081] Fusion strategy and model building:

[0082] Data layer fusion: pixel-level fusion method is used to fuse radar range image and photoelectric image after time and space calibration. Using the fusion algorithm based on wavelet transform, radar range image Ir and photoelectric image I o Decompose into sub-bands of different frequencies, and let the low-frequency sub-band after the j-th layer wavelet decomposition be and The high frequency subband is and

[0083] According to the energy criterion, the sub-band coefficients are selected for fusion, and the fused low-frequency sub-band and high frequency sub-band The calculation formula is as follows:

[0084]

[0085]

[0086] Where E() represents the energy of the subband. Then the fused image is reconstructed by inverse wavelet transform.

[0087] Feature layer fusion: combining radar distance, speed, and angle features F r =(r, v, θ) and the shape, color, and texture features of the photoelectric image F o , use the support vector machine (SVM) algorithm for feature selection and fusion. Assume that the training sample set is where x i is the eigenvector, y i ∈{-1, 1} is the category label, and the goal of SVM is to solve the following optimization problem:

[0088]

[0089] Where ω is the weight vector, b is the bias, ξ i is the slack variable, C is the penalty factor, and φ(x) is the feature mapping function. By solving this optimization problem, the optimal ω and b are obtained, thereby achieving feature selection and fusion.

[0090] Decision layer fusion: radar data and optoelectronic data are used to locate and identify targets respectively, and their respective decision results D are obtained. r and D o DS evidence theory is used for decision-making layer fusion, and the basic probability distribution function of radar decision results is assumed to be m r , the basic probability distribution function of the optoelectronic decision result is m o , the fused basic probability distribution function m can be obtained by D empster Combination rule calculation:

[0091]

[0092] Where K = ∑ B∩C＝φ m r (B)m o (C) is the conflict coefficient, and φ represents the empty set.

[0093] Target positioning and tracking:

[0094] Positioning algorithm: Based on the fused data, the extended Kalman filter (EKF) algorithm is used to locate the target.

[0095] Let the target state vector be x k , the state transfer equation is x k+1 =f(x k ,u k )+w k , the measurement equation is z k =h(x k )+v k , where u k is the control input, w k is the process noise, v k is the measurement noise. The steps of the EKF algorithm are as follows:

[0096] Prediction steps:

[0097]

[0098]

[0099] in, is the predicted state, P k|k-1 is the forecast error covariance matrix, is the state transfer matrix, Q k-1 is the process noise covariance matrix.

[0100] Update steps:

[0101]

[0102] P k|k =(IK k H k )P k|k-1 ;

[0103] where K k is the Kalman gain, is the measurement matrix, R k is the measurement noise covariance matrix.

[0104] Tracking Management: Build a target tracking management system and use the Multiple Hypothesis Tracking (MHT) algorithm to handle multi-target tracking. The MHT algorithm generates multiple hypothesized trajectories and updates, merges, and deletes them based on new measurement data.

[0105] Let the state vector of the i-th hypothesis trajectory be x i , the measurement vector is z, and the updated probability of the trajectory can be calculated by the Bayesian formula:

[0106]

[0107] Where P(z|x i ) is the likelihood function, P(x i ) is the prior probability.

[0108] Performance evaluation and feedback optimization:

[0109] Performance evaluation: Build a performance evaluation index system, including indicators such as positioning accuracy, recognition accuracy, and tracking stability.

[0110] Positioning accuracy: Let the actual position of the target be x true , the estimated position is x est , positioning error ε pos for:

[0111] ε pos =||X true -X est ||;

[0112] The average positioning error is Where N is the number of samples.

[0113] Recognition accuracy: Let the number of correctly identified targets be n correct , the total number of targets is n total , recognition accuracy A cc for:

[0114] A cc =n correct / n total ;

[0115] Tracking stability: It can be evaluated through indicators such as the continuity of the tracking trajectory and the number of times the target is lost.

[0116] Feedback optimization: Based on the performance evaluation results, an adaptive parameter adjustment algorithm is used to optimize the system parameters. For example, the process noise covariance Q and measurement noise covariance R of the Kalman filter are adjusted according to the positioning error.

[0117] Adjust the kernel parameters of the SVM model according to the recognition accuracy. Let the performance index be J, the parameter vector be θ, and use the gradient descent method to adjust the parameters:

[0118]

[0119] Where α is the learning rate.

[0120] The above is a detailed description of an embodiment of the present invention. However, the content described is only a preferred embodiment of the present invention and should not be considered to limit the scope of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A target positioning method based on radar optoelectronic fusion technology, characterized in that: The following steps are involved: S1: Integrates radar sensors and multiple optoelectronic sensors to collect radar data and image data of the target; S2: Perform wavelet denoising on the radar data, perform median filtering on the photoelectric image data, extract micro-Doppler features from the radar data, and perform PCA dimensionality reduction on the photoelectric image data; S3: GPS timing and hardware clock synchronization technology are used to achieve time synchronization, Zhang's calibration method is used to calibrate the photoelectric sensor, and a spatial conversion model of the radar and photoelectric sensor is established to achieve spatial registration; S4: pixel-level fusion method combined with wavelet transform is used for data layer fusion, SVM algorithm is used for feature layer fusion, and DS evidence theory is used for decision layer fusion; S5: Based on the fused data, the EKF algorithm is used for target positioning, and the MHT algorithm is used for multi-target tracking management; S6: Build a performance evaluation index system and use an adaptive parameter adjustment algorithm to optimize system parameters based on the performance evaluation results.

2. The target positioning method based on radar optoelectronic fusion technology according to claim 1, characterized in that: In said S2, the following steps are specifically included: Assume that the original radar signal is x(n), and the wavelet coefficient d of the jth layer is obtained through wavelet decomposition. j (n) and scale factor c j (n), the wavelet coefficients are processed according to the threshold rule λ, and the processed wavelet coefficients are The calculation formula is as follows: Then the denoised radar signal is reconstructed by inverse wavelet transform Assume that the neighborhood of a pixel point (i, j) in the image data is N(i, j), and the set of pixel values ​​in the neighborhood is {p k }, k=1,2,...,m, then the value of the pixel after filtering is for:

3. The target positioning method based on radar optoelectronic fusion technology according to claim 2, characterized in that: In said S2, it further includes: Assume that the radar echo signal is s(t). After time-frequency analysis, the time-frequency distribution S(t,f) is obtained. The micro-Doppler feature can be obtained by feature extraction from S(t,f). Let the image data matrix be X, and its mean is The covariance matrix is ​​C: Where N is the number of samples, x i is the i-th sample; Perform eigenvalue decomposition on the covariance matrix C to obtain the eigenvalue λ i and the corresponding eigenvector e i , select the eigenvectors corresponding to the first k largest eigenvalues ​​to form the projection matrix P, and project the original image data into the new low-dimensional space to obtain the principal component feature Y: T represents the transpose operator.

4. The target positioning method based on radar optoelectronic fusion technology according to claim 1, characterized in that: In the S3, the following steps are specifically included: Assume that the radar acquisition time is t r , the photoelectric acquisition time is t o , the time synchronization error Δt satisfies: |Δt|=|t r -t o |≤∈; Where ∈ is the maximum allowed time error; Assume that the coordinates of the point in the radar coordinate system are (x r ,y r ,z r ), the point coordinates in the photoelectric coordinate system are (x o ,y o ,z o ), the conversion relationship is: Where R represents the rotation matrix and T represents the translation vector.

5. The target positioning method based on radar optoelectronic fusion technology according to claim 1, characterized in that: In said S4, the following steps are specifically included: Using the fusion algorithm based on wavelet transform, the radar range image I r and photoelectric image I o Decompose into sub-bands of different frequencies, and let the low-frequency sub-band after the j-th layer wavelet decomposition be and The high frequency subband is and According to the energy criterion, the sub-band coefficients are selected for fusion, and the fused low-frequency sub-band and high frequency sub-band The calculation formula is as follows: Where E() represents the energy of the subband; The fused image is reconstructed by inverse wavelet transform 6. The target positioning method based on radar optoelectronic fusion technology according to claim 1, characterized in that: In said S4, it further includes: Assume that the training sample set is where x i is the eigenvector, y i ∈{-1,1} is the category label, and the following optimization problem is solved: Where ω is the weight vector, b is the bias, ξ i is the slack variable, C is the penalty factor, is the feature mapping function. By solving the optimization problem, we can get the optimal ω and b, thus achieving feature selection and fusion.

7. The target positioning method based on radar optoelectronic fusion technology according to claim 1, characterized in that: In said S4, it further includes: Use radar data and optoelectronic data to locate and identify targets respectively, and obtain their respective decision results D r and D o ; DS evidence theory is used for decision-making layer fusion, and the basic probability distribution function of radar decision results is assumed to be m r , the basic probability distribution function of the optoelectronic decision result is m o , the fused basic probability distribution function m can be obtained by D empster Combination rule calculation: Where K = ∑ B∩C＝φ m r (B)m o (C) is the conflict coefficient, and φ represents the empty set.

8. A target positioning system based on radar optoelectronic fusion technology, characterized in that: include: Data acquisition module: Integrates radar sensors and multiple photoelectric sensors to collect radar data and image data of the target; Data preprocessing module: performs wavelet denoising on radar data, median filtering on photoelectric image data, extracts micro-Doppler features from radar data, and performs PCA dimensionality reduction on photoelectric image data; Time and space calibration module: GPS timing and hardware clock synchronization technology are used to achieve time synchronization, Zhang calibration method is used to calibrate the photoelectric sensor, and a spatial conversion model of radar and photoelectric sensor is established to achieve spatial registration; Fusion processing module: pixel-level fusion method combined with wavelet transform is used for data layer fusion, SVM algorithm is used for feature layer fusion, and DS evidence theory is used for decision layer fusion; Target positioning and tracking module: Based on the fused data, the EKF algorithm is used for target positioning, and the MHT algorithm is used for multi-target tracking management; Performance evaluation and optimization module: Build a performance evaluation index system and use an adaptive parameter adjustment algorithm to optimize system parameters based on performance evaluation results.

Images