An intelligent target detection system based on multi-sensor data fusion

Abstract

The application discloses a kind of intelligent target detection systems based on multi-sensor data fusion, specifically related to intelligent target detection field, including heterogeneous sensor module, data fusion module, self-calibration module;The present application is at night or low light, infrared thermal imaging dominant target detection, visible light camera data weight is down-regulated;Under strong electromagnetic interference, laser radar enables polarization filtering, millimeter wave radar frequency hopping anti-interference;Data fusion uses three-level architecture: one is based on improved D-S evidence theory to solve cross-modal data conflict;Second, for sampling frequency and position difference, with space-time stamp compensation algorithm alignment;Third, use the weight distribution of bidirectional LSTM neural network of attention mechanism;When the confidence of sensor is low, start particle filtering compensation mechanism, correct sensor deviation, carry out health assessment and predictive maintenance, record calibration historical data, calculate health index, predict remaining useful life, below threshold value when triggering alarm.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent target detection, more particularly, to an intelligent target detection system based on multi-sensor data fusion. BACKGROUND

[0002] In many fields such as autonomous driving, intelligent security, military reconnaissance, etc., accurate detection and recognition of targets are of great importance. The limitations of traditional single-sensor systems in complex environments are increasingly evident. For example, in the autonomous driving scenario, relying solely on cameras may not accurately perceive the surrounding environment at night or in adverse weather conditions; while relying solely on radar may not accurately identify target details due to insufficient resolution. In the face of complex and variable environments such as strong electromagnetic interference, low light, high-speed moving targets, etc., single-sensor systems often fail to meet the high-precision, high-reliability detection requirements, with problems such as low detection accuracy, high false alarm rate, and poor adaptability.

[0003] The data obtained by different sensors often has complementary nature; multi-sensor data fusion technology emerges as the times require, by integrating information from multiple sensors, it can make full use of the advantages of each sensor, make up for the defects of single sensor, so as to improve the accuracy, reliability and robustness of target detection. SUMMARY

[0004] In order to overcome the above-mentioned defects of the prior art, the embodiments of the present application provide an intelligent target detection system based on multi-sensor data fusion to solve the problems raised in the background art.

[0005] To achieve the above-mentioned purpose, the present application provides the following technical scheme:

[0006] Heterogeneous sensor module: containing radar unit, infrared thermal imaging unit, laser radar unit and visible light camera unit;

[0007] Data fusion module: adopting a three-level fusion architecture:

[0008] First level: sensor-level data conflict resolution module based on improved D-S evidence theory, cross-modal data confidence assessment is realized by introducing dynamic trust weight factor;

[0009] Second level: multi-scale spatio-temporal alignment for target features, using spatio-temporal stamp compensation algorithm to eliminate the differences in sampling frequency and spatial position between sensors;

[0010] Third level: bidirectional LSTM neural network based on attention mechanism, dynamically allocating the contribution weight of different sensors in the feature extraction stage;

[0011] Self-calibration module: when detecting that the single sensor confidence is continuously lower than the threshold value, a sensor bias compensation mechanism based on particle filtering is started.

[0012] Preferably, in the isometric sensor module, under night or low light conditions, the sampling frequency of the infrared thermal imaging module is set to 30Hz, the non-uniformity correction algorithm is used to suppress thermal noise, and the data weight of the visible light camera is dynamically reduced to below 0.2 through a weighted fusion algorithm to ensure that the infrared data dominates target detection; if there is strong electromagnetic interference in the environment, the polarization filtering mode is immediately enabled for the laser radar unit, combined with Stokes parameter analysis to filter out stray light interference, and the millimeter wave radar triggers the frequency hopping anti-interference protocol, switches the carrier frequency in the 76-81GHz frequency band in a pseudo-random sequence, and assists with space-time adaptive processing technology to suppress sidelobe noise.

[0013] Preferably, in the data fusion module, the first level is based on an improved D-S evidence theory, and the cross-modal data conflict problem is solved through a dynamic trust weight factor; in the traditional D-S theory, the basic probability assignment function has limitations in processing conflicting evidence, and the real-time sensor confidence evaluation mechanism is introduced; the trust weight of the ith sensor is defined as a time-varying function, considering its signal-to-noise ratio, historical accuracy and environmental adaptability;

[0014] The second level designs a joint calibration algorithm based on space-time stamp compensation for the sampling frequency difference (radar 100Hz vs. infrared 30Hz) and installation position difference between sensors:

[0015] For sensor data that arrives with a delay, such as the delay caused by the scanning period of the laser radar, a cubic spline interpolation is used to reconstruct the synchronous time sequence: LiDAR represents the laser radar;

[0016]

[0017] where L(t) represents the estimated value of the laser radar data obtained by the interpolation method at time t; K represents the total number of reflection points received by the LiDAR before time t; φ k represents the basis function, a k represents the weight of the kth reflection point; t k represents the reception time of the kth reflection point, represents the time when the kth reflection point is received by the LiDAR; Δt represents the time scale parameter, which is used to normalize the time difference t-t k , controls the shape and decay rate of the basis function;

[0018] The bidirectional LSTM neural network of the attention mechanism in the third level includes:

[0019] Temporal feature extraction layer: local temporal features of multi-sensor data are extracted by using a cavity convolution layer;

[0020] Cross-modal attention layer: the feature correlation between different sensor channels is calculated through a learnable parameter matrix to generate dynamic attention weights;

[0021] Decision fusion layer: the weighted feature sequence is classified by the integrated gated recurrent unit to output structured data containing target type, spatial coordinates and motion vector.

[0022] Preferably, in the self-calibration module, after target recognition, the real-time confidence of each sensor is continuously evaluated;

[0023] The confidence calculation adopts a weighted exponential moving average method, and the calculation method is specifically as follows:

[0024] H i (t3)=μSNR i (t3)+(1-μ)H i (t3-1)

[0025] Wherein, H i (t) represents the confidence at time step t3, μ represents the forgetting factor, SNR i (t3) represents the signal-to-noise ratio of the i-th sensor at time step t3, H i (t3-1) represents the confidence at time step t3-1.

[0026] When the H i (t3) of a certain sensor is lower than the threshold value for 5 consecutive periods, it is determined that the confidence is abnormal, and the online calibration process is triggered;

[0027] When the sensor confidence is abnormal, the system starts a dynamic compensation mechanism driven by particle filtering. First, it is assumed that there is a systematic deviation between the sensor measurement value and the true value, and then the deviation estimate is output by particle sampling and updating. The method steps are specifically as follows:

[0028] Step A1: initialize M particles subject to Gaussian distribution

[0029] Step A2: calculate the weight of each particle;

[0030] Step A3: resample to avoid particle degradation and retain high-weight particles;

[0031] Step A4: output the deviation estimate;

[0032] The output deviation estimation value is fed back to the sensor data stream, the corrected observation value is calculated, the health assessment and predictive maintenance of the sensor are carried out after the sensor deviation compensation, the calibration history data of each sensor is recorded for a long time, the health index is calculated, and the remaining service life is predicted, the LSTM network is used to learn the sensor degradation trend, and the input includes: historical deviation sequence, environmental stress data; the output is the probability distribution of RUL, and the maintenance alarm is triggered when the predicted life is lower than the threshold.

[0033] Technical effects and advantages of the present application:

[0034] The present application can detect the target by infrared thermal imaging under low light, and the weight of the visible light camera data is reduced; under strong electromagnetic interference, the polarization filter of the laser radar is enabled, and the frequency hopping anti-interference of the millimeter wave radar; when the high-speed moving target is tracked, the radar-laser radar cooperates. The data fusion adopts a three-level architecture: the first level solves the cross-modal data conflict based on the improved D-S evidence theory; the second level aligns the time and space stamp compensation algorithm according to the sampling frequency and position difference; the third level uses the attention mechanism of the bidirectional LSTM neural network to allocate weights. The self-calibration module starts the particle filter compensation mechanism when the sensor confidence is low, corrects the sensor deviation, and then performs health assessment and predictive maintenance, records the calibration history data, calculates the health index, and predicts the remaining useful life, and triggers the alarm when it is lower than the threshold; the present application improves the accuracy and reliability of target detection and tracking through multi-modal sensor fusion, efficient data fusion architecture, sensor deviation compensation and self-calibration mechanism. BRIEF DESCRIPTION OF DRAWINGS

[0035] Figure 1 The module schematic diagram of the present application.

[0036] Figure 2 The bidirectional LSTM neural network schematic diagram of the present application.

[0037] Figure 3 The self-calibration module flowchart of the present application. DETAILED DESCRIPTION

[0038] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0039] Please refer to Figure 1 As shown in the figure, the present application provides an intelligent target detection system based on multi-sensor data fusion, which comprises a heterogeneous sensor module, a data fusion module and a self-calibration module.

[0040] Heterogeneous sensor module: containing radar unit, infrared thermal imaging unit, laser radar unit and visible light camera unit;

[0041] In the heterogeneous sensor module, under night or low light conditions, the sampling frequency of the infrared thermal imaging module is set to 30Hz, while the non-uniformity correction algorithm is used to suppress thermal noise, and the data weight of the visible light camera is dynamically reduced to below 0.2 through a weighted fusion algorithm to ensure that the infrared data dominates target detection; if there is strong electromagnetic interference in the environment, the laser radar unit immediately enables the polarization filtering mode, combines with the Stokes parameter analysis to filter out stray light interference, and at the same time, the millimeter wave radar triggers the frequency hopping anti-interference protocol, switches the carrier frequency in the 76-81GHz frequency band in a pseudo-random sequence, and assists with space-time adaptive processing technology to suppress sidelobe noise; when a high-speed moving target is detected, the radar-laser radar cooperative tracking mode is automatically activated, in which the millimeter wave radar provides high update rate radial velocity measurement, and the laser radar extracts target three-dimensional features through super-resolution point cloud clustering, both data inputs mixed filtering algorithm-linear motion component is processed by Kalman filter, nonlinear maneuvering is predicted by volume particle filter, and sensor contribution degree is optimized in real time through dynamic weight distribution mechanism;

[0042] Data fusion module: adopts a three-level fusion architecture:

[0043] First level: sensor-level data conflict resolution module based on improved D-S evidence theory, which realizes cross-modal data confidence assessment by introducing dynamic trust weight factor;

[0044] Second level: multi-scale spatio-temporal alignment for target features, using spatio-temporal stamp compensation algorithm to eliminate the difference in sampling frequency and spatial position between sensors;

[0045] Third level: bidirectional LSTM neural network based on attention mechanism, dynamically allocating the contribution weight of different sensors in the feature extraction stage;

[0046] In the data fusion module, the first level is based on improved D-S evidence theory, which solves the cross-modal data conflict problem through dynamic trust weight factor; in traditional D-S theory, the basic probability assignment function has limitations in dealing with conflicting evidence, and the real-time confidence assessment mechanism of the sensor is introduced; the trust weight of the i th sensor is defined as a time-varying function, considering its signal-to-noise ratio, historical accuracy and environmental adaptability; the calculation method of trust weight is as follows:

[0047]

[0048] Where, w i(t) represents the confidence weight of the ith sensor at time t, a, b, g represent the adjustment coefficients, SNR i (t) represents the signal-to-noise ratio of the ith sensor at time t, N represents the number of historical data points, f env (T, EMI) represents the environmental adaptation function of temperature, electromagnetic interference, represents the indicator function, whether the true value of the kth sample is consistent with the prediction value of the ith sensor, 1 if consistent, otherwise 0;

[0049] EMI represents electromagnetic interference;

[0050] The conflicting evidence of low-confidence sensors is suppressed by the weight factor, and the calculation method of the synthesized basic probability assignment is:

[0051]

[0052] where m(A) represents the synthesized BPA, i.e. the confidence in proposition A; A represents a proposition or hypothesis, which is usually the target of the basic probability assignment in D-S evidence theory, m i (A j ) represents the basic probability assignment of the ith sensor to the proposition A j i (t) represents the weight factor of the ith sensor at time t, ∩A j =A represents the intersection of all sensors as A, i.e. all sensors support the proposition A; represents the intersection of all sensors as the empty set, indicating that there is a conflict between the sensors;

[0053] BPA represents the basic probability assignment;

[0054] The second level is designed to compensate for the joint calibration algorithm based on space-time stamps for the differences in sampling frequency (radar 100 Hz vs infrared 30 Hz) and installation location between sensors:

[0055] For sensor data that arrives with a delay, such as the delay caused by the scanning period of the laser radar, a cubic spline interpolation is used to reconstruct the synchronous time series: LiDAR represents the laser radar;

[0056]

[0057] where L(t) represents the estimated value of the laser radar data at time t obtained by the interpolation method; K represents the total number of reflection points received by the LiDAR before time t, f k represents the basis function, a k represents the weight of the kth reflection point; t​k denotes the time when the k-th reflection point is received by the LiDAR; denotes the time scale parameter for normalizing the time difference t-t k , controls the shape and decay rate of the basis function;

[0058]

[0059] where S denotes the transformation matrix from the sensor coordinate system to the world coordinate system, S0 denotes the change matrix to be optimized, P LiDAR denotes the three-dimensional coordinates of the target point measured by the LiDAR sensor, P Radar denotes the three-dimensional coordinates of the target point measured by the Radar sensor, denotes the optimal transformation matrix by minimizing the objective function;

[0060] where S is an unknown variable during optimization, and is solved by minimizing the objective function;

[0061] Radar denotes the Radar sensor

[0062] The transformation matrix S contains rotation and translation parameters, which are automatically updated every 5 seconds through feature point matching;

[0063] The bidirectional LSTM neural network of the third attention mechanism comprises:

[0064] The time sequence feature extraction layer adopts a cavity convolution layer to extract local time sequence features of multi-sensor data;

[0065] The time sequence feature extraction layer captures multi-scale local features through cavity convolution. The original sensor data is first input into the cavity causal convolution layer to extract local time sequence patterns with different expansion coefficients;

[0066] The Radar signal adopts three layers of convolution with expansion coefficients d=1, 2, and 4 to capture short-time speed mutations (such as sudden braking), medium-range maneuvers (such as turning), and long-range trajectory trends, respectively. The output dimension calculation method of the convolution kernel is:

[0067] F(t)=ReLU(W dil *X radar (t-d,t)+b dil )

[0068] where F(t) denotes the output feature value, i.e., the feature value after expansion convolution and ReLU activation at time t; W dil denotes the weight matrix of the expansion convolution, and X radardenotes the radar data segment from time t-d to time t, * denotes the dilated convolution operation to avoid the time information compression of common convolution for high-speed targets; b dil denotes the bias term of dilated convolution; ReLU denotes the activation function;

[0069] Infrared and visible light data share the same set of convolution layers but independent parameters, and extract spatial-temporal joint features through 2D dilated convolution, such as target thermal radiation change and texture motion.

[0070] The output of this stage is the local temporal feature tensor of each sensor, with the dimension of [time step x channel number x feature dimension];

[0071] Cross-modal attention layer: calculate the feature correlation between different sensor channels through a learnable parameter matrix to generate dynamic attention weights.

[0072] Input the temporal features of each sensor into the cross-modal attention layer, and calculate the inter-modal correlation through a learnable parameter matrix.

[0073] Q i = W1F i

[0074] wherein Q i denotes the query vector of the i-th sensor, W1 denotes the weight matrix for generating the query vector, F i denotes the feature sequence of the i-th sensor

[0075] K j = W2F j

[0076] wherein K j denotes the key vector of the j-th sensor, W2 denotes the weight matrix for generating the key vector, F j denotes the feature sequence of the j-th sensor;

[0077] V j = W3F j

[0078] wherein V j denotes the value vector of the j-th sensor, W3 denotes the weight matrix for generating the value vector, F j denotes the feature sequence of the j-th sensor;

[0079] Calculate the attention score of sensor i to j, and the calculation method is as follows:

[0080]

[0081] wherein δ ij denotes the attention score of sensor i to j, Qi query vector of the i-th sensor, transpose of the key vector of the j-th sensor, g k dimension of the key vector,

[0082] weighted aggregated multi-sensor value vector, resulting in the modal-interacted feature, the calculation method of the modal-interacted feature is specifically:

[0083]

[0084] modal-interacted feature of the sensor i, δ ij attention score of the sensor i to the j-th sensor, V j value vector of the j-th sensor;

[0085] For example, when the target is partially occluded by trees, the attention weight of the infrared feature to visible light is increased, and the missing contour information is completed by using thermal radiation;

[0086] The output of this stage is the context-aware feature of each sensor, which integrates the complementary information of other modalities.

[0087] The decision fusion layer: the integrated gated recurrent unit classifies the weighted feature sequence to output structured data containing target type, spatial coordinates and motion vector;

[0088] The decision fusion layer inputs the enhanced multi-modal feature sequence into the gated recurrent unit for time series modeling and classification;

[0089] The calculation method of the update gate is specifically:

[0090]

[0091] wherein, z t update gate, W4 represents the weight matrix of the update gate, h t-1 previous hidden state, enhanced feature, and σ represents an activation function;

[0092]

[0093] wherein, r t reset gate, W5 represents the weight matrix of the reset gate, h t-1 previous hidden state, enhanced feature, and σ represents an activation function;

[0094]

[0095] wherein, ht denotes the candidate hidden state, W6 denotes the weight matrix of the candidate hidden state, r t denotes the reset gate, h t-1 denotes the previous time hidden state, denotes the enhanced feature, tanh denotes the activation function, i.e., hyperbolic tangent function;

[0096] h t0 = (1 - z t ) O h t-1 + z t O h t

[0097] wherein h t0 denotes the new hidden state, z t denotes the update gate, h t-1 denotes the previous time hidden state, h t denotes the candidate hidden state, O denotes the element-wise multiplication;

[0098] The bidirectional GRU further fuses the forward and backward context to capture the causality of the target motion;

[0099] The target type is classified, and the category probability is output based on the full connection layer of the new hidden state h t0 . The calculation method is specifically as follows:

[0100] P class = softmax(W7h t0 + b1)

[0101] wherein P class denotes the output category probability, W7 denotes the full connection layer weight matrix, h t0 denotes the new hidden state, b1 denotes the full connection layer bias vector, and softmax denotes the Softmax function, which is used to convert the full connection layer output into a probability distribution;

[0102] The linear layer is used to predict the target center point (x, y, z) and the bounding box size, and the calculation method is specifically as follows:

[0103] B t = W8h t0 + b2

[0104] wherein B t denotes the output target center point and bounding box size, W8 denotes the linear layer weight matrix, which is used to map the hidden state to the target center point and the bounding box size; b2 denotes the linear layer bias vector, which is used to adjust the output offset;

[0105] The speed and acceleration are calculated by time difference hidden state. The specific calculation method of speed is:

[0106]

[0107] wherein, denotes the speed at time step t2, h t0 denotes the new hidden state, denotes the previous hidden state, Δt0 denotes the time step length, and denotes the time interval between adjacent time steps;

[0108] The specific calculation method of acceleration is:

[0109]

[0110] wherein, denotes the acceleration at time step t, denotes the speed at time step, denotes the speed at previous time step, Δt0 denotes the time step length, and denotes the time interval between adjacent time steps;

[0111] The final output is in tuple form, and the specific tuple form is:

[0112]

[0113] wherein, P class denotes the category probability of output, B t denotes the target center point and bounding box size of output, v t denotes the speed at time step t, and a t denotes the acceleration at time step t.

[0114] For example, for a car that suddenly changes lanes, the output is:

[0115] (“Vehicle”, [x=12.3m, y=4.5m, z=0m], [v x =8.2m / s, v y

[0116] =3.1m / s], [a x =0.5m / s 2 , a y =2.3m / s 2 ])

[0117] Vehicle denotes a vehicle;

[0118] Self-calibration module: when it is detected that the single sensor confidence is continuously lower than the threshold, a sensor bias compensation mechanism based on particle filtering is started;

[0119] In the self-calibration module, after target recognition, the real-time confidence of each sensor is continuously evaluated;

[0120] The confidence calculation adopts a weighted exponential moving average method, and the calculation method is specifically as follows:

[0121] H i (t3) = μSNR i (t3) + (1-μ)H i (t3-1)

[0122] Wherein, H i (t) represents the confidence at time step t3, μ represents a forgetting factor, SNR i (t3) represents the signal-to-noise ratio of the i-th sensor at time step t3, H i (t3-1) represents the confidence at time step t3-1.

[0123] When the H i (t3) of a certain sensor is continuously lower than the threshold value for 5 consecutive periods, it is determined that the confidence is abnormal, and the online calibration process is triggered.

[0124] If the confidence continues to decrease (such as a slope < -0.1 / period), it is predicted that the sensor has a hardware aging or shielding problem, and an early warning is given.

[0125] When the sensor confidence is abnormal, the system starts a dynamic compensation mechanism driven by particle filtering, and first assumes that there is a systematic deviation between the sensor measurement value and the true value. The calculation method of the sensor measurement value is specifically as follows:

[0126] b t = c t + τ t + p t

[0127] Wherein, b t represents the sensor measurement value, c t represents the true value, τ t represents a time-varying deviation to be estimated, and p t represents the observation noise.

[0128] Then the bias estimate is output by particle sampling and updating, and the method steps are specifically as follows:

[0129] Step A1: initialize M particles subject to Gaussian distribution

[0130] Wherein, the state of the e-th particle, M represents the number of particles, represents the mean value at the previous time step, φ 2variance represented as a Gaussian distribution;

[0131] Step A2: Calculate the weight of each particle, the calculation method is as follows:

[0132]

[0133] where w t (e) represents the weight of the e th particle, represents the observation likelihood function, represents the probability of observing b t under the particle state ; R represents the observation noise variance, represents the state of the e th particle, b t represents the sensor measurement value, c t represents the true value;

[0134] Step A3: Resample to avoid particle degeneration and retain high-weight particles;

[0135] Step A4: Output the bias estimate, the calculation method is as follows:

[0136]

[0137] where τ t0 represents the output bias estimate value, w t (e) represents the weight of the e th particle, the state of the e th particle, and M represents the number of particles.

[0138] The output bias estimate value is fed back to the sensor data stream, and the corrected observation value is calculated, and the calculation method is as follows:

[0139]

[0140] where, represents the corrected observation value, i.e. the sensor data after bias correction; b t represents the sensor measurement value, τ t0 represents the output bias estimate value;

[0141] For example, the millimeter wave radar has a ranging bias of +0.5 m due to temperature drift, and the particle filter estimates τ t0 = 0.48 m, and the error is reduced to ±0.05 m after correction; the infrared thermal imager has a radiation bias due to lens contamination, and the compensation restores it to the calibrated value;

[0142] After compensating for the sensor bias, the sensor is evaluated for health and predictive maintenance, and the calibration history data of each sensor is recorded for a long time, and the health index is calculated, and the calculation method is as follows:

[0143] HI = 1 - ω1E - ω2G - ω3U

[0144] wherein HI represents a health index, ω1, ω2, ω3 represent weight factors, E represents a sensor operating time attenuation factor, G represents a cumulative root mean square of historical deviation, and U represents a frequency of recent calibration triggering;

[0145] Then, the remaining useful life prediction is performed, an LSTM network is used to learn the sensor degradation trend, input includes: historical deviation sequence, environmental stress data; and output is a probability distribution of RUL, when the predicted life is lower than a threshold, a maintenance alarm is triggered;

[0146] For example, an autonomous vehicle drives in a dusty environment:

[0147] Problem detection: the laser radar causes the point cloud density to decrease due to dust adhesion, and the confidence decreases from 0.8 to 0.4 in 3 cycles;

[0148] Online compensation: start particle filtering, estimate the ranging deviation of the laser radar by using radar and vision data; after correction, the multi-sensor fusion positioning error decreases from 1.2 m to 0.3 m;

[0149] Health management: the system records this abnormality, the HI index decreases by 5%, and a "laser radar lens cleaning suggestion" is prompted.

[0150] Finally, the above only describes the preferred embodiments of the present application and is not used to limit the present application, any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. An intelligent target detection system based on multi-sensor data fusion, characterized in that, Comprise: Heterogeneous sensor module: including radar unit, infrared thermal imaging unit, laser radar unit and visible light camera unit; Data fusion module: adopt three-level fusion architecture: First level: sensor level data conflict resolution module based on improved D-S evidence theory, through the introduction of dynamic trust weight factor to realize cross-modal data confidence assessment; In the data fusion module, the first level is constructed based on improved D-S evidence theory, the trust weight of the ith sensor is defined as a time-varying function, and the signal-to-noise ratio, historical accuracy and environmental adaptability are considered comprehensively; The calculation method of trust weight is as follows: wherein, represents the trust weight of the ith sensor at time t, , , represents the adjustment coefficient, represents the signal-to-noise ratio of the ith sensor at time t, N represents the number of historical data points, represents the environmental adaptation function of temperature, electromagnetic interference, represents the indicator function, whether the true value of the kth sample is consistent with the predicted value of the ith sensor, 1 if consistent, otherwise 0; Second level: multi-scale space-time alignment for target features, using space-time stamp compensation algorithm to eliminate the difference of sampling frequency and spatial position between sensors; Third level: bidirectional LSTM neural network based on attention mechanism, dynamically allocating the contribution weight of different sensors in the feature extraction stage; Self-calibration module: when it is detected that the confidence of a single sensor is continuously lower than the threshold, the sensor bias compensation mechanism based on particle filtering is started.

2. The intelligent target detection system based on multi-sensor data fusion according to claim 1, characterized in that: In the heterogeneous sensor module, under night or low light conditions, the sampling frequency of the infrared thermal imaging module is set to 30Hz, the non-uniformity correction algorithm is used to suppress thermal noise, and the data weight of the visible light camera is dynamically reduced to below 0.2 through a weighted fusion algorithm; When a high-speed moving target is detected, the radar-laser radar cooperative tracking mode is automatically activated, in which the millimeter wave radar provides high update rate radial velocity measurement, and the laser radar extracts target three-dimensional features through super-resolution point cloud clustering, both data inputs mixed filtering algorithm-linear motion component is processed by Kalman filter, nonlinear maneuvering is predicted by volume particle filter, and sensor contribution is optimized in real time through dynamic weight distribution mechanism.

3. The intelligent target detection system based on multi-sensor data fusion according to claim 1, characterized in that: The second level designs a joint calibration algorithm based on space-time stamp compensation for the difference of sampling frequency and installation position between sensors; For the delayed sensor data, the delay caused by the scanning period of the laser radar, a cubic spline interpolation is used to reconstruct the synchronous time sequence; wherein, represents an estimate of the LiDAR data obtained by an interpolation method at time t; K represents the total number of reflected points received by the LiDAR up to time t, represents a basis function, represents a weight of the kth reflected point; represents the reception time of the kth reflected point, which represents the time at which the kth reflected point was received by the LiDAR; represents a time scale parameter, which is used to normalize the time difference controls the shape and decay rate of the basis function.

4. The intelligent target detection system based on multi-sensor data fusion according to claim 1, characterized in that: The third level of the bidirectional LSTM neural network based on attention mechanism includes: Time series feature extraction layer: adopts a cavity convolution layer to extract local time series features of multi-sensor data; Cross-modal attention layer: calculates the feature correlation between different sensor channels through a learnable parameter matrix to generate dynamic attention weights; Decision fusion layer: integrates the gated recurrent unit to classify the weighted feature sequence and outputs structured data containing target type, spatial coordinates and motion vector.

5. The intelligent target detection system based on multi-sensor data fusion according to claim 4, characterized in that: The time series feature extraction layer captures multi-scale local features through a cavity convolution, and the original sensor data is first input into a cavity causal convolution layer to extract local time series patterns with different dilation coefficients; The radar signal adopts a three-layer convolution with an expansion coefficient , respectively capturing short-time speed mutation, medium-range maneuver and long-range trajectory trend; the output dimension calculation method of the convolution kernel is: wherein, represents the output feature value, i.e., the feature value after the dilated convolution and the ReLU activation at time t; represents the weight matrix of the dilated convolution, represents the radar data segment from time to time t, represents the empty convolution operation, avoiding the time sequence information compression of the ordinary convolution to the high-speed target; represents the bias term of the dilated convolution; represents the activation function.

6. The intelligent target detection system based on multi-sensor data fusion according to claim 4, characterized in that: The time series features of each sensor are input into the cross-modal attention layer to calculate the correlation between modalities through a learnable parameter matrix; Weighted aggregation of multi-sensor value vectors gives the feature after modal interaction, and the calculation method of the feature after modal interaction is as follows: post-interaction features of the sensor i, denoted as attention score of sensor i to j, denoted as value vector of sensor j.

7. The intelligent target detection system based on multi-sensor data fusion as claimed in claim 4, wherein: The decision fusion layer inputs the enhanced multi-modal feature sequence into the gated recurrent unit for time series modeling and classification; Classify the target type, based on the new hidden state of the fully connected layer + Softmax output category probability, the calculation method is: wherein, the class probabilities are represented as outputs, the fully connected layer weight matrix, the new hidden state is represented as, the fully connected layer bias vector is represented as, the Softmax function is represented as, which converts the fully connected layer output into a probability distribution; The target center point (x, y, z) and the bounding box size are calculated by linear layer prediction, and the calculation method is specifically as follows: wherein, represents a target center point and a bounding box size as output, represents a linear layer weight matrix for mapping the hidden state to the target center point and the bounding box size; represents a linear layer bias vector for adjusting the output offset. The speed and acceleration are calculated by time difference hidden state, and the final output is in the form of a tuple, and the tuple form is specifically: .

8. The intelligent target detection system based on multi-sensor data fusion as claimed in claim 1, wherein: In the self-calibration module, after target recognition, the real-time confidence of each sensor is continuously evaluated; The confidence calculation adopts a weighted exponential moving average method, and the calculation method is specifically as follows: wherein, the confidence at time step is represented as a forgetting factor, the signal-to-noise ratio of the ith sensor at time step the confidence at time step is represented as a forgetting factor, the signal-to-noise ratio of the ith sensor at time step is represented as a forgetting factor, When the sensor confidence is abnormal, the system starts a dynamic compensation mechanism driven by particle filtering, assumes that there is a systematic deviation between the sensor measurement value and the true value, and then outputs the deviation estimation through particle sampling and updating. The corrected observation value is calculated by feeding back the output deviation estimation value to the sensor data stream. After the sensor deviation compensation, the health of the sensor is evaluated and predictive maintenance is performed. The calibration history data of each sensor is recorded for a long time, the health index is calculated, and the remaining useful life is predicted. The LSTM network is used to learn the degradation trend of the sensor, and the input includes: historical deviation sequence, environmental stress data; the output is the probability distribution of RUL. When the predicted life is lower than the threshold, a maintenance alarm is triggered.

Images