FSO multi-sensor fusion method based on unscented Kalman filtering and RBF neural network

Abstract

The invention provides an FSO multi-sensor fusion method based on unscented Kalman filtering and an RBF neural network, and the method comprises the steps: carrying out the unscented Kalman filtering time updating, and carrying out the measurement of a noise covariance adjustment mechanism and an event triggering mechanism based on light intensity self-adaption; the unscented Kalman filter measures and adjusts confidence weights of the infrared camera and the four-quadrant detector according to the channel state provided by the avalanche photodiode, and processes data output by the infrared camera and the four-quadrant detector according to the confidence weights; then, the posterior state estimation of the unscented Kalman filter at the current moment is output, and a position component is extracted and transmitted. Normalized light intensity reflecting channel quality is used as a key state to carry out synchronous estimation, an RBF neural network is used to carry out dynamic modeling and compensation on nonlinear errors of a four-quadrant detector, and a self-adaptive unscented Kalman filtering framework with a line learning capability is constructed. And finally, a virtual sensor output with high update rate, high precision and high sensitivity is generated.

Description

Technical Field

[0001] This invention relates to the field of space optical communication technology, and in particular to an FSO multi-sensor fusion method based on unscented Kalman filtering and RBF neural network. Background Technology

[0002] The tracking performance of free-space optical communication (FSO) systems is limited by the inherent principles of the sensors and the depth of the fusion strategy. Existing solutions all face severe challenges in complex dynamic environments, making it difficult to achieve a balance between high frequency, high precision, and high sensitivity.

[0003] First, single sensors suffer from inherent contradictions that are difficult to reconcile. Infrared cameras, based on area array imaging, can achieve high-precision positioning in μrads using sub-pixel algorithms, but their data update rate is limited by the image acquisition and processing flow, typically only on the order of 100Hz, unable to respond to higher-frequency disturbances, and exhibiting significant tracking lag. Conversely, quadrant detectors (QDs), with their analog photoelectric conversion characteristics, can achieve high-speed position response in μs, but their difference-ratio-based calculation principle has inherent flaws: their accuracy and linearity heavily depend on the instantaneous received optical power. While maintaining high performance under strong light conditions, when the optical power approaches its sensitivity threshold due to link attenuation or turbulence, the measurement signal-to-noise ratio deteriorates sharply, and the positioning variance increases significantly. Furthermore, their position calculation model is inherently nonlinear, producing systematic distortion errors when the light spot deviates from the central region.

[0004] Secondly, existing dual-sensor composite solutions fail to achieve genuine intelligent fusion. The common strategy of switching between "camera for capture and QD for tracking" is merely a simple superposition of functions, rather than deep data fusion. Its core flaw lies in the lack of awareness and adaptive capability regarding channel conditions. The system fails to introduce and utilize the crucial physical quantity of real-time received light intensity, resulting in an inability to dynamically assess the measurement reliability of the QD (i.e., its noise statistics dynamically change with light intensity). Consequently, the filtering algorithm cannot fully trust the high-frequency data of the QD in strong light, and intelligently relies on the high-precision observation of the camera in weak light. More importantly, there is a complete lack of a mechanism for online calibration and compensation of QD nonlinear errors. The error of the QD is a complex nonlinear function that varies with the spot position and received light intensity. Static, offline calibration results cannot adapt to dynamic atmospheric channels, causing its effective accuracy under non-ideal conditions to be far lower than the theoretical value.

[0005] Therefore, under the existing technical architecture, the tracking performance of the system will degrade significantly when the light intensity fluctuates drastically, and it cannot stably give full play to the advantages of each sensor in the entire operating range, which has become a bottleneck restricting the high-performance tracking of the FSO system. Summary of the Invention

[0006] The purpose of this invention is to overcome the shortcomings of the above-mentioned technologies and provide an FSO multi-sensor fusion method based on unscented Kalman filtering and RBF neural network.

[0007] Therefore, this invention provides an FSO multi-sensor fusion method based on unscented Kalman filtering and RBF neural network, comprising the following steps:

[0008] S1. Initialize and configure the parameters of the FSO system;

[0009] After the S2 and FSO systems are started, the infrared camera, quadrant detector and avalanche photodiode work in parallel. The three output measurement data and mark the timestamp. The measurement data enters the queue and waits for the main processing thread to read it.

[0010] S3. After the main processing thread enters the loop, it first performs the unscented Kalman filter time update.

[0011] S4. Based on the light intensity adaptive measurement noise covariance adjustment mechanism and event triggering mechanism, the unscented Kalman filter adjusts the confidence weights of the infrared camera and the four-quadrant detector according to the channel state measurement data provided by the avalanche photodiode, and processes the measurement data output by the infrared camera and the four-quadrant detector according to the corresponding confidence weights.

[0012] S5. After processing the measurement data, output the posterior state estimate of the unscented Kalman filter at the current time, extract the position component, and transmit it to the beam deflection controller. This cycle ends and waits for the next cycle.

[0013] Preferably, in step S3, the unscented Kalman filter time update includes the following steps:

[0014] S31. Estimate based on initial state Or the posterior state estimate of the previous time step and initial covariance or the covariance at the previous time step Calculate a set of Sigma points:

[0015]

[0016] in, This is a scaling parameter used to adjust the distribution of Sigma points; Represent the i-th column of the square root of the matrix;

[0017] S32. Transform each Sigma point through the state transition matrix in the state model. spread:

[0018]

[0019] S33. Calculate the prior state estimate and covariance based on the propagated Sigma points:

[0020]

[0021] in, and The weights of the Sigma points.

[0022] Preferably, in step S4, the real-time channel state measurement data provided by the avalanche photodiode is updated based on the event triggering mechanism, including the following steps:

[0023] a) When the avalanche photodiode measurement value is received At that time, calculate the observation prediction for the Sigma point:

[0024]

[0025] in, For the observation matrix, only the light intensity state is selected;

[0026] b. Calculate the mean of the observed predictions:

[0027] ;

[0028] c. Calculate the innovation covariance and Kalman gain, perform unscented Kalman filtering time updates, and correct the normalized light intensity in the state model. .

[0029] Preferably, in step S4, the arriving infrared camera measurement data is processed based on the event-triggered mechanism, including the following steps:

[0030] a. Use a fixed high-precision covariance ;

[0031] b. Perform unscented Kalman filtering time update;

[0032] c. Unscented Kalman filter time updates provide posterior state estimates for online training of RBF neural networks. The training benchmark is used to calculate the system error samples of the four-quadrant detector and update the weights of the RBF neural network.

[0033] Preferably, in step S4, when processing the measurement data of the four-quadrant detector, the RBF neural network is first used to perform feedforward compensation on the original measurement data with nonlinear errors, and then adaptive fusion with the unscented Kalman filter is performed, including the following steps:

[0034] S41. Using the position measurement data provided by the infrared camera as a supervision signal, the RBF neural network is trained online, so that its weights are continuously updated online.

[0035] S42. Perform real-time compensation on the four-quadrant detector based on the updated weights to obtain effective observations. The obtained effective observations As input for subsequent unscented Kalman filter fusion.

[0036] Preferably, in step S41, the position measurement data provided by the infrared camera is used as a supervision signal to drive the online training of the RBF neural network, including the following steps:

[0037] S411, at the time of receipt Infrared camera position measurement values First, perform an unscented Kalman filter time update for the infrared camera measurements to obtain the posterior state estimate. ;

[0038] S412, Extraction and Infrared Camera Timing Aligned four-quadrant detector raw measurements Calculate the system error samples of the four-quadrant detectors at the current moment:

[0039]

[0040] in, Represents the observation matrix;

[0041] S413, Time Corresponding RBF neural network input and sample error Combined, they form a training data pair. ;

[0042] S414. The recursive least squares algorithm is used to update the output layer weights of the RBF neural network online.

[0043] Preferably, in step S42, real-time compensation is performed on the four-quadrant detector, including the following steps:

[0044] S421. Calculate the RBF neural network input at the current time. ;

[0045] S422, the obtained The RBF neural network, fed with the latest weights, obtains an estimate of the nonlinear error through forward propagation. ;

[0046] S423. Compensate the raw measurements of the four-quadrant detector to obtain effective observations. .

[0047] Preferably, in step S4, the light intensity adaptive measurement noise covariance adjustment mechanism includes the following steps:

[0048] A. Effective observations of the four-quadrant detector obtained after compensation The corresponding measurement noise covariance matrix is ​​obtained. Compared with the current prior estimate of light intensity Functional relationship:

[0049]

[0050] in, The baseline covariance matrix; function Designed as a monotonically decreasing function;

[0051] According to the function Determine the current estimated light intensity Size, if the current estimated light intensity If the value is smaller, the confidence in the measurement data of the current four-quadrant detector will decrease, while the confidence in the measurement data of the infrared camera will increase.

[0052] Preferably, in step S4, based on the event-triggered mechanism, the arriving measurement data from the four-quadrant detector is processed, including the following steps:

[0053] a. Estimating light intensity based on current prior knowledge Calculate the adaptive measurement noise covariance matrix ;

[0054] b. Calculate the observation prediction for the Sigma point:

[0055]

[0056] in, ;

[0057] c. Perform unscented Kalman filter time update, using As the measurement noise covariance matrix, the corrected position and speed state.

[0058] Preferably, the measurement data in step S2, after being processed by the measurement model, all use the infrared camera focal plane coordinate system as a unified spatial reference.

[0059] The beneficial effects of the present invention are as follows: The present invention provides an FSO multi-sensor fusion method based on unscented Kalman filtering and RBF neural network, which has the following beneficial effects.

[0060] This invention achieves a significant leap in tracking performance. The fused position estimation sequence output by this invention achieves a QD-level update rate (above 10kHz), effectively suppressing high-frequency beam jitter; and approaches the level of infrared cameras in terms of absolute accuracy and low-light sensitivity. It completely solves the core contradiction in traditional solutions where "high accuracy and high update rate are mutually exclusive," providing FSO systems with an ideal tracking signal that combines high-frequency response and high accuracy.

[0061] A robust, intelligent, and adaptive tracking capability across all operating conditions has been established. Through online learning of the RBF neural network, the system can dynamically model and compensate for the nonlinear errors of the QD (Quick Draw) as a function of position and light intensity, extending its effective linear operating range to the entire field of view. Combined with an adaptive noise adjustment mechanism based on light intensity estimation, the filter can fully rely on the high-frequency data of the QD under strong light and automatically enhance its dependence on high-precision data from the camera under weak light. This ensures stable high-precision tracking even under complex channel conditions such as link attenuation and atmospheric turbulence, significantly reducing the risk of link loss.

[0062] This results in a synergistic enhancement effect at the algorithm level. This invention is not a simple aggregation of sensors, but rather achieves deep information fusion and closed-loop optimization through the UKF framework. The APD provides channel state awareness, the RBF network utilizes state information for feedforward compensation, the QD provides high-frequency updates, the camera provides periodic accuracy calibration, and the UKF serves as the core for optimal estimation. This closed-loop collaborative architecture enables the overall system performance to surpass the performance limits of any single sensor or simple combination scheme, essentially creating a "virtual super sensor."

[0063] It possesses good engineering feasibility and versatility. The algorithm flow is clear, the computational load is controllable, and it is easy to implement on existing digital signal processor (DSP) or field-programmable gate array (FPGA) platforms. This solution does not depend on specific hardware and can be integrated as a core algorithm module into FSO tracking systems of different specifications, significantly improving their tracking performance ceiling, and has broad application prospects and significant engineering value. Detailed Implementation

[0064] The present invention will be further described below with reference to specific embodiments to aid in understanding its content. Unless otherwise specified, the methods used in this invention are conventional methods; the raw materials and apparatus used, unless otherwise specified, are conventional commercially available products.

[0065] This invention provides an FSO multi-sensor fusion method based on unscented Kalman filtering and RBF neural network, which is described in detail in the following aspects.

[0066] I. System State and Measurement Model.

[0067] 1. State Model.

[0068] To synchronously track the motion state of the light spot and channel information, the state vector Defined as:

[0069] (1)

[0070] in, The coordinates of the center of the light spot in the camera's image plane coordinate system; It represents the velocity of the image plane and is used to characterize short-term motion inertia; The normalized received optical intensity is obtained by dividing the instantaneous measurement of the APD by the maximum expected optical power calibrated by the system. It directly characterizes the signal-to-noise ratio level of the current channel.

[0071] Considering the high-frequency measurement of QD, the motion of the light spot can be approximated as uniform within this extremely short measurement time interval. Therefore, a discrete constant velocity model is used to characterize the motion of the light spot, and the state transition equation is:

[0072] (2)

[0073] in, Here is the state transition matrix. For zero-mean Gaussian process noise, its covariance matrix is: .

[0074] State transition matrix The specific details are as follows:

[0075] (3)

[0076] in, This is the system sampling time interval (usually consistent with the data rate of QD).

[0077] The upper left of the matrix The block implements the classic discrete constant-velocity motion model: the new position equals the old position plus the product of velocity and time. The speed remains constant. (The parameter in the lower right corner of the matrix is...) Used for modeling normalized received light intensity The first-order Markov process, namely , Represents the attenuation factor, coefficient Inertial properties characterizing light intensity The channel coherence time is caused by atmospheric turbulence. The closer to 1, the slower the change in light intensity.

[0078] Process noise covariance matrix The construction is based on a physical perturbation model and can take a block-based form:

[0079] (4)

[0080] in, Let be the covariance matrix of the noise during the motion process, and its derivation is based on a continuous-time stochastic acceleration model; The noise variance of the light intensity process.

[0081] Suppose that the light spot is subjected to random acceleration perturbation in continuous time, then the process noise covariance matrix of the motion state in discrete time is:

[0082] (5)

[0083] in, The spectral intensity representing the equivalent acceleration reflects the intensity of the random acceleration perturbation. The time power relationships among the elements in the matrix originate from the integration process over continuous-time white noise: The term represents the effect of random acceleration, after double integration, on the position state. The correlation between position noise and velocity noise stems from the fact that they are driven by the same random acceleration source. The term represents the effect of random acceleration on the velocity state after one integration.

[0084] Light intensity process noise variance The determination is based on statistical analysis of APD measurement data. Atmospheric turbulence and system noise can cause fluctuations in received light intensity, thus affecting the light intensity state. The evolution can be viewed as a first-order Markov process with process noise. Its process noise variance... The volatility characteristics of the sequence can be estimated using APD measurements, specifically as follows:

[0085] (6)

[0086] in, The steady-state variance of the normalized light intensity reflects the amplitude of light intensity fluctuations and can be calculated from historical measurement data of the APD. This is the attenuation factor in the aforementioned state transition matrix. This expression ensures that the variance of the light intensity state in steady state is... .

[0087] This is built based on physical mechanisms The matrix ensures the statistical characteristics of process noise and the sampling time. and the physical disturbance intensity of the system Strict matching provides prior information that accurately reflects the dynamic uncertainty of the system for subsequent UKF, and is the foundation for achieving high-performance state estimation.

[0088] 2. Measurement model.

[0089] The FSO system acquires observational information about the spot state at different dimensions and frequencies using three heterogeneous sensors. All sensor measurements must be unified to a common reference coordinate system—the infrared camera's focal plane coordinate system—and correlated with the system state vector through specific observation equations. The position defined in the state vector... This indicates the coordinates of the light spot in the camera's focal plane coordinate system.

[0090] Infrared cameras image directly on their own focal plane, capturing the two-dimensional intensity distribution of the light spot through an area array sensor. By using sub-pixel localization algorithms (such as the centroid method) to process this distribution, the coordinate observation values ​​of the spot center in the camera's focal plane coordinate system can be directly output. Its measurement model is direct and accurate, and can be expressed as follows:

[0091] (7)

[0092] in, This is the position vector directly observed by the camera in its own focal plane coordinate system at any given moment. The observation matrix indicates that the camera observations directly correspond to the position components in the state vector. and ; The noise vector measured by the camera follows a zero-mean Gaussian distribution, i.e. Its covariance matrix The value is very small, indicating its high precision. The camera data update rate is low ( N represents the ratio of the measurement cycle of the infrared camera to the measurement cycle of the four-quadrant detector. Since the measurement frequency of the infrared camera is usually much lower than that of the four-quadrant detector, N is generally between 10 and 100, indicating that the measurement frequency of the QD is a multiple higher than that of the infrared camera. However, it provides the true position reference of the entire system in the reference coordinate system.

[0093] A four-quadrant detector (QD) has a detector plane and coordinate system independent of the camera. It obtains the original position measurement of the light spot in its own detector plane coordinate system by calculating the photocurrent difference and ratio in the four quadrants. To achieve fusion with camera data, the local measurements of QD must be unified to the camera focal plane coordinate system through a pre-calibrated coordinate transformation. This transformation typically includes translation, rotation, and scaling, and can be represented by a single affine transformation model:

[0094] (8)

[0095] in, These are the original QD observations after being transformed to the camera coordinate system. This represents the transformation function obtained from calibration (e.g., , of which Rotation and scaling matrix, (This is a translation vector). The transformed QD measurement and the system state (defined in the camera coordinate system) have the following relationship:

[0096] (9)

[0097] in, This means that, under ideal error-free conditions, the transformed QD observation should be directly equal to the position in the state; This is a systematic nonlinear error. This error already exists in the original local coordinate system of QD and is further affected by coordinate transformation. Subsequently, in the camera coordinate system, it manifests as a complex nonlinear function of position and light intensity, the sources of which include the nonlinear response of the QD itself, non-ideal imaging of the light spot, etc. It is random measurement noise, and its covariance satisfies Furthermore, its magnitude strongly depends on the instantaneously normalized received light intensity. .when At lower levels, noise variance Significantly increased.

[0098] Avalanche photodiodes detect the total received optical power, and their measurements are independent of the coordinate system. This power value, after system calibration and normalization, is directly used to estimate the channel state. The model is as follows:

[0099] (10)

[0100] in, For a moment Scalar-normalized light intensity observations; It is the normalized true value of the received light intensity to be estimated in the state vector. Noise is measured for APD.

[0101] In summary, the measurement model established in this invention explicitly uses the infrared camera's focal plane coordinate system as a unified spatial reference. QD measurement is achieved through a defined coordinate transformation. Incorporation into this benchmark ensures spatial consistency with the state vector definition and camera observations. The APD provides channel state information decoupled from spatial coordinates. This modeling approach lays a solid foundation for subsequent sensor error compensation and data fusion within a unified framework.

[0102] II. QD dynamic error compensation based on RBF neural network and APD detection light intensity.

[0103] To address the systematic nonlinear error problem in four-quadrant detectors (QDs), this invention designs a dynamic error compensation mechanism based on a radial basis function (RBF) neural network. This mechanism uses online learning to model the complex relationship between QD nonlinear error and the position of the light spot and the received light intensity in real time, and performs feedforward compensation on the original QD measurements, thereby significantly improving the measurement accuracy of QDs in nonlinear regions and under low-light conditions.

[0104] 1. Structure and working principle of RBF neural network.

[0105] The RBF neural network is a three-layer feedforward network with strong nonlinear approximation capabilities and fast convergence characteristics, making it very suitable for online real-time modeling. The RBF network structure designed in this invention has an input layer containing three nodes, corresponding to the normalized location prediction and light intensity estimate, respectively; the hidden layer contains... One radial basis function neuron; the output layer contains two nodes, corresponding to... and Nonlinear error estimate in direction.

[0106] The network input vector is defined as the one-step prediction of the current state:

[0107] (11)

[0108] in, This is obtained from the prediction step of the state equation. Choosing the predicted value instead of the original measurement value as input can effectively reduce the impact of measurement noise on network training.

[0109] Hidden layer The output of each neuron uses a Gaussian function:

[0110] (12)

[0111] in, For the first The center vector of each hidden layer neuron Its width parameter. These parameters are determined during initialization using methods such as K-means clustering and can be fine-tuned during subsequent training.

[0112] The output layer is computed as a linear combination of the outputs of the hidden layers:

[0113] (13)

[0114] in, To connect the hidden layer The weight vector from each neuron to the output layer This is the output layer bias vector.

[0115] 2. Online supervised training mechanism based on infrared camera measurement.

[0116] To achieve adaptive compensation for QD nonlinearity errors during dynamic tracking, the weights of the RBF neural network need to be continuously updated online. This invention utilizes high-precision position measurements provided by the infrared camera cycle as a "supervision signal" to drive the network for online training.

[0117] The training process is triggered each time infrared camera measurement data arrives, and the specific steps are as follows:

[0118] (1) State update and truth retrieval: When the time is received Infrared camera measurement First, a complete UKF update (including predictions and measurement updates based on camera data) is performed to obtain a high-precision posterior state estimate. This estimated value, after correction by high-precision camera observation, can be regarded as the "quasi-true value" of the current spot position and light intensity state.

[0119] (2) Error sample generation: Extracting samples from the camera time Aligned raw QD measurements This value has been transformed by coordinates. (Transform to camera coordinate system). Calculate the system error sample of QD at the current time:

[0120] (14)

[0121] This sample reflects the situation at a specific location. He Guangqiang The actual measurement deviation of QD.

[0122] (3) Construction of training data pairs: time points Corresponding network input (From the prior state) (Calculated) and sample error Combined, they form a training data pair. .

[0123] (4) Online update of network weights: The recursive least squares (RLS) algorithm is used to update the output layer weights of the RBF network. Perform online updates. The recursive formula for the RLS algorithm is:

[0124] (15)

[0125] Where m represents the iteration time (the m-th weight update), and represents the "m-th data frame" during online runtime. This is the hidden layer output vector (including bias terms). The covariance matrix is ​​an intermediate matrix used to assist in weight updates and measures the "uncertainty" of the weights. λ is the forgetting factor, used to enable the algorithm to track time-varying systems. The closer λ is to 1, the higher the weight of old data; the smaller λ is, the more attention is paid to new data (adapting to the time-varying QD dynamic error).

[0126] K(m) is the Kalman gain (also called the gain matrix) of the RLS algorithm. Its core function is to balance the "weight information from the previous time step" and the "error information from the current time step," determining the contribution of the current error to the weight update. The larger K(m) is, the higher the "reference value of the current error," and the weights will be adjusted more significantly in the direction of "correcting the current error." The smaller K(m) is, the more it relies on the "weights from the previous time step," avoiding frequent fluctuations.

[0127] 3. Real-time compensation of QD measurement values.

[0128] For each arriving QD raw measurement Before sending it to UKF for fusion, the following compensation calculation is performed first:

[0129] (1) Calculate the network input at the current time. ;

[0130] (2) The RBF network with the latest weights is fed into it, and the estimate of the nonlinear error is obtained through forward propagation of the network. ;

[0131] (3) Compensate the original QD measurements to obtain the "cleaned" effective observations:

[0132] (16)

[0133] The compensated observation It will serve as the input for subsequent UKF fusion.

[0134] The QD dynamic error compensation mechanism of the present invention has the following outstanding advantages:

[0135] First, it has online adaptability. By continuously using the high-precision measurement of the infrared camera as a monitoring signal, the RBF network can track the slow drift of the QD nonlinear error characteristics with factors such as temperature, device aging, and changes in spot shape, and always maintain the effectiveness of compensation.

[0136] Secondly, it has light intensity sensing capabilities, and the network input includes normalized light intensity. This enables the RBF network to learn and compensate for the dependence of nonlinear errors on the received light intensity, which is particularly important for light intensity fluctuations (such as atmospheric scintillation) commonly found in FSO systems.

[0137] Third, it is computationally efficient. The forward propagation of the RBF network has a small computational cost, and the RLS training algorithm has controllable complexity, which fully meets the requirements of real-time processing (μs level).

[0138] Finally, it has synergy with the fusion framework. The compensation mechanism, as an independent module of the UKF fusion front end, has its input (state prediction value) and training signal (posterior state estimate) both from UKF, forming a tight closed loop with the fusion framework to jointly improve system performance.

[0139] Through this intelligent compensation mechanism, QD is effectively transformed into a high-precision, fast position sensor with near-linear performance across a wide dynamic range (position and light intensity), laying a solid foundation for subsequent high-performance data fusion.

[0140] III. Adaptive Asynchronous Fusion Based on UKF.

[0141] To achieve efficient fusion of data from three heterogeneous sensors—infrared cameras, QD, and APD—this invention employs unscented Kalman filtering (UKF) as the core fusion framework. This framework not only handles nonlinear system models but also achieves optimal state estimation with high frequency, high precision, and high sensitivity within a unified filtering framework through an innovative light intensity adaptive mechanism and asynchronous data fusion strategy.

[0142] 1. Basic principles and adaptive extension of the UKF framework.

[0143] Unscented Kalman Filter (UKF) approximates the probability distribution of the state and measurement of a nonlinear system through Unscented Transform (UT), avoiding the cumbersome Jacobian matrix calculations in Extended Kalman Filter (EKF), and thus has higher accuracy and stability when dealing with nonlinear problems.

[0144] Based on the standard UKF, this invention has made two key extensions:

[0145] (1) Asynchronous fusion mechanism: For the heterogeneous sampling rates of different sensors (infrared camera ~100Hz, QD ~10kHz, APD ~10kHz), an event-driven asynchronous update strategy was designed to ensure that all sensor data can be processed in a timely manner under a unified time base.

[0146] (2) Light intensity adaptive noise model: Based on the light intensity information provided by APD and the light intensity component in the state estimation, the noise covariance of QD measurement is dynamically adjusted to realize the filter's adaptation to the channel state.

[0147] 2. Light intensity adaptive measurement noise covariance adjustment mechanism.

[0148] The noise characteristics of QD measurements are highly dependent on the instantaneous received light intensity, which is one of the core problems that this fusion algorithm needs to solve. This invention designs an adaptive noise covariance adjustment mechanism based on the estimated light intensity.

[0149] For compensated QD measurement Its corresponding measurement noise covariance matrix It is no longer a fixed value, but rather a current estimated light intensity. Functions:

[0150] (17)

[0151] in, Let be the baseline covariance matrix, representing the measurement accuracy of QD under ideal high-intensity light conditions. Function Designed as a monotonically decreasing function, its specific form can be:

[0152] (18)

[0153] In the formula, , , This is an adjustable parameter.

[0154] The physical significance of this mechanism lies in: when estimating light intensity When the value is low, the function value Larger, leading to As the signal strength increases, the UKF filter gain decreases accordingly, thus reducing the confidence in the current QD measurement; conversely, under strong light conditions, the high-frequency QD measurement is fully trusted. This mechanism enables the filter to intelligently allocate sensor weights based on real-time channel conditions.

[0155] 3. Detailed implementation principles of asynchronous fusion process.

[0156] UKF's asynchronous fusion process uses the highest sampling rate (QD update rate) as the clock reference and processes data from different sensors through event triggering. The entire process follows a "prediction-update" loop structure, but the update steps vary depending on the type of data arriving.

[0157] In the prediction step (time update), in each system cycle (Based on QD data rate), time updates are performed to advance state estimation regardless of whether measurement data arrives:

[0158] (1) Sigma point generation: based on the posterior estimate of the previous time step Covariance Calculate a set of 2n+1 Sigma points (n=5 is the state dimension):

[0159] (19)

[0160] in, This is a scaling parameter used to adjust the distribution of Sigma points. The i-th column represents the square root of the matrix.

[0161] (2) Sigma point propagation: Each Sigma point is propagated through a state transition function. (corresponding state transition matrix) )spread:

[0162] (20).

[0163] (3) Calculate the prior statistics: Calculate the prior state estimate and covariance based on the propagated Sigma points:

[0164] (twenty one)

[0165] in, and These are the weights of the Sigma points, used to calculate the mean and covariance, respectively.

[0166] 4. Measurement update (asynchronous update step).

[0167] Measurement updates are triggered asynchronously based on the data type of the arriving sensor, and fall into three categories:

[0168] (1) APD measurement update principle.

[0169] When the APD measurement value is received At that time, perform an update on the light intensity state:

[0170] a. Calculate the observation prediction for the Sigma point:

[0171] (twenty two)

[0172] in, For the observation matrix, only the light intensity state is selected.

[0173] b. Calculate the mean of the observed predictions:

[0174] (twenty three).

[0175] c. Calculate the innovation covariance and Kalman gain, perform standard UKF updates, and primarily correct the intensity components in the state. This update ensures accurate real-time tracking of light intensity.

[0176] It should be noted that innovation refers to the difference between the actual measured value and the predicted measured value, i.e., z-Hx, where z is the actual measured value and Hx is the predicted measured value. The innovation covariance is calculated as S=E[(z-Hx)(z-Hx)^T]=HPH^T+R, where P is the state covariance matrix, R is the measurement noise covariance matrix, and the Kalman gain is K=PH^TS^{-1}, where P is the state covariance matrix, H is the observation matrix, and S is the innovation covariance.

[0177] (2) QD measurement update principle.

[0178] When the QD measurement after RBF compensation is received At that time, perform updates to the position and velocity states:

[0179] a) Based on the current prior light intensity estimation Calculate adaptive noise covariance .

[0180] b. Calculate the observation prediction for the Sigma point:

[0181] (twenty four)

[0182] in, .

[0183] c. Perform standard UKF updates, but use As the measurement noise covariance, this updates the high-frequency correction position. and speed Status is the primary source of updates for the tracking loop.

[0184] (3) Infrared camera measurement update principle.

[0185] When camera measurement is received At that time, perform high-precision calibration and update:

[0186] a. Use a fixed high-precision covariance (The value is very small).

[0187] b. Perform standard UKF update, which strongly corrects all state components and significantly improves the accuracy of state estimation.

[0188] c. Simultaneously, this update provides a high-quality training benchmark for online training of the RBF network: estimating the posterior state at this moment. Based on this, calculate the QD error samples and update the network weights.

[0189] IV. Status Output and Performance Characteristics.

[0190] After any measurement update, the system outputs the current posterior state estimate. Among them, the position component This is the high-frequency, high-precision beam tracking result ultimately provided by this invention.

[0191] This fusion framework has the following performance characteristics:

[0192] (1) Frequency characteristics: The output update rate is mainly determined by the QD update rate (up to 10kHz). The system only performs more complex processing at the moment when the camera data arrives (about once every 100 QD cycles), the computational load is evenly distributed and the real-time performance is strong.

[0193] (2) Accuracy characteristics: Under strong light conditions, the accuracy of the fusion result is close to the theoretical limit of QD; under weak light conditions, the weight of QD is reduced through the adaptive mechanism, and the high-precision measurement of the camera is periodically "injected" into the system through the UKF framework, ensuring that the tracking accuracy does not decrease due to the deterioration of channel conditions.

[0194] (3) Adaptive characteristics: through This function determines the system's ability to smoothly transition its dependence on different sensors when light intensity fluctuates. The function parameters are... It can be adjusted according to the actual system characteristics to adapt to different channel environments.

[0195] (4) Synergistic Enhancement Characteristics: The three sensors form a synergistic enhancement effect within the UKF framework. The APD provides channel state awareness, which the RBF network uses to compensate for QD errors. QD provides high-frequency updates, the camera provides an accuracy benchmark, and the UKF acts as the "intelligent brain" to organically integrate this information. This synergistic effect enables the overall performance to exceed the performance limits of any single sensor or simple combination scheme.

[0196] The implementation of this invention is described in detail below with reference to the algorithm flow. The entire method is executed in the digital signal processor of the beam tracking system according to the following steps:

[0197] (a) System initialization.

[0198] Set the initial state estimate of the unscented Kalman filter. With covariance matrix Configuration process noise covariance matrix (Its elements are composed of) , , and (Calculation), and the measurement noise parameters of each sensor: fixed covariance of the infrared camera. APD noise variance QD benchmark covariance and adaptive function Parameters ( Initialize the structural parameters of the RBF neural network, including the center vector. ,width and weight matrix .

[0199] (ii) Asynchronous data stream parallel acquisition and buffering.

[0200] After system startup, the infrared camera, QD, and APD begin operating in parallel, but outputting data asynchronously at their inherent frequencies. Three independent data buffer queues are established to receive and timestamp data from:

[0201] 1) Low-frequency, high-precision position data packets from infrared cameras;

[0202] 2) QD's high-speed raw location data stream;

[0203] 3) High-speed optical intensity data stream of APD.

[0204] After the data enters the queue, it waits for the main processing thread to read it.

[0205] (iii) Main processing loop (event-driven).

[0206] The system runs the main loop at a stable high clock frequency (not lower than the QD data rate), and processes data within the loop according to the following logic:

[0207] 1) Check and execute time updates: In each cycle, the UKF time update (prediction step) is executed first to calculate the prior state estimate at the current time. Covariance .

[0208] 2) Handling APD data events: If there is new data in the APD queue. Upon arrival, a UKF measurement update is performed, incorporating the light intensity observations into the state and updating the light intensity components. The estimate.

[0209] 3) Handling QD data events: If there is new raw data in the QD queue. Upon arrival, the core compensation and update process is executed. First, feedforward compensation is performed using an RBF neural network to estimate the current prior state. Calculate nonlinear error estimation After compensation, the observed values Then adaptive fusion is performed based on prior light intensity estimation. Calculate the current adaptive noise covariance Then with For observation, To address the noise covariance, perform UKF measurement updates to correct the position and velocity states.

[0210] 4) Handling camera data events: If there is new data in the camera queue. Upon arrival, a high-precision calibration is first performed to... For observation, To mitigate noise covariance, a UKF measurement update is performed to strongly correct all state components. Next, the network is trained online to obtain the updated high-precision posterior state estimate. Using this as a baseline, error training samples are generated by combining the original QD data at the same time, and the weights of the RBF network are updated using the RLS algorithm. Complete an online learning session.

[0211] (iv) Status output and control.

[0212] After processing any possible data events, estimate the posterior state of the UKF at the current time. As the final output, extract the positional components. The data is then transmitted to the beam deflection controller (such as a fast-reflecting mirror) to complete the closed-loop tracking control. The system then returns to the beginning of the loop, waiting to process the data for the next moment, forming a continuously running tracking closed loop.

[0213] To systematically address the inherent contradiction between high precision, high update rate, and high sensitivity in free-space optical communication tracking, as well as the insufficient depth of existing fusion schemes, this invention proposes a fusion architecture based on multi-dimensional information perception and intelligent signal processing. The purpose of this invention is not simply to superimpose sensor data, but to construct an intelligent tracking system with online learning, adaptive, and predictive capabilities. Its specific objectives are as follows:

[0214] First, a dynamic fusion core with channel state awareness is established. This aims to introduce precise instantaneous received optical power information from avalanche photodiodes (APDs) as a key environmental state observation variable, dynamically coupling it to the noise model of the main filter. Specifically, a four-quadrant detector measurement noise covariance function with real-time light intensity as the independent variable is designed. This allows the unscented Kalman filter, as the fusion core, to autonomously and continuously adjust the confidence weights of different sensors based on the instantaneous signal-to-noise ratio (SNR) of the channel. The goal is to enable the system to prioritize and fuse high-frequency QD data to maximize tracking bandwidth under strong light and high SNR conditions; and to automatically increase reliance on low-frequency, high-precision infrared camera data under weak light and low SNR conditions to ensure tracking accuracy and continuity, thereby achieving adaptive optimal matching of filter parameters to the dynamic channel.

[0215] Second, an online modeling and real-time compensation mechanism for nonlinear time-varying errors is created. Addressing the complex characteristics of nonlinear time-varying measurement errors in four-quadrant detectors with varying spot position and received light intensity, this invention aims to construct a radial basis function neural network dynamic compensator operating in a feedforward manner. This module utilizes the high-precision reference value of "true value" data provided by the infrared camera cycle as a supervisory signal, using the previous moment's fusion state estimate as a benchmark to generate training samples online. Through a recursive learning algorithm, the neural network can establish a high-dimensional nonlinear mapping model from "QD raw output - real-time light intensity" to "systematic error" in real time, and perform feedforward compensation for each QD measurement value about to enter the fusion process. Its fundamental purpose is to transform the QD from a sensor with significant nonlinear defects into a high-precision, fast measurement unit that is online calibrated and approximately linear within the effective field of view and light intensity range.

[0216] Third, this invention aims to achieve optimal state estimation and prediction for multi-rate, asynchronous, and heterogeneous sensor information streams. Building upon the aforementioned adaptive noise model and intelligent error compensation, the ultimate goal of this invention is to design and implement a complete unscented Kalman filter fusion framework. This framework aims to use the system's highest sampling rate (QD frequency) as a clock reference and, through an asynchronous update strategy, elegantly process low-frequency data from the infrared camera, compensated high-frequency QD data, and APD auxiliary information. Its objective is to output a statistically optimal sequence of spot position and motion state estimates in each fusion cycle. This sequence theoretically inherits both the infrared camera's maximum accuracy and sensitivity, as well as the quadrant detector's maximum update rate, thereby achieving near-hysteresis-free high-precision tracking of the beam dynamics at the algorithmic level.

[0217] In summary, the fundamental purpose of this invention is to deeply reconstruct and fuse multi-source heterogeneous sensors at the information level through algorithmic innovation, thereby generating a "virtual super sensor" with superior dynamic performance and environmental robustness that cannot be achieved by a single physical sensor.

[0218] Compared with existing technologies, this invention achieves the following significant benefits by deeply fusing information from three types of sensors: infrared camera, four-quadrant detector (QD), and avalanche photodiode (APD), and by introducing intelligent compensation and adaptive filtering mechanisms:

[0219] 1. A significant leap in tracking performance has been achieved.

[0220] The fused position estimation sequence output by this invention achieves a QD-level update rate (above 10kHz), effectively suppressing high-frequency beam jitter; and approaches the level of infrared cameras in terms of absolute accuracy and low-light sensitivity. It completely resolves the core contradiction in traditional solutions where "high accuracy and high update rate are mutually exclusive," providing an ideal tracking signal for FSO systems that combines high-frequency response and high accuracy.

[0221] 2. An intelligent, adaptive, and robust tracking capability under all operating conditions has been developed.

[0222] Through online learning of the RBF neural network, the system can dynamically model and compensate for the nonlinear errors of the QD (Quick Diffraction) as it varies with position and light intensity, extending its effective linear operating range to the entire field of view. Combined with an adaptive noise adjustment mechanism based on light intensity estimation, the filter can fully rely on the high-frequency data of the QD under strong light and automatically enhance its dependence on the high-precision data of the camera under weak light. This ensures stable high-precision tracking even under complex channel conditions such as link attenuation and atmospheric turbulence, significantly reducing the risk of link loss.

[0223] 3. A synergistic enhancement effect was formed at the algorithm level.

[0224] This invention is not a simple aggregation of sensors, but rather achieves deep information fusion and closed-loop optimization through the UKF framework. The APD provides channel state awareness, the RBF network uses state information for feedforward compensation, the QD provides high-frequency updates, the camera provides periodic accuracy calibration, and the UKF acts as the core for optimal estimation. This closed-loop collaborative architecture enables the overall system performance to surpass the performance limits of any single sensor or simple combination scheme, essentially creating a "virtual super sensor."

[0225] 4. It possesses good engineering feasibility and versatility.

[0226] The algorithm is clearly defined, computationally intensive, and easily implemented on existing digital signal processors (DSPs) or field-programmable gate arrays (FPGAs). This solution is not dependent on specific hardware and can be integrated as a core algorithm module into FSO tracking systems of different specifications, significantly improving their tracking performance ceiling. It has broad application prospects and significant engineering value.

[0227] In the description of this invention, it should be understood that the terms "left", "right", "up", "down", "top", "bottom", "front", "back", "inner", "outer", "back", "middle", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0228] However, the above description is merely a specific embodiment of the present invention and should not be construed as limiting the scope of the present invention. Therefore, any substitution of equivalent components or equivalent changes and modifications made in accordance with the scope of protection of the present invention should still fall within the scope of the claims of the present invention.

Claims

1. A multi-sensor fusion method for FSO based on unscented Kalman filtering and RBF neural network, characterized in that, Includes the following steps: S1. Initialize and configure the parameters of the FSO system; After the S2 and FSO systems are started, the infrared camera, quadrant detector and avalanche photodiode work in parallel. The three output measurement data and mark the timestamp. The measurement data enters the queue and waits for the main processing thread to read it. S3. After the main processing thread enters the loop, it first performs the unscented Kalman filter time update. S4. Based on the light intensity adaptive measurement noise covariance adjustment mechanism and event triggering mechanism, the unscented Kalman filter adjusts the confidence weights of the infrared camera and the four-quadrant detector according to the channel state measurement data provided by the avalanche photodiode, and processes the measurement data output by the infrared camera and the four-quadrant detector according to the corresponding confidence weights. S5. After processing the measurement data, output the posterior state estimate of the unscented Kalman filter at the current time, extract the position component, and transmit it to the beam deflection controller. This cycle ends and waits for the next cycle.

2. The FSO multi-sensor fusion method based on unscented Kalman filtering and RBF neural network according to claim 1, characterized in that, In step S3, the unscented Kalman filter time update includes the following steps: S31. Estimate based on initial state Or the posterior state estimate of the previous time step and initial covariance or the covariance at the previous time step Calculate a set of Sigma points: ； in, This is a scaling parameter used to adjust the distribution of Sigma points; Represent the i-th column of the square root of the matrix; S32. Transform each Sigma point through the state transition matrix in the state model. spread: ； S33. Calculate the prior state estimate and covariance based on the propagated Sigma points: ； in, and The weights of the Sigma points.

3. The FSO multi-sensor fusion method based on unscented Kalman filtering and RBF neural network according to claim 1, characterized in that, In step S4, based on the event-triggered mechanism, the real-time channel state measurement data provided by the avalanche photodiode is updated, including the following steps: a) When the avalanche photodiode measurement value is received At that time, calculate the observation prediction for the Sigma point: ； in, For the observation matrix, only the light intensity state is selected; b. Calculate the mean of the observed predictions: ； c. Calculate the innovation covariance and Kalman gain, perform unscented Kalman filtering time updates, and correct the normalized light intensity in the state model. .

4. The FSO multi-sensor fusion method based on unscented Kalman filtering and RBF neural network according to claim 1, characterized in that, In step S4, based on the event-triggered mechanism, the arriving infrared camera measurement data is processed, including the following steps: a. Use a fixed high-precision covariance ; b. Perform unscented Kalman filtering time update; c. Unscented Kalman filter time updates provide posterior state estimates for online training of RBF neural networks. The training benchmark is used to calculate the system error samples of the four-quadrant detector and update the weights of the RBF neural network.

5. The FSO multi-sensor fusion method based on unscented Kalman filtering and RBF neural network according to claim 1, characterized in that, In step S4, when processing the measurement data of the four-quadrant detector, the RBF neural network is first used to perform feedforward compensation on the original measurement data with nonlinear errors, and then adaptive fusion with the unscented Kalman filter is performed, including the following steps: S41. Using the position measurement data provided by the infrared camera as a supervision signal, the RBF neural network is trained online, so that its weights are continuously updated online. S42. Perform real-time compensation on the four-quadrant detector based on the updated weights to obtain effective observations. The obtained effective observations As input for subsequent unscented Kalman filter fusion.

6. The FSO multi-sensor fusion method based on unscented Kalman filtering and RBF neural network according to claim 5, characterized in that, In step S41, the position measurement data provided by the infrared camera is used as a supervision signal to drive the online training of the RBF neural network, including the following steps: S411, at the time of receipt Infrared camera position measurement values First, perform an unscented Kalman filter time update for the infrared camera measurements to obtain the posterior state estimate. ; S412, Extraction and Infrared Camera Timing Aligned four-quadrant detector raw measurements Calculate the system error samples of the four-quadrant detectors at the current moment: ； in, Represents the observation matrix; S413, Time Corresponding RBF neural network input and sample error Combined, they form a training data pair. ; S414. The recursive least squares algorithm is used to update the output layer weights of the RBF neural network online.

7. The FSO multi-sensor fusion method based on unscented Kalman filtering and RBF neural network according to claim 5, characterized in that, In step S42, real-time compensation is performed on the four-quadrant detector, including the following steps: S421. Calculate the RBF neural network input at the current time. ; S422, the obtained The RBF neural network, fed with the latest weights, obtains an estimate of the nonlinear error through forward propagation. ; S423. Compensate the raw measurements of the four-quadrant detector to obtain effective observations. .

8. The FSO multi-sensor fusion method based on unscented Kalman filtering and RBF neural network according to claim 1, characterized in that, In step S4, the light intensity adaptive measurement noise covariance adjustment mechanism includes the following steps: A. Effective observations of the four-quadrant detector obtained after compensation The corresponding measurement noise covariance matrix is ​​obtained. Compared with the current prior estimate of light intensity Functional relationship: ； in, The baseline covariance matrix; function Designed as a monotonically decreasing function; B. According to the function Determine the current estimated light intensity Size, if the current estimated light intensity If the value is smaller, the confidence in the measurement data of the current four-quadrant detector will decrease, while the confidence in the measurement data of the infrared camera will increase.

9. The FSO multi-sensor fusion method based on unscented Kalman filtering and RBF neural network according to claim 8, characterized in that, In step S4, based on the event-triggered mechanism, the arriving measurement data from the four-quadrant detectors is processed, including the following steps: a. Estimating light intensity based on current prior knowledge Calculate the adaptive measurement noise covariance matrix ; b. Calculate the observation prediction for the Sigma point: ； in, ; c. Perform unscented Kalman filter time update, using As the measurement noise covariance matrix, the corrected position and speed state.

10. The FSO multi-sensor fusion method based on unscented Kalman filtering and RBF neural network according to claim 1, characterized in that, The measurement data in step S2, after being processed by the measurement model, all use the infrared camera focal plane coordinate system as a unified spatial reference.

Images