Anti-shake imaging system based on dual-axis gyroscope-stabilized infrared gimbal pod

Abstract

This invention discloses an anti-shake imaging system for an infrared gimbal pod based on dual-axis gyroscope stabilization. It collects various types of raw data from the aircraft, preprocesses and extracts shake features, constructs a predictive shake model based on machine learning algorithms, loads a predetermined flight trajectory to generate a continuous reference trajectory, and combines the aircraft's health status with the output of high- and low-frequency shake sequences from the predictive shake model to generate pre-compensation commands and feedforward compensation. Based on real-time reading of state data to obtain vibration data and tracking errors, the difference error is calculated and then processed by a feedback control algorithm to obtain feedback control commands and generate feedback control. Combining feedforward compensation and feedback control, the infrared gimbal pod cancels out various types of shake during flight, maintains the stability of the infrared optical line of sight, overcomes physical delay, and improves imaging reliability and accuracy.

Description

Technical Field

[0001] This invention relates to the field of infrared imaging technology, and more specifically, to an anti-shake imaging system for an infrared gimbal pod based on dual-axis gyroscope stabilization. Background Technology

[0002] In mountainous regions with complex terrain, power operation and maintenance departments need to conduct regular infrared inspections on critical ultra-high-voltage transmission lines crossing canyons. The goal is to accurately identify early overheating risks caused by poor contact in key components such as conductor connectors. Since the temperature rise in early overheating is minimal, these tasks must be performed at night to improve detection sensitivity. The significantly lower background temperature at night effectively increases the contrast between the tiny heat generated by the equipment and the background temperature, thus more accurately capturing early overheating signals. In conventional techniques, the standard execution plan for these inspection tasks uses a medium-sized industrial-grade drone as the platform, equipped with a high-performance mid-wave infrared gimbal pod. The drone flies stably at a safe altitude along a pre-set line corridor, continuously and stably scanning core equipment such as insulator string clamps on the towers.

[0003] When performing such inspection operations in mountainous terrain, UAV systems face complex airframe vibration issues caused by multiple factors, affecting the stability and accuracy of infrared detection. Traditional gimbal stabilization systems use a single control logic of real-time sensor feedback and motor compensation to address this problem. However, due to the inherent response delay of the gimbal system, the feedback compensation process involves multiple stages, from sensor detection of changes in airframe attitude to motor execution of compensation commands. This delay can have different negative impacts depending on the vibration scenario.

[0004] The long period of low-frequency oscillations causes response delays, resulting in compensation commands being calculated based on past body attitude data. This makes it impossible to fully synchronize with real-time changes in low-frequency disturbances. The direct consequence is that the gimbal's line of sight cannot accurately lock onto the target, and it can only continuously oscillate around the target at a low frequency, failing to achieve absolute stillness. This leads to periodic target shifts in the infrared image. In high-frequency vibration scenarios, the phase lag problem caused by response delays is even more severe. The period of high-frequency vibration is much shorter than the gimbal's response time. The compensation motion not only fails to offset the vibration in time but may also be in phase or out of phase with the disturbance. In phase means that the compensation direction is consistent with the vibration direction, while out of phase means that there is a fixed phase difference between the compensation direction and the vibration direction. This not only fails to provide stabilization but may also amplify the jitter effect, resulting in severe blurring and jitter in the acquired infrared video sequence. Summary of the Invention

[0005] To address image quality issues caused by jitter, this invention provides an anti-jitter imaging system based on a dual-axis gyroscope-stabilized infrared gimbal pod. This system collects and preprocesses various types of raw data from the aircraft, extracts jitter features, constructs a predictive jitter model based on machine learning algorithms, generates feedforward compensation, and then generates feedback control based on real-time measurement data. By combining feedforward compensation and feedback control, the infrared gimbal pod can counteract various types of jitter during flight, maintain the stability of the infrared optical line of sight, overcome physical delay, and improve imaging reliability and accuracy.

[0006] The technical solution of this invention is as follows:

[0007] An anti-shake imaging system based on a dual-axis gyroscope-stabilized infrared gimbal pod is used. The imaging process includes...

[0008] Step S1. Collect raw data of the aircraft's flight process through multiple sensors. The raw data includes flight data, control data, status data, environmental data, and vibration data. After preprocessing all the raw data, convert it to a unified mathematical coordinate system. Perform feature extraction based on the vibration data, and calculate and obtain jitter feature data.

[0009] Step S2. Combine continuous flight data, control data, state data, and environmental data to form a flight state sequence, combine vibration data and vibration characteristic data to form jitter information data, and combine the flight state sequence and jitter information data to construct a predictive jitter model that feeds back the mapping relationship between the flight state sequence and jitter information data through machine learning algorithms.

[0010] Step S3. The predetermined flight trajectory is loaded into the core controller of the aircraft. The core controller of the aircraft performs smoothing interpolation based on the flight trajectory to generate a continuous time reference trajectory. The core controller of the aircraft loads the predictive jitter model. The core controller of the aircraft inputs the continuous time reference trajectory and the current health status parameters of the aircraft into the predictive jitter model. Based on the health status parameters of the aircraft and the continuous time reference trajectory, the predictive jitter model deduces the jitter situation that the aircraft may generate when executing the flight trajectory at each time step and outputs low-frequency jitter sequence and high-frequency jitter sequence. Based on the predicted low-frequency jitter sequence, the core controller of the aircraft converts it into a pre-compensation command for the gimbal pod. The pre-compensation command is precisely bound to the future time axis.

[0011] Step S4. Read the motion state data and flight state data of the aircraft in real time. The motion state data is analyzed and calculated by the signal processing unit of the aircraft core controller to obtain the real-time vibration data of the aircraft. The flight state data is analyzed and calculated by the calculation unit of the aircraft core controller to obtain the real-time tracking error representing the difference between the current flight state and the continuous time reference trajectory at the current moment. Calculate the difference between the real-time vibration data and the predicted jitter data of the jitter prediction model, and record it as the difference error. Input the difference error into the calculation unit of the aircraft core controller. After the feedback control algorithm of the calculation unit calculates, the feedback control command is obtained.

[0012] Step S5. The core controller of the aircraft superimposes the pre-compensation command generated by the jitter prediction model with the feedback control command to generate the gimbal control command, and sends the gimbal control command to the gimbal driver. The gimbal driver generates a movement opposite to the direction of the aircraft vibration according to the gimbal control command.

[0013] The aforementioned anti-shake imaging system for an infrared gimbal pod based on dual-axis gyroscope stabilization includes raw data such as flight data, control data, status data, environmental data, and vibration data.

[0014] Among them, flight data is the flight status and trajectory data of the aircraft, including three-dimensional spatial position, three-dimensional linear velocity, three-dimensional linear acceleration, roll angle, pitch angle, yaw angle, roll angular velocity, pitch angular velocity, yaw angular velocity, airspeed, track angle, and heading angle;

[0015] Control data refers to the control inputs and execution data of an aircraft, including aileron deflection angle, elevator deflection angle, rudder deflection angle, throttle position, engine power, and motor speed;

[0016] Status data refers to the airframe status data of an aircraft, including cumulative flight time, landing gear retraction and extension cycles, engine start cycles, battery voltage, current, remaining charge, engine vibration spectrum characteristics, engine temperature, total weight of the aircraft, and center of gravity position.

[0017] Environmental data refers to the environmental data of an aircraft during flight, including wind speed, wind direction, atmospheric temperature, humidity, pressure, and turbulence intensity.

[0018] Vibration data is raw data acquired by a high-precision inertial measurement unit installed on the aircraft, including raw three-axis specific force and raw three-axis angular velocity. The vibration data is strictly synchronized with a high-precision timestamp.

[0019] Furthermore, vibration data is extracted, and feature extraction is performed on the vibration data to calculate and obtain vibration feature data, including the root mean square value of vibration acceleration in each axis, the peak value of vibration acceleration in each axis, the peak-to-peak value of vibration acceleration in each axis, the amplitude of the dominant frequency in the vibration signal spectrum, the vibration energy in a specific frequency band, the skewness of the jitter signal, and the kurtosis of the jitter signal.

[0020] Specifically, time-domain statistical analysis is performed on the original accelerometer signal to calculate the root mean square value, peak value, and peak-to-peak value of vibration acceleration for each axis. After performing spectral analysis such as fast Fourier transform on the original vibration signal, the amplitude of the dominant frequency in the vibration signal spectrum is extracted. After bandpass filtering of the original vibration signal, its power is calculated or the vibration energy in a specific frequency band is obtained by integrating the spectrum. Higher-order statistics are performed on the original or preprocessed vibration signal to calculate the skewness and kurtosis of the jitter signal.

[0021] The aforementioned anti-shake imaging system for an infrared gimbal pod based on dual-axis gyroscope stabilization includes the following step S2: The process of constructing the jitter prediction model includes:

[0022] In the early stages of building the jitter prediction model, the definition and characteristics of aircraft jitter are clearly defined, and the target jitter feature dimension and jitter feature value are defined: the target jitter feature dimension is the number of different quantitative indicators that the jitter prediction model needs to predict to describe jitter, and the jitter feature value is the numerical value corresponding to the target jitter feature dimension;

[0023] The input data for the machine learning algorithm is a flight state sequence arranged in chronological order. The multidimensional feature vectors in the flight state sequence are linearly transformed through a standardization layer to form a standardized feature sequence.

[0024] The standardized feature sequence is fed into multiple sequentially connected long short-term memory layers. Each memory unit receives the input feature vector of the current time step, the hidden state vector of the previous time step, and the unit state vector at each time step.

[0025] The memory unit calculates three gate signals for the forget gate, input gate, and output gate, as well as a candidate state vector, based on the input data.

[0026] The forget gate of the memory unit generates a forget gate vector by concatenating the input feature vector with the hidden state vector of the previous time step, multiplying it with the weight matrix, adding a bias, and then passing it through a logic function.

[0027] The input gate generates an input gating vector and produces a candidate state vector;

[0028] The output gate generates the output gating vector;

[0029] The memory unit updates its internal state;

[0030] Multiple long short-term memory layers are stacked in series. In the last long short-term memory network, a fixed-dimensional context encoding vector is obtained. The context encoding vector is fed into a fully connected feedforward neural network for final transformation. Finally, the output layer maps it to a space with the same dimension as the target jitter feature to obtain the predicted jitter feature value.

[0031] Furthermore, the process of updating the internal state of a memory unit includes:

[0032] Step A1. Selective forgetting of old information: Multiply the cell state vector of the previous time step by the forgetting gate vector element by element. If the value of an element of the forgetting gate vector is close to 1, the corresponding old state information is retained. If it is close to 0, the information is discarded. This achieves the filtering and retention of historical cell state information, and the filtered old state information is obtained.

[0033] Step A2. New information extraction and integration: Multiply the input gating vector and the candidate state vector element by element. If the value of an element in the input gating vector is close to 1, retain the corresponding new state information; if it is close to 0, filter the information to be added.

[0034] Step A3. Generate new cell state: Add the old state information filtered in step A1 to the new information to be added filtered in step A2 to obtain the cell state vector at the current time.

[0035] In the aforementioned dual-axis gyroscope-stabilized infrared gimbal pod anti-shake imaging system, in step S3, before the actual flight mission is executed, a predetermined flight trajectory is loaded into the aircraft's core controller. The flight trajectory consists of a series of waypoints containing time, spatial position, and desired attitude. The aircraft's core controller performs smooth interpolation on these discrete waypoints to generate a high-density, high-precision continuous time reference trajectory.

[0036] The continuous-time reference trajectory precisely describes the spatial position, velocity vector, roll, pitch, and yaw angles that the aircraft should maintain at each moment.

[0037] In the aforementioned anti-shake imaging system for infrared gimbal pods based on dual-axis gyroscope stabilization, step S3 converts the low-frequency jitter sequence into a pre-compensation index for the gimbal pod based on the mechanical kinematic model and control response model of the gimbal pod.

[0038] Furthermore, the process of obtaining the mechanical kinematic model includes:

[0039] Measure the structural parameters of the gimbal, including the joint center distance of the two axes, rotation range, connecting rod dimensions, mass of the gimbal infrared camera, mass of the motor, center of mass position, and moment of inertia;

[0040] Using a laser tracker or a high-precision tilt sensor, measure the deviation between the actual installation position of each component and the design drawings, and record the deviation data;

[0041] Based on rigid body kinematics theory, a homogeneous transformation matrix is ​​established with the gimbal base as the fixed coordinate system and the infrared line-of-sight endpoint as the tool coordinate system. The motor output angle is used as the input of the homogeneous transformation matrix to calculate the attitude angle and spatial position of the infrared line-of-sight. Based on the target line-of-sight attitude, the required motor rotation angle is inversely calculated, so that the model can be mapped bidirectionally.

[0042] A fixed gimbal is used, and a known angle sequence is input through a stepper motor. A high-precision gyroscope and a laser positioning instrument are used to collect actual line-of-sight attitude data. The theoretical calculation value is compared with the actual measurement value, an error compensation function is fitted, the model deviation is corrected, and finally the attitude prediction error is less than or equal to the preset value.

[0043] Furthermore, the process of obtaining the control response model includes:

[0044] With the gimbal isolated from the aircraft body, multiple test excitation signals are applied to the gimbal in a series. At the same time, the minute vibration displacement of the final camera platform is directly measured using measuring tools.

[0045] Using the test excitation signal as the input and the small vibration displacement as the output, the system identification algorithm is used to estimate the model parameters.

[0046] The specific model is a second-order system transfer function that includes inertia, damping, and stiffness terms. Its model parameters correspond to the equivalent rotational inertia of the motor and load, the viscous damping coefficient of the bearing and transmission mechanism, and the stiffness coefficient exhibited by gear clearance and structural flexibility.

[0047] The final form of the model is a discrete-time transfer function or state-space equation.

[0048] In the aforementioned anti-shake imaging system based on dual-axis gyroscope-stabilized infrared gimbal pod, in step S4, the inertial measurement unit installed on the main body of the aircraft continuously measures the three-axis acceleration and three-axis angular velocity of the aircraft at high frequency. The real-time measured motion state data of the three-axis acceleration and three-axis angular velocity are sent to the signal processing unit of the core controller for processing.

[0049] The signal processing unit first performs coordinate transformation on the original real-time measurement values, converting them from the coordinate system of the sensor to the coordinate system of the gimbal base, removing the gravitational acceleration component, and obtaining real-time vibration data representing the dynamic vibration acceleration of the aircraft's vibration state.

[0050] According to the above-described solution, the beneficial effects of this invention are as follows:

[0051] 1. By uploading a file containing a predicted jitter model, the system can deduce possible aircraft jitters several seconds or even longer in advance based on the flight plan. This allows the gimbal servo system to generate and execute compensation commands in advance, thereby initiating countermeasures before the aircraft disturbance actually reaches the gimbal. This fundamentally compensates for the inherent time delays in sensor measurement, controller calculation, and motor response, achieving near real-time precision and stability.

[0052] 2. The predictive jitter model of this invention is not fixed; its parameters are refreshed by uploading updated data. The system can utilize real-time tracking error and jitter data collected during actual flight as new training samples, and fine-tune and iterate the model periodically or after specific missions. This allows the system to continuously adapt to the slow changes in the aircraft's own performance (such as component wear) and new mission modes, enabling its anti-jitter performance to continuously improve with the accumulation of usage time, exhibiting significant continuous learning advantages. Detailed Implementation

[0053] To make the technical problems, technical solutions, and beneficial effects of this invention clearer, the invention will be further described in detail below with reference to embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0054] An anti-shake imaging system for an infrared gimbal pod based on dual-axis gyroscope stabilization, the imaging process including

[0055] Step S1. Collect raw data of the aircraft's flight process through multiple sensors. The raw data includes flight data, control data, status data, environmental data, and vibration data. After preprocessing, all raw data is converted to a unified mathematical coordinate system. Based on the vibration data, feature extraction is performed, and jitter feature data is calculated and obtained.

[0056] The raw data includes flight data, control data, status data, environmental data, and vibration data. Flight data includes the aircraft's flight status and trajectory data, such as three-dimensional spatial position, three-dimensional linear velocity, three-dimensional linear acceleration, roll angle, pitch angle, yaw angle, roll angular velocity, pitch angular velocity, yaw angular velocity, airspeed, track angle, and heading angle. Control data includes the aircraft's control inputs and execution data, such as aileron deflection angle, elevator deflection angle, rudder deflection angle, throttle position, engine power, and motor speed. State data includes the aircraft's airframe state-related data, such as cumulative flight time, landing gear retraction / extension cycles, engine start cycles, battery voltage, current, remaining battery power, engine vibration spectrum characteristics, engine temperature, aircraft gross weight, and center of gravity position. Environmental data refers to the environmental data encountered by the aircraft during flight, including wind speed, wind direction, atmospheric temperature, humidity, pressure, and turbulence intensity (usually estimated values ​​or obtained from meteorological data). Vibration data is the raw data obtained by the high-precision inertial measurement unit installed on the aircraft, including three-axis raw specific force and three-axis raw angular velocity. This vibration data is strictly synchronized with high-precision timestamps.

[0057] After the raw data is collected, it enters the preprocessing stage. First, any abnormal values ​​are identified and corrected. Then, information from different sensors is precisely aligned on the timeline to ensure millisecond-level synchronization. Finally, a mathematical coordinate system is established with the spacecraft's center of mass as the origin, and its three axes pointing forward, right, and downward, respectively. The raw data is then transformed to allow for direct calculations and comparisons between different physical quantities. All preprocessed raw data is marked and stored with a unified timestamp, forming a multi-source data set with strict temporal correspondence.

[0058] Subsequently, vibration data was extracted separately, and feature extraction was performed on the vibration data to calculate and obtain vibration feature data, including the root mean square value of vibration acceleration for each axis, the peak value of vibration acceleration for each axis, the peak-to-peak value of vibration acceleration for each axis, the amplitude of the dominant frequency in the vibration signal spectrum, the vibration energy in a specific frequency band, the skewness of the jitter signal, and the kurtosis of the jitter signal. Specifically, time-domain statistical analysis was performed on the raw accelerometer signal to calculate the root mean square value of vibration acceleration for each axis, the peak value of vibration acceleration for each axis, and the peak-to-peak value of vibration acceleration for each axis. After spectral analysis such as Fast Fourier Transform, the amplitude of the dominant frequency in the vibration signal spectrum was extracted. After bandpass filtering, the power of the raw vibration signal was calculated, or the vibration energy in a specific frequency band was obtained by integrating the spectrum. Higher-order statistics were used to calculate the skewness and kurtosis of the jitter signal from the raw or preprocessed vibration signal.

[0059] Step S2. Combine continuous flight data, control data, state data, and environmental data to form a flight state sequence, combine vibration data and vibration characteristic data to form jitter information data, and combine the flight state sequence and jitter information data to construct a predictive jitter model that feeds back the mapping relationship between the flight state sequence and jitter information data through machine learning algorithms.

[0060] In the initial stage of jitter prediction model construction, it is necessary to clarify the definition and characteristics of aircraft jitter, that is, to define the target jitter feature dimensions and jitter feature values. The target jitter feature dimensions are the number of different quantitative indicators that the jitter prediction model needs to predict to describe jitter, while the jitter feature values ​​are the specific predicted values ​​of these quantitative indicators. In this invention, the target jitter feature dimensions are divided into three categories, including basic vibration feature dimensions: root mean square values ​​of vibration acceleration in each axis (X-axis, Y-axis, Z-axis, a total of 3 sub-dimensions), peak values ​​of vibration acceleration in each axis (X-axis, Y-axis, Z-axis, a total of 3 sub-dimensions), peak-to-peak values ​​of vibration acceleration in each axis (X-axis, Y-axis, Z-axis, a total of 3 sub-dimensions), amplitude of the dominant frequency in the vibration signal spectrum, vibration energy in a specific frequency band, skewness of the jitter signal, and kurtosis of the jitter signal; low-frequency jitter feature dimensions: maximum amplitude of low-frequency jitter, period of low-frequency jitter, and root mean square value of low-frequency jitter; and high-frequency jitter feature dimensions: high-frequency jitter frequency, standard deviation of high-frequency jitter amplitude, and energy density of high-frequency jitter. The jitter feature value is the specific quantified value corresponding to the above target jitter feature dimensions, with each dimension corresponding to one feature value.

[0061] The input data for machine learning algorithms is a sequence arranged in chronological order. Each time step in the sequence contains multiple flight state features at that moment, which are actually combinations of flight data, control data, state data, and environmental data corresponding to each time step. These multidimensional feature vectors of flight data, control data, state data, and environmental data combinations first pass through a standardization layer. The standardization layer performs a linear transformation on each feature vector based on the mean and standard deviation of each feature vector in the training dataset, transforming it into a value with a mean of zero and a standard deviation of one, so that all feature vectors are at similar numerical scales, forming a standardized feature sequence.

[0062] The standardized feature sequence is fed into multiple sequentially connected long short-term memory layers in the machine learning algorithm, where the first long short-term memory network receives the standardized feature sequence.

[0063] Each long short-term memory layer contains multiple memory units with identical structures. Each memory unit performs a series of defined mathematical operations at each time step, as shown below.

[0064] Each memory unit receives three input data at each time step: the input feature vector of the current time step, the hidden state vector generated by the same unit in the previous time step, and the unit state vector of the previous time step. The input feature vector is the feature vector input to the machine learning algorithm; the hidden state vector and the unit state vector are calculated by the memory unit. In the initial time step of the sequence (i.e., the first time step), since there is no calculation result from the previous time step, the hidden state vector and the unit state vector usually adopt the same initialization strategy: generally initialized as a vector of all zeros, or a very small random vector (random initialization avoids the gradient vanishing problem in the early stages of model training), thus providing a starting baseline for the entire time-series calculation.

[0065] The memory unit calculates three gating signals (the output vectors of the forget gate, the input gate, and the output gate) and a candidate state vector based on the input data.

[0066] The forgetting gate of a memory unit is created by concatenating the input feature vector (the feature vector within the feature sequence) with the hidden state vector from the previous time step, multiplying it by a weight matrix, adding a bias vector, and then passing the result through a logic function to produce a vector with a value between zero and one, called the forgetting gate vector (here, the first set of weight matrices, the first set of bias vectors, and the first set of logic functions are used). The forgetting gate vector determines how much information from the previous time step's unit state is retained.

[0067] The input gate of the memory unit uses a structure similar to the forget gate, concatenating the input feature vector (the feature vector within the feature sequence) with the hidden state vector from the previous time step. It then uses another set of weight matrices and bias vectors, and similarly, passes a logic function (here, a second set of weight matrices, a second set of bias vectors, and a second set of logic functions) to generate an input gating vector with a value between zero and one. This input gating vector regulates how much new information will be stored in the unit state. Simultaneously, the input gate of the memory unit substitutes the result calculated using the weight matrix and bias vectors into the hyperbolic tangent function to generate a candidate state vector. This candidate state vector contains potential new state information from the current input and the hidden state obtained from the hyperbolic tangent function.

[0068] The output gate of the memory cell generates an output gating vector through a third set of weight matrices, a bias vector, and a logic function (here, the second set of weight matrices, a second set of bias vectors, and a second set of logic functions are used). The output gating vector controls how much information from the current cell state is output to the hidden state.

[0069] Next, the memory unit updates its internal state, and the specific update process is as follows.

[0070] Step A1. Selective forgetting of old information: The cell state vector from the previous time step is multiplied element-wise with the forgetting gate vector (that is, the corresponding elements of the two vectors are multiplied separately to obtain a new vector with the same dimension as the original vector, and each element of the new vector is the product of the corresponding elements of the two original vectors). If the value of a certain element of the forgetting gate vector is close to 1, the corresponding old state information is retained; if it is close to 0, the information is discarded. This achieves the filtering and retention of historical cell state information, and the filtered old state information is obtained.

[0071] Step A2. New information extraction and integration: Multiply the input gating vector and the candidate state vector element by element. If the value of an element in the input gating vector is close to 1, retain the corresponding new state information; if it is close to 0, filter the information to be added.

[0072] Step A3. Generate new cell state: Add the old state information filtered in step A1 to the new information to be added filtered in step A2 to obtain the cell state vector at the current time.

[0073] After updating the internal state of the memory unit, the hidden state vector for the current time step is generated based on the current unit state vector. This is done by performing a nonlinear transformation on the current unit state vector using the hyperbolic tangent function, mapping its value to between -1 and 1. The result is then multiplied element-wise with the output gating vector to produce the hidden state vector for the current time step. This hidden state vector serves as the output of this memory unit at the current time step and is passed to the next time step. It also serves as the output feature of this layer and is passed to the next layer of the network. The current unit state vector itself is only passed within the memory unit and is not directly output.

[0074] Multiple Long Short-Term Memory (LSTM) layers are stacked in series as described above. The output of the previous LSM layer—that is, the sequence of hidden states at all time steps (i.e., the sequence of hidden state vectors at all time steps)—serves as the input sequence for the next LSM layer. This deep structure allows the model to extract features from the input sequence layer by layer, from low-order to temporal correlations to high-order abstract features. In the final LSM layer, the hidden state vector at the final time step is typically used, or a global average pooling is performed on the hidden state vectors at all time steps to obtain a fixed-dimensional context encoding vector. This vector condenses the dynamic behavioral information contained in the entire input sequence.

[0075] The context encoding vector obtained after processing by the last long short-term memory layer is fed into a fully connected feedforward neural network for final transformation.

[0076] Fully connected feedforward neural networks typically consist of multiple linear layers interleaved with nonlinear activation functions, with each layer operating as follows.

[0077] The first linear layer operates on the context encoding vector. It uses matrix multiplication to map the context encoding vector to a higher or lower dimension feature space, thus obtaining the output vector of the first linear layer.

[0078] Taking the output vector of the first linear layer as the object, a nonlinear transformation is performed on it through a nonlinear activation function (such as a modified linear unit), thereby introducing nonlinear feature mapping capability to capture more complex feature associations in the context encoding vector.

[0079] The above process is repeated once or multiple times according to the model design requirements. Each time it is repeated, the output vector of the nonlinear transformation of the previous linear layer will be used as the operation object of the next linear layer, and undergo a new round of dimension mapping and nonlinear processing.

[0080] Finally, the output vector of the last nonlinear transformation is used as the operation object, and the last linear output layer performs dimensional mapping on it. The feature space of this vector is mapped to a space that is exactly the same as the target jitter feature dimension, so as to obtain the predicted jitter feature values ​​of various dimensions, such as the root mean square value of vibration acceleration in each axis, vibration energy density in each frequency band, and other specific jitter-related feature indicators.

[0081] Throughout the computational process, from the standardization of input features (operating on multi-dimensional feature vectors at each time step), to the gating computation and state propagation of each long short-term memory layer (operating on the standardized feature sequence, input feature vectors of each memory unit, hidden state vectors of the previous time step, and unit state vectors of the previous time step), and then to the layer-by-layer transformation of the fully connected feedforward neural network (operating on the context encoding vector and the output vectors of each layer), the weight matrices and bias vectors involved in all steps together constitute the learnable parameters of this machine learning algorithm. Specifically, the weights of each gate refer to the weight matrices corresponding to the forget gate, input gate, and output gate in the long short-term memory layer; the weights of the candidate states refer to the weight matrices used by the input gate in the long short-term memory layer to generate candidate state vectors; and the weights of the feedforward network refer to the weight matrices corresponding to each linear layer in the fully connected feedforward neural network. These learnable parameters are continuously iteratively optimized based on the prediction error during model training.

[0082] The specific values ​​of these parameters are not manually set, but are automatically learned by an optimization algorithm during a phase called training. During training, a large number of known flight state sequences are input into the algorithm. The algorithm calculates the predicted jitter feature values ​​according to the steps described above, compares the predicted values ​​with the actual jitter feature labels obtained from actual measurements, and calculates the difference between the two, typically using mean squared error as a quantification of the difference. Then, using the backpropagation algorithm, starting from the output layer, the gradient of the error with respect to each learnable parameter in the network is calculated in reverse, i.e., the degree of influence of parameter changes on the error. Next, the gradient descent optimization algorithm is used to update all parameters in the opposite direction of the gradient with a small step size, reducing the prediction error. This process is iterated repeatedly on a large training dataset until the prediction error converges to an acceptable low level. At this point, the parameters within the algorithm are fixed, forming a mathematical model that can accurately capture the complex relationship between dynamic behavior and jitter features from flight state sequences. The trained algorithm can then be used for new flight missions, predicting the corresponding aircraft jitter based on real-time or preset flight state sequences through the aforementioned forward calculation steps.

[0083] Step S3. The predetermined flight trajectory is loaded into the core controller of the aircraft. The core controller performs smoothing interpolation based on the flight trajectory to generate a continuous time reference trajectory. The core controller loads the predictive jitter model and inputs the continuous time reference trajectory and the current health status parameters of the aircraft into the predictive jitter model. Based on the health status parameters of the aircraft and the continuous time reference trajectory, the predictive jitter model deduces the jitter that the aircraft may generate when executing the flight trajectory time by time and outputs low-frequency jitter sequences and high-frequency jitter sequences. Based on the predicted low-frequency jitter sequence, the core controller converts it into a pre-compensation command for the gimbal pod. The pre-compensation command is precisely bound to the future time axis.

[0084] Before the actual flight mission, a predetermined flight trajectory is loaded into the aircraft's core controller. This trajectory consists of a series of waypoints containing time, spatial position, and desired attitude. The core controller smoothly interpolates these discrete waypoints to generate a high-density, high-precision continuous-time reference trajectory. This trajectory precisely describes the aircraft's spatial position, velocity vector, and roll, pitch, and yaw angles at each moment. Finally, the core controller inputs the aircraft's current health status parameters, such as cumulative flight time and performance degradation coefficients of critical components, along with the generated continuous-time reference trajectory sequence, into a pre-trained jitter prediction calculation model.

[0085] The jitter prediction model, based on the input state and a continuous-time reference trajectory, extrapolates the potential jitter of the aircraft as it executes that trajectory, time-by-time. The extrapolation results include fluctuation sequences of three-dimensional linear acceleration and three-dimensional angular velocity, which are clearly distinguished into low-frequency and high-frequency components. The low-frequency components are associated with slow deviations from the flight path and corrective maneuvers, while the high-frequency components are associated with rapid disturbances such as air turbulence and engine vibration.

[0086] The aircraft's core controller converts predicted low-frequency jitter sequences into pre-compensation commands (feedforward control commands) for the gimbal pod. The conversion process calculates the angle and angular velocity command sequences that the gimbal motors need to move in advance to counteract the predicted jitter, based on the gimbal pod's mechanical kinematics and control response models. These feedforward control commands are pre-calculated and stored, precisely bound to the future timeline.

[0087] The process of obtaining the mechanical kinematic model is shown below.

[0088] Measure the gimbal's structural parameters, including the joint center distance, rotation range, linkage dimensions, gimbal infrared camera mass, motor mass, center of gravity position, and moment of inertia (measured by weighing method and inertial measurement instrument) of the dual axes (pitch axis and roll axis). Measure the deviation between the actual installation position of each component and the design drawings using a laser tracker or high-precision tilt sensor, such as axis parallelism and mounting plane perpendicularity, and record the deviation data for subsequent correction.

[0089] Based on rigid body kinematics theory (using a rotation matrix to describe the single-axis rotation of the gimbal, a homogeneous transformation matrix to handle the total transformation from the gimbal base coordinate system to the camera coordinate system caused by attitude rotation and position offset, and describing the control commands of the two motors through chain multiplication of continuous coordinate system transformations), a homogeneous transformation matrix is ​​established with the gimbal base as the fixed coordinate system and the infrared line-of-sight endpoint as the tool coordinate system. The motor output angle is used as the input to calculate the attitude angles (roll angle, pitch angle) and spatial position (output) of the infrared line-of-sight axis. Based on the target line-of-sight axis attitude (desired output), the required motor rotation angle (control input) is inversely calculated to ensure bidirectional mapping of the model.

[0090] A fixed gimbal is used, and a known angle sequence is input via a stepper motor. A high-precision gyroscope and laser positioning device are used to collect actual line-of-sight attitude data. The theoretical calculations are compared with the actual measurements. An error compensation function (such as a nonlinear correction term) is fitted using the least squares method to correct model deviations caused by machining errors and component deformation, ultimately ensuring that the attitude prediction error is ≤0.1 degrees.

[0091] The process of obtaining the control response model is shown below.

[0092] With the gimbal isolated from the aircraft body, multiple series of test excitation signals are applied to the gimbal, such as linear sweep signals containing multiple frequency components or pseudo-random binary sequences. At the same time, a high-precision encoder is used to measure the actual angular position of the motor rotor, or a non-contact laser displacement sensor, high-speed vision system, etc., are used to directly measure the minute vibration displacement of the final camera platform.

[0093] Then, based on the input and output data, a system identification algorithm is used to estimate the model parameters. The specific model can be a second-order system transfer function containing inertia, damping, and stiffness terms, whose parameters correspond to the equivalent rotational inertia of the motor and load, the viscous damping coefficient of the bearings and transmission mechanism, and the stiffness coefficient exhibited by gear backlash and structural flexibility. The model can further consider nonlinear factors such as actuator saturation, static friction, cogging torque fluctuation, and cable flexibility, and may be described using nonlinear model structures such as Wiener or Hammerstein.

[0094] The final form of this model is a discrete-time transfer function or state-space equation that can accurately predict the actual angular position, angular velocity, and even response delay of the gimbal axis under a given input command.

[0095] Step S4. Read the motion state data and flight state data of the aircraft in real time. The motion state data is analyzed and calculated by the signal processing unit of the aircraft core controller to obtain the real-time vibration data of the aircraft. The flight state data is analyzed and calculated by the calculation unit of the aircraft core controller to obtain the real-time tracking error representing the difference between the current flight state and the continuous time reference trajectory at the current moment. Calculate the difference between the real-time vibration data and the predicted jitter data of the jitter prediction model, and record it as the difference error. Input the difference error into the calculation unit of the aircraft core controller. After the feedback control algorithm of the calculation unit calculates, the feedback control command is obtained.

[0096] Step S5. The core controller of the aircraft superimposes the pre-compensation command generated by the jitter prediction model with the feedback control command to generate the gimbal control command, and sends the gimbal control command to the gimbal driver. The gimbal driver generates a movement opposite to the direction of the aircraft vibration according to the gimbal control command.

[0097] The aircraft actually began flying along the predetermined trajectory.

[0098] An inertial measurement unit (IMU) mounted on the aircraft's main body continuously measures the aircraft's triaxial acceleration and triaxial angular velocity at high frequency. This real-time motion data is sent to the signal processing unit of the core controller for processing. The signal processing unit first performs a coordinate transformation on the raw real-time measurements, converting them from the sensor's coordinate system to the gimbal base's coordinate system, and removes the gravitational acceleration component to obtain the dynamic vibration acceleration representing the aircraft's vibration state, i.e., the real-time vibration data. Subsequently, the signal processing unit applies a digital filter to separate the real-time vibration data into low-frequency and high-frequency components that are consistent with the prediction model's frequency band.

[0099] Simultaneously, information such as the current aircraft attitude, velocity, and control surface position (i.e., the real-time physical deflection angle of the controllable aerodynamic surfaces on the aircraft) read from the flight controller in real time is compared with the expected values ​​in the continuous-time reference trajectory at the current moment to obtain the real-time tracking error of the flight state. The real-time tracking error is used as data for learning and training in the jitter prediction model to improve the accuracy of the jitter prediction model.

[0100] The computing unit in the core controller of the aircraft will measure and separate the real-time vibration data in real time and compare it point by point with the jitter data (wave sequence) predicted by the jitter prediction model at the corresponding time. The difference between the two is calculated, which is the error. This difference represents the inaccurate part of the model prediction and the unforeseen external interference.

[0101] The differential error is input into the feedback control algorithm of the computing unit. This algorithm uses proportional-integral-derivative control law or its variant. Based on the current value, historical cumulative value and trend of the error, it calculates additional gimbal control commands (since the predictive jitter model has already calculated a series of predicted jitters, it has also calculated the corrected feedforward control commands, i.e., pre-compensation commands. At the same time, because of the existence of the feedback control algorithm, the feedback control signal generated in the previous moment has also been executed. Therefore, the current gimbal control command is a control command for the future moment that is corrected again after the gimbal has acted according to the pre-compensation command and feedback control signal). The purpose is to further reduce this real-time error and suppress the expansion of the error.

[0102] Ultimately, the control commands sent to the gimbal servo driver are the sum of the pre-calculated feedforward compensation command (for the current moment) and the real-time generated feedback correction command (for the previous moment). The feedforward command is responsible for dealing with the predictable main jitter, while the feedback command is responsible for suppressing the unpredictable residual disturbance. The two work together to drive the gimbal motor to produce a movement opposite to the vibration direction of the body, thereby making the infrared optical line of sight mounted on the gimbal stable in the air relative to the inertial space.

[0103] This invention provides a gimbal stability control method that, through model-based feedforward control, enables the aircraft and infrared gimbal to anticipate disturbances based on the flight plan and issue compensation commands in advance. This fundamentally overcomes the inherent physical delays in sensor measurements, controller calculations, and motor responses (i.e., jitter caused by the aircraft itself, known disturbances inherent in the flight trajectory, and, depending on the amount of data, jitter caused by external influences such as climate and weather). This allows the gimbal compensation action to achieve near real-time and precise offsetting with aircraft disturbances, which is beneficial for suppressing mid-to-high frequency jitter. Furthermore, the superposition of feedback control can handle the occurrence of sudden events and suppress any disturbances not covered by the feedforward, ensuring the final stability and reliability of the system in complex real-world environments.

[0104] Real-time tracking errors measured in real time can not only be used for feedback correction, but also serve as continuous training data for online fine-tuning or periodic updates to the predictive jitter model, enabling both the predictive jitter model and the aircraft to continuously learn. As flight data accumulates, the predictive jitter model becomes increasingly accurate, the proportion of feedforward compensation increases, and the overall performance of the aircraft continuously improves over time.

[0105] In this invention, the sensor that measures the angular velocity of the gimbal (or body) around its pitch and roll axes in real time is a dual-axis gyroscope.

[0106] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An anti-shake imaging system for an infrared gimbal pod based on dual-axis gyroscope stabilization, characterized in that, The imaging process includes Step S1. Collect raw data of the aircraft's flight process through multiple sensors. The raw data includes flight data, control data, status data, environmental data, and vibration data. After preprocessing all the raw data, convert it to a unified mathematical coordinate system. Perform feature extraction based on the vibration data, and calculate and obtain jitter feature data. Step S2. Combine continuous flight data, control data, state data, and environmental data to form a flight state sequence, combine vibration data and vibration characteristic data to form jitter information data, and combine the flight state sequence and jitter information data to construct a predictive jitter model that feeds back the mapping relationship between the flight state sequence and jitter information data through machine learning algorithms. Step S3. The predetermined flight trajectory is loaded into the core controller of the aircraft. The core controller of the aircraft performs smoothing interpolation based on the flight trajectory to generate a continuous time reference trajectory. The core controller of the aircraft loads the predictive jitter model. The core controller of the aircraft inputs the continuous time reference trajectory and the current health status parameters of the aircraft into the predictive jitter model. Based on the health status parameters of the aircraft and the continuous time reference trajectory, the predictive jitter model deduces the jitter situation that the aircraft may generate when executing the flight trajectory at each time step and outputs low-frequency jitter sequence and high-frequency jitter sequence. Based on the predicted low-frequency jitter sequence, the core controller of the aircraft converts it into a pre-compensation command for the gimbal pod. The pre-compensation command is precisely bound to the future time axis. Step S4. Read the motion state data and flight state data of the aircraft in real time. The motion state data is analyzed and calculated by the signal processing unit of the aircraft core controller to obtain the real-time vibration data of the aircraft. The flight state data is analyzed and calculated by the calculation unit of the aircraft core controller to obtain the real-time tracking error representing the difference between the current flight state and the continuous time reference trajectory at the current moment. Calculate the difference between the real-time vibration data and the predicted jitter data of the jitter prediction model, and record it as the difference error. Input the difference error into the calculation unit of the aircraft core controller. After the feedback control algorithm of the calculation unit calculates, the feedback control command is obtained. Step S5. The core controller of the aircraft superimposes the pre-compensation command generated by the jitter prediction model with the feedback control command to generate the gimbal control command, and sends the gimbal control command to the gimbal driver. The gimbal driver generates a movement opposite to the direction of the aircraft vibration according to the gimbal control command.

2. The anti-shake imaging system for an infrared gimbal pod based on dual-axis gyroscope stabilization as described in claim 1, characterized in that, Raw data includes flight data, control data, status data, environmental data, and vibration data; Among them, flight data is the flight status and trajectory data of the aircraft, including three-dimensional spatial position, three-dimensional linear velocity, three-dimensional linear acceleration, roll angle, pitch angle, yaw angle, roll angular velocity, pitch angular velocity, yaw angular velocity, airspeed, track angle, and heading angle; Control data refers to the control inputs and execution data of an aircraft, including aileron deflection angle, elevator deflection angle, rudder deflection angle, throttle position, engine power, and motor speed; Status data refers to the airframe status data of an aircraft, including cumulative flight time, landing gear retraction and extension cycles, engine start cycles, battery voltage, current, remaining charge, engine vibration spectrum characteristics, engine temperature, total weight of the aircraft, and center of gravity position. Environmental data refers to the environmental data of an aircraft during flight, including wind speed, wind direction, atmospheric temperature, humidity, pressure, and turbulence intensity. Vibration data is raw data acquired by a high-precision inertial measurement unit installed on the aircraft, including raw three-axis specific force and raw three-axis angular velocity. The vibration data is strictly synchronized with a high-precision timestamp.

3. The anti-shake imaging system for an infrared gimbal pod based on dual-axis gyroscope stabilization as described in claim 2, characterized in that, Vibration data is extracted, and features are extracted from the vibration data. Vibration feature data is calculated and obtained, including the root mean square value of vibration acceleration in each axis, the peak value of vibration acceleration in each axis, the peak-to-peak value of vibration acceleration in each axis, the amplitude of the dominant frequency in the vibration signal spectrum, the vibration energy in a specific frequency band, the skewness of the jitter signal, and the kurtosis of the jitter signal. Specifically, time-domain statistical analysis is performed on the original accelerometer signal to calculate the root mean square value, peak value, and peak-to-peak value of vibration acceleration for each axis. After performing spectral analysis such as fast Fourier transform on the original vibration signal, the amplitude of the dominant frequency in the vibration signal spectrum is extracted. After bandpass filtering of the original vibration signal, its power is calculated or the vibration energy in a specific frequency band is obtained by integrating the spectrum. Higher-order statistics are performed on the original or preprocessed vibration signal to calculate the skewness and kurtosis of the jitter signal.

4. The anti-shake imaging system for an infrared gimbal pod based on dual-axis gyroscope stabilization as described in claim 1, characterized in that, In step S2, the process of constructing the jitter prediction model includes: In the early stages of building the jitter prediction model, the definition and characteristics of aircraft jitter are clearly defined, and the target jitter feature dimension and jitter feature value are defined: the target jitter feature dimension is the number of different quantitative indicators that the jitter prediction model needs to predict to describe jitter, and the jitter feature value is the numerical value corresponding to the target jitter feature dimension; The input data for the machine learning algorithm is a flight state sequence arranged in chronological order. The multidimensional feature vectors in the flight state sequence are linearly transformed through a standardization layer to form a standardized feature sequence. The standardized feature sequence is fed into multiple sequentially connected long short-term memory layers. Each memory unit receives the input feature vector of the current time step, the hidden state vector of the previous time step, and the unit state vector at each time step. The memory unit calculates three gate control signals for the forget gate, input gate, and output gate, as well as a candidate state vector, based on the input data. The forget gate of the memory unit generates a forget gate vector by concatenating the input feature vector with the hidden state vector of the previous time step, multiplying it with the weight matrix, adding a bias, and then passing it through a logic function. The input gate generates an input gating vector and produces a candidate state vector; The output gate generates the output gating vector; The memory unit updates its internal state; Multiple long short-term memory layers are stacked in series. In the last long short-term memory network, a fixed-dimensional context encoding vector is obtained. The context encoding vector is fed into a fully connected feedforward neural network for final transformation. Finally, the output layer maps it to a space with the same dimension as the target jitter feature to obtain the predicted jitter feature value.

5. The anti-shake imaging system for an infrared gimbal pod based on dual-axis gyroscope stabilization as described in claim 4, characterized in that, The process of updating the internal state of a memory unit includes: Step A1. Selective forgetting of old information: Multiply the cell state vector of the previous time step by the forgetting gate vector element by element. If the value of an element of the forgetting gate vector is close to 1, the corresponding old state information is retained. If it is close to 0, the information is discarded. This achieves the filtering and retention of historical cell state information, and the filtered old state information is obtained. Step A2. New information extraction and integration: Multiply the input gating vector and the candidate state vector element by element. If the value of an element in the input gating vector is close to 1, retain the corresponding new state information; if it is close to 0, filter the information to be added. Step A3. Generate new cell state: Add the old state information filtered in step A1 to the new information to be added filtered in step A2 to obtain the cell state vector at the current time.

6. The anti-shake imaging system for an infrared gimbal pod based on dual-axis gyroscope stabilization as described in claim 1, characterized in that, In step S3, before the actual flight mission is executed, the predetermined flight trajectory is loaded into the core controller of the aircraft. The flight trajectory consists of a series of waypoints containing time, spatial position, and desired attitude. The core controller of the aircraft performs smooth interpolation on these discrete waypoints to generate a high-density, high-precision continuous time reference trajectory. The continuous-time reference trajectory precisely describes the spatial position, velocity vector, roll, pitch, and yaw angles that the aircraft should maintain at each moment.

7. The anti-shake imaging system for an infrared gimbal pod based on dual-axis gyroscope stabilization as described in claim 1, characterized in that, In step S3, the low-frequency jitter sequence is converted into a pre-compensation index for the gimbal pod based on the mechanical kinematic model and control response model of the gimbal pod.

8. The anti-shake imaging system for an infrared gimbal pod based on dual-axis gyroscope stabilization as described in claim 7, characterized in that, The process of obtaining the mechanical kinematic model includes: Measure the structural parameters of the gimbal, including the joint center distance of the two axes, rotation range, connecting rod dimensions, mass of the gimbal infrared camera, mass of the motor, center of mass position, and moment of inertia; Using a laser tracker or a high-precision tilt sensor, measure the deviation between the actual installation position of each component and the design drawings, and record the deviation data; Based on rigid body kinematics theory, a homogeneous transformation matrix is ​​established with the gimbal base as the fixed coordinate system and the infrared line-of-sight endpoint as the tool coordinate system. The motor output angle is used as the input of the homogeneous transformation matrix to calculate the attitude angle and spatial position of the infrared line-of-sight. Based on the target line-of-sight attitude, the required motor rotation angle is inversely calculated, so that the model can be mapped bidirectionally. A fixed gimbal is used, and a known angle sequence is input through a stepper motor. A high-precision gyroscope and a laser positioning instrument are used to collect actual line-of-sight attitude data. The theoretical calculation value is compared with the actual measurement value, an error compensation function is fitted, the model deviation is corrected, and finally the attitude prediction error is less than or equal to the preset value.

9. The anti-shake imaging system for an infrared gimbal pod based on dual-axis gyroscope stabilization as described in claim 7, characterized in that, The process of obtaining the control response model includes: With the gimbal isolated from the aircraft body, multiple test excitation signals are applied to the gimbal in a series. At the same time, the minute vibration displacement of the final camera platform is directly measured using measuring tools. Using the test excitation signal as the input and the small vibration displacement as the output, the system identification algorithm is used to estimate the model parameters. The specific model is a second-order system transfer function that includes inertia, damping, and stiffness terms. Its model parameters correspond to the equivalent rotational inertia of the motor and load, the viscous damping coefficient of the bearing and transmission mechanism, and the stiffness coefficient exhibited by gear clearance and structural flexibility. The final form of the model is a discrete-time transfer function or state-space equation.

10. The anti-shake imaging system for an infrared gimbal pod based on dual-axis gyroscope stabilization as described in claim 1, characterized in that, In step S4, the inertial measurement unit installed on the main body of the aircraft continuously measures the three-axis acceleration and three-axis angular velocity of the aircraft at high frequency. The real-time measured motion state data of the three-axis acceleration and three-axis angular velocity are sent to the signal processing unit of the core controller for processing. The signal processing unit first performs coordinate transformation on the original real-time measurement values, converting them from the coordinate system of the sensor to the coordinate system of the gimbal base, removing the gravitational acceleration component, and obtaining real-time vibration data representing the dynamic vibration acceleration of the aircraft's vibration state.