Airborne camera jitter image compensation method based on artificial intelligence

Abstract

The invention discloses an airborne camera jitter image compensation method based on artificial intelligence, and the method is characterized in that the method comprises a sensor module which comprises a gyroscope and an accelerometer and is used for collecting the attitude data and motion data of an airborne camera in real time, and the attitude data comprises a triaxial angular velocity, a triaxial acceleration and a quaternion attitude matrix; the image acquisition module is provided with a 1 / 1.8 inch CMOS (Complementary Metal-Oxide-Semiconductor Transistor) sensor, supports dual modes of phase focusing and laser focusing, and is used for acquiring a real-time image shot by an airborne camera; the processor module is configured to execute the following steps: step 1, performing Kalman filtering processing on the attitude data and the motion data through a sensor fusion algorithm; through cooperative work of the sensor module, the image acquisition module and the processor module and in combination with Kalman filtering and a deep learning model, high-precision anti-shake in a dynamic scene is realized, dependence on traditional optical or mechanical anti-shake hardware is reduced, and equipment weight and cost are reduced.

Description

Technical Field

[0001] The present invention relates to the technical field related to airborne cameras, and in particular to an airborne camera jitter image compensation method based on artificial intelligence. Background Art

[0002] Computer vision is a key branch of artificial intelligence, enabling computers to "see" and understand the content of images and videos. With the rapid development of deep learning and big data technologies, computer vision algorithms have made significant progress in areas such as image recognition, object detection, face recognition, and autonomous driving. Deep learning is a major breakthrough in computer vision, building deep neural networks that can automatically learn complex features in images.

[0003] Aircraft cameras play a vital role in flight safety monitoring. By monitoring the aircraft's flight status, external environment, and the behavior of passengers and crew members in real time, they can promptly identify potential safety hazards and take appropriate action.

[0004] Aircraft cameras operate in complex and harsh environments. During flight, they may encounter severe vibrations and sharp turns, which can cause unstable flight conditions to produce jittery and blurry images. To ensure a stable and clear video stream, airborne camera anti-shake technology is crucial, directly impacting the stability and clarity of the final image. However, current airborne camera anti-shake designs have the following drawbacks: optical image stabilization devices are fragile, while mechanical image stabilization devices are bulky and costly. Summary of the Invention

[0005] In order to solve the defects of the prior art, the present invention provides an airborne camera jitter image compensation method based on artificial intelligence.

[0006] In order to solve the above technical problems, the present invention provides the following technical solutions: The present invention provides an artificial intelligence-based airborne camera image jitter compensation method, comprising: The sensor module includes a gyroscope and an accelerometer, which is used to collect the attitude data and motion data of the airborne camera in real time. The attitude data includes three-axis angular velocity, three-axis acceleration and quaternion attitude matrix; The image acquisition module is equipped with a 1 / 1.8-inch CMOS sensor and supports both phase focus and laser focus modes, and is used to obtain real-time images captured by the onboard camera. A processor module configured to perform the following steps: Step 1: Perform Kalman filtering on the posture data and motion data through the sensor fusion algorithm to generate a spatiotemporal synchronized motion feature vector; Step 2: Input the motion feature vector and the real-time image into a pre-trained anti-shake deep learning model, and predict the image jitter pattern through a spatiotemporal attention mechanism; Step 3: Generate a dynamic compensation strategy based on the predicted jitter pattern, wherein the dynamic compensation strategy includes an image translation matrix, rotation parameters, and deformation weights; Step 4: Perform bilinear interpolation digital correction on the real-time image based on the compensation strategy to output a stable image.

[0007] As an optimal technical solution of the present invention, the anti-shake deep learning model is trained using a training set containing on-machine jitter image data. The training process includes image data preprocessing, model forward propagation, loss calculation, backpropagation and parameter optimization.

[0008] As a preferred technical solution of the present invention, the sensor module includes an IMU406 gyroscope, which has a built-in three-axis angular rate meter and a three-axis accelerometer, supporting a dynamic attitude algorithm to output attitude data in real time.

[0009] As a preferred technical solution of the present invention, the image data preprocessing includes: Image data cleaning to remove duplicate, invalid or erroneous image data; Image data annotation, labeling image data; Image normalization keeps the image data values ​​within a reasonable range; Image data enhancement, by rotating, scaling, flipping, etc., to expand the image dataset.

[0010] As a preferred technical solution of the present invention, the anti-shake deep learning model is trained using the Adam optimizer to update parameters, and the gradient is calculated through the back-propagation algorithm to minimize the loss function.

[0011] As a preferred technical solution of the present invention, the dynamic compensation strategy includes: Perform polar coordinate transformation on the image according to the dithering pattern; Based on the heat map, the key point coordinates are divided and the jitter distance difference between adjacent frames is calculated; Compensation parameters are generated based on lighting conditions and motion status to perform intelligent synthesis of images.

[0012] As a preferred technical solution of the present invention, the training process of the anti-shake deep learning model includes: Input image data: Provide training image dataset; Forward propagation: Image data passes through the computational layer of the anti-shake deep learning model to generate a prediction value; Calculate loss: compare the predicted value and the true value to measure the error; Backpropagation: The backpropagation algorithm efficiently calculates the gradient of the loss function with respect to the parameters of each layer of the anti-shake deep learning model. Parameter update: The Adam optimizer, as a leader among them, can continuously update the model parameters based on the calculated gradient; Repeated iteration: During the continuous training of the anti-shake model, the following steps need to be repeatedly performed: Load and preprocess image data; Feed the image data into the anti-shake deep learning model in batches; Calculate gradients and update model parameters; Monitor and adjust the learning rate as appropriate.

[0013] As a preferred technical solution of the present invention, based on the anti-shake deep learning model and its dynamic compensation strategy, combined with the common deep learning task loss function, the calculation formulas for position loss, confidence loss, and classification loss can be defined as follows: Position loss: used to evaluate the deviation between the predicted motion compensation parameters (translation matrix, rotation parameters) and the true value, and uses Smooth L1 Loss to improve robustness to outliers: ; in: : The i-th motion parameter predicted by the model (such as translation, rotation angle); The corresponding real motion parameters; : The total number of motion parameters to be regressed (such as translation X, Y, rotation When there are 3 parameters in total, ); Confidence loss: If the model predicts the confidence of the jittered area (such as whether the key point is affected by the jitter), binary cross entropy loss is used: ; in The confidence level of the model prediction for the jth region (range ); : True label (1 indicates a significant jitter area, 0 otherwise); M: the number of regions or key points divided in the image; Classification loss: If you need to classify the jitter pattern (such as high-frequency vibration, low-frequency offset, etc.), use cross-entropy loss: ; in: : The probability of the Kth type of jitter pattern predicted by the model; : One-hot encoding of the true category (for example, when category 3 is the true label, 1, the rest are 0); K: the total number of categories of jitter patterns; Total loss function: When training the model, the above losses can be weighted and summed according to task requirements: ; in, is a hyperparameter used to balance the importance of each loss term.

[0014] As a preferred technical solution of the present invention, the processor module includes a Rockchip RV1126 chip, which has a built-in neural network processor with a computing power of not less than 2.0TOPs and supports TensorFlow and PyTorch deep learning frameworks.

[0015] As a preferred technical solution of the present invention, the control method of the airborne camera includes the following steps: Collect attitude data and motion data through gyroscopes and accelerometers; Acquire real-time images and input them into pre-trained deep learning models; using the model to predict jitter patterns and generate compensation strategies; Digitally correct the real-time image and output a stable video stream.

[0016] The beneficial effects of the present invention are: This AI-based airborne camera jitter image compensation method achieves high-precision anti-shake in dynamic scenes through the collaborative work of the sensor module, image acquisition module and processor module, combined with Kalman filtering and deep learning models, reducing dependence on traditional optical or mechanical anti-shake hardware, reducing equipment weight and cost, and achieving better stability in dynamic scenes and improved performance in low-light environments. It learns various jitter patterns through a large amount of training data and performs real-time compensation through model inference in actual applications. It can handle complex jitter patterns and has strong adaptability. The effect can be continuously improved as the model is improved. It is suitable for video shooting that needs to handle complex jitter scenes, so that the airborne camera can output stable and clear images. The airborne camera is small and light, achieving the purpose of weight reduction and reducing the manufacturing and maintenance costs of the airborne camera. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings: Figure 1Schematic diagram of the structure of the airborne camera jitter image compensation method based on artificial intelligence of the present invention; Figure 2 The present invention is a schematic diagram of the image data preprocessing structure of the airborne camera jitter image compensation method based on artificial intelligence. DETAILED DESCRIPTION

[0018] The preferred embodiments of the present invention are described below with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0019] Example: Figure 1 As shown, the present invention provides an artificial intelligence-based airborne camera image jitter compensation method, comprising: The sensor module includes a gyroscope and an accelerometer, which is used to collect the attitude data and motion data of the airborne camera in real time. The attitude data includes three-axis angular velocity, three-axis acceleration and quaternion attitude matrix; The image acquisition module is equipped with a 1 / 1.8-inch CMOS sensor and supports both phase focus and laser focus modes, and is used to obtain real-time images captured by the onboard camera. A processor module configured to perform the following steps: Step 1: Perform Kalman filtering on the posture data and motion data through the sensor fusion algorithm to generate a spatiotemporal synchronized motion feature vector; Step 2: Input the motion feature vector and the real-time image into a pre-trained anti-shake deep learning model, and predict the image jitter pattern through a spatiotemporal attention mechanism; Step 3: Generate a dynamic compensation strategy based on the predicted jitter pattern, wherein the dynamic compensation strategy includes an image translation matrix, rotation parameters, and deformation weights; Step 4: Perform bilinear interpolation digital correction on the real-time image based on the compensation strategy to output a stable image; The processor module is further configured to perform real-time multi-threaded parallel computing optimization, assigning Kalman filtering and deep learning reasoning to independent threads through a task partitioning strategy to improve processing efficiency; The parallel speedup formula is: ; in, The serial calculation time is To parallelize the computation time, ; GPU-accelerated bilinear interpolation correction uses CUDA kernel functions to parallelize image compensation calculations, increasing the interpolation speed to ≥10^6 pixels per second; Multi-camera collaborative anti-shake uses a spatiotemporal alignment algorithm to synchronize the compensation parameters of multiple cameras. The alignment error formula is: ; in, is the translation matrix of the i-th camera, is the global average matrix, requiring ϵ≤0.1 pixels; Ensure synchronization of multi-camera anti-shake parameters to avoid image tearing; It also includes an abnormal jitter detection and adaptive recovery module, which determines abnormal jitter through statistical outlier analysis. The formula is: ; in, is the motion feature vector of the current frame; is the historical mean; is the standard deviation, if , then the compensation parameters are reset; Dynamic learning rate adjustment temporarily increases the model learning rate in abnormal conditions to quickly adapt to the new jitter pattern. The adjustment formula is: ; in, is the basic learning rate; Through the coordinated work of the sensor module, image acquisition module and processor module, combined with Kalman filtering and deep learning models, high-precision anti-shake in dynamic scenes is achieved, which reduces the dependence on traditional optical or mechanical anti-shake hardware, reduces the weight and cost of equipment, has better stability in dynamic scenes, and improves performance in dark environments. It learns various jitter patterns through a large amount of training data and performs real-time compensation through model inference in actual applications. It can handle complex jitter patterns and has strong adaptability. The effect can be continuously improved as the model is improved. It is suitable for video shooting that needs to handle complex jitter scenes, so that the onboard camera can output stable and clear images. The onboard camera is small and light, achieving the goal of reducing The purpose is to reduce the manufacturing and maintenance costs of airborne cameras. Through Kalman filtering (state equation and observation equation), multi-sensor data is fused and processed to generate spatiotemporal synchronized motion feature vectors, effectively eliminate noise interference, and improve motion prediction accuracy. Combined with dynamic compensation strategies, it can achieve image jitter compensation in high-speed flight scenarios and output a stable image peak signal-to-noise ratio; based on pre-trained anti-shake deep learning models (such as GRU time series modeling and spatiotemporal attention mechanism), it can accurately capture key point displacements through heat map analysis (Gaussian kernel function S(x,y)), predict complex jitter patterns (such as high-frequency vibration and low-frequency offset), and dynamically generate compensation parameters. The response time of the compensation strategy is ≤5 ms, which is suitable for high-speed motion platforms such as drones and aircraft; image data preprocessing combined with the Adam optimizer and the cosine annealing learning rate strategy significantly improves model training efficiency and enhances the model's generalization ability to multi-scene jitter (such as different lighting and motion trajectories).

[0020] Among them, the anti-shake deep learning model is trained using a training set containing on-board jitter image data. The training process includes image data preprocessing, model forward propagation, loss calculation, backpropagation and parameter optimization, ensuring that the anti-shake deep learning model can learn the complex jitter patterns in the real flight environment and improve the generalization ability and adaptability of the anti-shake algorithm.

[0021] Among them, the sensor module includes an IMU406 gyroscope, which has a built-in three-axis angular rate meter and a three-axis accelerometer, supports dynamic attitude algorithm to output attitude data in real time, and adopts IMU406 gyroscope with a built-in high-precision three-axis sensor, which can output attitude data in real time, enhance the accuracy of motion feature vectors, and thus improve the reliability of jitter prediction.

[0022] Specifically, such as Figure 2 As shown, the image data preprocessing includes: Image data cleaning to remove duplicate, invalid or erroneous image data; Image data annotation, labeling image data; Image normalization keeps the image data values ​​within a reasonable range; Image data enhancement, by rotating, scaling, flipping, etc., to expand the image dataset; Through preprocessing steps such as data cleaning, labeling, normalization and enhancement, the quality of training data is optimized, noise interference is reduced, and the efficiency and effectiveness of model training are improved.

[0023] Among them, the training of the anti-shake deep learning model uses the Adam optimizer to update parameters, and calculates the gradient through the back propagation algorithm to minimize the loss function. The Adam optimizer is combined with the back propagation algorithm to accelerate model convergence, reduce the gradient instability problem during training, and improve the accuracy of parameter updates.

[0024] The dynamic compensation strategy includes: Perform polar coordinate transformation on the image according to the dithering pattern; Based on the heat map, the key point coordinates are divided and the jitter distance difference between adjacent frames is calculated; Compensation parameters are generated based on lighting conditions and motion status to perform intelligent synthesis of images.

[0025] The training process of the anti-shake deep learning model includes: Input image data: Provide training image dataset; Forward propagation: Image data passes through the computational layer of the anti-shake deep learning model to generate a prediction value; Calculate loss: compare the predicted value and the true value to measure the error; Backpropagation: The backpropagation algorithm efficiently calculates the gradient of the loss function with respect to the parameters of each layer of the anti-shake deep learning model. Parameter update: The Adam optimizer, as a leader among them, can continuously update the model parameters based on the calculated gradient; Repeated iteration: During the continuous training of the anti-shake model, the following steps need to be repeatedly performed: Load and preprocess image data; Feed the image data into the anti-shake deep learning model in batches; Calculate gradients and update model parameters; Monitor and adjust learning rate appropriately; Through the iterative process of batch training, gradient calculation and learning rate adjustment, the model is gradually optimized during continuous training, avoiding overfitting and improving robustness.

[0026] Among them, based on the anti-shake deep learning model and its dynamic compensation strategy, combined with the common deep learning task loss function, the calculation formulas for position loss, confidence loss, and classification loss can be defined as follows: Position loss: used to evaluate the deviation between the predicted motion compensation parameters (translation matrix, rotation parameters) and the true value, and uses Smooth L1 Loss to improve robustness to outliers: ; in: : The i-th motion parameter predicted by the model (such as translation, rotation angle); The corresponding real motion parameters; : The total number of motion parameters to be regressed (such as translation X, Y, rotation When there are 3 parameters in total, ); Confidence loss: If the model predicts the confidence of the jittered area (such as whether the key point is affected by the jitter), binary cross entropy loss is used: ; in: The confidence level of the model prediction for the jth region (range ); : True label (1 indicates a significant jitter area, 0 otherwise); M: the number of regions or key points divided in the image; Classification loss: If you need to classify the jitter pattern (such as high-frequency vibration, low-frequency offset, etc.), use cross-entropy loss: ; in: : The probability of the Kth type of jitter pattern predicted by the model; : One-hot encoding of the true category (for example, when category 3 is the true label, 1, the rest are 0); K: the total number of categories of jitter patterns; Total loss function: When training the model, the above losses can be weighted and summed according to task requirements: ; in, is a hyperparameter used to balance the importance of each loss term; A multi-task loss function (position loss, confidence loss, and classification loss) is used to balance the optimization objectives of different tasks through weighted summation, thereby improving the model's comprehensive prediction ability for jitter patterns.

[0027] Based on the NPU computing power characteristics of the Rockchip RV1126 chip, this embodiment quantizes and accelerates the 6-DOF compensation matrix of the anti-shake model. The specific implementation is as follows: ; Quantization function : Mapping floating-point matrices to INT8 precision (-128~127), using the NPU's INT8 acceleration unit, increases matrix multiplication efficiency by 3 times; Rotation matrix parameter range: , matching the maximum offset of the airborne jitter; Real-time performance guarantee: On the RV1126, the single-frame compensation calculation takes less than 2ms, meeting the requirements for real-time processing of 1080p@60fps.

[0028] Among them, the processor module includes the Rockchip RV1126 chip, which has a built-in neural network processor with a computing power of no less than 2.0TOPs, supports TensorFlow and PyTorch deep learning frameworks, and uses the Rockchip RV1126 chip with a built-in high-performance neural network processor to support mainstream deep learning frameworks, ensuring low latency and high efficiency in real-time image processing.

[0029] The control method of the airborne camera includes the following steps: Collect attitude data and motion data through gyroscopes and accelerometers; Acquire real-time images and input them into pre-trained deep learning models; using the model to predict jitter patterns and generate compensation strategies; Digitally correct the real-time image and output a stable video stream; Through modular control method design, the system operation process is simplified, and end-to-end processing from data acquisition to stable output is achieved, which improves the convenience of actual deployment.

[0030] During operation, through the coordinated work of the sensor module, image acquisition module and processor module, combined with Kalman filtering and deep learning models, high-precision anti-shake in dynamic scenes is achieved, which reduces the dependence on traditional optical or mechanical anti-shake hardware, reduces the weight and cost of equipment, has better stability in dynamic scenes, and improves performance in dark environments. It learns various jitter patterns through a large amount of training data, and performs real-time compensation through model inference in actual applications. It can handle complex jitter patterns and has strong adaptability. The effect can be continuously improved with the improvement of the model. It is suitable for video shooting that needs to handle complex jitter scenes, so that the onboard camera can output stable and clear images. The onboard camera is small and light, which achieves the purpose of weight reduction and reduces the manufacturing and maintenance costs of the onboard camera. In general, the specific implementation steps of artificial intelligence anti-shake are: Collect jitter image data: Jitter image data is the "fuel" of artificial intelligence anti-shake technology. To train an artificial intelligence anti-shake model, a large amount of high-quality image data is required. Image data comes from two sources: one is manually collected on-board jitter image data; the other is image data transmitted in real time on-board; image data preprocessing: raw image data is usually chaotic and needs to be cleaned and standardized to improve the efficiency of model training; anti-shake deep learning model: This is the core stage of AI anti-shake learning. It is a process of adjusting the parameters of the anti-shake model to the optimal state by providing image data and targets. Its goal is to allow the anti-shake model to learn patterns from image data so that it can make accurate predictions or decisions on new image data; identify and predict jitter patterns: The anti-shake model is used to analyze the jitter trajectory, lighting conditions, and motion status of the onboard camera in real time to identify and predict jitter patterns; real-time inference and generation of compensation strategies: By collecting multiple frames of images, combining sensor data (gyroscope, accelerometer) and jitter patterns to infer the jitter status of the image data, and then generating an image compensation strategy, the image is intelligently synthesized to reduce image blur and jitter.

[0031] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. An artificial intelligence-based airborne camera image jitter compensation method, characterized in that: include: The sensor module includes a gyroscope and an accelerometer, which is used to collect the attitude data and motion data of the airborne camera in real time. The attitude data includes three-axis angular velocity, three-axis acceleration and quaternion attitude matrix; The image acquisition module is equipped with a CMOS sensor and supports both phase focus and laser focus modes, and is used to obtain real-time images captured by the onboard camera; A processor module configured to perform the following steps: Step 1: Perform Kalman filtering on the posture data and motion data through the sensor fusion algorithm to generate a spatiotemporal synchronized motion feature vector; Step 2: Input the motion feature vector and the real-time image into a pre-trained anti-shake deep learning model, and predict the image jitter pattern through a spatiotemporal attention mechanism; Step 3: Generate a dynamic compensation strategy based on the predicted jitter pattern, wherein the dynamic compensation strategy includes an image translation matrix, rotation parameters, and deformation weights; Step 4: Perform bilinear interpolation digital correction on the real-time image based on the compensation strategy to output a stable image.

2. The method for compensating for image jitter of an airborne camera based on artificial intelligence according to claim 1, wherein: The anti-shake deep learning model is trained using a training set containing on-machine jittered image data. The training process includes image data preprocessing, model forward propagation, loss calculation, backpropagation and parameter optimization.

3. The method for compensating for image jitter of an airborne camera based on artificial intelligence according to claim 1, wherein: The sensor module includes an IMU406 gyroscope, which has a built-in three-axis angular rate meter and a three-axis accelerometer, and supports a dynamic attitude algorithm to output attitude data in real time.

4. The method for compensating for image jitter of an airborne camera based on artificial intelligence according to claim 2, wherein: The image data preprocessing includes: Image data cleaning to remove duplicate, invalid or erroneous image data; Image data annotation, labeling image data; Image normalization keeps the image data values ​​within a reasonable range; Image data augmentation, by rotating, scaling, and flipping images to expand the image dataset.

5. The method for compensating for image jitter of an airborne camera based on artificial intelligence according to claim 1, wherein: The anti-shake deep learning model is trained using the Adam optimizer to update parameters and calculate gradients through the back-propagation algorithm to minimize the loss function.

6. The method for compensating for airborne camera jitter image based on artificial intelligence according to claim 1, characterized in that: The dynamic compensation strategy includes: Perform polar coordinate transformation on the image according to the dithering pattern; Based on the heat map, the key point coordinates are divided and the jitter distance difference between adjacent frames is calculated; Compensation parameters are generated based on lighting conditions and motion status to perform intelligent synthesis of images.

7. The method for compensating for image jitter of an airborne camera based on artificial intelligence according to claim 1, wherein: The training process of the anti-shake deep learning model includes: Input image data: Provide training image dataset; Forward propagation: Image data passes through the computational layer of the anti-shake deep learning model to generate a prediction value; Calculate loss: compare the predicted value and the true value to measure the error; Backpropagation: The backpropagation algorithm efficiently calculates the gradient of the loss function with respect to the parameters of each layer of the anti-shake deep learning model. Parameter update: The Adam optimizer, as a leader among them, can continuously update the model parameters based on the calculated gradient; Repeated iteration: During the continuous training of the anti-shake model, the following steps need to be repeatedly performed: Load and preprocess image data; Feed the image data into the anti-shake deep learning model in batches; Calculate gradients and update model parameters; Monitor and adjust the learning rate as appropriate.

8. The method for compensating for image jitter of an airborne camera based on artificial intelligence according to claim 7, wherein: Based on the anti-shake deep learning model and its dynamic compensation strategy, combined with the common deep learning task loss function, the calculation formulas for position loss, confidence loss, and classification loss can be defined as follows: Position loss: used to evaluate the deviation between the predicted motion compensation parameters (translation matrix, rotation parameters) and the true value, and uses Smooth L1 Loss to improve robustness to outliers: ； in: : The i-th motion parameter predicted by the model (such as translation, rotation angle); The corresponding real motion parameters; : The total number of motion parameters to be regressed; Confidence loss: If the model predicts the confidence of the jittered area (such as whether the key point is affected by the jitter), binary cross entropy loss is used: ； in The confidence level of the model prediction for the jth region (range ); : True label (1 indicates a significant jitter area, 0 otherwise); M: the number of regions or key points divided in the image; Classification loss: If you need to classify the jitter pattern (such as high-frequency vibration, low-frequency offset, etc.), use cross-entropy loss: ； in: : The probability of the Kth type of jitter pattern predicted by the model; : One-hot encoding of the true category; K: the total number of categories of jitter patterns; Total loss function: When training the model, the above losses can be weighted and summed according to task requirements: ； in, is a hyperparameter used to balance the importance of each loss term.

9. The method for compensating for airborne camera jitter image based on artificial intelligence according to claim 1, characterized in that: The processor module includes a Rockchip RV1126 chip, which has a built-in neural network processor with a computing power of no less than 2.0TOPs and supports TensorFlow and PyTorch deep learning frameworks.

10. The method for compensating for airborne camera jitter images based on artificial intelligence according to claim 1, wherein: The control method of the airborne camera comprises the following steps: Collect attitude data and motion data through gyroscopes and accelerometers; Acquire real-time images and input them into pre-trained deep learning models; using the model to predict jitter patterns and generate compensation strategies; Digitally correct the real-time image and output a stable video stream.

Images