Live broadcast picture stabilizing method based on artificial intelligence

Abstract

The invention discloses a live broadcast picture stabilizing method based on artificial intelligence, relates to the technical field of artificial intelligence, and can further improve the picture stability through picture stability evaluation and filter dynamic adjustment. The stability degree of the current picture is evaluated by calculating a picture stability index Sx (t), and when an output result is lower than 0.7, a filter dynamic adjustment mechanism is triggered. And executing a dynamic adjustment mechanism of the filter by adjusting the size Nnew of the window, and further smoothing the image jitter. After the step S1 and the step S2 are executed twice, the jitter amplitude delta P (t) is compared with the jitter threshold Pthreshold, an early warning mechanism is triggered to remind a photographer to adjust the posture of the equipment, and abnormal image jitter caused by the fact that the motion of the equipment exceeds the limit is avoided in time. According to the feedback process of stability evaluation and filtering optimization, the adaptability and stability are remarkably improved, and it is ensured that the live broadcast picture quality is kept stable and smooth in a high-dynamic motion scene.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a live broadcast picture stabilization method based on artificial intelligence. Background Art

[0002] Artificial intelligence technology is widely used in image and video stabilization. Especially in the field of high-dynamic video shooting, real-time live broadcast image stabilization technology has become a research focus and difficulty. In extreme sports scenes, especially in the live broadcast of extreme skiing, the camera equipment will be affected by rapidly changing movements and environmental interference such as vibration, wind speed and angle changes, resulting in unstable images. Therefore, through the live broadcast image stabilization method based on artificial intelligence, it is possible to analyze and predict the movement trend of the camera equipment in real time, dynamically adjust and compensate for the image jitter, and provide viewers with a clear and smooth extreme sports live broadcast experience.

[0003] At present, due to the fast speed, changing terrain and complex external environment in extreme skiing, traditional hardware anti-shake and general video stabilization algorithms are difficult to cope with this high-dynamic scene. When the device shakes frequently, the picture will be discontinuous, distorted and other abnormal effects, and the audience experience will be significantly reduced. Such defects will not only make the picture unable to clearly capture the wonderful movements of the athletes, but may also make the audience feel uncomfortable due to the shaking of the picture, affecting the smoothness and viewing of the live broadcast, and reducing the overall quality of the extreme skiing live broadcast. Through the picture stabilization method based on artificial intelligence, these problems can be effectively solved, and the picture stability and visual experience can be improved. Summary of the invention

[0004] In view of the deficiencies in the prior art, the present invention provides a live broadcast picture stabilization method based on artificial intelligence, which solves the problems mentioned in the background technology.

[0005] To achieve the above objectives, the present invention is implemented through the following technical solutions: A live broadcast picture stabilization method based on artificial intelligence comprises the following steps:

[0006] S1. By installing a dynamic monitoring device group on the live camera equipment, the environment and motion data in the extreme skiing live broadcast are collected in real time, and the collected environment and motion data are transmitted to the central processing module, the environment and motion data are pre-processed in the central processing module to obtain a standard data set, and then the standard data set is stored in a time series database;

[0007] S2. Use the AI ​​prediction model to analyze the movement trend of the live camera equipment during skiing, predict the motion jitter amplitude Mpred(t) of the live camera equipment at time t in the future, and then calculate the cumulative jitter amplitude △P(t) of the live camera equipment at time t;

[0008] S3. According to the accumulated jitter amplitude △P(t), the lens trajectory Ts(t) is calculated and output for compensation adjustment, and the live camera equipment is adjusted and compensated in real time. Based on the output adjusted lens trajectory Ts(t), the optimized feedback function Fopt(t) is calculated and outputted according to the real-time collected picture data through the real-time feedback formula, and dynamic adjustment and compensation are performed to optimize the picture stability.

[0009] S4. After further optimization, the picture stability index Sx(t) is calculated, and the stability of the current picture is evaluated based on the output result of the picture stability index Sx(t), and a dynamic adjustment filter mechanism is executed based on the corresponding evaluation result, and the dynamically adjusted filter window Nnew is output based on the AI ​​algorithm to further smooth the live picture;

[0010] S5. After the dynamic adjustment filter mechanism is executed, S1 and S2 are executed for the second time to output the cumulative jitter amplitude △P (t), and the jitter threshold Pthreshold is optimized and evaluated, and an early warning mechanism is generated based on the evaluation results.

[0011] Preferably, S1 includes S11, S12 and S13;

[0012] S11. By installing a dynamic monitoring device group on the live video camera, the environment and motion data of the extreme skiing live broadcast are collected in real time;

[0013] The dynamic monitoring equipment group includes acceleration sensors, wind speed sensors, GPS locators, gyroscopes and vibration sensors;

[0014] Environmental and motion data include the acceleration a of the live camera device in the horizontal direction x , acceleration a in the vertical direction y , acceleration in the vertical direction a z , angular velocity w in the horizontal direction x , angular velocity w in the vertical direction y , angular velocity w in the vertical direction z , wind speed change data Ws, athlete's sliding speed Vs and vibration frequency Vf;

[0015] S12, directly connecting the dynamic monitoring device to the central processing module, and then using an analog-to-digital converter ADC to convert the sensor signal into a digital signal and send it to the central processing module;

[0016] The central processing module receives environmental and motion data in real time, pre-processes and timestamps the environmental and motion data, and summarizes them after pre-processing to obtain a standard data set. The standard data set is then denoised and smoothed using a sliding average filter to optimize data noise caused by sensor fluctuations or environmental changes. The standard data set includes the gravity change index G(t) at time t, the wind speed change index Ws(t) at time t, and the attitude angle of the live camera at time t. , the athlete's sliding acceleration As(t) at time t and the vibration frequency Vf(t) at time t;

[0017] The gravity change index G(t) is calculated by the acceleration a of the horizontal axis of the live camera at time t. x (t), acceleration a in the longitudinal direction y (t) and the acceleration a in the vertical direction z (t) Use gravity component processing and calculation to extract the gravity change of the live camera equipment. The specific algorithm formula is: , where g represents the standard acceleration of gravity, which is 9.81 m / s 2 ;

[0018] Live camera equipment posture angle The angular velocity w of the live camera device in the horizontal direction at time t is collected x (t), angular velocity w in the longitudinal direction y (t), angular velocity w in the vertical direction z (t), and the specific algorithm formula is as follows: Where, dt represents the time-integrated variable;

[0019] The athlete's sliding acceleration As(t) is obtained by integrating the collected athlete's sliding speed Vs(t) at time t. The specific algorithm formula is: , dVs(t) represents the small change in the athlete's sliding speed;

[0020] S13. Build a time series database in the central processing module, and set a write port, a write port, a real-time data table and a historical data table. Write the standard data set obtained in real time after preprocessing into the real-time data table of the time series database through the write port for data storage. At the same time, transfer the standard data set in the original real-time data table to the historical data table and sort them according to the marked timestamps.

[0021] Preferably, S2 includes S21, S22 and S23;

[0022] S21. Build an AI prediction model through a neural network and a long short-term memory network (LSTM), extract the standard data set in the historical data table through the time series database write port, learn the standard data set in the historical data table, identify the jitter pattern of the live camera equipment, and train the AI ​​prediction model.

[0023] S22, extracting a standard data set from a real-time data table through a time series database write port, inputting the data set into an AI prediction model, and calculating and outputting a prediction of the motion jitter amplitude Mpred(t) of the live camera device at time t in the future;

[0024] The output formula of the AI ​​prediction model is:

[0025] ;

[0026] In the formula, f represents the prediction function learned by the AI ​​prediction model, which shows how the model combines various input parameters. Through learning, the AI ​​model can identify the influence weight and interaction relationship of each parameter on the device jitter, and thus output the jitter amplitude of the device at the future moment.

[0027] Preferably, S23, predicting the motion jitter amplitude Mpred(t) of the live camera device at time t in the future based on the output of the AI ​​prediction model, analyzing the cumulative impact of the jitter amplitude in the entire time interval, and calculating and outputting the cumulative jitter amplitude △P(t) of the live camera device at time t;

[0028] The cumulative jitter amplitude △P(t) at time t is calculated and output by the following algorithm formula:

[0029] .

[0030] Preferably, S3 includes S31 and S32;

[0031] S31, based on the acquired cumulative jitter amplitude △P(t), the correction algorithm is used to perform real-time correction of the lens of the live camera device, the lens trajectory Ts(t) is adjusted by calculating the output compensation, and then the mechanical adjustment function of the pan / tilt is used to adjust the lens angle and position of the camera device in real time to compensate for the jitter caused by the movement of the device;

[0032] The compensation adjustment lens trajectory Ts(t) is calculated and output by the following correction algorithm:

[0033] ;

[0034] Where Pcamera(t) represents the original trajectory of the live camera device, which is obtained through the GPS locator.

[0035] Preferably, S32, after the live camera device lens is corrected, each frame is output, and then the difference between the compensated adjustment lens trajectory Ts(t) and the original trajectory Pcamera(t) of the live camera device is calculated through a real-time feedback formula, and an optimized feedback function Fopt(t) is output, and the stability result of the current picture at each moment is analyzed to dynamically adjust the compensation to optimize the picture stability;

[0036] The optimization feedback function Fopt(t) calculates the output through the following real-time feedback formula:

[0037] ;

[0038] In the formula, represents the relative error between the compensated trajectory and the original trajectory, represents the adjustment coefficient of the relative error between the compensated trajectory and the original trajectory, Indicates the adjustment coefficient of the accumulated jitter amplitude.

[0039] Preferably, S4 includes S41, S42 and S43;

[0040] S41, after dynamically adjusting and compensating to optimize the image stability, the stability of the live broadcast image after compensation is analyzed by calculating the image stability index Sx(t);

[0041] The image stability index Sx(t) is calculated and output by the following algorithm formula;

[0042] .

[0043] Preferably, S42, based on the output result of the picture stability index Sx(t), evaluating the picture stability after the dynamic adjustment compensation of the current picture, and generating a filter dynamic adjustment mechanism based on the evaluation, the specific evaluation content is as follows;

[0044] When the image stability index Sx(t) is ≥ 0.7, it means that the jitter amplitude is normal relative to the original motion trajectory of the device, indicating that the compensated image is stable;

[0045] When the image stability index Sx(t) is less than 0.7, it indicates that the jitter amplitude is abnormal relative to the motion trajectory of the device, indicating that the live broadcast image of the live camera device is unstable after compensation, and the dynamic adjustment filter mechanism is triggered at this time.

[0046] Preferably, S43, after the dynamic adjustment filter mechanism is triggered, the window size n of the sliding average filter is adjusted to obtain the dynamically adjusted filter window Nnew, and the preprocessed standard data set is smoothed to eliminate noise and small jitter;

[0047] The dynamically adjusted filter window Nnew is calculated and outputted by the following algorithm formula;

[0048] ;

[0049] In the formula, Pthreshold represents the preset jitter threshold. The AI ​​prediction model automatically learns the optimal Pthreshold setting based on historical jitter data, scene changes, and user preferences for picture quality. Indicates the difference between the current jitter amplitude and the jitter threshold. Represents the filter window adjustment coefficient of the sliding average filter.

[0050] Preferably, S5 includes S51;

[0051] S51, after the dynamic adjustment filter mechanism is executed, S1 and S2 are executed for the second time, the output cumulative jitter amplitude △P (t) and the jitter threshold Pthreshold are calculated for optimization evaluation, and based on the optimization evaluation results, an early warning prompt is triggered. The specific evaluation content is as follows;

[0052] When the cumulative jitter amplitude △P(t)>jitter threshold Pthreshold, it indicates jitter abnormality. At this time, an early warning prompt is generated in the live broadcast screen to remind the cameraman to adjust the equipment posture and correct the shooting strategy;

[0053] When the cumulative jitter amplitude △P (t) ≤ the jitter threshold Pthreshold, it indicates that the live screen temperature is high and no warning is required.

[0054] Beneficial Effects

[0055] The present invention provides a method for stabilizing live broadcast images based on artificial intelligence, which has the following beneficial effects:

[0056] (1) This method uses a dynamic monitoring device group to collect environmental and motion data in real time and perform preprocessing in an extreme skiing live broadcast environment, so as to obtain multi-dimensional dynamic data of the equipment under complex motion conditions. The central processing module generates a standard data set by denoising and smoothing the real-time data, and stores it in a time series database, making the data call in subsequent steps more accurate and efficient. This method ensures real-time response to external environments such as wind speed changes, equipment attitude angles, athlete sliding acceleration, etc., providing a stable and reliable data source for the AI ​​prediction model, thereby greatly reducing the jitter frequency of the picture during the live broadcast and enhancing the overall picture smoothness.

[0057] (2) This method uses the AI ​​prediction model to analyze the movement trend of the device and predict the jitter amplitude Mpred(t) and the cumulative jitter amplitude ΔP(t) at future moments. The correction algorithm calculates and outputs the lens compensation adjustment trajectory Ts(t) in real time, and uses the mechanical adjustment function of the gimbal to correct the lens angle and position of the camera device in real time to compensate for the jitter in the live broadcast. In addition, the real-time feedback function Fopt(t) dynamically analyzes the difference between the compensation trajectory and the original trajectory and continuously optimizes the compensation effect, effectively suppressing the jitter and offset of the lens and making the picture performance more stable. The AI ​​prediction model and feedback optimization mechanism can adapt to the changes in complex motion scenes to ensure that the audience obtains clear and stable visual effects.

[0058] (3) This method can further improve the stability of the picture through picture stability evaluation and filter dynamic adjustment. Specifically, by calculating the picture stability index Sx(t) and evaluating the stability of the current picture based on the stability index, the filter dynamic adjustment mechanism is triggered when the index is lower than the threshold. The filter further smoothes the picture jitter by adjusting the window size Nnew, while maintaining the continuity of the picture while eliminating noise and small jitter. After executing steps S1 and S2 twice, by comparing the jitter amplitude ΔP(t) with the jitter threshold Pthreshold, the early warning mechanism is triggered to remind the cameraman to adjust the equipment posture and avoid abnormal picture jitter caused by excessive equipment movement in time. This feedback process of stability evaluation and filter optimization significantly improves adaptability and stability, ensuring that the live picture quality remains stable and smooth in high-dynamic motion scenes. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 The figure is a schematic diagram of the steps of the live broadcast picture stabilization method based on artificial intelligence of the present invention. DETAILED DESCRIPTION

[0060] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] Example 1

[0062] The present invention provides a method for stabilizing live broadcast images based on artificial intelligence. Figure 1 , including the following steps:

[0063] S1. By installing a dynamic monitoring device group on the live camera equipment, the environment and motion data in the extreme skiing live broadcast are collected in real time, and the collected environment and motion data are transmitted to the central processing module, the environment and motion data are pre-processed in the central processing module to obtain a standard data set, and then the standard data set is stored in a time series database;

[0064] S2. Use the AI ​​prediction model to analyze the movement trend of the live camera equipment during skiing, predict the motion jitter amplitude Mpred(t) of the live camera equipment at time t in the future, and then calculate the cumulative jitter amplitude △P(t) of the live camera equipment at time t;

[0065] S3. According to the accumulated jitter amplitude △P(t), the lens trajectory Ts(t) is calculated and output for compensation adjustment, and the live camera equipment is adjusted and compensated in real time. Based on the output adjusted lens trajectory Ts(t), the optimized feedback function Fopt(t) is calculated and outputted according to the real-time collected picture data through the real-time feedback formula, and dynamic adjustment and compensation are performed to optimize the picture stability.

[0066] S4. After further optimization, the picture stability index Sx(t) is calculated, and the stability of the current picture is evaluated based on the output result of the picture stability index Sx(t), and a dynamic adjustment filter mechanism is executed based on the corresponding evaluation result, and the dynamically adjusted filter window Nnew is output based on the AI ​​algorithm to further smooth the live picture;

[0067] S5. After the dynamic adjustment filter mechanism is executed, S1 and S2 are executed for the second time to output the cumulative jitter amplitude △P (t), and the jitter threshold Pthreshold is optimized and evaluated, and an early warning mechanism is generated based on the evaluation results.

[0068] In this embodiment, the method uses a dynamic monitoring device group composed of multiple sensors to collect environmental and motion data in real time and preprocess them, generate a standard data set and store it in a time series database, and provide accurate real-time data support for the subsequent AI prediction model; the AI ​​prediction model is used to analyze and predict the jitter trend of the device, and accurate data is provided for the picture stabilization adjustment by predicting the jitter amplitude Mpred(t) and the cumulative jitter amplitude ΔP(t). The compensation adjustment algorithm outputs the compensation adjustment lens trajectory Ts(t) of the lens trajectory correction, and generates the optimized feedback function Fopt(t) in combination with the real-time feedback formula, which effectively improves the flexibility of dynamic compensation and picture stability. Subsequently, the picture is further smoothed by the picture stability index Sx(t) and the filter dynamic adjustment mechanism, providing a guarantee for high-quality picture output. By executing the data collection and prediction model twice, and realizing the optimization evaluation and early warning mechanism based on the jitter threshold Pthreshold, the cameraman can intervene in time and optimize the equipment posture. Unlike traditional anti-shake equipment, this method realizes adaptive and high-precision picture stabilization through AI algorithm and dynamic feedback optimization, especially in complex motion scenes, it can better cope with various uncertain factors such as wind speed, vibration and rapid steering. Compared with current stabilization methods, this method not only reduces image blur and jitter, but also improves image clarity and stability, significantly improving the audience's visual experience and comprehensively improving the stability and viewing experience of extreme sports live broadcasts.

[0069] Example 2

[0070] This embodiment is explained in Example 1, please refer to Figure 1 , specifically: S1 includes S11, S12 and S13;

[0071] S11. By installing a dynamic monitoring device group on the live video camera, the environment and motion data of the extreme skiing live broadcast are collected in real time;

[0072] The dynamic monitoring equipment group includes acceleration sensors, wind speed sensors, GPS locators, gyroscopes and vibration sensors;

[0073] Environmental and motion data include the acceleration a of the live camera device in the horizontal direction x , acceleration a in the vertical direction y , acceleration in the vertical direction a z , angular velocity w in the horizontal direction x , angular velocity w in the vertical direction y , angular velocity w in the vertical direction z , wind speed change data Ws, athlete's sliding speed Vs and vibration frequency Vf;

[0074] S12, directly connecting the dynamic monitoring device to the central processing module, and then using an analog-to-digital converter ADC to convert the sensor signal into a digital signal and send it to the central processing module;

[0075] The central processing module receives environmental and motion data in real time, pre-processes and timestamps the environmental and motion data, and summarizes them after pre-processing to obtain a standard data set. The standard data set is then denoised and smoothed using a sliding average filter to optimize data noise caused by sensor fluctuations or environmental changes. The standard data set includes the gravity change index G(t) at time t, the wind speed change index Ws(t) at time t, and the attitude angle of the live camera at time t. , the athlete's sliding acceleration As(t) at time t and the vibration frequency Vf(t) at time t;

[0076] The gravity change index G(t) is calculated by the acceleration a of the horizontal axis of the live camera at time t. x (t), acceleration a in the longitudinal direction y (t) and the acceleration a in the vertical direction z (t) Use gravity component processing and calculation to extract the gravity change of the live camera equipment. The specific algorithm formula is: , where g represents the standard acceleration of gravity, which is 9.81 m / s 2 ;

[0077] Live camera equipment posture angle The angular velocity w of the live camera device in the horizontal direction at time t is collected x (t), angular velocity w in the longitudinal direction y (t), angular velocity w in the vertical direction z (t), and the specific algorithm formula is as follows: Where, dt represents the time-integrated variable;

[0078] The athlete's sliding acceleration As(t) is obtained by integrating the collected athlete's sliding speed Vs(t) at time t. The specific algorithm formula is: , dVs(t) represents the small change in the athlete's sliding speed;

[0079] S13. Build a time series database in the central processing module, and set a write port, a write port, a real-time data table and a historical data table. Write the standard data set obtained in real time after preprocessing into the real-time data table of the time series database through the write port for data storage. At the same time, transfer the standard data set in the original real-time data table to the historical data table and sort them according to the marked timestamps.

[0080] In this embodiment, the method can collect environmental and motion data in real time through the installed dynamic monitoring equipment group, ensuring that the equipment can accurately perceive external environmental changes and motion status in high-dynamic scenes. By converting analog signals into digital signals, and preprocessing, denoising and smoothing the environmental data in the central processing module, a standardized motion parameter data set is obtained, which effectively reduces the noise interference caused by sensor fluctuations and external environmental changes. The data is then stored in the real-time data table and historical data table of the time series database to facilitate subsequent calls and analysis, ensuring the integrity and timing of the data. This method not only realizes the structured storage of stability data and the traceability of historical records, but also provides reliable data support for AI prediction models and real-time compensation.

[0081] Example 3

[0082] This embodiment is explained in Example 2. Please refer to Figure 1 , specifically: S2 includes S21, S22 and S23;

[0083] S21. Build an AI prediction model through a neural network and a long short-term memory network (LSTM), extract the standard data set in the historical data table through the time series database write port, learn the standard data set in the historical data table, identify the jitter pattern of the live camera equipment, and train the AI ​​prediction model.

[0084] S22, extracting a standard data set from a real-time data table through a time series database write port, inputting the data set into an AI prediction model, and calculating and outputting a prediction of the motion jitter amplitude Mpred(t) of the live camera device at time t in the future;

[0085] The output formula of the AI ​​prediction model is:

[0086] ;

[0087] In the formula, f represents the prediction function learned by the AI ​​prediction model, which shows how the model combines various input parameters. Through learning, the AI ​​model can identify the influence weight and interaction relationship of each parameter on the device jitter, and thus output the jitter amplitude of the device at the future moment.

[0088] S23, predicting the motion jitter amplitude Mpred(t) of the live camera device at time t in the future based on the output of the AI ​​prediction model, analyzing the cumulative impact of the jitter amplitude in the entire time interval, and calculating and outputting the cumulative jitter amplitude △P(t) of the live camera device at time t;

[0089] The cumulative jitter amplitude △P(t) at time t is calculated and output by the following algorithm formula:

[0090] .

[0091] In this embodiment, the method constructs an AI prediction model of a neural network and a long short-term memory network LSTM, and uses the historical data set in the time series database to train the AI ​​prediction model, so that it can accurately identify the interaction between the jitter pattern of the camera device and different environmental parameters. Next, the real-time data is input into the AI ​​prediction model, and the jitter amplitude Mpred(t) at the future time t is output, so as to predict the movement trend of the device in advance and ensure that compensation preparations can be made before the actual jitter occurs. Further, the cumulative jitter amplitude ΔP(t) of the device within time t is calculated based on the predicted jitter amplitude Mpred(t), providing an accurate adjustment basis for the subsequent compensation algorithm. Compared with the current single filtering or static prediction method, the AI ​​prediction model realizes dynamic learning and prediction of complex jitter, is more sensitive to changes in multi-dimensional parameters, and can more effectively cope with complex motion environments. This AI-based prediction scheme has achieved significant improvements in accuracy and flexibility, not only improving the smoothness and stability of the picture, but also greatly reducing the picture blur and delay caused by jitter.

[0092] Example 4

[0093] This embodiment is explained in Example 3, please refer to Figure 1 , specifically: S3 includes S31 and S32;

[0094] S31, based on the acquired cumulative jitter amplitude △P(t), the correction algorithm is used to perform real-time correction of the lens of the live camera device, the lens trajectory Ts(t) is adjusted by calculating the output compensation, and then the mechanical adjustment function of the pan / tilt is used to adjust the lens angle and position of the camera device in real time to compensate for the jitter caused by the movement of the device;

[0095] The compensation adjustment lens trajectory Ts(t) is calculated and output by the following correction algorithm:

[0096] ;

[0097] Where Pcamera(t) represents the original trajectory of the live camera device, which is obtained through the GPS locator.

[0098] S32, after the live camera lens is corrected, each frame is output, and then the difference between the compensated adjusted lens trajectory Ts(t) and the original trajectory Pcamera(t) of the live camera is calculated through a real-time feedback formula, and an optimized feedback function Fopt(t) is output, and the stability result of the current picture at each moment is analyzed to dynamically adjust the compensation to optimize the picture stability;

[0099] The optimization feedback function Fopt(t) calculates the output through the following real-time feedback formula:

[0100] ;

[0101] In the formula, represents the relative error between the compensated trajectory and the original trajectory, represents the adjustment coefficient of the relative error between the compensated trajectory and the original trajectory, Indicates the adjustment coefficient of the accumulated jitter amplitude.

[0102] In this embodiment, the method generates a compensation adjustment trajectory Ts(t) based on the cumulative jitter amplitude ΔP(t) using a correction algorithm, and corrects the lens angle and position of the camera device in real time through the mechanical adjustment function of the pan / tilt, thereby effectively offsetting the jitter caused by external motion and keeping the picture stable. The real-time feedback function Fopt(t) dynamically analyzes the difference between the compensated trajectory Ts(t) and the original trajectory Pcamera(t), and adjusts the compensation strength and optimization strategy in real time. This feedback mechanism can flexibly adjust the compensation strength according to the real-time picture stability result, thereby ensuring that the picture always maintains the best state in a changing motion environment. Compared with the traditional static compensation method, real-time compensation and feedback optimization achieve more accurate and flexible jitter control, greatly improving the adaptability in complex motion scenes. The real-time feedback mechanism has adaptive capabilities, and can automatically increase the compensation strength when the jitter amplitude increases to ensure that the picture quality is not affected. This improvement significantly improves the clarity and smoothness of the live picture, avoids picture jumping and blurring, and brings a better viewing experience.

[0103] Example 5

[0104] This embodiment is explained in Example 4. Please refer to Figure 1 , specifically: S4 includes S41, S42 and S43;

[0105] S41, after dynamically adjusting and compensating to optimize the image stability, the stability of the live broadcast image after compensation is analyzed by calculating the image stability index Sx(t);

[0106] The image stability index Sx(t) is calculated and output by the following algorithm formula;

[0107] .

[0108] S42, based on the output result of the picture stability index Sx(t), evaluating the picture stability after the current picture dynamic adjustment compensation, and generating a filter dynamic adjustment mechanism based on the evaluation, the specific evaluation content is as follows;

[0109] When the image stability index Sx(t) is ≥ 0.7, it means that the jitter amplitude is normal relative to the original motion trajectory of the device, indicating that the compensated image is stable;

[0110] When the image stability index Sx(t) is less than 0.7, it indicates that the jitter amplitude is abnormal relative to the motion trajectory of the device, indicating that the live broadcast image of the live camera device is unstable after compensation, and the dynamic adjustment filter mechanism is triggered at this time.

[0111] S43, after the dynamic adjustment filter mechanism is triggered, the window size n of the sliding average filter is adjusted to obtain the dynamically adjusted filter window Nnew, and the preprocessed standard data set is smoothed to eliminate noise and small jitter;

[0112] The dynamically adjusted filter window Nnew is calculated and outputted by the following algorithm formula;

[0113] ;

[0114] In the formula, Pthreshold represents the preset jitter threshold. The AI ​​prediction model automatically learns the optimal Pthreshold setting based on historical jitter data, scene changes, and user preferences for picture quality. Indicates the difference between the current jitter amplitude and the jitter threshold. Represents the filter window adjustment coefficient of the sliding average filter.

[0115] In this embodiment, the method realizes dynamic evaluation and adaptive filtering adjustment based on the picture stability index Sx(t), further improving the stability and quality of the live picture. Specifically, the picture stability index Sx(t) is calculated, the compensation effect of the current picture is analyzed, and a stability feedback basis is provided. By evaluating the picture stability index, the threshold value 0.7 is used to determine whether the picture is stable. When Sx(t) is lower than the threshold value, the dynamic adjustment mechanism of the filter is triggered, thereby enhancing the processing capability of unstable pictures. According to the dynamically adjusted filter window Nnew, the smoothing effect of the sliding average filter is optimized, small jitters and noise are suppressed, and the picture is ensured to maintain high smoothness and clarity. Compared with the traditional static filtering method, the step has adaptive capability by dynamically adjusting the filter window, and can automatically optimize the filtering strength according to the real-time jitter situation. This adaptive filtering mechanism based on AI prediction and real-time feedback can automatically adjust the filtering strategy in a violent motion environment, ensuring the stability and visual effect of the picture under complex conditions. Finally, the improved method effectively reduces the picture blur and jumping caused by small jitters, allowing the audience to experience a smoother and clearer live picture effect.

[0116] Example 6

[0117] This embodiment is explained in Example 5. Please refer to Figure 1 , specifically: S5 includes S51;

[0118] S51, after the dynamic adjustment filter mechanism is executed, S1 and S2 are executed for the second time, the output cumulative jitter amplitude △P (t) and the jitter threshold Pthreshold are calculated for optimization evaluation, and based on the optimization evaluation results, an early warning prompt is triggered. The specific evaluation content is as follows;

[0119] When the cumulative jitter amplitude △P(t)>jitter threshold Pthreshold, it indicates jitter abnormality. At this time, an early warning prompt is generated in the live broadcast screen to remind the cameraman to adjust the equipment posture and correct the shooting strategy;

[0120] When the cumulative jitter amplitude △P (t) ≤ the jitter threshold Pthreshold, it indicates that the live screen temperature is high and no warning is required.

[0121] In this embodiment, the method further realizes real-time jitter monitoring and intelligent early warning functions after completing the filter adjustment, and improves the ability to cope with extreme jitter. Specifically, after the dynamic filter adjustment is performed, the cumulative jitter amplitude ΔP(t) is recalculated and compared with the jitter threshold Pthreshold by performing steps S1 and S2 twice, so as to achieve a comprehensive evaluation of the stability of the picture. When ΔP(t) exceeds the threshold Pthreshold, an early warning prompt will be generated to remind the cameraman to adjust the equipment posture to prevent severe jitter from continuously affecting the picture quality; when ΔP(t) is within the threshold range, there is no need to trigger an early warning to ensure the smooth output of the live picture. Compared with current technical means, this method improves the picture protection capability in complex environments through secondary jitter evaluation and intelligent early warning, so that the cameraman can intervene and optimize the shooting strategy in advance. This improvement not only enhances stability and fault tolerance, but also minimizes the impact of jitter through the early warning mechanism, ensures the clarity and smoothness of the live picture, and brings a higher quality live experience to the audience.

[0122] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and alterations may be made to the embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A live video stabilization method based on artificial intelligence, characterized by: The following steps are involved: S1. By installing a dynamic monitoring device group on the live camera device, the environment and motion data in the extreme skiing live broadcast are collected in real time, and the collected environment and motion data are transmitted to the central processing module, the environment and motion data are pre-processed in the central processing module to obtain a standard data set, and then the standard data set is stored in a time series database; S2. Use the AI ​​prediction model to analyze the movement trend of the live camera equipment during skiing, predict the motion jitter amplitude Mpred(t) of the live camera equipment at time t in the future, and then calculate the cumulative jitter amplitude △P(t) of the live camera equipment at time t; S3. According to the accumulated jitter amplitude △P(t), the lens trajectory Ts(t) is calculated and output for compensation adjustment, and the live camera equipment is adjusted and compensated in real time. Based on the output adjusted lens trajectory Ts(t), the optimized feedback function Fopt(t) is calculated and outputted according to the real-time collected picture data through the real-time feedback formula, and dynamic adjustment and compensation are performed to optimize the picture stability. S4. After further optimization, the picture stability index Sx(t) is calculated, and the stability of the current picture is evaluated based on the output result of the picture stability index Sx(t), and a dynamic adjustment filter mechanism is executed based on the corresponding evaluation result, and the dynamically adjusted filter window Nnew is output based on the AI ​​algorithm to further smooth the live picture; S5. After the dynamic adjustment filter mechanism is executed, S1 and S2 are executed for the second time to output the cumulative jitter amplitude △P (t), and the jitter threshold Pthreshold is optimized and evaluated, and an early warning mechanism is generated based on the evaluation results.

2. The method for stabilizing live broadcast images based on artificial intelligence according to claim 1, characterized in that: Said S1 includes S11, S12 and S13; S11. By installing a dynamic monitoring device group on the live video camera, the environment and motion data of extreme skiing live broadcast are collected in real time; The dynamic monitoring equipment group includes an acceleration sensor, a wind speed sensor, a GPS locator, a gyroscope and a vibration sensor; The environment and motion data include the acceleration a of the live camera device in the horizontal direction x , acceleration a in the vertical direction y , acceleration in the vertical direction a z , angular velocity w in the horizontal direction x , angular velocity w in the vertical direction y , angular velocity w in the vertical direction z , wind speed change data Ws, athlete's sliding speed Vs and vibration frequency Vf; S12, directly connecting the dynamic monitoring device to the central processing module, and then using an analog-to-digital converter ADC to convert the sensor signal into a digital signal and send it to the central processing module; The central processing module receives the environment and motion data in real time, pre-processes and timestamps the environment and motion data, and summarizes them after pre-processing to obtain a standard data set, and then uses a sliding average filter to reduce noise and smooth the standard data set. The standard data set includes the gravity change index G(t) at time t, the wind speed change index Ws(t) at time t, and the attitude angle of the live camera at time t. , the athlete's sliding acceleration As(t) at time t and the vibration frequency Vf(t) at time t; S13. Build a time series database in the central processing module, and set a write port, a write port, a real-time data table and a historical data table. Write the standard data set obtained in real time after preprocessing into the real-time data table of the time series database through the write port for data storage. At the same time, transfer the standard data set in the original real-time data table to the historical data table and sort them according to the marked timestamps.

3. The method for stabilizing live broadcast images based on artificial intelligence according to claim 1, characterized in that: The S2 includes S21, S22 and S23; S21. Build an AI prediction model through a neural network and a long short-term memory network (LSTM), extract the standard data set in the historical data table through the time series database write port, learn the standard data set in the historical data table, identify the jitter pattern of the live camera equipment, and train the AI ​​prediction model. S22, extracting a standard data set from a real-time data table through a time series database write port, inputting the data set into an AI prediction model, and calculating and outputting a prediction of the motion jitter amplitude Mpred(t) of the live camera device at time t in the future; The output formula of the AI ​​prediction model is: ； Where f represents the prediction function learned by the AI ​​prediction model.

4. The method for stabilizing live broadcast images based on artificial intelligence according to claim 3, characterized in that: S23, predicting the motion jitter amplitude Mpred(t) of the live camera device at time t in the future based on the output of the AI ​​prediction model, analyzing the cumulative impact of the jitter amplitude in the entire time interval, and calculating and outputting the cumulative jitter amplitude △P(t) of the live camera device at time t; The cumulative jitter amplitude △P(t) at the time t is calculated and output by the following algorithm formula: 。 5. The method for stabilizing live broadcast images based on artificial intelligence according to claim 4, characterized in that: The S3 includes S31 and S32; S31, based on the acquired cumulative jitter amplitude △P(t), the correction algorithm is used to perform real-time correction of the lens of the live camera device, the lens trajectory Ts(t) is adjusted by calculating the output compensation, and then the mechanical adjustment function of the pan / tilt is used to adjust the lens angle and position of the camera device in real time to compensate for the jitter caused by the movement of the device; The compensation adjustment lens trajectory Ts(t) is calculated and outputted by the following correction algorithm: ； Where Pcamera(t) represents the original trajectory of the live camera device.

6. The method for stabilizing live broadcast images based on artificial intelligence according to claim 4, characterized in that: S32, after the live camera lens is corrected, each frame is output, and then the difference between the compensated adjusted lens trajectory Ts(t) and the original trajectory Pcamera(t) of the live camera is calculated through a real-time feedback formula, and an optimized feedback function Fopt(t) is output, and the stability result of the current picture at each moment is analyzed to dynamically adjust the compensation to optimize the picture stability; The optimization feedback function Fopt(t) is calculated and outputted by the following real-time feedback formula: ； In the formula, represents the relative error between the compensated trajectory and the original trajectory, represents the adjustment coefficient of the relative error between the compensated trajectory and the original trajectory, Indicates the adjustment coefficient of the accumulated jitter amplitude.

7. The method for stabilizing live broadcast images based on artificial intelligence according to claim 6, characterized in that: The S4 includes S41, S42 and S43; S41, after dynamically adjusting and compensating to optimize the image stability, the stability of the live broadcast image after compensation is analyzed by calculating the image stability index Sx(t); The image stability index Sx(t) is calculated and outputted by the following algorithm formula; 。 8. The method for stabilizing live broadcast images based on artificial intelligence according to claim 7, characterized in that: S42, based on the output result of the picture stability index Sx(t), evaluating the picture stability after the current picture dynamic adjustment compensation, and generating a filter dynamic adjustment mechanism based on the evaluation, the specific evaluation content is as follows; When the image stability index Sx(t) is ≥ 0.7, it means that the jitter amplitude is normal relative to the original motion trajectory of the device, indicating that the compensated image is stable; When the image stability index Sx(t) is less than 0.7, it indicates that the jitter amplitude is abnormal relative to the motion trajectory of the device, indicating that the live broadcast image of the live camera device is unstable after compensation, and the dynamic adjustment filter mechanism is triggered at this time.

9. The method for stabilizing live broadcast images based on artificial intelligence according to claim 8, characterized in that: S43, after the dynamic adjustment filter mechanism is triggered, the window size n of the sliding average filter is adjusted to obtain the dynamically adjusted filter window Nnew, and the preprocessed standard data set is smoothed to eliminate noise and small jitter; The dynamically adjusted filter window Nnew is calculated and outputted by the following algorithm formula; ； Where Pthreshold represents the preset jitter threshold. Indicates the difference between the current jitter amplitude and the jitter threshold. Represents the filter window adjustment coefficient of the sliding average filter.

10. The method for stabilizing live broadcast images based on artificial intelligence according to claim 9, characterized in that: The S5 includes S51; S51, after the dynamic adjustment filter mechanism is executed, S1 and S2 are executed for the second time, the output cumulative jitter amplitude △P (t) and the jitter threshold Pthreshold are calculated for optimization evaluation, and based on the optimization evaluation results, an early warning prompt is triggered. The specific evaluation content is as follows; When the cumulative jitter amplitude △P(t)>jitter threshold Pthreshold, it indicates jitter abnormality. At this time, an early warning prompt is generated in the live broadcast screen to remind the cameraman to adjust the equipment posture and correct the shooting strategy; When the cumulative jitter amplitude △P (t) ≤ the jitter threshold Pthreshold, it indicates that the live screen temperature is high and no warning is required.

Images