Multi-channel video real-time processing method and system based on embedded computer platform

Abstract

The invention discloses a multi-channel video real-time processing method and a multi-channel video real-time processing system based on an embedded computer platform, and relates to the technical field of multi-channel video processing. Obtaining rotation dynamic distortion analysis information in a video collected by a camera in the state in real time, and respectively generating a rotation track deviation index and an edge contour distortion coefficient; constructing a rotation distortion prediction model for the generated rotation track deviation index and the edge contour distortion coefficient, and generating a rotation distortion prediction coefficient; and performing prediction analysis based on the generated rotation distortion prediction coefficient, and predicting the distortion degree of the image in the video collected by the camera when the target rotates at a high speed and the rotation direction changes nonlinearly. According to the method, the problem of image distortion caused by nonlinear change of a high-speed rotating target is solved, and the image quality and the processing efficiency are improved through real-time prediction and dynamic optimization of a motion compensation strategy.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-channel video processing, and particularly to a method and system for real-time processing of multi-channel videos based on an embedded computer platform. Background Art

[0002] Multi-channel video refers to the simultaneous processing of video streams from multiple video sources, which may have different formats, resolutions, and frame rates, covering data from multiple perspectives or sensors. Especially in the fields of security monitoring, intelligent transportation, remote medical treatment, etc., real-time processing of multi-channel video streams is crucial. Real-time processing based on an embedded computer platform can make full use of its characteristics of low power consumption, high integration, and high reliability, integrating video acquisition, decoding, processing, and display into a compact device, thereby improving processing efficiency, ensuring system stability, and enhancing the ability and application universality of real-time data processing.

[0003] Existing multi-channel video real-time processing technologies based on an embedded computer platform usually adopt a way of cooperation between hardware and software to achieve efficient real-time processing. First, the embedded platform accelerates the acquisition and decoding of video signals by integrating a dedicated video processing chip. Signals from multiple video sources are simultaneously input into the system, and are preliminarily decoded and preprocessed through hardware acceleration to ensure low latency of the video stream. Subsequently, the system performs real-time analysis and processing on the video signals through an embedded processor, including image enhancement, target detection, motion analysis, etc. To ensure efficient processing, the embedded platform is usually equipped with a multi-core processor or adopts parallel processing technology to ensure that multi-channel video streams can be processed simultaneously without jamming or frame loss. To further improve efficiency, the system also compresses and optimizes the data during the processing, only transmitting and storing key information to reduce bandwidth and storage pressure. Finally, the processing results are displayed in real time or transmitted to the control center to achieve real-time monitoring and response to multiple scenarios. This technology combines hardware acceleration and real-time computing capabilities, and can complete complex video processing tasks on a low-power, high-performance embedded platform, and is widely used in the fields of security monitoring, intelligent transportation, etc.

[0004] The existing technology has the following deficiencies:

[0005] When the drone monitoring equipment processes multiple video streams, when the camera captures a high-speed rotating target, such as a propeller or mechanical part, the direction of the rotating target changes nonlinearly, resulting in the accumulation of errors in the calculation of the motion vector between frames, making it difficult for the motion compensation algorithm to correctly match the target position. Due to the fast rotation speed, the displacement of adjacent frames changes dramatically, and the traditional block matching or optical flow method is difficult to accurately estimate the motion trajectory, resulting in deformation, blur or ghosting of the rotating target in the picture. The existing multi-channel video real-time processing technology based on embedded computer platforms cannot predict the degree of distortion of the image in the video captured by the camera when the target rotates at high speed and the rotation direction changes nonlinearly, resulting in the inability of motion compensation to effectively correct the target contour, blurring the boundary or generating a smear, which in turn affects the accuracy of target classification or defect detection, increases the computational burden and reduces real-time performance.

[0006] The above information disclosed in this Background section is only for enhancement of understanding of the background of the present disclosure and therefore it may contain information that does not constitute the prior art that is already known to one of ordinary skill in the art. Summary of the invention

[0007] The purpose of the present invention is to provide a multi-channel video real-time processing method and system based on an embedded computer platform to solve the problems in the above-mentioned background technology.

[0008] In order to achieve the above object, the present invention provides the following technical solution: a multi-channel video real-time processing method based on an embedded computer platform, specifically comprising the following steps:

[0009] When the UAV monitoring equipment processes multiple video streams, it determines the target captured by the camera based on the embedded computer platform and the fusion of multi-channel video data streams, and detects its rotation state in real time to determine whether the target is in a state of high-speed rotation and nonlinear change in rotation direction;

[0010] When it is detected that the target is in a state of high-speed rotation and nonlinear change in the rotation direction, the rotation dynamic distortion analysis information in the video captured by the camera in this state is obtained in real time, and analyzed after acquisition to generate the rotation trajectory deviation index and edge profile distortion coefficient respectively;

[0011] A rotation distortion prediction model is constructed based on the generated rotation trajectory deviation index and edge profile distortion coefficient to generate a rotation distortion prediction coefficient;

[0012] Based on the generated rotation distortion prediction coefficient, a prediction analysis is performed to predict the degree of distortion of the image in the video captured by the camera when the target rotates at a high speed and the rotation direction changes nonlinearly, and the degree of distortion is divided into low distortion degree, medium distortion degree and high distortion degree;

[0013] According to the prediction results, corresponding optimization measures are taken for the motion compensation optimization strategies under different distortion levels;

[0014] After the implementation of the motion compensation optimization measures, continuously monitor the rotation state of the target, and adjust the rotation distortion prediction model according to the rotation dynamic distortion analysis information updated in real time, so as to dynamically optimize the motion compensation strategy.

[0015] Preferably, when it is detected that the target is in a state of high-speed rotation with a non-linear change in the rotation direction, the rotation dynamic distortion analysis information in the video collected by the camera in this state is obtained in real time, and after obtaining it, analysis is carried out to respectively generate a rotation trajectory deviation index and an edge contour distortion coefficient, which specifically includes the following steps:

[0016] When it is detected that the target is in a state of high-speed rotation with a non-linear change in the rotation direction, the rotation dynamic distortion analysis information in the video collected by the camera in this state is obtained in real time, and after obtaining it, preprocessing is carried out;

[0017] Extract the rotation trajectory dynamic analysis information and the edge contour deformation analysis information from the preprocessed rotation dynamic distortion analysis information;

[0018] Analyze the extracted rotation trajectory dynamic analysis information and the edge contour deformation analysis information to respectively generate a rotation trajectory deviation index and an edge contour distortion coefficient.

[0019] Preferably, the acquisition logic of the rotation trajectory deviation index is as follows:

[0020] Extract the rotation trajectory dynamic analysis information from the preprocessed rotation dynamic distortion analysis information, which specifically includes the angular velocity, trajectory curvature, and change rate of the rotation angular velocity of each frame of the target in the video collected by the camera when the target is in a state of high-speed rotation with a non-linear change in the rotation direction, and are respectively calibrated as 、 and , represents the angular velocity of the target in the th frame of the video collected by the camera when the target is in a state of high-speed rotation with a non-linear change in the rotation direction, represents the trajectory curvature of the target in the th frame of the video collected by the camera when the target is in a state of high-speed rotation with a non-linear change in the rotation direction, represents the change rate of the rotation angular velocity of the target in the th frame of the video collected by the camera when the target is in a state of high-speed rotation with a non-linear change in the rotation direction, , is a positive integer;

[0021] Calculate the rotation trajectory deviation index, and the specific calculation formula is as follows:

[0022]

[0023] In the formula, is the rotation trajectory deviation index.

[0024] Preferably, the acquisition logic of the edge contour distortion coefficient is as follows:

[0025] Extract the edge contour deformation analysis information from the pre - processed rotation dynamic distortion analysis information, specifically including the average edge pixel gradient, contour area, and edge direction deviation angle of each frame of the target in the video when the target collected by the camera is in a state of high - speed rotation with non - linear change in the rotation direction, and calibrate them as , and , represents the average edge pixel gradient of the target in the th frame of the video when the target collected by the camera is in a state of high - speed rotation with non - linear change in the rotation direction, represents the contour area of the target in the th frame of the video when the target collected by the camera is in a state of high - speed rotation with non - linear change in the rotation direction, represents the edge direction deviation angle of the target in the th frame of the video when the target collected by the camera is in a state of high - speed rotation with non - linear change in the rotation direction, , is a positive integer;

[0026] Calculate the edge contour distortion coefficient, and the specific calculation formula is as follows:

[0027]

[0028] In the formula, is the edge contour distortion coefficient.

[0029] Preferably, for the generated rotation trajectory deviation index and the edge contour distortion coefficient construct a rotation distortion prediction model, and generate a rotation distortion prediction coefficient through weighted summation. The specific calculation formula is as follows:

[0030]

[0031] In the formula, is the rotation distortion prediction coefficient, and are the non - zero weight coefficients of the rotation trajectory deviation index and the edge contour distortion coefficient respectively, and .

[0032] Preferably, a pre-set threshold interval of the rotational distortion prediction coefficient is determined and compared with the generated rotational distortion prediction coefficient after determination. According to the comparison result, the distortion degree of the images in the video captured by the camera when the target rotates at a high speed and the rotation direction changes non-linearly is predicted, and it is classified into a low distortion degree, a medium distortion degree, and a high distortion degree. The specific comparison and analysis are as follows:

[0033] If , the distortion degree of the images in the video captured by the camera when the target rotates at a high speed and the rotation direction changes non-linearly is a low distortion degree;

[0034] If , the distortion degree of the images in the video captured by the camera when the target rotates at a high speed and the rotation direction changes non-linearly is a medium distortion degree;

[0035] If , the distortion degree of the images in the video captured by the camera when the target rotates at a high speed and the rotation direction changes non-linearly is a high distortion degree.

[0036] Preferably, according to the prediction result, corresponding optimization measures are taken for the motion compensation optimization strategies in different distortion degree cases, specifically:

[0037] For the motion compensation optimization strategy in the case of low distortion degree, the specific optimization measures are: maintaining the settings of the current motion compensation algorithm, maintaining a low computational complexity; according to the low distortion prediction result, maintaining the existing compensation accuracy; continuously monitoring the target rotation state to ensure the stability of the image quality;

[0038] For the motion compensation optimization strategy in the case of medium distortion degree, the specific optimization measures are: starting an enhanced motion compensation strategy, combining multi-frame fusion technology, increasing the compensation accuracy, and optimizing the matching accuracy of the rotation trajectory; by dynamically adjusting the compensation parameters, improving the accuracy of the rotation trajectory prediction; increasing the allocation of computational resources as needed;

[0039] For the motion compensation optimization strategy in the case of high distortion degree, the specific optimization measures are: starting a high-level motion compensation mechanism, adopting a motion estimation algorithm based on deep learning and an adaptive correction algorithm to precisely repair the image distortion; strengthening the improvement of the compensation accuracy, using advanced image reconstruction technology to process the deformation caused by rotation; increasing the allocation of computational resources.

[0040] Preferably, a multi-channel video real-time processing system based on an embedded computer platform includes a target detection and data fusion module, a distortion analysis and feature extraction module, a distortion prediction model construction module, a distortion prediction and classification module, a motion compensation optimization module, and a real-time adjustment and optimization module;

[0041] The target detection and data fusion module, when the UAV monitoring device processes multiple video streams, determines the targets captured by the camera based on the embedded computer platform and the fusion of multi-channel video data streams, and real-time detects its rotation state to judge whether the target is in a state of high-speed rotation and non-linear change in the rotation direction;

[0042] The distortion analysis and feature extraction module, when it detects that the target is in a state of high-speed rotation and non-linear change in the rotation direction, real-time obtains the rotational dynamic distortion analysis information in the video captured by the camera in this state, and analyzes it after obtaining, and respectively generates a rotation trajectory deviation index and an edge contour distortion coefficient;

[0043] The distortion prediction model construction module constructs a rotation distortion prediction model for the generated rotation trajectory deviation index and edge contour distortion coefficient, and generates a rotation distortion prediction coefficient;

[0044] The distortion prediction and classification module performs prediction analysis based on the generated rotation distortion prediction coefficient, predicts the distortion degree of the images in the video captured by the camera when the target is rotating at high speed and the rotation direction changes non-linearly, and classifies it into low distortion degree, medium distortion degree and high distortion degree;

[0045] The motion compensation optimization module takes corresponding optimization measures for the motion compensation optimization strategies in different distortion degree cases according to the prediction results;

[0046] The real-time adjustment and optimization module, after the motion compensation optimization measures are executed, continuously monitors the rotation state of the target, and adjusts the rotation distortion prediction model according to the real-time updated rotational dynamic distortion analysis information to dynamically optimize the motion compensation strategy.

[0047] In the above technical solution, the technical effects and advantages provided by the present invention:

[0048] 1. The present invention can accurately fuse and real-time detect the rotation state of the target in multiple video streams. Especially when the target rotates at a high speed and the direction changes non-linearly, it can accurately judge the rotation characteristics of the target. In this way, the system avoids the decrease in compensation accuracy caused by the accumulation of rotation trajectory errors in traditional motion compensation methods, thus ensuring the accurate estimation of motion vectors and the correct matching of target positions. This is particularly important for monitoring and detection in high-dynamic scenarios, especially in application scenarios such as UAV monitoring and industrial inspection that require high precision, and can significantly improve the quality of video images.

[0049] 2. The present invention constructs a rotational distortion prediction model to accurately predict image distortion based on the rotational trajectory deviation index and the edge contour distortion coefficient, and divides the distortion degree into three categories: low, medium, and high. This solution can not only dynamically evaluate the impact of the rotational state of the target on the image quality in a real-time video stream, but also adopt targeted motion compensation optimization strategies according to different distortion levels. In the case of low distortion, the system maintains an efficient and low-resource-consuming compensation algorithm; in the case of medium distortion, an enhanced compensation strategy is activated to improve accuracy; while in the case of high distortion, an adaptive correction algorithm based on deep learning is used for precise restoration. This hierarchical and flexible response mechanism can ensure optimal image quality under various rotational states, avoiding overcompensation or resource waste.

[0050] 3. After the compensation optimization strategy is executed, the present invention continuously adjusts the rotational distortion prediction model by continuously monitoring the rotational state of the target and the real-time updated rotational dynamic distortion analysis information to dynamically optimize the motion compensation strategy. Through this real-time update and dynamic adjustment, the system can flexibly optimize the compensation parameters according to the immediate changes during the target rotation, ensuring that the motion compensation strategy always maintains the best state. This efficient dynamic adjustment ability not only improves the adaptability of the system under high-speed rotation and non-linear change conditions, but also significantly reduces distortion phenomena such as image blurring, ghosting, or double imaging, ensures the accuracy of target classification and defect detection, and effectively reduces the computational burden and optimizes the real-time processing ability of the system. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments recorded in the present invention. For those of ordinary skill in the art, other drawings can also be obtained according to these drawings.

[0052] Figure 1 It is a schematic flowchart of a multi-channel video real-time processing method and system based on an embedded computer platform according to the present invention.

[0053] Figure 2 It is a schematic diagram of the modules of a multi-channel video real-time processing method and system based on an embedded computer platform according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0054] Now, the exemplary embodiments will be described more fully with reference to the accompanying drawings. However, the exemplary embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these exemplary embodiments are provided so that the present disclosure will be more thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art.

[0055] The present invention provides a multi-channel video real-time processing method based on an embedded computer platform as shown in Figure 1 , specifically including the following steps:

[0056] When the UAV monitoring device processes multi-channel video streams, based on the embedded computer platform and the fusion of multi-channel video data streams, determine the targets captured by the cameras, and real-time detect their rotation states to determine whether the targets are in a state of high-speed rotation and non-linear change in the rotation direction;

[0057] When the UAV monitoring device processes multi-channel video streams, accurate identification and tracking of the targets can be achieved by combining a deep learning-based object detection algorithm (such as YOLO, Faster R-CNN) with a multi-channel video data stream fusion strategy. Specifically, the embedded computer platform first performs spatio-temporal synchronization alignment on the video data from multiple cameras to ensure that the multi-channel video data streams can be fused in the same time frame. Then, using a multi-view object fusion algorithm based on feature matching (such as SIFT, ORB feature point matching), extract the feature points of the target images collected by each camera, and calculate the global feature vector of the target from different perspectives through a weighted fusion strategy to ensure the consistency of the target information. Next, through an object association and matching mechanism (such as based on the Kalman filter or multi-object tracking (MOT) algorithm), combined with the motion trajectory and appearance features of the target, fuse the detection results of the same target in different cameras, and finally determine the targets captured by the cameras.

[0058] After determining the targets, real-time detection and analysis of the rotation states of the targets can be achieved through the method of inter-frame angle change analysis + motion trajectory curvature analysis. First, the embedded computer platform performs key point detection (such as FAST corner detection, Harris corner detection) on the targets in consecutive frames, extracts the corner points or key structure features of the targets, and calculates their offsets in adjacent frames. Then, use the inter-frame angle difference analysis method to calculate the inter-frame rotation angle change rate of the targets, and combine the motion trajectory curvature fitting method to analyze the change of the motion trajectory curvature of the targets. If the angle change rate exceeds the preset threshold and the motion trajectory curvature shows non-linear fluctuations, it is determined that the targets are in a state of high-speed rotation and non-linear change in the rotation direction. In addition, the optical flow method or 3D reconstruction technology (such as Structure-from-Motion, SfM) can be used to extract the rotation vector information of the targets to further enhance the accuracy of the detection.

[0059] After determining the target through multi-channel video fusion and deep learning target detection algorithms (such as YOLO or Faster R-CNN), the system further uses the temporal changes of the target's corner points or feature points in consecutive image frames to analyze and determine the target's rotation state. Specifically, the embedded computer platform extracts the local structural features of the target through corner point detection methods such as FAST or Harris, and combines Kalman filtering or MOT algorithms to maintain its stable tracking between consecutive frames. Subsequently, the rotation angle difference of the target image between each frame is calculated through the spatial transformation relationship of the key point set between frames (such as rotation matrix fitting), and the inter-frame rotation angular velocity is derived. When the angular velocity exceeds the set threshold (such as 90° / s) in several consecutive frames, the target is considered to be in a "high-speed rotation" state. At the same time, in order to determine whether the rotation direction changes nonlinearly, the system further analyzes the change trend of the rotation speed over time. If its first-order derivative (i.e., rotation acceleration) has periodic sign changes or significant mutations, it indicates that the rotation direction of the target has nonlinear changes. For example, in 5 consecutive frames of images, the target's rotation angular velocity suddenly increases from 90° / s to 130° / s and then drops back to 95° / s, accompanied by a dramatic fluctuation in the trajectory curvature. It can be comprehensively determined that the target is in a state of "high-speed rotation and nonlinear change in rotation direction".

[0060] After the embedded platform realizes target detection and fusion recognition, it starts the dynamic monitoring mechanism of the rotation state of continuous frames. The system uses the target bounding box after the above fusion recognition to track the target image area between frames and extract its structural key points in real time. In each frame of the image, the target edge and center structural feature points are extracted by methods such as FAST or ORB, and the rotation angle change and the rotation center offset are estimated by combining the matching relationship of the previous and next frame point sets. The Kalman filter is used to dynamically track the rotation angular velocity curve, and the sliding window fitting is performed with the historical rotation data (for example, 5 frames as a cycle) to achieve continuous modeling of the target rotation state and abnormal fluctuation detection. The system automatically records the rotation state parameters for each frame and dynamically updates them, including key indicators such as angular velocity, angular acceleration, and rotation direction stability. Based on this, the platform can judge in real time whether the current rotation state is in a stable, high-speed or irregular change state, and trigger the corresponding compensation strategy adjustment.

[0061] After completing the cross-view fusion of the target and determining its position in each frame, the embedded computing platform can calculate the curvature of its two-dimensional spatial motion trajectory based on the temporal distribution of the target center point in consecutive image frames. Specifically, the calculation of trajectory curvature is based on the trajectory of the target's spatial position change in consecutive frame images. The embedded platform first extracts the target's centroid coordinates or the geometric center of the key point set in each frame to form a set of motion path points for the target. Then, the discrete curvature calculation method is used to calculate the curvature of the trajectory points in any three adjacent frames. The following curvature formula is applied:

[0062]

[0063] where Area(·) represents the triangle area calculation function, represents the Euclidean distance. This method can quantify the degree of curvature of the target's path within these three frames, thereby reflecting whether there are significant changes in the motion trajectory. When the curvature values continuously deviate from the stable interval in multiple consecutive frames (such as oscillations exceeding the set threshold), the system determines that the target trajectory has a non-linear rotation trend. For example, if the curvatures of a certain target in 5 frames are 0.03, 0.04, 0.07, 0.12, and 0.08 respectively, which are significantly higher than the stable threshold of 0.05 set in the system, then this trajectory is recognized as a "non-linear change trajectory".

[0064] The purpose of doing this is to solve the problem of motion compensation failure caused by high-speed rotating targets, and ensure that the embedded computer platform can accurately identify rotating targets and predict the degree of image distortion. The existing multi-channel video real-time processing technology based on the embedded computer platform is prone to be affected by motion blur and the accumulation of frame-interframe motion vector calculation errors when processing high-speed rotating targets, resulting in the inability of motion compensation to correctly match the position of the rotating target, thus causing image deformation, edge blurring or ghosting. Through multi-channel video data fusion, the loss of target information caused by the perspective limitation of a single camera can be reduced, and the robustness of target detection can be improved. At the same time, by real-time detecting the rotation state of the target, the motion mode of the target can be accurately identified, and it can be determined whether there is a non-linear change in the rotation direction, providing data support for subsequent rotation distortion prediction. Finally, by constructing a rotation distortion prediction model, the degree of image distortion can be predicted in advance according to the motion mode of the target, the motion compensation strategy can be dynamically adjusted, the compensation accuracy can be improved, and it can be ensured that high-quality video images can still be obtained in complex rotation scenarios, thereby improving the accuracy of target detection, defect recognition and behavior analysis.

[0065] When it is detected that the target is in a state of high-speed rotation with a non-linear change in the rotation direction, the rotational dynamic distortion analysis information in the video captured by the camera in this state is obtained in real time, and after obtaining it, an analysis is carried out to generate a rotation trajectory deviation index and an edge contour distortion coefficient respectively;

[0066] In this embodiment, when it is detected that the target is in a state of high-speed rotation with a non-linear change in the rotation direction, the rotational dynamic distortion analysis information in the video captured by the camera in this state is obtained in real time, and after obtaining it, an analysis is carried out to generate a rotation trajectory deviation index and an edge contour distortion coefficient respectively, which specifically includes the following steps:

[0067] When it is detected that the target is in a state of high-speed rotation with non-linear change in the rotation direction, the rotational dynamic distortion analysis information in the video captured by the camera in this state is obtained in real time and preprocessed after acquisition;

[0068] When it is detected that the target is in a state of high-speed rotation with non-linear change in the rotation direction, the rotational dynamic distortion analysis information in the video captured by the camera in this state can be obtained in real time by means of frame feature extraction and time series analysis. Specifically, the embedded computer platform performs frame synchronization processing on the original video stream captured by the camera to ensure that the timestamps between consecutive frames are consistent for subsequent calculation of motion features. Subsequently, the optical flow method (Optical Flow) or a deep learning-based key point detection algorithm (such as SIFT, ORB, SuperPoint) is used to extract the inter-frame motion features of the target, and key motion parameters such as the rotational angular velocity, trajectory curvature, and edge gradient change between adjacent frames are calculated. Then, time series analysis methods (such as sliding window filtering, long short-term memory network (LSTM)) are used to perform time series modeling on these parameters, extract their change trends at different time points, and further generate rotational dynamic distortion analysis information, which includes the motion trajectory of the target in the rotational state, the degree of edge distortion, and its characteristics over time. Finally, the system stores this information in the buffer and provides it to the subsequent preprocessing module for data optimization and noise reduction.

[0069] The purpose of preprocessing is to remove the noise interference in the rotational dynamic distortion analysis information, enhance the stability of the data, and improve the accuracy of calculating the rotational trajectory deviation index and the edge contour distortion coefficient. Since the video captured by the camera may be affected by factors such as environmental light changes, motion blur, and camera jitter, resulting in outliers or noise in the original data, preprocessing must be carried out. The specific preprocessing methods include: First, bilateral filtering or non-local means filtering is used to perform spatial noise reduction on the inter-frame motion feature data to reduce the interference of edge textures; Second, Kalman filtering or exponential smoothing algorithm is used to perform dynamic denoising on the time series data to eliminate short-term abnormal changes caused by camera jitter; Then, PCA (Principal Component Analysis) or wavelet transform is used to perform dimensionality reduction and feature optimization on the motion parameter data to remove redundant information and retain the most important feature components for rotational distortion prediction. After these preprocessing steps, the noise in the rotational dynamic distortion analysis information is effectively suppressed, and the data quality is improved, thus ensuring that the subsequent calculated rotational trajectory deviation index and edge contour distortion coefficient are more accurate and improving the reliability of the final rotational distortion prediction.

[0070] Extract the rotational trajectory dynamic analysis information and the edge contour deformation analysis information from the preprocessed rotational dynamic distortion analysis information;

[0071] Extract the rotational trajectory dynamic analysis information and the edge contour deformation analysis information from the preprocessed rotational dynamic distortion analysis information. Data extraction and reconstruction can be performed through feature decomposition and pattern recognition methods to ensure that the extracted information can accurately characterize the rotational motion characteristics of the target and its deformation in the image. Specifically, first, use principal component analysis (PCA) or independent component analysis (ICA) to reduce the dimensionality of the preprocessed data to remove redundant information and enhance the separation of rotation-related features. Then, for the rotational trajectory dynamic analysis information, adopt a time-series feature decomposition method (such as Fourier transform (FFT) or empirical mode decomposition (EMD)) to extract the trajectory change pattern of the target, calculate key indicators such as the change in rotational angular velocity between frames and the curvature of the motion trajectory to construct a rotational trajectory feature vector and remove noise interference. At the same time, for the edge contour deformation analysis information, adopt a feature extraction method based on edge gradient and morphological analysis. First, extract the edge contour of the target through Canny edge detection or Sobel operator, and enhance the integrity of the edge structure by combining morphological closing operation. Subsequently, calculate key indicators such as the edge gradient change rate, local deformation area ratio, and edge direction deviation to construct an edge contour deformation feature vector. Finally, use a feature clustering method (such as K-Means or Gaussian mixture model (GMM)) to classify the rotational trajectory feature vector and the edge contour deformation feature vector, and extract the information most relevant to rotational distortion from them. Finally, form the rotational trajectory dynamic analysis information and the edge contour deformation analysis information respectively for subsequent calculation of the rotational trajectory deviation index and the edge contour distortion coefficient.

[0072] Analyze the extracted rotational trajectory dynamic analysis information and edge contour deformation analysis information to generate a rotational trajectory deviation index and an edge contour distortion coefficient respectively.

[0073] In this embodiment, the acquisition logic of the rotational trajectory deviation index is as follows:

[0074] Extract the rotational trajectory dynamic analysis information from the preprocessed rotational dynamic distortion analysis information, specifically including the angular velocity, trajectory curvature, and change rate of rotational angular velocity of each frame of the target in the video collected by the camera when the target is rotating at high speed and the rotation direction is non-linearly changing, and calibrate them as 、 and , represents the angular velocity of the target in the th frame of the video collected by the camera when the target is rotating at high speed and the rotation direction is non-linearly changing, Indicates the trajectory curvature of the target in the th frame of the video when the target collected by the camera is in a state of high-speed rotation with a non-linear change in the rotation direction, Indicates the rate of change of the rotational angular velocity of the target in the th frame of the video when the target collected by the camera is in a state of high-speed rotation with a non-linear change in the rotation direction, , where

[0075] is a positive integer; To obtain the angular velocity, trajectory curvature, and rate of change of rotational angular velocity of each frame in real time in the video of the target in a state of high-speed rotation with a non-linear change in the rotation direction, first, the video frames collected by the camera will extract the target area in each frame in real time through an object detection algorithm (such as YOLO or Faster R-CNN based on deep learning). Subsequently, for the rotation state of the target, the optical flow method can be used to estimate the movement of the target in consecutive frames. Specifically, the optical flow method analyzes the movement of each pixel between adjacent frames, calculates the motion vector of each pixel, thereby obtaining the local motion information of the target, and then estimates the angular velocity of the target (estimated by the angular change amount of the target between adjacent frames).

[0076] For the trajectory curvature, the motion trajectory of the target is modeled through curve fitting techniques (such as least squares fitting or spline interpolation method) to extract the curvature change of the trajectory. First, the position points of the target in consecutive frames are extracted and connected into a trajectory, and then the curvature of the trajectory is calculated, which is a measure of the bending of the trajectory and reflects the non-linear change of the target's movement.

[0077] To calculate the trajectory curvature of the target in a state of high-speed rotation, the embedded computing platform first extracts the center point (or the center of gravity of key feature points) of the target in each frame image to form an ordered two-dimensional spatial trajectory point set , , …, . Then, based on the least squares fitting or spline interpolation algorithm, continuous curve modeling is performed on this discrete point set to obtain the fitted trajectory function or parametric curve form . Taking the parametric form as an example, the target trajectory can be expressed as a sequence of smooth functions with respect to the time frame sequence . The calculation formula for the trajectory curvature is as follows:

[0078]

[0079] where and are the first-order derivatives of the trajectory in the horizontal and vertical directions respectively, representing the instantaneous velocity components of the target; , is its corresponding second derivative, representing the acceleration component. This curvature formula measures the degree of curvature of the trajectory at a certain point, and its geometric meaning is the reciprocal of the curvature of the circular arc fitting at that point.

[0080] In actual calculations, the system uses the five-point difference method or the three-point central difference method for the discrete point sequence to obtain the first and second derivatives, so as to obtain the curvature value of each frame. For example, if the center point coordinates of the target in 5 frames of images are (100, 100), (102, 103), (105, 107), (109, 112), (114, 118) respectively, the system will perform spline interpolation on these point sequences to generate a smooth curve, and numerically differentiate at the points corresponding to the 3rd frame, and substitute into the above formula to calculate the curvature.

[0081] Finally, the system can construct a curvature sequence corresponding to the frame sequence , and analyze its fluctuation trend to judge whether the target is in a rotating non-linear state. If the curvature shows periodic mutations or rapid oscillations over time (such as the change rate of exceeds the set threshold), then the trajectory can be determined to have strong non-linear characteristics.

[0082] As for the change rate of the rotational angular velocity, it can be achieved by calculating the time derivative of the target's rotational speed. The specific method is to calculate the change in the rotational angle of the target between consecutive frames through the optical flow method mentioned above, and then combine these angle change values with the time difference to obtain the rotational angular velocity of each frame. Finally, the change rate of the rotational angular velocity is obtained through differential calculation (for example, the difference between the rotational angular velocity of the current frame and the previous frame divided by the time interval). These data can be processed in real time through an embedded computing platform and continuously updated on the basis of each frame. This method ensures that the motion characteristics of the target can be accurately captured within each frame and can adapt to the dynamic environment of the target's high-speed rotation and non-linear change in the rotational direction.

[0083] Calculate the rotation trajectory deviation index, and the specific calculation formula is as follows:

[0084]

[0085] In the formula, is the rotation trajectory deviation index.

[0086] Calculate the rotation trajectory deviation index When, the purpose of using this formula is to accurately quantify the degree of deviation of the target's motion trajectory in the state of high-speed rotation and non-linear change in the rotational direction, and effectively capture the rotational characteristics of the target at different time points. First, in the formula The deviation of the angular velocity ratio is partially amplified by exponential operation, which helps to sensitively capture the sudden change of angular velocity. Angular velocity is an important indicator of the change of rotation state. Rapidly changing angular velocity often means the instability of the target rotation state. Exponential operation can effectively enhance the influence of this change and reflect the nonlinear characteristics of the target motion. Next, The curvature fluctuations of the trajectory are smoothed by logarithmic operations, and the curvature changes are weighted. The purpose of this operation is to avoid extreme values from having too much influence on the calculation results, to ensure a reasonable response to the changes in the curvature of the target trajectory, and to accurately describe the deformation of the rotation trajectory. On this basis, The rotation acceleration data is processed by square root operation, which can smooth the instantaneous rotation acceleration changes and avoid the impact of too drastic acceleration changes on the overall calculation results. Finally, all these parts are accumulated and averaged to obtain the rotation trajectory deviation index. This index can quantify the degree of nonlinear change of the trajectory of the target during rotation and provide effective data support for subsequent image distortion prediction.

[0087] Rotation trajectory deviation index There is a direct correlation between the size of and the degree of image distortion. When the rotation trajectory deviation index is large, it means that the target has experienced greater nonlinear changes during the rotation process, that is, the target's rotation speed, trajectory curvature, and the rate of change of rotation angular velocity all show large fluctuations. This usually indicates that the rotation behavior of the target is unstable, resulting in more severe motion blur, deformation, or edge ghosting in the image. Specifically, a larger A value of indicates that the target's motion trajectory deviates from the ideal smooth rotation mode, and the changes involved in the rotation process are more complex, which will increase the error of the motion compensation algorithm and make the target edge in the image blurred or distorted. The larger the value, the higher the image distortion. A smaller value indicates that the rotation trajectory of the target is close to the ideal linear motion and the rotation state is relatively stable. At this time, the image distortion is lighter, the motion compensation accuracy is higher, and the image quality is better.

[0088] In this embodiment, the logic for obtaining the edge contour distortion coefficient is as follows:

[0089] The edge contour deformation analysis information is extracted from the preprocessed rotational dynamic distortion analysis information, including the edge pixel gradient mean, contour area and edge direction deviation angle of each frame of the target in the video captured by the camera when the target is in a high-speed rotation and the rotation direction changes nonlinearly, and is calibrated as , and , It represents the average edge pixel gradient of the target in the th frame in the video when the target captured by the camera is in a state of high-speed rotation with a non-linear change in the rotation direction. It represents the contour area of the target in the th frame in the video when the target captured by the camera is in a state of high-speed rotation with a non-linear change in the rotation direction. It represents the deviation angle of the edge direction of the target in the th frame in the video when the target captured by the camera is in a state of high-speed rotation with a non-linear change in the rotation direction. , where is a positive integer.

[0090] To obtain the average edge pixel gradient, contour area, and deviation angle of the edge direction of each frame of the target in the video when the target captured by the camera is in a state of high-speed rotation with a non-linear change in the rotation direction in real time, data extraction can be performed through the following methods. First, for the average edge pixel gradient, an edge detection algorithm (such as the Canny edge detection algorithm or Sobel operator) can be used to extract the target edge and calculate the gradient magnitude of the target edge in each frame. After obtaining the edge image, calculate the average gradient of all edge pixels in the image to obtain the average edge pixel gradient of each frame of the target, which is used to reflect the clarity and change of the target edge. Second, for the contour area, a contour extraction method (such as the findContours function in OpenCV) can be used to extract the boundary contour of the target and calculate the area of the target contour in each frame. By calculating the area of the extracted contour region, the shape information of the target edge can be obtained, reflecting the edge stability and deformation of the target. Finally, the deviation angle of the edge direction can be obtained by calculating the normal direction of the target edge points. In each frame, each pixel point of the target edge is obtained through the edge detection algorithm, and then the normal vector direction of each edge point is calculated and compared with the normal vector of the previous frame to obtain the deviation angle of the edge direction. Finally, by combining these data, the average edge pixel gradient, contour area, and deviation angle of the edge direction in each frame can be obtained in real time, providing the necessary basic data for calculating the edge contour distortion coefficient in the subsequent process.

[0091] Calculate the edge contour distortion coefficient, and the specific calculation formula is as follows:

[0092]

[0093] In the formula, is the edge contour distortion coefficient.

[0094] The calculation formula of the edge contour distortion coefficient is designed to quantify the deformation degree of the edge of the target during rotation to predict image distortion. First, in the formula, Part processes the rate of change of the contour area through square root operations to measure the amplitude of the morphological changes of the target edge during rotation. The change in the contour area directly reflects whether the target edge undergoes stretching or contraction, and using the square root can smooth out excessive changes and avoid the overinfluence of extreme values on the results. Next, Part amplifies the influence of the edge direction deviation through exponential operations. The edge direction deviation reflects the irregular changes of the target edge during rotation. Exponential operations can give a stronger response to large changes in the edge direction and enhance the sensitivity to larger deviations. Finally, Part smooths the change in the edge gradient through logarithmic operations, so that the gradient fluctuations of the target during rotation will not have an excessive impact on the calculation results. The logarithmic function effectively suppresses extreme values and ensures more stable calculations. Through the combination of these three steps, the formula can comprehensively evaluate the complex changes of the target edge during rotation, thereby accurately predicting the degree of image distortion and further optimizing subsequent image processing and motion compensation algorithms.

[0095] Edge contour distortion coefficient There is a direct relationship between the magnitude of and the degree of image distortion. When the value is large, it means that the deformation degree of the target edge during rotation is relatively significant, usually accompanied by strong image distortion, such as blurring, stretching or distortion of the target edge. Specifically, a large change in the contour area means that the target edge has undergone a large degree of deformation, usually resulting in the edge of the target in the image no longer being clear, producing obvious distortion or blurring effects. The deviation of the edge direction also reflects the irregular changes of the target edge during rotation. A large value usually indicates that the target edge has undergone severe distortion or discontinuity, further exacerbating the deformation of the image. When the value is small, it indicates that the deformation of the target edge is relatively light, and the edge of the image remains relatively stable during rotation, with a low degree of distortion. Therefore, the edge contour distortion coefficient can be effectively used to predict the degree of image distortion during target rotation, the larger the value, the more serious the distortion, and vice versa, the lighter the distortion, further providing a quantitative basis for subsequent image restoration and compensation.

[0096] Construct a rotation distortion prediction model for the generated rotation trajectory deviation index and edge contour distortion coefficient to generate a rotation distortion prediction coefficient;

[0097] In this embodiment, for the generated rotation trajectory deviation index and edge contour distortion coefficient construct a rotation distortion prediction model, and generate a rotation distortion prediction coefficient through weighted summation. The specific calculation formula is as follows:

[0098]

[0099] In the formula, is the rotational distortion prediction coefficient, and are respectively the non-zero weight coefficients of the rotational trajectory deviation index and the edge contour distortion coefficient and .

[0100] To calculate the rotational distortion prediction coefficient , first, it is necessary to perform a weighted sum of the rotational trajectory deviation index and the edge contour distortion coefficient . The key to this process is to select appropriate non-zero weight coefficients ( and ), which determine the contribution degree of each index to the final prediction coefficient. Specifically, when implementing, these two weight coefficients can be determined according to experimental data or optimization algorithms (such as the least squares method or cross-validation). The weight coefficients are usually obtained through data fitting or empirical formulas, where represents the influence weight of the rotational trajectory deviation index on the prediction result, while represents the influence weight of the edge contour distortion coefficient. It is necessary to ensure , which means that the sum of the contributions of the rotational trajectory deviation index and the edge contour distortion coefficient to the final prediction coefficient is 1. By adjusting these two coefficients, it is possible to dynamically adjust their respective weights under different rotational and distortion conditions, thereby achieving accurate prediction of the image distortion degree.

[0101] Based on the generated rotational distortion prediction coefficient, perform prediction analysis to predict the distortion degree of the images in the video captured by the camera when the target rotates at high speed and the rotation direction changes non-linearly, and classify it into low distortion degree, medium distortion degree, and high distortion degree;

[0102] In this embodiment, determine the preset rotational distortion prediction coefficient threshold interval , and after determination, compare it with the generated rotational distortion prediction coefficient . According to the comparison result, predict the distortion degree of the images in the video captured by the camera when the target rotates at high speed and the rotation direction changes non-linearly, and classify it into low distortion degree, medium distortion degree, and high distortion degree. The specific comparison analysis is as follows:

[0103] If , the distortion degree of the images in the video captured by the camera when the target rotates at high speed and the rotation direction changes non-linearly is the low distortion degree;

[0104] This situation means that during the process where the target is rotating at high speed and the rotation direction is changing non-linearly, its rotation characteristics are relatively stable, the trajectory changes and edge deformations during rotation are relatively minor, and the distortion degree of the image is relatively low. This usually indicates that the quality of the video image captured by the camera is good, and the motion compensation algorithm can correct the rotation trajectory of the target more accurately, and the image maintains a relatively high clarity and stability. For subsequent processing, images with a low distortion degree do not require complex restoration operations, so the requirements for computing resources are relatively low, which helps to improve the efficiency of real-time processing.

[0105] If , during the process where the target is rotating at high speed and the rotation direction is changing non-linearly, the distortion degree of the image in the video captured by the camera is a medium distortion degree;

[0106] This situation indicates that there is a certain degree of non-linear change in the rotation trajectory of the target, resulting in a slight deformation or blurring of the image edges. In this case, the correction effect of the motion compensation algorithm may be slightly insufficient, and the image quality is affected to a certain extent, but the basic usability can still be guaranteed. Images with a medium distortion degree may require further optimization processing, such as multi-frame fusion or adaptive compensation algorithms, to improve the compensation accuracy. The demand for computing resources is relatively high, but still within an acceptable range, and the system can find a balance between real-time performance and image quality.

[0107] If , during the process where the target is rotating at high speed and the rotation direction is changing non-linearly, the distortion degree of the image in the video captured by the camera is a high distortion degree.

[0108] This situation means that the rotation process of the target has extremely strong non-linear characteristics, the rotation trajectory deviates significantly from the ideal orbit, resulting in serious distortion or deformation of the image. The edges of the image may be significantly distorted, blurred or ghosted, affecting subsequent target detection, recognition and analysis. At this time, the motion compensation algorithm may not be able to effectively correct the distortion of the image, resulting in the need for more complex restoration techniques, such as high-order motion estimation or deep learning-based restoration algorithms. Images with such a high distortion degree require a large amount of computing resources for correction, so it poses relatively high requirements for the real-time processing ability of the system and may affect the overall performance and response speed.

[0109] Determining the pre-set threshold interval of the rotational distortion prediction coefficient can be achieved through data analysis and optimization algorithms. First, collect sample data of the rotational distortion prediction coefficient under different target rotational states. Through the analysis of these samples, calculate the minimum and maximum values in various rotational situations, that is, the actual range of the distortion coefficient. This can be achieved through statistical analysis methods (such as standard deviation, mean analysis) to determine a suitable threshold range to ensure accurate prediction of different types of target rotations and image distortion degrees. Secondly, machine learning optimization methods such as K-means clustering can be used to group the sample data and automatically divide the coefficient ranges corresponding to low distortion, medium distortion, and high distortion. On this basis, use the cross-validation method to evaluate the accuracy of different threshold intervals and determine the final threshold interval by minimizing the error. This process can be completed in real time through an embedded computing platform to ensure that the threshold interval can be dynamically adjusted according to different scenarios to meet the image distortion prediction requirements under various rotational states.

[0110] According to the prediction results, corresponding optimization measures are taken for the motion compensation optimization strategies under different distortion degrees;

[0111] In this embodiment, according to the prediction results, corresponding optimization measures are taken for the motion compensation optimization strategies under different distortion degrees. Specifically:

[0112] For the motion compensation optimization strategy under the condition of low distortion degree, the specific optimization measures are: maintaining the settings of the current motion compensation algorithm, maintaining low computational complexity to ensure real-time performance; according to the low distortion prediction results, maintaining the existing compensation accuracy without additional computational resource overhead; continuously monitoring the target rotational state to ensure stable image quality;

[0113] For the case of low distortion degree, the optimization goal of the motion compensation algorithm is to maintain low computational complexity while ensuring real-time performance. Specifically, this strategy can be achieved through the adjustment of the existing motion compensation algorithm. First, use traditional compensation algorithms based on block matching or optical flow. These methods have high computational efficiency and are suitable for real-time processing in the case of low distortion. By continuously monitoring the rotational state of the target, the system can maintain the existing compensation accuracy without increasing additional computational overhead in the case of low distortion. For further optimization, the image can be denoised through lightweight filtering algorithms (such as mean filtering or Gaussian filtering) to eliminate small-scale image noise, thereby ensuring the stability and clarity of the image. In addition, the real-time monitoring module can regularly adjust the inter-frame difference threshold and flexibly adjust the compensation strategy according to the changes in image features to avoid unnecessary computational burdens. Through these optimization means, relatively accurate image compensation can be ensured in the case of low distortion, while ensuring the real-time performance and computational efficiency of the system.

[0114] For the motion compensation optimization strategy under medium distortion conditions, the specific optimization measures are as follows: Activate the enhanced motion compensation strategy, combine multi-frame fusion technology, increase the compensation accuracy, and optimize the matching accuracy of the rotation trajectory; Improve the accuracy of rotation trajectory prediction by dynamically adjusting compensation parameters; Increase the allocation of computing resources as needed to ensure that the image quality reaches an acceptable level;

[0115] For the case of medium distortion, the goal is to increase the compensation accuracy, optimize the accuracy of rotation trajectory matching, and appropriately increase the allocation of computing resources. The way to implement this strategy is to activate the enhanced motion compensation algorithm, and multi-frame fusion technology is an effective means. Specifically, through multi-frame interpolation algorithms (such as bilinear interpolation or cubic convolution interpolation), the information of multiple adjacent frames is combined to obtain a more accurate motion vector estimate. Further optimization can be achieved through adaptive compensation parameter adjustment. For example, according to the rotation state of the current image, the search range and step size in the matching algorithm are adjusted to improve the prediction accuracy of the rotation trajectory. At the same time, in this process, the dynamic allocation of computing resources is particularly important. The number of processing threads and computing priorities can be dynamically adjusted according to the complexity and distortion degree of the current image, avoiding unnecessary resource waste while ensuring the full utilization of computing resources. This optimization strategy can effectively improve the compensation accuracy and ensure that the image quality reaches an acceptable level.

[0116] For the motion compensation optimization strategy under high distortion conditions, the specific optimization measures are as follows: Activate the high-level motion compensation mechanism, adopt the motion estimation algorithm and adaptive correction algorithm based on deep learning to accurately repair image distortion; Strengthen the improvement of compensation accuracy, and use advanced image reconstruction technology to process the deformation caused by rotation; Increase the allocation of computing resources to ensure real-time performance and image quality under high distortion.

[0117] For the case of high distortion levels, the optimization goal of motion compensation is to precisely repair image distortion and ensure real-time performance under high distortion. To this end, deep learning-driven motion estimation algorithms can be adopted, such as using convolutional neural networks (CNNs) for motion compensation prediction. These algorithms can automatically identify and repair image distortion by learning a large amount of rotated image data in complex rotation scenarios, significantly improving the compensation accuracy. In addition, the adaptive correction algorithm can dynamically adjust the parameters during the compensation process and make real-time adjustments according to the distortion level of each frame. For example, the step size or compensation intensity of the correction algorithm can be adjusted based on the image distortion level to optimize the position and shape of the target. For image reconstruction, advanced image reconstruction techniques, such as variational methods or image super-resolution reconstruction algorithms, can be introduced to finely repair the distorted edges and restore details. At the same time, since images with high distortion levels require higher computing resources for processing, through multi-core parallel processing and GPU acceleration technologies, computing resources can be efficiently utilized to ensure real-time performance and image quality even under high distortion. Through these measures, the system can handle high-distortion scenarios, provide high-quality image compensation, and ensure the image quality and processing efficiency of the final output.

[0118] After the motion compensation optimization measures are executed, continuously monitor the rotation state of the target, and adjust the rotation distortion prediction model according to the rotation dynamic distortion analysis information updated in real time to dynamically optimize the motion compensation strategy and improve the adaptability and stability of real-time processing.

[0119] To continuously monitor the rotation state of the target and adjust the rotation distortion prediction model according to the rotation dynamic distortion analysis information updated in real time, it can be achieved through a dynamic update algorithm and a feedback mechanism. First, the rotation dynamic distortion analysis information (such as angular velocity, trajectory curvature, edge deformation, etc.) can be continuously updated by a real-time video processing module, which can be realized through optical flow methods and image feature tracking algorithms (such as SIFT, SURF) to continuously track the rotation changes of the target. Through these methods, the platform can obtain the rotation state and related distortion data of the target in real time and analyze them. Using time series modeling techniques (such as sliding window models, Kalman filters), the changes in rotation dynamics can be updated and predicted in real time, thereby continuously optimizing the parameters of the rotation distortion prediction model to ensure that the model can adapt to the changes in the rotation state of the target.

[0120] The rotation state monitoring of the target is achieved by constructing a multi-frame time-series feature tracking mechanism. Specifically, after the embedded computing platform determines the target, it performs optical flow matching based on the image feature points (such as SIFT or SURF key points) extracted from consecutive frames to obtain the pixel-level displacement vector field of the target between adjacent frames. The platform focuses on analyzing the motion direction and angle changes of the high-gradient feature points within the target area, calculates the rotation amount of the main direction of the target in each frame (i.e., the angle change of the centroid of the local feature set around the rotation center), and generates an inter-frame rotation angle sequence. .

[0121] To improve the anti-interference ability, the system dynamically estimates the rotational angular velocity (i.e., ) through Kalman filtering, filters out noise and instantaneous abnormal jumps, and combines the sliding window technique to calculate the average rotation rate and the change rate of the rotation trend for the recent frames. In addition, to identify whether the rotation direction is stable, the platform further calculates the first-order difference sequence of the continuous-frame rotation angle sequence and extracts statistical indicators such as the maximum fluctuation amplitude and the number of sign changes to determine whether there is a non-linear direction change.

[0122] For example, during the monitoring process, if the angle changes of a certain target in 5 consecutive frames are , , , , , the system can detect a sudden change in its rotation direction and a reverse trend, thus identifying it as non-linear rotation. This identification process will trigger the adaptive adjustment of the parameters of the rotation distortion prediction model and affect the strategy selection of the subsequent motion compensation module, realizing the feedforward control of the image distortion trend.

[0123] Through the above methods, the present invention can not only accurately model the rotation state of the target, but also dynamically respond to the instability during the rotation change process, ensuring that the motion compensation model is always in the optimal compensation state, and improving the stability and real-time performance of image processing.

[0124] Based on the real-time update of the rotation dynamic distortion analysis information, the dynamic optimization of the rotation distortion prediction model is achieved through model adaptive adjustment. Through machine learning or adaptive algorithms, the weight coefficients of the rotation distortion prediction model can be automatically adjusted according to the continuously changing distortion information during the target rotation process. For example, when it is detected that a large change has occurred in the rotation state of the target, the various parameters in the model can be dynamically adjusted to optimize the accuracy of the motion compensation algorithm. The advantage of adopting this real-time feedback mechanism is that it can automatically adapt to different rotation modes and changes in the distortion degree, enabling the motion compensation strategy to always remain in the best state, thereby improving the stability and adaptability of the system and ensuring high-quality image processing results in various environments.

[0125] As Figure 2 shown in the multi-channel video real-time processing system based on an embedded computer platform, which includes a target detection and data fusion module, a distortion analysis and feature extraction module, a distortion prediction model construction module, a distortion prediction and classification module, a motion compensation optimization module, and a real-time adjustment and optimization module;

[0126] The target detection and data fusion module, when the UAV monitoring device processes multi-channel video streams, determines the targets captured by the camera based on the embedded computer platform and the fusion of multi-channel video data streams, and real-time detects its rotation state to determine whether the target is in a state of high-speed rotation and non-linear change in the rotation direction;

[0127] The distortion analysis and feature extraction module, when it is detected that the target is in a state of high-speed rotation and non-linear change in the rotation direction, real-time obtains the rotational dynamic distortion analysis information in the video captured by the camera in this state, and analyzes it after obtaining, and respectively generates a rotation trajectory deviation index and an edge contour distortion coefficient;

[0128] The distortion prediction model construction module constructs a rotation distortion prediction model for the generated rotation trajectory deviation index and edge contour distortion coefficient, and generates a rotation distortion prediction coefficient;

[0129] The distortion prediction and classification module conducts prediction analysis based on the generated rotation distortion prediction coefficient, predicts the distortion degree of the images in the video captured by the camera when the target is rotating at high speed and the rotation direction is non-linearly changing, and classifies it into low distortion degree, medium distortion degree, and high distortion degree;

[0130] The motion compensation optimization module, according to the prediction results, takes corresponding optimization measures for the motion compensation optimization strategies in different distortion degree cases;

[0131] The real-time adjustment and optimization module, after the motion compensation optimization measures are executed, continuously monitors the rotation state of the target, and adjusts the rotation distortion prediction model according to the real-time updated rotational dynamic distortion analysis information to dynamically optimize the motion compensation strategy.

[0132] The above formulas are all dimensionless and take their numerical calculations. The formula is a formula obtained by collecting a large amount of data for software simulation to get the closest to the real situation. The preset parameters in the formula are set by those skilled in the art according to the actual situation.

[0133] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired or wireless (such as infrared, wireless, microwave, etc.) manner. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more collections of available media. The available media can be magnetic media (such as floppy disks, hard disks, magnetic tapes), optical media (such as DVDs), or semiconductor media. The semiconductor media can be a solid-state drive.

[0134] It should be understood that in various embodiments of the present application, the order of the above processes does not mean the order of execution. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0135] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0136] In several embodiments provided by the present application, it should be understood that the disclosed systems and methods can be implemented in other ways. For example, the above-described embodiments are merely illustrative. For example, the division of the units is only a logical function division, and there can be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be indirect couplings or communication connections through some interfaces, devices, or units, and can be in an electrical, mechanical, or other form.

[0137] The unit described as a separation component may or may not be physically separated. The component shown as a unit may or may not be a physical unit, that is, it may be located in one place or may be distributed over multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0138] In addition, each functional unit in various embodiments of the present application may be integrated into one processing unit, may exist separately as individual physical units, or two or more units may be integrated into one unit.

[0139] As mentioned above, the above are only specific embodiments of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed in the present application can easily think of changes or substitutions, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method for real-time processing of multi-channel videos based on an embedded computer platform, characterized in that, Specifically, the following steps are included: When the UAV monitoring device processes multiple video streams, based on the embedded computer platform and the fusion of multi-channel video data streams, determine the targets captured by the camera, and real-time detect its rotation state to judge whether the target is in a state of high-speed rotation and non-linear change in the rotation direction; When it is detected that the target is in a state of high-speed rotation and non-linear change in the rotation direction, real-time obtain the rotational dynamic distortion analysis information in the video captured by the camera in this state, and analyze it after obtaining, and respectively generate a rotation trajectory deviation index and an edge contour distortion coefficient; Construct a rotation distortion prediction model for the generated rotation trajectory deviation index and edge contour distortion coefficient, and generate a rotation distortion prediction coefficient; Based on the generated rotation distortion prediction coefficient, conduct a prediction analysis to predict the distortion degree of the images in the video captured by the camera when the target is rotating at high speed and the rotation direction is non-linearly changing, and classify it into low distortion degree, medium distortion degree, and high distortion degree; According to the prediction results, take corresponding optimization measures for the motion compensation optimization strategies in different distortion degree cases; After the motion compensation optimization measures are executed, continuously monitor the rotation state of the target, and adjust the rotation distortion prediction model according to the real-time updated rotational dynamic distortion analysis information to dynamically optimize the motion compensation strategy.

2. The real-time multi-channel video processing method based on an embedded computer platform according to claim 1, characterized in that, When it is detected that the target is in a state of high-speed rotation and non-linearly changing rotation direction, real-time obtain the rotational dynamic distortion analysis information in the video captured by the camera in this state, and analyze it after obtaining, and respectively generate a rotation trajectory deviation index and an edge contour distortion coefficient. Specifically, the following steps are included: When it is detected that the target is in a state of high-speed rotation and non-linearly changing rotation direction, real-time obtain the rotational dynamic distortion analysis information in the video captured by the camera in this state, and preprocess it after obtaining; Extract the rotational trajectory dynamic analysis information and edge contour deformation analysis information from the preprocessed rotational dynamic distortion analysis information; Analyze the extracted rotational trajectory dynamic analysis information and edge contour deformation analysis information, and respectively generate a rotation trajectory deviation index and an edge contour distortion coefficient.

3. The real-time multi-channel video processing method based on an embedded computer platform according to claim 2, wherein, The acquisition logic of the rotation trajectory deviation index is as follows: Extract the rotational trajectory dynamic analysis information from the preprocessed rotational dynamic distortion analysis information, specifically including the angular velocity, trajectory curvature, and the change rate of the rotational angular velocity of each frame of the target in the video when the target collected by the camera is in a state of high-speed rotation and non-linear change of the rotation direction, and calibrate them respectively as , and , represents the angular velocity of the target in the -th frame of the video when the target collected by the camera is in a state of high-speed rotation and non-linear change of the rotation direction, represents the trajectory curvature of the target in the -th frame of the video when the target collected by the camera is in a state of high-speed rotation and non-linear change of the rotation direction, represents the change rate of the rotational angular velocity of the target in the -th frame of the video when the target collected by the camera is in a state of high-speed rotation and non-linear change of the rotation direction, , is a positive integer; Calculate the rotation trajectory deviation index. The specific calculation formula is as follows: In the formula, is the rotation trajectory deviation index.

4. The real-time multi-channel video processing method based on an embedded computer platform according to claim 3, wherein The acquisition logic of the edge contour distortion coefficient is as follows: Extract the edge contour deformation analysis information from the preprocessed rotational dynamic distortion analysis information, specifically including the average edge pixel gradient, contour area, and edge direction deviation angle of each frame of the target in the video when the target collected by the camera is in a state of high-speed rotation with a non-linearly changing rotation direction, and calibrate them respectively as , and , represents the average edge pixel gradient of the target in the -th frame of the video when the target collected by the camera is in a state of high-speed rotation with a non-linearly changing rotation direction, represents the contour area of the target in the -th frame of the video when the target collected by the camera is in a state of high-speed rotation with a non-linearly changing rotation direction, represents the edge direction deviation angle of the target in the -th frame of the video when the target collected by the camera is in a state of high-speed rotation with a non-linearly changing rotation direction, , is a positive integer; Calculate the edge contour distortion coefficient. The specific calculation formula is as follows: In the formula, is the edge contour distortion coefficient.

5. The real-time multi-channel video processing method based on an embedded computer platform according to claim 4, characterized in that The generated deviation index of the rotation trajectory and the distortion coefficient of the edge contour Construct a rotation distortion prediction model, and generate a rotation distortion prediction coefficient through weighted summation. The specific calculation formula is as follows: wherein, is the rotational distortion prediction coefficient, and are respectively the non-zero weight coefficients of the rotational trajectory deviation index and the edge profile distortion coefficient , and .

6. The real-time multi-channel video processing method based on an embedded computer platform according to claim 5, wherein Determine the preset threshold interval of the rotational distortion prediction coefficient , and after determination, compare it with the generated rotational distortion prediction coefficient . According to the comparison result, predict the distortion degree of the images in the video captured by the camera when the target rotates at high speed and the rotation direction changes non-linearly, and classify it into low distortion degree, medium distortion degree, and high distortion degree. The specific comparison and analysis are as follows: If , when the target rotates at a high speed and the rotation direction changes non-linearly, the distortion degree of the images in the video captured by the camera is a low distortion degree; If , when the target rotates at a high speed and the rotation direction changes non-linearly, the distortion degree of the images in the video captured by the camera is medium distortion degree; If , when the target rotates at a high speed and the rotation direction changes non-linearly, the distortion degree of the images in the video captured by the camera is a high distortion degree.

7. The real-time multi-channel video processing method based on an embedded computer platform according to claim 6, characterized in that, According to the prediction results, take corresponding optimization measures for the motion compensation optimization strategies in different distortion degree cases. Specifically: For the motion compensation optimization strategy in the case of low distortion degree, the specific optimization measures taken are: maintain the settings of the current motion compensation algorithm, maintain low computational complexity; according to the low distortion prediction results, maintain the existing compensation accuracy; continuously monitor the rotation state of the target to ensure stable image quality; For the motion compensation optimization strategy in the case of medium distortion degree, the specific optimization measures taken are: start the enhanced motion compensation strategy, combine multi-frame fusion technology, increase the compensation accuracy, and optimize the matching accuracy of the rotation trajectory; improve the accuracy of rotation trajectory prediction by dynamically adjusting the compensation parameters; increase the allocation of computing resources as needed; For the motion compensation optimization strategy in the case of high distortion degree, the specific optimization measures are as follows: Start the high-level motion compensation mechanism, adopt the motion estimation algorithm and adaptive correction algorithm based on deep learning to accurately repair the image distortion; Strengthen the improvement of compensation accuracy, and use advanced image reconstruction technology to process the deformation caused by rotation; Increase the allocation of computing resources.

8. A multi-channel video real-time processing system based on an embedded computer platform is used to implement the multi-channel video real-time processing method based on the embedded computer platform according to any one of claims 1-7 above, characterized in that It includes a target detection and data fusion module, a distortion analysis and feature extraction module, a distortion prediction model construction module, a distortion prediction and classification module, a motion compensation optimization module, and a real-time adjustment and optimization module; The target detection and data fusion module, when the UAV monitoring device processes multiple video streams, determines the target captured by the camera based on the embedded computer platform and multi-channel video data stream fusion, and real-time detects its rotation state to determine whether the target is in a state of high-speed rotation and non-linear change in the rotation direction; The distortion analysis and feature extraction module, when it is detected that the target is in a state of high-speed rotation and non-linear change in the rotation direction, real-time obtains the rotation dynamic distortion analysis information in the video collected by the camera in this state, and analyzes it after obtaining it, and respectively generates a rotation trajectory deviation index and an edge contour distortion coefficient; The distortion prediction model construction module constructs a rotation distortion prediction model for the generated rotation trajectory deviation index and edge contour distortion coefficient, and generates a rotation distortion prediction coefficient; The distortion prediction and classification module conducts prediction analysis based on the generated rotation distortion prediction coefficient, predicts the distortion degree of the image in the video collected by the camera when the target is in high-speed rotation and non-linear change in the rotation direction, and classifies it into low distortion degree, medium distortion degree, and high distortion degree; The motion compensation optimization module, according to the prediction results, takes corresponding optimization measures for the motion compensation optimization strategy in different distortion degree cases; The real-time adjustment and optimization module, after the motion compensation optimization measures are executed, continuously monitors the rotation state of the target, and adjusts the rotation distortion prediction model according to the real-time updated rotation dynamic distortion analysis information to dynamically optimize the motion compensation strategy.

Images