Monitoring camera image stabilization method and device

Abstract

The invention provides a monitoring camera image stabilization method and device, and the method comprises the steps: collecting the vibration data of a monitoring camera, extracting vibration characteristic parameters through time-frequency analysis, and generating environment vibration related data; according to the environment vibration associated data, calculating a weight distribution proportion of the monitoring camera between mechanical anti-shake and electronic anti-shake to obtain a cooperative work strategy; predicting a vibration track according to the vibration characteristic parameters, and compensating the vibration track in advance through mechanical anti-shake; and calculating the residual jitter amount after compensation in advance according to the cooperative work strategy, and carrying out electronic anti-jitter based on the residual jitter amount to realize stable output of images of the monitoring camera. By analyzing the characteristics of different vibration sources, the dynamic cooperative work of mechanical anti-shake and electronic anti-shake is realized, the compensation conflict between the two systems is eliminated, and the stability of monitoring camera images in a complex environment is effectively improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method and device for stabilizing a surveillance camera image. Background Art

[0002] With the widespread adoption of intelligent surveillance systems in urban security management, the installation environments for surveillance cameras are becoming increasingly complex and diverse. Traditional image stabilization technologies for surveillance cameras primarily rely on either mechanical or electronic image stabilization. Each approach has its own advantages and disadvantages: Mechanical image stabilization uses physical compensation within the pan / tilt motor system, effectively handling low-frequency, large-amplitude vibrations, but suffers from slow response speed; electronic image stabilization uses algorithms to optimize the image, effectively handling high-frequency, small-amplitude vibrations, but can compromise image quality. In complex urban surveillance environments, cameras are exposed to multiple vibration sources simultaneously, including wind, traffic vibrations, and the natural sway of buildings. Relying solely on a single image stabilization solution is difficult to adapt to. While there are products on the market that utilize both mechanical and electronic image stabilization technologies, the two technologies lack effective synergy and can even interfere with each other, resulting in reduced image stability in certain complex vibration environments. Summary of the Invention

[0003] The main purpose of the present invention is to solve the technical problem of insufficient coordination between mechanical anti-shake and electronic anti-shake in existing surveillance camera image stabilization technology; A first aspect of the present invention provides a method for stabilizing an image of a surveillance camera, the method comprising: Collecting vibration data from a surveillance camera, extracting vibration characteristic parameters of the vibration data through time-frequency analysis, and generating environmental vibration correlation data based on the vibration characteristic parameters; Calculating a weight distribution ratio between mechanical image stabilization and electronic image stabilization for the surveillance camera based on the environmental vibration correlation data to obtain a coordinated working strategy for mechanical-electronic image stabilization; predicting a vibration trajectory according to the vibration characteristic parameters, and compensating the vibration trajectory in advance through mechanical anti-shake; The residual jitter amount after advance compensation of mechanical anti-shake is calculated according to the collaborative working strategy, and electronic anti-shake is performed based on the residual jitter amount to achieve stable output of the monitoring camera image.

[0004] Optionally, in a first implementation of the first aspect of the present invention, collecting vibration data from a surveillance camera, extracting vibration characteristic parameters of the vibration data through time-frequency analysis, and generating environmental vibration-related data based on the vibration characteristic parameters includes: Vibration data is collected using the three-axis gyroscope and three-axis acceleration sensor built into the surveillance camera; Applying a wavelet transform algorithm to the vibration data to perform time-frequency domain decomposition, extracting frequency distribution, amplitude change, and duration characteristics, and dividing the vibration data into frequency bands to calculate the vibration energy distribution of each frequency band; The frequency distribution, amplitude change, duration characteristics, and vibration energy distribution are used as vibration characteristic parameters, and cluster analysis is performed on the vibration characteristic parameters based on an improved K-means clustering algorithm to obtain corresponding vibration types; Environmental vibration related data is generated according to the correspondence between the installation location information of the monitoring camera and the vibration type.

[0005] Optionally, in a second implementation of the first aspect of the present invention, calculating, based on the environmental vibration correlation data, a weight distribution ratio between mechanical image stabilization and electronic image stabilization for the surveillance camera to obtain a coordinated working strategy for mechanical-electronic image stabilization includes: Constructing a three-dimensional boundary function using the vibration frequency, vibration amplitude and environment type in the environmental vibration correlation data; Performing frequency domain decomposition of the input vibration signal according to the three-dimensional boundary function, calculating processing weights of mechanical image stabilization and electronic image stabilization for each frequency component, and dynamically adjusting the processing weights based on the temperature and operating time data of the pan / tilt motor of the surveillance camera; Calculating a correspondence between a mechanical anti-shake response parameter and a vibration frequency based on instantaneous strong vibration characteristics in the environmental vibration correlation data, and generating a parameter adjustment matrix for mechanical anti-shake; The processing weights are combined with the parameter adjustment matrix to form a cooperative working strategy of mechanical-electronic image stabilization.

[0006] Optionally, in a third implementation of the first aspect of the present invention, performing frequency domain decomposition of the input vibration signal according to the three-dimensional boundary function, calculating processing weights of mechanical image stabilization and electronic image stabilization for each frequency component, and dynamically adjusting the processing weights based on the pan / tilt motor temperature and operating time data of the surveillance camera includes: Applying a fast Fourier transform algorithm to the input vibration signal to obtain signal amplitudes of different frequency components; Substituting the signal amplitudes of the different frequency components into the three-dimensional boundary function, calculating the processing weights of mechanical image stabilization and electronic image stabilization corresponding to each frequency component, and obtaining the pan / tilt motor temperature data and operating time data from the sensor module of the surveillance camera; Obtaining a temperature adjustment factor based on the PTZ motor temperature data and a mapping relationship between temperature and mechanical response efficiency, and obtaining a duration adjustment factor based on the operating time data and a corresponding relationship between the operating time and mechanical wear; The temperature adjustment factor and the duration adjustment factor are weightedly combined to generate a comprehensive adjustment coefficient, and the processing weight is nonlinearly transformed using the comprehensive adjustment coefficient to obtain a dynamically adjusted processing weight.

[0007] Optionally, in a fourth implementation of the first aspect of the present invention, predicting a vibration trajectory according to the vibration characteristic parameters and pre-compensating the vibration trajectory through mechanical anti-shake includes: Using an improved Kalman filter algorithm to perform time series processing on the vibration characteristic parameters to predict the vibration trajectory in a future time period; Convert the predicted vibration trajectory into a gimbal compensation angle command and calculate the response delay time of the mechanical execution system; Performing time advance processing on the gimbal compensation angle instruction based on the response delay time to generate a mechanical anti-shake execution instruction; The mechanical anti-shake execution instruction is sent to the pan / tilt control system of the surveillance camera, and the timestamp, target compensation angle and actual completion angle of the mechanical anti-shake execution instruction are recorded to generate a compensation execution record table.

[0008] Optionally, in a fifth implementation of the first aspect of the present invention, converting the predicted vibration trajectory into a pan / tilt compensation angle instruction and calculating the response delay time of the mechanical execution system includes: Decomposing the predicted vibration trajectory in a three-dimensional space coordinate system to obtain displacement components in the X-axis, Y-axis, and Z-axis directions; According to the installation posture matrix of the surveillance camera, the displacement components in the X-axis, Y-axis and Z-axis directions are converted into horizontal rotation angle, vertical pitch angle and axial rotation angle; Applying a nonlinear magnification factor to the horizontal rotation angle and the vertical pitch angle based on the optical zoom ratio of the surveillance camera to generate compensation angle data; Utilizing the motor characteristic parameters of the surveillance camera, a motor control pulse sequence corresponding to the compensation angle data is calculated to form a pan / tilt compensation angle instruction; By sending a test pulse to the gimbal control system, the time difference from the gimbal compensation angle command to the actual gimbal movement is measured to obtain the basic delay value in the current working state; Combined with the current data of the pan / tilt load and the angular velocity of the surveillance camera, the basic delay value is corrected and calculated to generate the response delay time of the mechanical execution system.

[0009] Optionally, in a sixth implementation of the first aspect of the present invention, calculating the residual jitter amount after advance compensation for mechanical anti-shake according to the collaborative working strategy, and performing electronic anti-shake based on the residual jitter amount to achieve stable output of the surveillance camera image includes: querying the compensation execution record table according to the timestamp of the current image frame to obtain the actual compensation displacement data of the mechanical anti-shake at the corresponding moment; Calculating the difference between the actual compensation displacement data of the mechanical anti-shake and the ideal compensation displacement of the vibration trajectory to obtain a residual shake amount; Applying a compensation envelope comparison technique to the residual jitter amount to identify a phase conflict region that may be caused by mechanical image stabilization and electronic image stabilization; In the phase conflict area, the phase of the remaining jitter is adjusted, and electronic image displacement compensation is performed on the adjusted remaining jitter according to the electronic anti-shake weight in the collaborative working strategy to achieve stable output of the surveillance camera image.

[0010] A second aspect of the present invention provides a surveillance camera image stabilization device, the surveillance camera image stabilization device comprising: A vibration analysis module is used to collect vibration data from a surveillance camera, extract vibration characteristic parameters of the vibration data through time-frequency analysis, and generate environmental vibration correlation data based on the vibration characteristic parameters; a boundary division module, configured to calculate a weight distribution ratio between mechanical image stabilization and electronic image stabilization for the surveillance camera based on the environmental vibration correlation data, and obtain a cooperative working strategy of mechanical-electronic image stabilization; a trajectory prediction module, configured to predict a vibration trajectory based on the vibration characteristic parameters, and compensate the vibration trajectory in advance through mechanical anti-shake; The collaborative compensation module is used to calculate the residual jitter amount after the mechanical anti-shake is compensated in advance according to the collaborative working strategy, and perform electronic anti-shake based on the residual jitter amount to achieve stable output of the monitoring camera image.

[0011] The above-mentioned surveillance camera image stabilization method and device collects vibration data from the surveillance camera, extracts vibration characteristic parameters through time-frequency analysis, and generates environmental vibration correlation data; based on the environmental vibration correlation data, calculates the weight distribution ratio between mechanical anti-shake and electronic anti-shake for the surveillance camera to obtain a collaborative working strategy; predicts the vibration trajectory based on the vibration characteristic parameters, and pre-compensates the vibration trajectory through mechanical anti-shake; calculates the residual jitter amount after pre-compensation according to the collaborative working strategy, and performs electronic anti-shake based on the residual jitter amount to achieve stable output of the surveillance camera image. The present invention realizes dynamic collaborative work of mechanical anti-shake and electronic anti-shake by analyzing the characteristics of different vibration sources, eliminating compensation conflicts between the two systems, and effectively improving the stability of surveillance camera images in complex environments.

[0012] Other features and advantages of the present invention will be described in the following description, and in part will become apparent from the description, or understood by practicing the present invention. The purposes and other advantages of the present invention are realized and obtained by the structures particularly pointed out in the description, claims and drawings.

[0013] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, preferred embodiments are given below and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 Schematic diagram of a first embodiment of a surveillance camera image stabilization method according to an embodiment of the present invention; Figure 2 FIG. 1 is a schematic diagram of an embodiment of a surveillance camera image stabilization device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0015] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0016] The terms "including," "having," and any variations thereof, as used in the embodiments of the present invention are intended to cover non-exclusive inclusions. For example, a process, method, apparatus, product, or device comprising a series of steps or units is not limited to the listed steps or units, but may optionally include other steps or units not listed, or may optionally include other steps or units inherent to the process, method, product, or device.

[0017] To facilitate understanding of this embodiment, a surveillance camera image stabilization method disclosed in an embodiment of the present invention is first introduced in detail. Figure 1 As shown, this method includes the following steps: 101. Collect vibration data from a surveillance camera, extract vibration characteristic parameters of the vibration data through time-frequency analysis, and generate environmental vibration correlation data based on the vibration characteristic parameters; In one embodiment of the present invention, the collecting vibration data of the monitoring camera, extracting vibration characteristic parameters of the vibration data through time-frequency analysis, and generating environmental vibration related data based on the vibration characteristic parameters include: using the three-axis gyroscope and three-axis acceleration sensor built into the monitoring camera to collect vibration data; applying a wavelet transform algorithm to the vibration data for time-frequency domain decomposition, extracting frequency distribution, amplitude change and duration characteristics, and dividing the vibration data into frequency bands, and calculating the vibration energy distribution of each frequency band; using the frequency distribution, amplitude change, duration characteristics and the vibration energy distribution as vibration characteristic parameters, and performing cluster analysis on the vibration characteristic parameters based on an improved K-means clustering algorithm to obtain corresponding vibration types; generating environmental vibration related data based on the correspondence between the installation location information of the monitoring camera and the vibration type.

[0018] Specifically, vibration data from the surveillance camera is first collected using the camera's built-in three-axis gyroscope and three-axis accelerometer. Upon system startup, the microcontroller initializes the sensor modules, setting the gyroscope's range to ±250° / s and sensitivity to 0.01° / s; the accelerometer's range to ±8g and sensitivity to 0.002g. The system then configures data acquisition parameters, setting the sampling frequency to 200Hz, acquiring sensor data every 5ms. Data acquisition is interrupt-triggered. When a timer interrupt occurs, the system reads raw data from the sensor registers via the I2C bus, including angular velocity and acceleration values ​​along the three axes. After reading, the system performs unit conversion and coordinate system alignment on the raw data, converting the ADC values ​​to physical units. The system then processes this data using a Kalman filter algorithm to remove high-frequency noise and sensor drift errors. The processed data, along with a timestamp, is stored in a circular buffer with a buffer size of 2 seconds, or 400 data points. The system also reads auxiliary information such as ambient temperature and wind speed from other interfaces and creates a correlated record.

[0019] Specifically, the system performs a time-frequency analysis process on the collected vibration data. First, it reads the most recent 1-second data sample from the buffer, containing angular velocity and acceleration values ​​at 200 time points. It then initializes the wavelet transform parameters, selects the db4 wavelet basis function as the analysis basis function, and sets the decomposition level to 5, covering the frequency range from 0.1 Hz to 100 Hz. It then performs a discrete wavelet transform calculation, processing the data for each sensor axis separately to generate wavelet coefficient matrices at different scales. The system traverses the wavelet coefficient matrix and identifies significant coefficient points, which represent characteristic moments and frequencies in the signal. Based on these coefficients, the system statistically calculates the frequency distribution, including the energy ratio and dominant frequency value within each frequency band; calculates the amplitude variation characteristics, obtains the signal envelope through wavelet reconstruction, and measures the maximum value, mean value, and rate of change; extracts duration characteristics, marks the vibration start, peak, and end points, and calculates the time interval between each characteristic point. To further analyze frequency band characteristics, the system implements four bandpass filters, one for each frequency band: 0-5Hz, 5-15Hz, 15-30Hz, and above 30Hz. Passing the original signal through these filters yields four sub-band signals. The RMS value of each sub-band signal is calculated and squared to yield an energy estimate. The percentage of each band's energy relative to the total energy is then calculated to form a description of the band's energy distribution.

[0020] Specifically, when combining the extracted features, the system first normalizes the eigenvalues ​​to ensure consistent dimensioning. Frequency distribution features include the energy percentages of four frequency bands and three dominant frequency values, for a total of seven parameters. Amplitude variation features include the maximum amplitude, root mean square amplitude, peak-to-valley ratio, and number of mutations, for a total of four parameters. Duration features include the average period, maximum duration, and decay time constant, for a total of three parameters. The system then arranges these parameters sequentially to form a 14-dimensional feature vector. Once the feature vector is constructed, an improved K-means clustering process is initiated. First, the system loads a preprocessed sample set from the historical database, containing feature vectors of various typical vibration modes. Density peak detection is then performed, calculating the sample point density in the feature space and identifying four local points of highest density as initial cluster centers. Feature weighting is then applied, dynamically adjusting the weight coefficients of different feature dimensions based on the preliminary analysis results of the current vibration. For example, frequency features are weighted more heavily for low-frequency vibration and amplitude features are weighted more heavily for impact vibration. The K-means algorithm then iterates, calculating the weighted distance from each sample point to each cluster center, reassigning the sample point to its category, and updating the cluster center location until the classification stabilizes or the maximum number of iterations is reached. Finally, the class label and distance to the cluster center for each sample point are output.

[0021] Specifically, when generating environmental vibration correlation data, the system first reads the current surveillance camera's installation information from the configuration database, including its installation height, structure type code, installation method parameters, and GPS coordinates. It then creates a data structure, designing a multidimensional array containing four key fields: location information, vibration type, time period, and characteristic parameters. The system associates the vibration type results obtained through cluster analysis with the installation location information and records the frequency statistics of each vibration type at the current location. For long-term systems, time series analysis is performed, dividing a day into 24-hour segments. The probability of occurrence and typical characteristic values ​​of each vibration type are calculated hourly to construct a vibration pattern model in the time dimension. Furthermore, the system identifies vibration patterns under special conditions, such as those with distinctive vibration characteristics during rainy, windy, or rush hour traffic, and identifies the relevance of these conditions. All this information is integrated into structured environmental vibration correlation data, which is stored in non-volatile memory and continuously updated and refined as the system operates, gradually forming a comprehensive description of the vibration characteristics of the specific installation environment.

[0022] 102. Calculate the weight distribution ratio between mechanical image stabilization and electronic image stabilization for the surveillance camera based on the environmental vibration correlation data, and obtain a coordinated working strategy for mechanical and electronic image stabilization. In one embodiment of the present invention, the weight distribution ratio between mechanical anti-shake and electronic anti-shake of the surveillance camera is calculated based on the environmental vibration correlation data to obtain the collaborative working strategy of mechanical-electronic anti-shake, including: constructing a three-dimensional boundary function using the vibration frequency, vibration amplitude and environment type in the environmental vibration correlation data; performing frequency domain decomposition on the input vibration signal according to the three-dimensional boundary function, and calculating the processing weights of mechanical anti-shake and electronic anti-shake for each frequency component, and dynamically adjusting the processing weights based on the pan-tilt motor temperature and operating time data of the surveillance camera; calculating the correspondence between mechanical anti-shake response parameters and vibration frequency based on the instantaneous strong vibration characteristics in the environmental vibration correlation data, and generating a parameter adjustment matrix for mechanical anti-shake; combining the processing weights with the parameter adjustment matrix to form a collaborative working strategy for mechanical-electronic anti-shake.

[0023] Specifically, the implementation process for constructing a three-dimensional boundary function using vibration frequency, amplitude, and environment type from environmental vibration data first uses data extraction technology to load the environmental vibration data from a database. The system then uses structured query operations to retrieve frequency characteristic records, amplitude distribution information, and environment type codes from the environmental vibration data. A grid mapping technique is then used to construct a three-dimensional spatial structure, with frequency, amplitude, and environment type as the three orthogonal coordinate axes. Based on this grid, the system runs a support vector machine algorithm, using historical anti-shake effect data as training samples to learn the optimal weight distribution pattern under different vibration conditions. Cross-validation is used during the learning process to ensure model generalization and avoid overfitting. After model training, the system executes an interpolation and filling algorithm, using trilinear interpolation to calculate weights at grid midpoints to ensure continuity of the weight distribution throughout the three-dimensional space. For areas with sparse data, radial basis functions are applied for smoothing to avoid boundary discontinuities. The constructed three-dimensional boundary function is essentially a multidimensional lookup table that maps vibration conditions to weights, achieving a precise mapping from vibration characteristics to anti-shake strategies. Finally, the system uses data compression technology to encode and store boundary functions to reduce memory usage, and designs a fast retrieval algorithm to ensure real-time query efficiency.

[0024] Specifically, the process of frequency-domain decomposition of the input vibration signal and calculating processing weights based on a three-dimensional boundary function employs a multi-stage signal processing technology chain. First, the system performs windowing, applying a Hanning window function to the vibration data to reduce spectral leakage. Then, the time-domain signal is converted to the frequency domain using a fast Fourier transform algorithm, and peak detection based on energy thresholds is used to identify the primary frequency components. For each significant frequency component detected, the system uses a three-dimensional boundary function query technique and trilinear interpolation to accurately calculate the theoretical weight corresponding to the current combination of frequency, amplitude, and environmental type. Next, the system applies sensor fusion technology to acquire real-time gimbal motor temperature data and employs a thermal effect compensation algorithm to assess the impact of temperature on motor performance. Furthermore, the system tracks motor life using cumulative runtime statistics and applies a wear model to estimate mechanical performance degradation. Based on these input parameters, the system executes an adaptive weight adjustment algorithm to dynamically adjust the theoretical weights according to device status. This algorithm uses a nonlinear decay function to account for the impact of temperature and usage time on performance, and performs constrained optimization to ensure that the adjusted weights still meet system stability requirements. The entire process is processed in parallel on a microcontroller, ensuring real-time responsiveness.

[0025] Specifically, the process of calculating mechanical anti-shake response parameters based on transient strong vibration characteristics in environmental vibration correlation data employs pattern recognition and control parameter optimization techniques. First, the system uses time series analysis to extract the time-frequency characteristics of typical strong vibration patterns from the environmental vibration correlation data. Principal component analysis is then applied to reduce dimensionality, retaining key feature dimensions. A pattern matching algorithm then compares the current vibration signal with a pre-set template to identify the current vibration type. The system then performs control theory analysis to establish a dynamic model of the gimbal mechanical system, including physical parameters such as mass, inertia, damping, and friction. Based on this model, the system runs a parameter optimization algorithm to calculate the control parameter combination that minimizes residual vibration for vibration inputs of different frequencies. This optimization process uses a genetic algorithm to explore the parameter space, with the objective function set to minimize residual vibration energy. The system performs optimization at each frequency point, generating a mapping between frequency and control parameters. Finally, numerical fitting techniques are used to convert the discrete optimization results into a continuous function, forming a smooth mapping from frequency to control parameters and constructing a complete parameter adjustment matrix. This matrix, essentially a control parameter lookup table, enables the system to rapidly adjust the mechanical control strategy based on the frequency characteristics of the input vibration.

[0026] Specifically, the process of combining processing weights and parameter adjustment matrices to form a collaborative working strategy utilizes data integration and multi-layer control strategy design techniques. First, the system uses data structure design methods to create a composite configuration object consisting of two main components: mechanical and electronic stabilization. Weight parameter fields, control parameter fields, and timing control fields are set within these components to store the results calculated in the previous steps. The system then performs timing analysis, measuring the response delay characteristics of the mechanical system to establish a temporal relationship model between the mechanical and electronic stabilization systems. Based on this model, the system applies an optimization scheduling algorithm to calculate the optimal operating sequence for the two subsystems, ensuring complementary collaboration rather than mutual interference. The system then implements frequency-scaling processing strategy design, optimizing the allocation of processing resources based on the characteristics of different vibration frequencies and the processing strengths of each subsystem. For special case handling, the system employs rule-based expert system technology to define trigger conditions and corresponding processing logic to cope with complex vibration environments. Finally, through data encapsulation technology, the system converts all configuration information into a structured collaborative working strategy data packet. A communication interface is implemented to regularly transmit updated strategies to the stabilization control unit. The entire combination process is essentially a multi-level control strategy construction process, from the parameter layer to the strategy layer, and finally forms a complete collaborative anti-shake control solution.

[0027] Furthermore, the frequency domain decomposition of the input vibration signal according to the three-dimensional boundary function, and the calculation of the processing weights of mechanical anti-shake and electronic anti-shake for each frequency component, and the dynamic adjustment of the processing weights based on the pan-tilt motor temperature and operating time data of the monitoring camera include: applying a fast Fourier transform algorithm to the input vibration signal to obtain signal amplitudes of different frequency components; substituting the signal amplitudes of the different frequency components into the three-dimensional boundary function, calculating the processing weights of mechanical anti-shake and electronic anti-shake corresponding to each frequency component, and obtaining the pan-tilt motor temperature data and operating time data from the sensor module of the monitoring camera; obtaining a temperature adjustment factor based on the mapping relationship between the pan-tilt motor temperature data and temperature and mechanical response efficiency, and obtaining a time adjustment factor based on the operating time data and the corresponding relationship between usage time and mechanical wear; performing a weighted combination of the temperature adjustment factor and the time adjustment factor to generate a comprehensive adjustment coefficient, and using the comprehensive adjustment coefficient to perform a nonlinear transformation on the processing weight to obtain a dynamically adjusted processing weight.

[0028] Specifically, the process of applying the Fast Fourier Transform (FFT) algorithm to the input vibration signal to obtain frequency components first employs data segmentation. The system uses overlapping windows to segment the continuously acquired vibration data stream into multiple analysis segments, each containing a fixed number of data points. After segmentation, each data segment is subjected to windowing, using a Hanning window function applied to the raw data to reduce leakage effects during spectral analysis. After preprocessing, the system executes a butterfly-based Fast Fourier Transform (FFT) algorithm, which simplifies the complex discrete Fourier transform (DFT) calculation into a series of basic units through recursive decomposition. In actual implementation, the system uses a radix-2 decimation FFT algorithm to convert the time-domain signal to a frequency-domain representation. After the transformation, the system applies amplitude calculation techniques to convert the complex Fourier coefficients into real amplitudes and calculate the energy of each frequency component. Spectral peak detection is then implemented, using an adaptive threshold method to identify significant components in the spectrum, using statistical characteristics of the energy distribution as the criteria. For the detected frequency peaks, the system performs frequency band classification technology to classify each frequency component into different processing groups according to the preset frequency boundaries, forming a set of frequency-amplitude data pairs and completing the frequency domain decomposition of the vibration signal.

[0029] Specifically, the implementation of the 3D boundary function to calculate weights by substituting the signal amplitudes of different frequency components into the processing weights integrates multidimensional data query and sensor integration technologies. The system extracts characteristic parameters for each identified frequency component, recording its frequency value and normalized amplitude. The system then applies environmental perception technology to extract the environment type code from the current working environment information. Combining the frequency value, amplitude, and environment type parameters, the system performs a 3D table lookup. Because the 3D boundary function is stored as a discrete grid, the system uses multidimensional interpolation technology to calculate the precise weight value for a given point. Specifically, the system locates the grid cell where the target point is located, obtains the pre-calculated weight values ​​for the cell vertices, and then calculates the weight at any location using a trilinear interpolation algorithm. In parallel processing, the system implements sensor data acquisition technology, reading data from a temperature sensor near the pan / tilt motor through a dedicated interface. The temperature sensor uses a thermistor design and provides digitized temperature readings via analog-to-digital conversion circuitry. Simultaneously, the system uses runtime tracking technology to access motor usage records stored in persistent memory. This record is updated through cumulative timekeeping, enabling long-term monitoring of the mechanical system's usage history. These sensor data, together with the vibration frequency characteristics, constitute the input parameter set for weight calculation.

[0030] Specifically, device characteristic modeling and performance degradation analysis techniques are applied to the mapping of gimbal motor temperature data and temperature to mechanical response efficiency. The system performs a temperature normality analysis, comparing the current temperature data with the normal operating temperature range defined in the motor specifications. The system then applies a motor thermal characteristic model, which describes how temperature changes affect the motor's physical properties. This model is based on the thermal expansion characteristics of the motor material, the viscosity variation of the lubricating oil, and the temperature coefficient of resistance, comprehensively considering the impact of these factors on the motor's response time. The system loads temperature-response curves from a pre-set characteristic curve database. These curves, obtained through motor performance testing, store the changes in motor response efficiency at different temperature points. The system then uses a curve interpolation algorithm to calculate the precise response efficiency value at the current temperature and converts the result into a temperature adjustment factor. Parallel to this, a usage time analysis technique applies a mechanical wear model to assess the impact of long-term use on hardware performance. Based on the principles of mechanical tribology, this model considers various mechanical aging factors, such as bearing wear, gear wear, and brush wear, and uses statistical analysis to establish a correlation between usage time and performance degradation. The system calculates the mechanical performance retention rate based on the current cumulative operating time and converts this value into a duration adjustment factor.

[0031] Specifically, the process of combining the temperature adjustment factor and the duration adjustment factor to generate a comprehensive adjustment coefficient utilizes multi-factor fusion and adaptive adjustment technology. The system first performs a factor weight analysis, assessing the relative importance of the temperature and duration factors based on the current operating status. This assessment dynamically determines the appropriate weight distribution ratio, taking into account the degree of variability of the two factors and their sensitivity to system performance. The system then applies a weighted fusion algorithm to linearly combine the two adjustment factors according to the determined weights to generate a preliminary comprehensive adjustment value. The system then implements a clipping technique to ensure that the comprehensive adjustment coefficient remains within a reasonable range, preventing extreme values ​​from causing system instability. After calculating the adjustment coefficient, the system applies a nonlinear transformation technique to adjust the original weights. This transformation is based on a power function relationship, taking into account the different response characteristics of the mechanical and electronic systems to parameter changes. For the mechanical stabilization weights, the system applies a smooth transformation function to ensure the continuity of weight changes. For the electronic stabilization weights, the system maintains the constraint that the sum of the mechanical and electronic weights is always 1, maintaining the overall stabilization capability. In this way, the system generates dynamically adjusted weight values ​​that adapt to the current hardware status, completing the conversion process from static theoretical weights to actual executable weights, ensuring that the anti-shake system maintains optimal performance under various working conditions.

[0032] 103. Predict the vibration trajectory based on the vibration characteristic parameters, and compensate the vibration trajectory in advance through mechanical anti-shake; In one embodiment of the present invention, predicting the vibration trajectory based on the vibration characteristic parameters and compensating the vibration trajectory in advance through mechanical anti-shake includes: using an improved Kalman filter algorithm to perform time series processing on the vibration characteristic parameters to predict the vibration trajectory in a future time period; converting the predicted vibration trajectory into a pan-tilt compensation angle instruction, and calculating the response delay time of the mechanical execution system; performing time advance processing on the pan-tilt compensation angle instruction based on the response delay time to generate a mechanical anti-shake execution instruction; sending the mechanical anti-shake execution instruction to the pan-tilt control system of the surveillance camera, and recording the timestamp, target compensation angle and actual completion angle of the mechanical anti-shake execution instruction to generate a compensation execution record table.

[0033] Specifically, the improved Kalman filter algorithm uses state-space modeling and recursive estimation techniques to perform time-series processing on vibration characteristic parameters to predict vibration trajectories. The system first organizes the collected vibration characteristic parameters into a time series format, including multidimensional feature vectors such as frequency distribution, amplitude variation, and duration characteristics. It then constructs a state-space model of the vibration system, representing the vibration state as a combination of position, velocity, and acceleration. A state transition equation is established to describe the temporal evolution of the vibration state. The improved Kalman filter algorithm adds three key technologies to the traditional algorithm: adaptive noise estimation, which dynamically adjusts the process noise and measurement noise covariance matrices by analyzing the statistical characteristics of recent data samples; multi-model fusion, which runs multiple filters with different parameter settings in parallel and automatically selects the optimal model based on prediction error; and vibration pattern recognition, which uses a classifier trained on historical data to identify the current vibration type and provide prior knowledge for the filter. When performing predictions, the system first applies a filtering algorithm to the historical data to estimate the current vibration state. It then uses the state transition equation to predict future vibration states. The system typically predicts vibration trajectories within 50 milliseconds, a period sufficient to cover the response delay of the mechanical anti-shake system. The prediction result is a trajectory sequence containing time points, three-dimensional spatial positions, and posture angles, which describes the expected vibration motion path of the camera.

[0034] Specifically, the process of converting the predicted vibration trajectory into gimbal compensation angle commands utilizes coordinate transformation and mechanical control technologies. The system first performs a coordinate transformation on the predicted vibration trajectory, converting the vibration displacement in the global coordinate system to the gimbal's mechanical coordinate system. This transformation takes into account the camera's mounting posture. After this transformation, the system applies an inverse vibration suppression transformation to the trajectory data to calculate the compensating motion required to offset the vibration—a motion trajectory in the opposite direction of the predicted vibration. The system then performs optical compensation, converting the displacement compensation into an angle compensation based on the current lens focal length and field of view. A longer focal length requires less angle compensation for the same linear displacement. The system then generates control signals, converting the angle compensation value into a control command for the gimbal's drive motor, including a pulse train for the stepper motor or a PWM signal for the servo motor. In parallel, the system measures the response delay of the mechanical actuator system. This process utilizes closed-loop testing techniques. The system sends standard test commands to the gimbal while simultaneously monitoring the actual start of motion using a gyroscope sensor, calculating the time difference between command issuance and execution. Delay measurement takes into account multiple factors, including communication delay, signal processing time, motor starting characteristics, and mechanical inertia. These factors are combined to obtain the system response delay time under the current operating state.

[0035] Specifically, the implementation of time-advancing the gimbal compensation angle command based on the response delay utilizes predictive compensation and trajectory optimization techniques. The system first performs a time shift operation, advancing the calculated compensation angle command on the time axis by an amount equal to the measured response delay. This ensures that mechanical compensation is in place when vibration actually occurs. The system then applies dynamic characteristic compensation technology, taking into account the gimbal motor's acceleration performance and maximum speed limit, to correct the advanced motion trajectory, ensuring that the command is within the execution capabilities of the mechanical system. Specifically, the system uses a step response model to describe the dynamic characteristics of the gimbal system and adjusts the timing and amplitude of the control commands based on this model. The system then applies a smoothing technique, applying a low-pass filtering algorithm to the compensation trajectory to reduce high-frequency components in the command, thereby preventing mechanical system resonance and excessive wear. Based on this processed trajectory, the system generates time-series mechanical anti-shake execution commands. Each command contains the execution time, horizontal compensation angle, and vertical compensation angle. These commands are encoded in a format recognizable by the micro-motion controller and include a command type identifier, target position parameters, and an execution timestamp, forming a complete command sequence ready for transmission to the gimbal control system.

[0036] Specifically, the process of sending mechanical stabilization execution commands to the surveillance camera's pan-tilt control system and recording their execution status integrates communication control and feedback recording technologies. The system first establishes a communication connection with the pan-tilt controller, typically using a serial interface (such as RS-485 or UART) or a network interface (such as Ethernet or CAN bus) to ensure low-latency and highly reliable command transmission. The communication protocol uses a customized real-time control protocol that includes synchronization mechanisms, error detection, and flow control. Command transmission utilizes queue management technology. The system places generated command sequences into a priority queue and dispatches them according to their urgency. The system also maintains a command tracking database that records detailed information about each command sent, including the timestamp, command content, and sequence number. After a command is issued, the system continuously monitors the pan-tilt feedback channel to receive execution status information. This feedback information, obtained via a sensor network, includes actual angle information and execution completion signals from the pan-tilt encoder. The system correlates the received feedback data with the command data to construct a complete compensation execution record. This record table is organized in a time series format. Each record contains the timestamp of the original command, the timestamp of the command arriving at the gimbal controller, the timestamp of execution completion, the target compensation angle value, and the actual angle value. The record table is stored in system memory and periodically written to persistent storage for performance analysis and system optimization. The data structure of the execution record table supports efficient queries, allowing the system to quickly locate gimbal status information at a specific moment using a time index.

[0037] Furthermore, converting the predicted vibration trajectory into a pan-tilt compensation angle instruction and calculating the response delay time of the mechanical execution system includes: decomposing the predicted vibration trajectory in a three-dimensional space coordinate system to obtain displacement components in the X-axis, Y-axis and Z-axis directions; converting the displacement components in the X-axis, Y-axis and Z-axis directions into horizontal rotation angles, vertical pitch angles and axial rotation angles according to the installation posture matrix of the monitoring camera; applying a nonlinear amplification factor to the horizontal rotation angle and vertical pitch angle based on the optical zoom ratio of the monitoring camera to generate compensation angle data; using the motor characteristic parameters of the monitoring camera to calculate the corresponding motor control pulse sequence for the compensation angle data to form a pan-tilt compensation angle instruction; by sending a test pulse to the pan-tilt control system, measuring the time difference from the sending of the pan-tilt compensation angle instruction to the actual start of the pan-tilt movement, and obtaining a basic delay value in the current working state; combining the current pan-tilt load data and angular velocity of the monitoring camera, correcting and calculating the basic delay value to generate the response delay time of the mechanical execution system.

[0038] Specifically, the process of decomposing the predicted vibration trajectory into a three-dimensional spatial coordinate system first defines a unified global coordinate system, typically with the initial installation position of the surveillance camera as the origin, the horizontal axis as the X-axis, the vertical axis as the Y-axis, and the optical axis as the Z-axis. The system receives vibration trajectory data from the prediction module, which consists of a series of time points and their corresponding predicted positions. The system then performs a vector decomposition operation, projecting the three-dimensional position vector of each time point onto three orthogonal coordinate axes to obtain displacement components in the X, Y, and Z directions. In specific implementation, the system uses a vector projection algorithm to perform coordinate axis separation on the original trajectory point sequence, generating three independent time-displacement sequences representing the time-varying displacement values ​​in the X, Y, and Z directions. This decomposition process takes into account the anisotropic nature of vibration. In the actual operating environment of surveillance cameras, vibration characteristics often differ in different directions. For example, vibration caused by building shaking is often more pronounced in the horizontal direction than in the vertical direction. The decomposed data is filtered. The system applies a Butterworth low-pass filter to remove high-frequency noise while retaining the main vibration components. The filter cutoff frequency is adaptively set based on the vibration characteristics, typically within the 15-30Hz range. The processed triaxial displacement component data reflects the independent variation of the predicted vibration in the three orthogonal directions.

[0039] Specifically, spatial geometric transformation techniques are employed to convert displacement components into angles based on the surveillance camera's installation pose matrix. The system first loads the installation pose matrix from the camera configuration database. This matrix, generated by a calibration process during installation, describes the rotation and translation of the camera coordinate system relative to the global coordinate system. The pose matrix is ​​typically represented as a 4×4 matrix, containing both rotation and translation information. The system then performs a coordinate transformation to convert the decomposed X-, Y-, and Z-axis displacement components from the global coordinate system to the camera's local coordinate system. This transformation involves matrix multiplication, applying the same transformation matrix to the displacement vector at each time point. After the transformation is complete, the system performs an inverse kinematic solution to convert the displacement components into the required rotation angles for the camera pan / tilt. This inverse solution is based on the pan / tilt's mechanical structure model and takes into account its kinematic constraints. For a typical two-degree-of-freedom pan / tilt, the system calculates the horizontal rotation and vertical pitch angles; for a three-degree-of-freedom pan / tilt, the system also calculates the axial rotation angles. The calculation considers nonlinear mechanical characteristics, such as gear backlash and bearing friction, and corrects for these using a compensation model. The converted angle data is represented as a series of time points and their corresponding three-axis rotation angle values, describing the angle changes that the gimbal needs to perform to offset the predicted vibration effect.

[0040] Specifically, the implementation of applying a nonlinear magnification factor to the rotation angle based on the surveillance camera's optical zoom ratio combines optical imaging principles with control compensation technology. The system first reads the current optical zoom ratio from the lens control module. This value reflects the camera's current field of view and focal length. A higher zoom ratio indicates a longer focal length and a smaller field of view, resulting in greater image displacement for the same angular change. The system then applies an optical compensation model to calculate the nonlinear magnification factor for angle compensation based on the current zoom ratio. In its implementation, the system uses a piecewise function to describe the relationship between zoom ratio and magnification factor. For example, in the low zoom ratio range (1-5x), the magnification factor changes nearly linearly; in the medium zoom ratio range (5-20x), the magnification factor gradually slows down; and in the high zoom ratio range (above 20x), the magnification factor approaches saturation. This nonlinear relationship is stored in a lookup table, and the system interpolates and calculates the precise magnification factor based on the current zoom ratio. The system then applies the calculated magnification factor to the horizontal rotation angle and vertical pitch angle to obtain compensated angle data. This dynamic adjustment based on zoom ratio ensures consistent image stabilization compensation at different focal lengths, avoiding overcompensation at high zoom levels and undercompensation at low zoom levels. The generated compensation angle data maintains a time series structure, including timestamps and corrected three-axis rotation angle values.

[0041] Specifically, the implementation process of calculating the control pulse sequence using the surveillance camera's motor characteristic parameters involves motor control theory and signal generation technology. The system first loads the pan / tilt motor's characteristic parameters from the hardware configuration database, including key parameters such as motor type (stepper or servo), drive mode, step angle, reduction ratio, and encoder resolution. For commonly used stepper motors, the system also loads subdivision settings and torque-speed curve data. The system then executes the motor motion planning algorithm, converting the compensation angle data into angular velocity and acceleration curves, taking into account the motor's starting characteristics, maximum speed limit, and acceleration limit. A trapezoidal or S-shaped velocity curve is used during the planning process to ensure smooth starting and stopping. The system then generates specific control signals based on the motor drive mode. For stepper motors, the system calculates the number of pulses required at each time point and converts this into an actual pulse sequence based on the subdivision settings. For servo motors, the system generates a combination of position and velocity commands. The control signal generation process takes into account the motor's response characteristics, smoothing rapidly changing commands to avoid mechanical resonance. The system also performs motion limit checks to ensure that the generated control sequence does not exceed the physical limitations of the motors, truncating or replanning any parts that exceed the limits. The resulting gimbal compensation angle command is a command sequence containing a timestamp, control signal type, and specific parameter values, ready to be sent to the gimbal control system through the communication interface.

[0042] Specifically, the implementation of measuring basic delay values ​​by sending test pulses to the pan-tilt control system utilizes timing analysis and delay measurement techniques. The system first designs a dedicated delay measurement experiment, generating a set of standard test pulse sequences with predefined amplitudes and forms for accurate identification in feedback signals. The system then enters a measurement loop, sending test pulses to the pan-tilt control system and simultaneously starting a high-precision timer. The system monitors the outputs of the gyroscope and accelerometer to detect the actual start of pan-tilt motion. In its implementation, the system applies a threshold detection algorithm, marking the start of motion when the sensor output changes above a preset threshold. To improve measurement accuracy, the system performs multiple repeated measurements, typically 10-20 times, and applies statistical analysis to the measurement results, removing outliers and calculating the average delay. The measurement process accounts for motion characteristics of varying directions and amplitudes, measuring response delays in the horizontal and vertical directions, as well as the difference in response to large and small amplitude motions. The system also analyzes the components of communication delay, signal processing delay, and mechanical actuation delay, enabling detailed measurement by inserting time markers at each stage. Through this series of measurements, the system obtains the basic delay value under the current working state, which reflects the time difference between the issuance of the command and the actual start of movement, usually in the range of 20-50 milliseconds.

[0043] Specifically, load adaptation and dynamic compensation techniques are employed to correct the base delay value based on the gimbal's current load data and angular velocity. The system first obtains current load data from the gimbal status monitoring module. This data comes from multiple sources, including current monitoring of the motor driver, output from the torque sensor, and load estimates from the motor controller. This load data reflects the gimbal's current mechanical load and is affected by the weight of the camera and accessories, as well as external factors such as wind. The system then obtains angular velocity information from the motion planning module for the control command to be executed. A higher angular velocity requires a higher drive torque and a faster response time. The system applies a load-delay mapping model, which describes how the system's response delay varies under different load conditions. This model, based on experimental data, generally demonstrates a positive correlation between higher load and longer delay. Furthermore, a speed-delay mapping model is used to describe the relationship between angular velocity and startup delay, which typically exhibits shorter startup delays for high-speed motion and longer delays for low-speed motion. The system combines the outputs of these two models through a weighted combination to calculate a comprehensive correction factor. The system then multiplies the base delay value by the correction factor to obtain a response delay time adjusted for the current operating state. This dynamic correction accounts for variations in actual operating conditions, making predictive compensation more accurate. This is particularly advantageous in applications with large load fluctuations or the need to execute diverse motion patterns.

[0044] 104. Calculate the residual jitter amount after the mechanical anti-shake is compensated in advance according to the collaborative working strategy, perform electronic anti-shake based on the residual jitter amount, and achieve stable output of the surveillance camera image.

[0045] In one embodiment of the present invention, the calculation of the residual jitter amount after advance compensation of mechanical anti-shake according to the collaborative working strategy, and the electronic anti-shake based on the residual jitter amount to achieve stable output of the surveillance camera image include: querying the compensation execution record table according to the timestamp of the current image frame to obtain the actual compensation displacement data of the mechanical anti-shake at the corresponding moment; calculating the difference between the actual compensation displacement data of the mechanical anti-shake and the ideal compensation displacement of the vibration trajectory to obtain the residual jitter amount; applying the compensation envelope comparison technology to the residual jitter amount to identify the phase conflict area that may be generated by mechanical anti-shake and electronic anti-shake; in the phase conflict area, performing phase adjustment on the residual jitter amount, and performing electronic image displacement compensation on the adjusted residual jitter amount according to the electronic anti-shake weight in the collaborative working strategy to achieve stable output of the surveillance camera image.

[0046] Specifically, the process of querying the compensation execution record table based on the timestamp of the current image frame to obtain the actual compensation displacement data for mechanical image stabilization involves time synchronization and data retrieval technologies. The system first receives the currently processed video frame from the image acquisition module. Each frame has a precise timestamp information. This timestamp is generated based on the system's unified clock, ensuring consistency with the time base of the mechanical image stabilization system. The system then performs a timestamp matching operation, converting the image frame timestamp into a query index. Because the sampling frequencies of the image acquisition and mechanical image stabilization systems typically differ, the system uses a nearest neighbor search algorithm to find the closest record in terms of time. Specifically, the system first calculates the difference between the image frame timestamp and the timestamps of each record in the execution record table, then selects the record with the smallest difference or performs time interpolation. The query process utilizes an efficient index structure, organizing the compensation execution record table in chronological order and establishing a time index to support fast positioning. For each retrieved record, the system extracts the actual compensation displacement data for mechanical image stabilization, including horizontal and vertical angular displacement values. These values ​​reflect the actual compensation actions performed by the mechanical system, taking into account practical factors such as mechanical limitations, execution delays, and precision errors. The query results also include an execution status flag, indicating whether the compensation action was fully executed, partially executed, or failed. If an exact match is not found, the system uses interpolation technology to generate an estimated value based on records from previous and subsequent time points to ensure continuous processing.

[0047] Specifically, the process of calculating the difference between the actual compensation displacement data for mechanical image stabilization and the ideal compensation displacement of the vibration trajectory utilizes precise comparison and error analysis techniques. The system first obtains the ideal compensation displacement data corresponding to the timestamp of the current image frame from the vibration prediction module. This data represents the theoretical compensation value required to completely offset the predicted vibration. This acquisition also utilizes a timestamp matching mechanism to ensure the consistency of the comparison time points. The system then performs coordinate alignment, converting the actual and ideal compensation displacements into the same reference coordinate system. After alignment, the system calculates the vector difference between the two sets of data, calculating the horizontal and vertical differences. In its implementation, the system accounts for the specificities of angle representation and normalizes differences exceeding 180 degrees to ensure that the calculated results reflect the shortest angular path. The difference calculation results are expressed as angles, and the system then performs an optical equivalent transformation to convert the angle differences into pixel displacements on the image plane based on the current camera focal length and image sensor parameters. This conversion process takes into account lens distortion characteristics, particularly for wide-angle lenses, applying a nonlinear mapping model for precise conversion. The calculated residual jitter is expressed as a pixel displacement vector, which includes horizontal and vertical components and describes the residual vibration that still needs to be processed by electronic image stabilization after the mechanical image stabilization system compensates.

[0048] Specifically, the process of identifying phase conflict areas by applying compensation envelope comparison to residual jitter combines signal analysis and pattern recognition techniques. The system first builds a jitter history buffer to store the most recently calculated residual jitter sequence, forming a time-continuous jitter trajectory. The system then performs a frequency domain transformation to convert the jitter trajectory into a frequency domain representation and analyze its frequency composition and phase characteristics. This transformation utilizes short-time Fourier transform (SFT) technology, processing the jitter data using a sliding window to produce a time-varying spectrum. The system then applies phase analysis to calculate the phase information of each frequency component and generate a time-varying phase trajectory. The system also overlays and analyzes the transfer characteristics of the mechanical and electronic image stabilization systems to construct the frequency and phase response functions of the two systems. By comparing the phase difference between the two systems at each frequency point, the system identifies frequency ranges where phase conflict may occur—i.e., frequency bands where the phase difference between the two systems approaches 180 degrees. The system then locates the contribution of these frequency components to the current jitter data in the time domain, uses a bandpass filter bank to separate the jitter components in each frequency band, and marks the phase conflict areas. The recognition process takes vibration amplitude into account, focusing only on jitter components that significantly impact image quality and ignoring components with smaller amplitudes. Ultimately, the system generates a map of phase conflict regions, pinpointing frequency components and time periods requiring special attention.

[0049] Specifically, selective filtering and phase reconstruction techniques are employed to phase-adjust the residual jitter in phase-conflicting regions. The system first constructs a customized filter bank based on the identified phase-conflicting region map, with each filter targeting conflicting components within a specific frequency range. The system then applies frequency band decomposition to decompose the residual jitter into multiple frequency band components, representing jitter components in different frequency ranges. For frequency bands marked as phase-conflicting, the system performs phase adjustment, using methods such as phase reversal, phase delay, or phase advance. In implementation, the system calculates the optimal phase adjustment, typically adjusting the conflicting component by approximately 90 degrees to avoid direct interference with the mechanical image stabilization system. Phase adjustment is performed using an all-pass filter, which can shift the signal phase without affecting the amplitude characteristics. After the adjustment is complete, the system resynthesizes the components in each frequency band to generate the phase-adjusted residual jitter. The system then weights the residual jitter according to the electronic image stabilization weights in the collaborative working strategy. The weights reflect the contribution of the electronic image stabilization to the overall image stabilization system. The weighted residual jitter is then used directly for electronic image displacement compensation, which is achieved by shifting image pixels inversely. Sub-pixel interpolation technology is applied during the compensation process to ensure smooth and precise image translation. Ultimately, the system outputs a fully stabilized image. This image combines the large-scale displacement compensation capabilities of mechanical image stabilization with the fine-grained correction capabilities of electronic image stabilization, effectively suppressing all types of vibration and ensuring the stability and clarity of the surveillance image.

[0050] In this embodiment, vibration data from a surveillance camera is collected, and vibration characteristic parameters are extracted through time-frequency analysis to generate environmental vibration correlation data. Based on the environmental vibration correlation data, the weight distribution ratio between mechanical and electronic image stabilization for the surveillance camera is calculated to obtain a collaborative working strategy. The vibration trajectory is predicted based on the vibration characteristic parameters, and the vibration trajectory is pre-compensated through mechanical image stabilization. The residual jitter after pre-compensation is calculated according to the collaborative working strategy, and electronic image stabilization is performed based on the residual jitter, thereby achieving stable output of the surveillance camera image. By analyzing the characteristics of different vibration sources, the present invention achieves dynamic collaborative work between mechanical and electronic image stabilization, eliminates compensation conflicts between the two systems, and effectively improves the stability of surveillance camera images in complex environments.

[0051] The above describes the monitoring camera image stabilization method according to the embodiment of the present invention. The following describes the monitoring camera image stabilization device according to the embodiment of the present invention. Figure 2 In one embodiment of the present invention, a device for stabilizing an image of a surveillance camera includes: A vibration analysis module 201 is configured to collect vibration data from a surveillance camera, extract vibration characteristic parameters of the vibration data through time-frequency analysis, and generate environmental vibration correlation data based on the vibration characteristic parameters; a boundary division module 202 for calculating a weight distribution ratio between mechanical image stabilization and electronic image stabilization for the surveillance camera based on the environmental vibration correlation data, and obtaining a cooperative working strategy of mechanical-electronic image stabilization; A trajectory prediction module 203 is configured to predict a vibration trajectory based on the vibration characteristic parameters and to compensate the vibration trajectory in advance through mechanical anti-shake; The collaborative compensation module 204 is used to calculate the residual jitter amount after the mechanical anti-shake is compensated in advance according to the collaborative working strategy, and perform electronic anti-shake based on the residual jitter amount to achieve stable output of the surveillance camera image.

[0052] In an embodiment of the present invention, the surveillance camera image stabilization device runs the above-mentioned surveillance camera image stabilization method. The surveillance camera image stabilization device collects vibration data of the surveillance camera, extracts vibration characteristic parameters through time-frequency analysis, and generates environmental vibration correlation data; calculates the weight distribution ratio between mechanical anti-shake and electronic anti-shake of the surveillance camera based on the environmental vibration correlation data, and obtains a collaborative working strategy; predicts the vibration trajectory based on the vibration characteristic parameters, and pre-compensates the vibration trajectory through mechanical anti-shake; calculates the residual jitter amount after pre-compensation based on the collaborative working strategy, and performs electronic anti-shake based on the residual jitter amount to achieve stable output of the surveillance camera image. The present invention realizes the dynamic collaborative work of mechanical anti-shake and electronic anti-shake by analyzing the characteristics of different vibration sources, eliminates the compensation conflict between the two systems, and effectively improves the stability of the surveillance camera image in complex environments.

[0053] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described systems, devices, and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0054] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program code, such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0055] As described above, the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that the technical solutions described in the above embodiments can still be modified, or some of the technical features thereof can be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A surveillance camera image stabilization method, characterized in that: The surveillance camera image stabilization method comprises: Collecting vibration data from a surveillance camera, extracting vibration characteristic parameters of the vibration data through time-frequency analysis, and generating environmental vibration correlation data based on the vibration characteristic parameters; Calculating a weight distribution ratio between mechanical image stabilization and electronic image stabilization for the surveillance camera based on the environmental vibration correlation data to obtain a coordinated working strategy for mechanical-electronic image stabilization; predicting a vibration trajectory according to the vibration characteristic parameters, and compensating the vibration trajectory in advance through mechanical anti-shake; The residual jitter amount after advance compensation of mechanical anti-shake is calculated according to the collaborative working strategy, and electronic anti-shake is performed based on the residual jitter amount to achieve stable output of the monitoring camera image.

2. The surveillance camera image stabilization method according to claim 1, characterized in that: The collecting vibration data of the monitoring camera, extracting vibration characteristic parameters of the vibration data through time-frequency analysis, and generating environmental vibration related data according to the vibration characteristic parameters includes: Vibration data is collected using the three-axis gyroscope and three-axis acceleration sensor built into the surveillance camera; Applying a wavelet transform algorithm to the vibration data to perform time-frequency domain decomposition, extracting frequency distribution, amplitude change, and duration characteristics, and dividing the vibration data into frequency bands to calculate the vibration energy distribution of each frequency band; The frequency distribution, amplitude change, duration characteristics, and vibration energy distribution are used as vibration characteristic parameters, and cluster analysis is performed on the vibration characteristic parameters based on an improved K-means clustering algorithm to obtain corresponding vibration types; Environmental vibration related data is generated according to the correspondence between the installation location information of the monitoring camera and the vibration type.

3. The surveillance camera image stabilization method according to claim 1, wherein: Calculating the weight distribution ratio between mechanical image stabilization and electronic image stabilization of the surveillance camera based on the environmental vibration correlation data to obtain a coordinated working strategy of mechanical-electronic image stabilization includes: Constructing a three-dimensional boundary function using the vibration frequency, vibration amplitude and environment type in the environmental vibration correlation data; Performing frequency domain decomposition of the input vibration signal according to the three-dimensional boundary function, calculating processing weights of mechanical image stabilization and electronic image stabilization for each frequency component, and dynamically adjusting the processing weights based on the temperature and operating time data of the pan / tilt motor of the surveillance camera; Calculating a correspondence between a mechanical anti-shake response parameter and a vibration frequency based on instantaneous strong vibration characteristics in the environmental vibration correlation data, and generating a parameter adjustment matrix for mechanical anti-shake; The processing weights are combined with the parameter adjustment matrix to form a cooperative working strategy of mechanical-electronic image stabilization.

4. The surveillance camera image stabilization method according to claim 3, characterized in that: The step of performing frequency domain decomposition of the input vibration signal according to the three-dimensional boundary function, calculating processing weights of mechanical image stabilization and electronic image stabilization for each frequency component, and dynamically adjusting the processing weights based on the temperature and operating time data of the pan / tilt motor of the surveillance camera includes: Applying a fast Fourier transform algorithm to the input vibration signal to obtain signal amplitudes of different frequency components; Substituting the signal amplitudes of the different frequency components into the three-dimensional boundary function, calculating the processing weights of mechanical image stabilization and electronic image stabilization corresponding to each frequency component, and obtaining the pan / tilt motor temperature data and operating time data from the sensor module of the surveillance camera; Obtaining a temperature adjustment factor based on the PTZ motor temperature data and a mapping relationship between temperature and mechanical response efficiency, and obtaining a duration adjustment factor based on the operating time data and a corresponding relationship between the operating time and mechanical wear; The temperature adjustment factor and the duration adjustment factor are weightedly combined to generate a comprehensive adjustment coefficient, and the processing weight is nonlinearly transformed using the comprehensive adjustment coefficient to obtain a dynamically adjusted processing weight.

5. The surveillance camera image stabilization method according to claim 1, wherein: Predicting the vibration trajectory according to the vibration characteristic parameters and compensating the vibration trajectory in advance through mechanical anti-shake includes: Using an improved Kalman filter algorithm to perform time series processing on the vibration characteristic parameters to predict the vibration trajectory in a future time period; Convert the predicted vibration trajectory into a gimbal compensation angle command and calculate the response delay time of the mechanical execution system; Performing time advance processing on the gimbal compensation angle instruction based on the response delay time to generate a mechanical anti-shake execution instruction; The mechanical anti-shake execution instruction is sent to the pan / tilt control system of the surveillance camera, and the timestamp, target compensation angle and actual completion angle of the mechanical anti-shake execution instruction are recorded to generate a compensation execution record table.

6. The surveillance camera image stabilization method according to claim 5, characterized in that: Converting the predicted vibration trajectory into a pan / tilt compensation angle command and calculating the response delay time of the mechanical execution system includes: Decomposing the predicted vibration trajectory in a three-dimensional space coordinate system to obtain displacement components in the X-axis, Y-axis, and Z-axis directions; According to the installation posture matrix of the surveillance camera, the displacement components in the X-axis, Y-axis and Z-axis directions are converted into horizontal rotation angle, vertical pitch angle and axial rotation angle; Applying a nonlinear magnification factor to the horizontal rotation angle and the vertical pitch angle based on the optical zoom ratio of the surveillance camera to generate compensation angle data; Utilizing the motor characteristic parameters of the surveillance camera, a motor control pulse sequence corresponding to the compensation angle data is calculated to form a pan / tilt compensation angle instruction; By sending a test pulse to the gimbal control system, the time difference from the gimbal compensation angle command to the actual gimbal movement is measured to obtain the basic delay value in the current working state; Combined with the current data of the pan / tilt load and the angular velocity of the surveillance camera, the basic delay value is corrected and calculated to generate the response delay time of the mechanical execution system.

7. The surveillance camera image stabilization method according to claim 5, characterized in that: The step of calculating the residual jitter amount after the mechanical anti-shake is compensated in advance according to the collaborative working strategy, and performing electronic anti-shake based on the residual jitter amount to achieve stable output of the surveillance camera image includes: querying the compensation execution record table according to the timestamp of the current image frame to obtain the actual compensation displacement data of the mechanical anti-shake at the corresponding moment; Calculating the difference between the actual compensation displacement data of the mechanical anti-shake and the ideal compensation displacement of the vibration trajectory to obtain a residual shake amount; Applying a compensation envelope comparison technique to the residual jitter amount to identify a phase conflict region that may be caused by mechanical image stabilization and electronic image stabilization; In the phase conflict area, the phase of the remaining jitter is adjusted, and electronic image displacement compensation is performed on the adjusted remaining jitter according to the electronic anti-shake weight in the collaborative working strategy to achieve stable output of the surveillance camera image.

8. A surveillance camera image stabilization device, characterized in that: The surveillance camera image stabilization device comprises: A vibration analysis module is used to collect vibration data from a surveillance camera, extract vibration characteristic parameters of the vibration data through time-frequency analysis, and generate environmental vibration correlation data based on the vibration characteristic parameters; a boundary division module, configured to calculate a weight distribution ratio between mechanical image stabilization and electronic image stabilization for the surveillance camera based on the environmental vibration correlation data, and obtain a cooperative working strategy of mechanical-electronic image stabilization; a trajectory prediction module, configured to predict a vibration trajectory based on the vibration characteristic parameters, and compensate the vibration trajectory in advance through mechanical anti-shake; The collaborative compensation module is used to calculate the residual jitter amount after the mechanical anti-shake is compensated in advance according to the collaborative working strategy, and perform electronic anti-shake based on the residual jitter amount to achieve stable output of the monitoring camera image.

Images