A four-legged robot on-board camera image stabilization and exposure coordination control method and system

Abstract

The application belongs to the technical field of robot vision and digital image processing, and discloses a four-legged robot airborne camera image stabilization and exposure cooperative control method and system, which comprises the following steps: obtaining the measurement data of an IMU, and constructing a shock signal sequence; performing periodic analysis and shock event detection on the shock signal sequence, and constructing a frequency-phase closed-loop tracking; constructing a shock proximity function, obtaining a reference attitude, and correcting an image; setting a shock avoidance interval, making an exposure period avoid the shock avoidance interval, adjusting an exposure time length and an ISO sensitivity; determining an exposure center time, and completing time alignment of the IMU measurement data and the image exposure. The application adopts the above method and system, uses the periodicity prior of the four-legged robot gait shock in the cooperative feedforward control of the camera imaging link and the digital image stabilization link, completes parameter scheduling before the shock arrives, simultaneously suppresses the inter-frame geometric jitter and the single-frame motion blur, and is suitable for a lightweight four-legged robot platform without a mechanical gimbal.

Description

Technical Field

[0001] This invention relates to the field of robot vision and digital image processing technology, and in particular to a method and system for coordinated control of image stabilization and exposure of an airborne camera on a quadruped robot. Background Technology

[0002] During walking and running, the feet of a quadruped robot perform periodic reciprocating motions. The impact force when the feet touch the ground is transmitted through the body structure to the onboard camera, subjecting the camera to two types of image quality degradation: geometric jitter, where the camera's posture changes rapidly and periodically with the gait, causing the video frame to shake; and motion blur, where significant changes in camera posture or scene within the exposure time window cause motion blur in a single frame. These two types of image degradation have the same physical source, but current technologies often treat them in isolation.

[0003] To address geometric jitter, existing solutions include mechanical gimbal stabilization, purely visual electronic image stabilization, and IMU-based digital image stabilization. Existing technologies utilize gait phase-time triggering for image capture, but still rely on foot force sensors and mechanical gimbals. IMU-based digital image stabilization typically smooths the calculated pose (e.g., using spherical linear interpolation or low-pass filtering with a fixed smoothing coefficient) to obtain a reference pose, which is then used to correct the image. Existing technologies mostly use fixed smoothing parameters, and a few rely on post-processing adjustments based on actual pose disturbances. This reactive control approach is ill-suited for scenarios involving reciprocating and impact-type disturbances, such as those experienced by quadruped robots.

[0004] To address motion blur, existing automatic exposure technologies mostly adjust exposure parameters based on image content feedback. This feedback implies lag and cannot distinguish between motion blur and changes in scene brightness. Although external trigger synchronous imaging technology has been widely used in industrial inspection, the triggering time is usually generated at a fixed cycle, independent of the machine's movement.

[0005] The gait of quadruped robots reveals periodicity or short-term periodicity in their movement. Foot impacts occur regularly, and the fundamental frequency of these impacts varies with speed but can be considered constant within a short time window. This prior information has been widely applied in motion planning and control for quadruped robots, but it has not yet been used for collaborative predictive control of the camera imaging link (trigger time, exposure parameters) and the digital image stabilization link (smoothing parameter scheduling).

[0006] In summary, the existing technology has at least the following problems: First, the image stabilization smoothing parameters lack feedforward scheduling. During the peak impact period, the fixed smoothing coefficient will cause the correction transformation to touch the image cropping boundary. After the impact, the reference pose catches up with the actual pose, resulting in a visible visual jump. Second, the exposure control is disconnected from the motion state. Automatic exposure based on image content has a lag. Fixed periodic triggering may expose at the peak impact time, which will simultaneously worsen single-frame blur and inter-frame jitter. Third, the image stabilization and exposure links are independent of each other and do not use the same predictor for coordinated control, which limits the overall image quality gains. Summary of the Invention

[0007] The purpose of this invention is to provide a method and system for coordinated control of image stabilization and exposure of an airborne camera on a quadruped robot. For four sets of robot-camera-IMU rigid connections without mechanical gimbals, the method uses only a single measurement data source from the IMU to predict foot impacts online based on periodic prior information of gait impacts. The same predicted value drives the coordinated adjustment of stabilization parameters and imaging parameters. Through feedforward means, geometric jitter and motion blur are suppressed simultaneously, avoiding visual jumps caused by improper parameter settings during impact, and ensuring the temporal consistency of variable frame rate output.

[0008] To achieve the above objectives, the present invention provides a method for coordinated control of image stabilization and exposure of an airborne camera on a quadruped robot, comprising the following steps: Step S1, External Trigger Acquisition and Spatiotemporal Alignment: Acquire the measurement data of the inertial measurement unit and the timestamp of the external trigger pulse through offline calibration, determine the exposure center time of any frame image, and align the measurement data of the inertial measurement unit to the exposure center time. Step S2, Inertial Measurement Unit Calibration and Preprocessing: Perform deterministic error compensation on the measurement data, and perform attitude calculation to obtain the attitude of the inertial measurement unit. Remove the gravity component from the acceleration data based on the attitude to obtain the dynamic acceleration. Step S3, Online Prediction of Foot Impact: Construct an impact signal sequence reflecting foot impact using dynamic acceleration; perform periodic analysis on the impact signal sequence to obtain the impact fundamental frequency estimate; detect impact events in the impact signal sequence to obtain the impact event time; calculate the impact phase using the most recent impact event time as the phase zero point, and extrapolate the impact phase to obtain the predicted time of the next foot impact; use the phase error between the detected impact event time and the predicted time to correct the impact fundamental frequency estimate, forming a frequency-phase closed-loop tracking; estimate the impact intensity and prediction confidence based on the impact signal sequence. Step S4, Corrected Gain Predictive Gating: Construct an impact proximity function, which varies between 0 and 1 depending on the proximity between the current time and the predicted time. Based on the impact proximity function, pre-lower the attitude calculated according to the impact proximity function before the predicted impact arrives. Step S5, smoothing coefficient double-layer amplitude limiting adaptive smoothing: adjust the smoothing coefficient of attitude smoothing according to the impact proximity function and impact intensity to smooth the attitude and obtain the reference attitude. Step S6, Image Correction: Calculate the correction transformation based on the difference between the pose and the reference pose, and correct the image acquired by the camera; Step S7, Trigger Control: Set the impact avoidance time interval with the predicted time as the center, estimate the nominal trigger time, and select the actual trigger time within the allowable adjustment interval near the nominal trigger time, so that the exposure center after deducting the trigger delay and half the exposure time falls into the calm window between adjacent impact avoidance time intervals. Step S8, Exposure Parameter Allocation: Adjust the exposure time and ISO gain based on the estimated fundamental frequency and impact intensity; Step S9, Failure Rollback: When the prediction confidence is lower than the preset threshold, switch to fixed frame rate free trigger, fixed smoothing coefficient and default automatic exposure strategy, and exit after the prediction confidence recovers. The switching process is set with hysteresis.

[0009] Preferably, in step S1, the external trigger acquisition is aligned with spatiotemporal synchronization, and the specific process is as follows: Let the timestamp of the external trigger pulse be... The time delay between the camera receiving the pulse and the actual exposure is... The exposure duration is Then the exposure of the current frame begins. Exposure ends With the center of exposure As shown below: ; ; ; in, Exposure duration; trigger delay With exposure duration The data is provided by the camera or obtained through offline calibration; the IMU data is interpolated or integrated to the exposure center time, and the calculated pose corresponding to the exposure center time is used as the base pose for image stabilization.

[0010] Preferably, in step S2, the inertial measurement unit (IMU) is calibrated and preprocessed, and the specific process is as follows: Step S21: In the offline calibration stage, perform six-sided static calibration on the IMU to obtain the deterministic error parameters of the accelerometer and gyroscope, including zero bias. and The calibration coefficient matrix and installation misalignment matrix were obtained; Allan variance analysis was performed on the IMU stationary data to obtain random error parameters, including random walk coefficients and zero-bias instability; extrinsic parameter calibration of the IMU and camera was performed to obtain the rotation matrix from the IMU coordinate system to the camera coordinate system. And install the lever arm vector, and complete the time alignment; Step S22, during the online operation phase, the deterministic error parameters of the accelerometer and gyroscope are used to compensate for the real-time acquired IMU data point by point, as shown below: ; ; in, Acceleration detected in real time; The acceleration after compensation; This refers to the actual angular velocity collected. The compensated angular velocity, and This is the compensation matrix that includes the scale coefficients and misalignment corrections; The random error parameters obtained from Allan calibration are used to set the bandpass filter parameters, the adaptive threshold coefficient for event detection, and the fusion noise covariance. Step S23, based on the compensated , The data is used to calculate the attitude, and the attitude from the IMU coordinate system to the world coordinate system is output in real time. ; First, the gravity is transformed to the IMU coordinate system, as shown below: ; in, This is the gravity vector in the world coordinate system; This is the gravity vector transformed into the IMU coordinate system; This represents the transformation matrix from the IMU coordinate system to the world coordinate system; its transpose is its inverse. Then, after subtracting the gravity component in the IMU coordinate system, it is then rotated using a rotation matrix. Transform to the camera coordinate system to obtain the dynamic acceleration, as shown below: ; in, It is dynamic acceleration; When the IMU's sensing center does not coincide with the camera's optical center, the machine's rotation will introduce additional acceleration at the camera. The complete form of the compensated dynamic acceleration is shown below: ; in, For the body's angular acceleration, Obtained by difference; For tangential terms; For the centripetal term; This indicates the installation arm.

[0011] Preferably, in step S3, the impact fundamental frequency refers to the fundamental frequency of the foot impact event sequence. The quadruped robot impact event sequence is related to gait: each trot gait cycle contains 2 diagonal foot impacts, and the impact fundamental frequency is twice the step frequency; each walk gait cycle contains 4 impact events; when the impact events are not uniformly distributed within the gait cycle, the time sequence of each impact event is directly used to replace the single fundamental frequency for impact prediction and exposure scheduling. Step S31: Based on the dynamic acceleration in the camera coordinate system Constructing an impact signal sequence As shown below: ; in, Indicates bandpass filtering; Step S32: Perform periodic analysis on the impact signal sequence to obtain the estimated fundamental frequency of the impact. ; First, the shock signal sequence Sliding analysis windows of varying lengths are applied, and these windows are updated as sampling progresses. Next, periodic analysis is performed on the sequences within each window, searching for the main peak within a preset frequency band and using parabolic interpolation to obtain the fundamental frequency estimate and main peak significance for each window. Then, the estimates from each window are weighted and fused according to their significance, and the fused fundamental frequency estimate is output. When the significance of the main peak in each window is below the preset threshold, freeze the output and retain the most recent valid value; estimate the fundamental frequency of the fused impulse. Apply a first-order low-pass filter and limit the rate of change; Step S33: Perform impact event detection on the impact signal sequence to obtain the impact event time; When the impact signal sequence When the adaptive threshold is exceeded and a local maximum is reached, a foot impact event is determined to have occurred; parabolic interpolation is performed on the sampling points near the peak to obtain the event time with sub-sampling precision. ; No new events will be responded to within a preset interval after an event is detected, in order to avoid repeated counting caused by the oscillation of a single impact; Step S34: Using the most recently confirmed impact event time Calculate the current impact phase using the phase zero point. As shown below: ; in, The smoothed fundamental frequency of the impact is estimated; the phase at future time Δ is obtained by linear extrapolation, as shown below: ; Wherein, Δ is the lead time for prediction, which is not greater than the estimated value of the current impact cycle; The predicted time of the next impact event is obtained by extrapolating the impact phase. As shown below: ; Step S35: Record the time of each detected impact event. Event timing predicted by phase extrapolation By comparison, the phase error is obtained. ;when Maintain phase continuity within the preset tolerance, and Proportional feedback correction of fundamental frequency estimation This constitutes a frequency-phase closed-loop tracking; when When the tolerance is exceeded, Reset the phase zero point; when If no new impact event is detected after a preset time period, the event is considered missed, and the current status is maintained. Continue extrapolating the phase; if there are no valid events for several consecutive impact cycles, set the unlock flag and freeze the output; Step S36: Take the moving average or envelope of the peak values ​​of the N impact cycles as the current intensity estimate, denoted as... And extrapolate the intensity of the next cycle according to the trend, and denot it as normalized. ; The reliability of the prediction is measured by the sliding statistics of the deviation between the predicted time and the actual time of the impact event over several periods, and is mapped to the prediction confidence level. .

[0012] Preferably, in step S4, the gain prediction gating is corrected, and the specific process is as follows: An attitude calculation algorithm is used to estimate the real-time attitude. Its update equation is as follows: ; in, The measured values ​​of the gyroscope after compensation; Normalized accelerometer measurements; The gradient of the attitude error function is constructed from the accelerometer gravity reference; To correct the gain; Predictive gating is applied to the modified gain, as shown below: ; in, Adjust the gain for normal operating conditions; The impact proximity function takes values ​​[0,1], with a value of 1 at the predicted impact time and approaching 0 during the calm phase. The formula is defined as follows: ; in, The angular distance from the impact phase to the nearest impact phase; It is a Gaussian window function; This refers to the gating depth.

[0013] Preferably, in step S5, the smoothing coefficient undergoes double-layer amplitude limiting adaptive smoothing, the specific process of which is as follows: The calculated pose is smoothed to separate the intended motion from jitter. The basic form is quaternion spherical linear interpolation, as shown below: ; in, For smoothing coefficients; According to the predicted impact Implement a two-layer amplitude limiting adaptive adjustment, as shown below: ; ; ; in, and These are the maximum and minimum values ​​of the smoothing coefficient; The strength coefficient; The impact proximity function value is calculated based on the extrapolated impact phase. To predict lead time; This is the impact strength gain coefficient; This is the upper limit of the gain adjustment; The predicted impact strength; Basic upper limit; , These are the lower cutoff interval and the upper cutoff interval, respectively; The maximum change in the smoothing coefficient in a single step; In step S6, a correction transformation is calculated based on the difference between the pose and the reference pose to correct the image acquired by the camera. The specific process is as follows: The relative rotation is calculated based on the difference between the actual pose and the reference pose, and then the corrected transformation and affine image are calculated, as shown below: ; in, This represents the relative rotation between the actual attitude and the reference attitude. for The reference attitude rotation matrix at time t; for The actual attitude rotation matrix at any given time; This is the camera intrinsic parameter matrix.

[0014] Preferably, in step S7, the specific process of triggering control is as follows: Step S71: Generate the nominal trigger time of the k-th frame recursively. As shown below: ; in, Set the output frame rate for the k-th frame; Impact avoidance zone and calm window: based on the predicted impact time Define the impact avoidance zone around the center as follows: ; in, , Set according to the statistical characteristics of the impact transient duration; The adjacent avoidance zones form a calm window, with the point furthest from the impact on either side, centered at the impact phase π. As shown below: ; ; in, Indicates the impact period at the foot; Step S72: Set the allowable adjustment range near the nominal trigger time, as shown below: ; in, The adjustment should not exceed one-quarter of the current estimated shock cycle to ensure that the adjustment does not cross adjacent calm windows. Within the allowable adjustment range Select the trigger time and align the exposure center with the center of the nearest calm window, as shown below: ; in, The time from the nominal trigger time The most recent calm window number; For trigger delay; For exposure duration; This indicates that the time within the parentheses will be projected onto the allowable adjustment range. The time in parentheses means that the trigger is activated earlier than expected after deducting the trigger delay and half the exposure time, so that the exposure center falls exactly in the center of the calm window. To trigger earlier after deducting the trigger delay and half the exposure time, so that the exposure center falls exactly in the center of the calm window; Step S73: Degradation of infeasible operating conditions; When the adjustment range is allowed If the avoidance zone is completely covered and no triggering time satisfying the avoidance constraint can be found, the nominal triggering time shall be used. Triggered, a degradation strategy is executed simultaneously: the exposure time is shortened, the ISO is increased accordingly, and the corresponding smoothing coefficient is increased. .

[0015] Preferably, in step S8, the exposure time and ISO are adjusted based on the estimated fundamental frequency and impact intensity. The specific process is as follows: The number of pixels in motion blur is approximately proportional to the image plane angular displacement during the exposure time, as shown below: ; in, The number of pixels for motion blur; The expected angular velocity of the exposure window; The focal length is the pixel value. From normalized impact strength With impulse fundamental frequency estimation The calculation is as follows: ; Where A is the scaling factor for offline calibration; Given a fuzzy budget The upper limit of exposure time is solved using a closed-loop method, as shown below: ; The insufficient exposure caused by shortened exposure time is compensated by ISO, as shown below: ; in, The camera's baseline ISO. and These are the lower and upper limits for adjusting the camera's ISO sensitivity, respectively.

[0016] Preferably, in step S9, when the prediction confidence level Below the preset threshold Time: Triggers switch to fixed frame rate free running, smoothing coefficient Restore fixed value Exposure parameters are restored to the default automatic exposure strategy; pending After several consecutive impact cycles exceeding the threshold, the system will automatically exit the rollback mode.

[0017] A quadruped robot airborne camera image stabilization and exposure collaborative control system is used to execute the above-mentioned quadruped robot airborne camera image stabilization and exposure collaborative control method, comprising: a six-axis inertial measurement unit (IMU) rigidly mounted on the camera module, a global shutter external trigger camera supporting external trigger mode, and an airborne computing unit; The airborne computing unit includes an online impact prediction module, an image stabilization branch, and an exposure control branch. The IMU should be installed as close as possible to the camera's optical center and rigidly connected to the camera; the trigger pulse is generated by an onboard computing unit or a dedicated timer; the IMU and the camera triggering system share the same hardware clock reference; the IMU-camera extrinsic parameter rotation matrix is ​​calibrated offline. With camera intrinsic parameter K.

[0018] Therefore, the present invention employs the above-mentioned method and system for coordinated image stabilization and exposure control of an airborne camera on a quadruped robot, and the beneficial effects are as follows: (1) This invention uses a single impact prediction to drive the image stabilization link and the imaging link in a unified way, and incorporates the two types of damage, geometric jitter and motion blur, into a unified prediction and control framework. Parameter scheduling is completed before the impact arrives, which overcomes the inherent lag of reactive adjustment. During the impact, the image does not change at the top level without correction, and the overall image quality gain is greater than the sum of the independent optimization of the two links.

[0019] (2) By triggering the phase-locked calm window, the exposure window avoids the impact transient, thus reducing jitter and blur from the physical source. When the phase-lock is triggered, the exposure window is in the calm phase, the expected angular velocity is significantly reduced, the exposure time can be extended accordingly, the image signal-to-noise ratio is improved, and the synergistic benefit of the two branches sharing the prediction quantity is reflected.

[0020] (3) This invention protects the attitude reference from the transient contamination of the impact during the impact by modifying the gain prediction gating and the smoothing coefficient dual-layer amplitude limiting scheduling, and constrains the correction transformation amount within the clipping capability range to avoid visual jump caused by the reference attitude chasing after the impact; the confidence backoff mechanism ensures that the system behavior is predictable under gait disorder conditions such as sudden stop, turning, and slipping.

[0021] (4) The external trigger timestamp of this invention, combined with the calibrated trigger delay and exposure duration, directly gives the frame exposure center time, realizes the precise time alignment between IMU measurement and image exposure, eliminates the main source of image stabilization residuals such as IMU-image time deviation, and ensures the timing consistency of variable frame rate output.

[0022] (5) This invention does not rely on mechanical gimbals and foot force sensors, but only requires external trigger interfaces of IMU and camera body, and is suitable for low-cost, gimbal-free lightweight quadruped robot platforms.

[0023] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0024] Figure 1 This is a general block diagram of a quadruped robot airborne camera image stabilization and exposure collaborative control system according to the present invention; Figure 2 This is a flowchart of the overall process of the present invention: a method for coordinated control of image stabilization and exposure of an airborne camera on a quadruped robot. Figure 3 This is a block diagram illustrating the principle of frequency-phase closed-loop tracking in a quadruped robot airborne camera image stabilization and exposure collaborative control method and system according to the present invention. Detailed Implementation

[0025] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0026] like Figure 1 As shown, the present invention discloses a quadruped robot-mounted camera-stabilized exposure collaborative control system, which is mounted on the quadruped robot body. It includes a six-axis inertial measurement unit (IMU) rigidly mounted on the camera module (sampling rate not less than 200Hz, preferably above 400Hz to cover the spectrum of impact transients), a global shutter external trigger camera supporting external trigger mode (one external pulse enables one exposure imaging), and an onboard computing unit (including a trigger pulse output port, such as GPIO / PWM). The onboard computing unit includes an online impact prediction module, an image stabilization branch, and an exposure control branch.

[0027] The IMU is installed as close as possible to the camera's optical center and rigidly connected to the camera; the trigger pulse is generated by the onboard computing unit or a dedicated timer, and the IMU and camera triggering system share the same hardware clock reference. Offline calibration of the IMU-camera extrinsic parameter rotation matrix. With camera intrinsic parameter K.

[0028] like Figure 2 As shown, based on the above system, the specific implementation process of the present invention of a quadruped robot airborne camera image stabilization and exposure cooperative control method is as follows: Step S1, External Trigger Acquisition and Spatiotemporal Alignment: Through offline calibration, acquire the measurement data of the inertial measurement unit (IMU) rigidly connected to the camera and the timestamp of the external trigger pulse. For any frame of image, determine the exposure center time of the frame based on the timestamp of the external trigger pulse, the trigger delay, and the exposure duration, and align the IMU measurement data to the exposure center time.

[0029] Let the timestamp of the external trigger pulse be... The time delay between the camera receiving the pulse and the actual exposure is... The exposure duration is Then the exposure of the current frame begins. Exposure ends With the center of exposure As shown below: ; ; ; in, Exposure duration; trigger delay With exposure duration The data is provided by camera data or obtained through offline calibration. The IMU data is interpolated or integrated to the exposure center time, and the calculated pose corresponding to the exposure center time is used as the base pose for image stabilization.

[0030] Step S2, Inertial Measurement Unit (IMU) Calibration and Preprocessing: Deterministic error compensation is performed on the measurement data. The attitude of the IMU is obtained by attitude calculation based on the compensated measurement data. The gravity component in the acceleration data is removed based on the attitude to obtain the dynamic acceleration.

[0031] Step S21: In the offline calibration stage, perform six-sided static calibration on the IMU (multi-position calibration is optional) to obtain the deterministic error parameters of the accelerometer and gyroscope, including zero bias ( , ), calibration coefficient matrix and installation misalignment matrix; perform Allan variance analysis on IMU stationary data to obtain random error parameters, including random walk coefficients and zero-bias instability; perform extrinsic parameter calibration on the IMU and camera to obtain the rotation matrix from the IMU coordinate system to the camera coordinate system. (and install the lever arm vector), and complete the time alignment.

[0032] Step S22, during the online operation phase, the deterministic error parameters of the accelerometer and gyroscope are used to compensate for the real-time acquired IMU data point by point, as shown below: ; ; in, Acceleration detected in real time; The acceleration after compensation; This refers to the actual angular velocity collected. The compensated angular velocity; and This is a compensation matrix that includes the scale coefficients and misalignment corrections.

[0033] The random error parameters obtained from Allan calibration are used to set subsequent bandpass filter parameters, event detection adaptive threshold coefficients, and fusion noise covariance.

[0034] Step S23, based on the compensated , The data is used to calculate the attitude (e.g., Madgwick algorithm, optionally fused with visual / magnetic information), and the attitude is output in real time from the IMU coordinate system to the world coordinate system. .

[0035] First, the gravity is transformed to the IMU coordinate system, as shown below: ; in, This is the gravity vector in the world coordinate system; This is the gravity vector transformed into the IMU coordinate system; This represents the transformation matrix from the IMU coordinate system to the world coordinate system; its transpose is its inverse.

[0036] Then, after subtracting the gravity component in the IMU coordinate system, it is then rotated using a rotation matrix. Transform to the camera coordinate system to obtain the dynamic acceleration, as shown below: ; in, This is dynamic acceleration. Substitute the values ​​from the stationary state for verification. , Close to 0.

[0037] When the IMU sensitive center does not coincide with the camera optical center (mounting lever arm) When expressed in IMU coordinates, the rotation of the machine introduces additional acceleration at the camera. The complete form of the compensated dynamic acceleration is shown below: ; in, For the body's angular acceleration (for (obtained by difference); For tangential terms; For the centripetal term; This represents the relative position vector of the mounting arm, i.e., the IMU sensing center to the camera optical center.

[0038] Step S3, Online Prediction of Foot Impact: Construct an impact signal sequence reflecting foot impact using dynamic acceleration; perform periodic analysis on the impact signal sequence to obtain the impact fundamental frequency estimate; detect impact events in the impact signal sequence to obtain the impact event time; calculate the impact phase using the most recent impact event time as the phase zero point, and extrapolate the impact phase to obtain the predicted time of the next foot impact; use the phase error between the detected impact event time and the predicted time to correct the impact fundamental frequency estimate, forming a frequency-phase closed-loop tracking; and estimate the impact intensity and prediction confidence based on the impact signal sequence.

[0039] In this invention, the fundamental frequency of the impact refers to the fundamental frequency of the foot impact event sequence. For example... Figure 3 As shown, the impact event sequence of the quadruped robot is related to gait: each gait cycle of the trot gait contains two diagonal foot impacts, with the impact fundamental frequency being twice the step frequency; each gait cycle of the walk gait contains four impact events. When the impact events are not uniformly distributed within the gait cycle (such as in the walk gait), the time sequence of each impact event can be directly used to replace the single fundamental frequency for impact prediction and exposure scheduling.

[0040] Step S31: Based on the dynamic acceleration in the camera coordinate system obtained in step S2 Constructing an impact signal sequence As shown below: ; in, This indicates a bandpass filter, with the passband preset according to the expected step frequency range (e.g., 0.5Hz~5Hz), used to suppress low-frequency swaying and high-frequency structural noise of the machine body, as an equivalent implementation method.

[0041] Step S32: Perform periodic analysis on the impact signal sequence to obtain the impact fundamental frequency estimate. ).

[0042] First, the shock signal sequence Three sliding analysis windows of lengths of 1s, 2s, and 4s are applied, and the windows are updated as the sampling progresses. Short windows are used to track rapid changes in speed, while long windows are used to provide steady-state estimation accuracy. Each window is effective only if its length covers at least about two impact cycles.

[0043] Secondly, periodic analysis (normalized autocorrelation or zero-filled short-time Fourier transform) is performed on the sequence within each window. The main peak is searched within the preset frequency band, and parabolic interpolation is performed on the position of the main peak to obtain the fundamental frequency estimate of each window and the significance of its main peak.

[0044] Then, the estimates of each window are weighted and fused according to their significance (or long window estimation is used in steady state, and short window estimation is switched when a frequency change is detected), and the fused impulse fundamental frequency estimate is output. .

[0045] Finally, when the significance of the main peak in each window is below a preset threshold, the output is frozen and the most recent valid value is retained. This is used for estimating the fundamental frequency of the fused impulse. Apply a first-order low-pass filter and limit the rate of change.

[0046] Step S33: Perform impact event detection on the impact signal sequence to obtain the impact event time.

[0047] When the impact signal sequence A foot impact event is determined to have occurred when the value exceeds an adaptive threshold (e.g., k times the root mean square (RMS) of the signal within the analysis window, where k is a preset coefficient) and is a local maximum. Parabolic interpolation is then performed on the sampling points near the peak to obtain the event time with subsampling precision. ; No new events will be responded to within a preset interval after an event is detected (e.g., 0.3 to 0.5 times the current impact cycle estimate) to avoid repeated counting caused by the oscillation of a single impact.

[0048] Step S34: Using the most recently confirmed impact event time Calculate the current impact phase using the phase zero point. As shown below: ; in, The smoothed fundamental frequency of the impulse is estimated. The phase at future time Δ is obtained by linear extrapolation, as shown below: ; Wherein, Δ is the prediction lead time, preferably 20ms~100ms, and not greater than the estimated value of the current impact cycle.

[0049] The predicted time of the next impact event is obtained by extrapolating the impact phase. (at any time) ), as shown below: ; Step S35: Record the time of each detected impact event. Event timing predicted by phase extrapolation By comparison, the phase error is obtained. ;when Maintain phase continuity within the preset tolerance, and The impulse fundamental frequency estimation is corrected by feedback proportionally (optional proportional-integral). This constitutes a frequency-phase closed-loop tracking; when When the tolerance is exceeded, Reset the phase zero point.

[0050] when If no new impact event is detected after a preset time period (e.g., 1.5 times the estimated impact period), the event is considered missed, and the current status is maintained. Continue extrapolating the phase; if there are no valid events for several consecutive impact cycles, set the unlock flag and freeze the output.

[0051] Step S36: Take the moving average or envelope of the peak values ​​of the most recent N impact cycles (preferably N=4~8) as the current intensity estimate, denoted as... And extrapolate the intensity of the next cycle according to the trend, and denot it as normalized. .

[0052] The reliability of the prediction is measured by the moving statistics (such as root mean square) of the deviations between the predicted time and the actual time of the impact event over a number of recent periods, which is then mapped to the prediction confidence level. Gait disturbances (sudden stops, turns, slippage) increase deviation. It descends automatically.

[0053] Step S4, Corrected Gain Predictive Gating: Construct an impact proximity function, which varies between 0 and 1 depending on the proximity between the current time and the predicted time. Based on the impact proximity function, pre-lower the attitude calculated according to the impact proximity function before the predicted impact arrives.

[0054] Attitude estimation algorithms such as Madgwick, Mahony, or complementary filtering are used to estimate the real-time attitude. Its update equation is as follows: ; in, The measured values ​​of the gyroscope after compensation; Normalized accelerometer measurements; The gradient of the attitude error function is constructed from the accelerometer gravity reference; To correct the gain (Madgwick / Mahony gain or equivalent complementary coefficient).

[0055] The accelerometer output is a specific force, which is the sum of gravity and dynamic acceleration. During the foot impact, the dynamic acceleration is much greater than that of normal walking. The gravity reference is contaminated by the impact transient. If the attitude is still corrected according to the normal gain, the impact will be mistakenly introduced into the attitude calculation as an attitude change, contaminating the attitude reference used for subsequent degravity, impact signal construction and image correction.

[0056] Therefore, predictive gating is applied to the modified gain, as shown below: ; in, Adjust the gain for normal operating conditions; The impact proximity function takes values ​​[0,1], with a value of 1 at the predicted impact time and approaching 0 during the calm phase. The formula is defined as follows: ; in, The angular distance from the impact phase to the nearest impact phase (0 or 2π); Use a Gaussian window function (or a triangular window function if optional), with the window width set according to the duration of the impact transient. For gating depth, The greater the impact, the more thorough the gravity reference shielding.

[0057] When the shock is predicted to arrive The attitude was pre-suppressed, and attitude updates during the impact were primarily based on gyro integrals; the gravity reference was shielded before the impact contamination arrived; after the impact... Smooth recovery The rate of change of the gain is limited to prevent the gain jump itself from introducing attitude disturbances. The impact transient is short (approximately 10ms~30ms), during which the attitude drift of the gyroscope integral is much smaller than the attitude change caused by the impact. Therefore, the rate of change of the gain is limited. The gating will not significantly affect the attitude accuracy during and after the impact, nor will it affect the construction of the impact signal and the accuracy of event detection.

[0058] Step S5, Double-layer amplitude limiting adaptive smoothing: Adjust the smoothing coefficient of attitude smoothing according to the impact proximity function and impact intensity to smooth the attitude and obtain the reference attitude.

[0059] The calculated pose is smoothed to separate the intended motion from jitter. The basic form is quaternion spherical linear interpolation, as shown below: ; in, For smoothing coefficients, The larger the reference attitude, the faster it follows the actual attitude, and the weaker the smoothness.

[0060] According to the predicted impact Implement a two-layer amplitude limiting adaptive adjustment, as shown below: ; ; ; in, and These are the maximum and minimum values ​​of the smoothing coefficient; The strength coefficient; The impact proximity function value calculated based on the extrapolated impact phase is the same as the impact proximity function used in step S4 to correct the gain gating. To predict lead time; This is the impact strength gain coefficient; This is the upper limit of the gain adjustment; The predicted impact strength; The upper limit is the base (preferably 0.15~0.3); , These are the lower cutoff interval and the upper cutoff interval, respectively. Preferably not exceeding 0.5); This represents the maximum change in the smoothing coefficient in a single step.

[0061] The physical meaning of the control law is: stationary phase. Maintain small value (Preferred value 0.03~0.08), the reference attitude is highly smooth, and jitter is fully suppressed; when the predicted impact is near its peak... The correction is moderately increased, and the reference posture moderately follows the actual posture during the impact, so that the amount of correction is constrained within the clipping capability. The correction amount changes symmetrically before and after the impact, avoiding the visual jump caused by the reference posture "catching up" with the actual posture after the impact; after the impact... Pullback. Upper limit. It adapts to the impact intensity (the greater the impact, the higher the allowable following speed), forming a double-layer amplitude limiting; the rate of change limit ensures... It does not change on its own.

[0062] As an equivalent embodiment, the smoothing coefficient can also be defined as the smoothing intensity (the larger the value, the smoother the smoothing). In this case, the adjustment direction is reversed (the smoothing intensity is reduced when the impact is close to the peak), and the technical essence is the same.

[0063] Step S6, Image Correction: Calculate the correction transformation based on the difference between the pose and the reference pose, and correct the image acquired by the camera.

[0064] The relative rotation is calculated based on the difference between the actual pose and the reference pose, and then the corrected transformation and affine image are calculated, as shown below: ; in, This represents the relative rotation between the actual attitude and the reference attitude. for The reference attitude rotation matrix at time t; for The actual attitude rotation matrix at any given time; This is the camera intrinsic parameter matrix.

[0065] Under conditions of small field of view and distant scenes, the correction transformation can degenerate into an affine transformation. It should be noted that when the trigger interval adapts to the impulse fundamental frequency, the output video is a variable frame rate sequence: each frame carries a trigger timestamp, and the time axis is reconstructed according to the timestamp during downstream encoding or display, or it is resampled into a fixed frame rate output by the frame rate transformation module; the image stabilization correction itself is independent frame by frame and is not affected by frame rate changes.

[0066] Step S7, Trigger Control: Set the impact avoidance time interval with the predicted time as the center, estimate the nominal trigger time, and select the actual trigger time within the allowable adjustment interval near the nominal trigger time, so that the exposure center after deducting the trigger delay and half the exposure time falls into the calm window between adjacent impact avoidance time intervals.

[0067] Step S71: Generate the nominal trigger time of the k-th frame recursively. As shown below: ; in, Set the output frame rate for the k-th frame.

[0068] Impact avoidance zone and calm window: based on the predicted impact time Define the impact avoidance zone around the center as follows: ; in, , Statistical settings based on the duration of the impact transient (e.g., taking historical impact events at...) The statistical value of the pulse width is preferably 10ms~30ms.

[0069] The adjacent avoidance zones form a calm window, with the point furthest from the impact on either side, centered at the impact phase π. As shown below: ; ; in, This indicates the impact cycle at the foot.

[0070] Step S72: Set the allowable adjustment range near the nominal trigger time, as shown below: ; in, The adjustment should not exceed one-quarter of the current estimated shock cycle to ensure that the adjustment does not cross adjacent calm windows.

[0071] Within the allowable adjustment range Select the trigger time and align the exposure center with the center of the nearest calm window, as shown below: ; in, The time from the nominal trigger time The most recent calm window number; For trigger delay; Exposure duration (given by step S8); This indicates that the time interval within the parentheses will be projected (truncated) to the allowable adjustment range. The time in parentheses means that the trigger is activated earlier than expected after deducting the trigger delay and half the exposure time, so that the exposure center falls exactly in the center of the calm window. To pre-trigger the exposure before deducting the trigger delay and half the exposure time, the exposure center is positioned precisely in the center of the calm window.

[0072] Step S73: Downgrade processing for infeasible operating conditions.

[0073] When the adjustment range is allowed If the avoidance zone is completely covered (the impact period is too short, and the calm window width is insufficient to accommodate the exposure window), and a triggering moment that satisfies the avoidance constraint cannot be found, the nominal triggering moment will be used. Triggered, and simultaneously executed degradation strategy: shorten the exposure time and increase the ISO in step S8, and increase the smoothing coefficient corresponding to the frame in step S5. .

[0074] There is an optional variable frame rate implementation method, namely, in the optional implementation method, With the fundamental frequency of the impact Adaptive adjustment (e.g.) , (for the preset scaling factor), and for Applying a first-order low-pass filter and a rate-of-change constraint allows the frame rate to smoothly follow the motion rhythm; in the basic implementation... Take a fixed value .

[0075] Step S8, Exposure Parameter Allocation: Adjust the exposure time and gain (ISO) based on the estimated fundamental frequency and impact intensity.

[0076] The number of pixels in motion blur is approximately proportional to the image plane angular displacement during the exposure time, as shown below: ; in, The number of pixels for motion blur. The expected angular velocity of the exposure window; It represents the focal length per pixel.

[0077] From normalized impact strength With impulse fundamental frequency estimation The calculation is as follows: ; Where A is the proportional coefficient for offline calibration. During calibration, the robot is controlled to walk at multiple known speeds at a constant speed. The measured peak angular velocity and the corresponding impact intensity and impact fundamental frequency within the exposure window are recorded under each working condition, and a lookup table is constructed.

[0078] Given a fuzzy budget (Preferably 1~3 pixels), the closed-loop solution for the upper limit of exposure time is as follows: ; The insufficient exposure caused by shortened exposure time is compensated by ISO, as shown below: ; in, The camera's baseline ISO. and These are the lower and upper limits for adjusting the camera's ISO sensitivity, respectively. Set to acceptable noise level.

[0079] When ISO reaches its upper limit and still cannot meet the blur budget, the blur budget is prioritized, and moderate underexposure (or supplemental lighting) is accepted. Hysteresis ranges and rate-of-change limits are applied to both exposure duration and ISO adjustments to prevent exposure parameter breathing between adjacent frames. When phase lock is triggered, the exposure window itself is already in a calm phase. The signal-to-noise ratio is significantly reduced, and the exposure time can be relaxed accordingly, which benefits the signal-to-noise ratio. This is the synergistic benefit brought about by the two branches sharing the prediction amount.

[0080] Step S9, Failure rollback strategy: When the prediction confidence is lower than the preset threshold, switch to fixed frame rate free trigger, fixed smoothing coefficient and default automatic exposure strategy, and exit after the prediction confidence recovers. The switching process is set with hysteresis.

[0081] When prediction confidence Below the preset threshold (During gait disturbances such as sudden stops, turns, and slippage): triggers a switch to fixed frame rate free-running, with smoothing coefficient... Restore fixed value Exposure parameters are restored to the default automatic exposure strategy; pending After several consecutive impact cycles exceeding the threshold, the system automatically exits the rollback mode. Hysteresis is incorporated into the switching between rollback and recovery modes to prevent repeated jittering near the threshold.

[0082] Example 1 This embodiment describes the specific implementation of the image stabilization and exposure collaborative control method for an airborne camera on a quadruped robot proposed in this invention.

[0083] (1) Uniform walking: The fundamental frequency of the impact is stable at 2~4Hz (corresponding to a step frequency of about 1~2Hz), and the prediction confidence level is... High; the trigger is locked in the calm window of each cycle, the target frame rate is set to 30fps, the interval between adjacent nominal trigger times is about 33ms, and the actual trigger time is fine-tuned within the allowable adjustment range to make the exposure center aligned with the center of the calm window and avoid the impact avoidance zone.

[0084] (2) Running: The impact fundamental frequency rises to above 6Hz (corresponding to a step frequency of above 3Hz), and the normalized impact intensity Approaching 1; the trigger interval automatically shortens, increasing the frame rate; As the intensity increases, the impact period accelerates accordingly; the exposure time is reduced to 3ms~5ms, and the ISO is adjusted accordingly but does not exceed the preset maximum gain.

[0085] (3) Accelerated transition: The impact fundamental frequency changes slowly, the frequency-phase closed-loop tracking follows smoothly, and the trigger phase is always locked in the calm window; the frame rate changes smoothly during the speed change process without sudden changes.

[0086] (4) Sudden stop / turn: Prediction bias increases, prediction confidence decreases. If the gait falls below the threshold, the system enters fallback mode (fixed frame rate and fixed parameters), and automatically re-enters predictive control after the gait returns to normal.

[0087] Example 2 This embodiment describes the working principle of a quadruped robot airborne camera image stabilization and exposure collaborative control system proposed in this invention.

[0088] During operation, the IMU acquires body motion data at high frequency, and after deterministic error compensation, attitude calculation, gravity removal and external parameter transformation, it constructs an impact signal sequence reflecting the impact on the foot end.

[0089] The online impact prediction module estimates the fundamental frequency, phase, predicted time of the next foot impact, impact intensity, and prediction confidence level. It also establishes a frequency-phase closed-loop tracking by analyzing the deviation between the measured impact event and the predicted time, ensuring the continuity and accuracy of the prediction under variable speed conditions.

[0090] The prediction quantity acts on the image stabilization branch in one aspect: it reduces the correction gain of the attitude calculation before the predicted impact arrives, and shields the gravity reference from the transient contamination caused by the impact; at the same time, it implements a two-layer amplitude limiting scheduling of the smoothing coefficient according to the impact proximity function and the impact intensity, so that the reference attitude follows appropriately during the impact and the correction transformation quantity is constrained within the clipping capability range.

[0091] On the other hand, it acts on the exposure control branch: setting the impact avoidance range with the predicted time as the center, fine-tuning the actual trigger time within the allowable adjustment range near the nominal trigger time, so that the exposure center is aligned with the center of the calm window; and allocating the exposure time and ISO according to the fuzzy budget model.

[0092] The trigger timestamp, combined with the trigger delay and exposure duration, provides the exposure center moment, achieving precise alignment between IMU measurement and image exposure. When gait disturbance causes a decrease in prediction confidence, the system reverts to a safe mode with a fixed frame rate and fixed parameters, automatically returning to predictive control after gait recovery. This achieves collaborative feedforward control with "one-time prediction and dual-path drive," simultaneously suppressing inter-frame geometric jitter and single-frame motion blur on a lightweight quadruped platform without a mechanical gimbal.

[0093] Therefore, the present invention adopts the above-mentioned quadruped robot airborne camera image stabilization exposure collaborative control method and system. For four sets of robot-camera-IMU rigid connection, without mechanical gimbal, only the IMU single measurement data source is used to predict the foot impact online based on the periodic prior information of gait impact. The same prediction quantity drives the coordinated adjustment of image stabilization parameters and imaging parameters. Through feedforward means, geometric jitter and motion blur are suppressed at the same time, avoiding visual jumps caused by improper parameter settings during the impact process, and ensuring the temporal consistency of variable frame rate output.

[0094] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit them. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the technical solutions of the present invention, and these modifications or equivalent substitutions cannot cause the modified technical solutions to deviate from the spirit and scope of the technical solutions of the present invention.

Claims

1. A method for coordinated control of image stabilization and exposure of an airborne camera on a quadruped robot, characterized in that, Includes the following steps: Step S1, External Trigger Acquisition and Spatiotemporal Alignment: Acquire the measurement data of the inertial measurement unit and the timestamp of the external trigger pulse through offline calibration, determine the exposure center time of any frame image, and align the measurement data of the inertial measurement unit to the exposure center time. Step S2, Inertial Measurement Unit Calibration and Preprocessing: Perform deterministic error compensation on the measurement data, and perform attitude calculation to obtain the attitude of the inertial measurement unit. Remove the gravity component from the acceleration data based on the attitude to obtain the dynamic acceleration. Step S3, Online Prediction of Foot Impact: Construct an impact signal sequence reflecting foot impact using dynamic acceleration; perform periodic analysis on the impact signal sequence to obtain the impact fundamental frequency estimate; detect impact events in the impact signal sequence to obtain the impact event time; calculate the impact phase using the most recent impact event time as the phase zero point, and extrapolate the impact phase to obtain the predicted time of the next foot impact; use the phase error between the detected impact event time and the predicted time to correct the impact fundamental frequency estimate, forming a frequency-phase closed-loop tracking; estimate the impact intensity and prediction confidence based on the impact signal sequence. Step S4, Corrected Gain Predictive Gating: Construct an impact proximity function, which varies between 0 and 1 depending on the proximity between the current time and the predicted time. Based on the impact proximity function, pre-lower the attitude calculated according to the impact proximity function before the predicted impact arrives. Step S5, smoothing coefficient double-layer amplitude limiting adaptive smoothing: adjust the smoothing coefficient of attitude smoothing according to the impact proximity function and impact intensity to smooth the attitude and obtain the reference attitude. Step S6, Image Correction: Calculate the correction transformation based on the difference between the pose and the reference pose, and correct the image acquired by the camera; Step S7, Trigger Control: Set the impact avoidance time interval with the predicted time as the center, estimate the nominal trigger time, and select the actual trigger time within the allowable adjustment interval near the nominal trigger time, so that the exposure center after deducting the trigger delay and half the exposure time falls into the calm window between adjacent impact avoidance time intervals. Step S8, Exposure Parameter Allocation: Adjust the exposure time and ISO gain based on the estimated fundamental frequency and impact intensity; Step S9, Failure Rollback: When the prediction confidence is lower than the preset threshold, switch to fixed frame rate free trigger, fixed smoothing coefficient and default automatic exposure strategy, and exit after the prediction confidence recovers. The switching process is set with hysteresis.

2. The method for coordinated control of image stabilization and exposure of an airborne camera on a quadruped robot according to claim 1, characterized in that, In step S1, the external trigger acquisition is aligned with spatiotemporal synchronization, and the specific process is as follows: Let the timestamp of the external trigger pulse be... The time delay between the camera receiving the pulse and the actual exposure is... The exposure duration is Then the exposure of the current frame begins. Exposure ends With the center of exposure As shown below: ； ； ； in, Exposure duration; trigger delay With exposure duration The data is provided by the camera or obtained through offline calibration; the IMU data is interpolated or integrated to the exposure center time, and the calculated pose corresponding to the exposure center time is used as the base pose for image stabilization.

3. The method for coordinated control of image stabilization and exposure of an airborne camera on a quadruped robot according to claim 2, characterized in that, In step S2, the inertial measurement unit (IMU) is calibrated and preprocessed, and the specific process is as follows: Step S21: In the offline calibration stage, perform six-sided static calibration on the IMU to obtain the deterministic error parameters of the accelerometer and gyroscope, including zero bias. and The calibration coefficient matrix and installation misalignment matrix were obtained; Allan variance analysis was performed on the IMU stationary data to obtain random error parameters, including random walk coefficients and zero-bias instability; extrinsic parameter calibration of the IMU and camera was performed to obtain the rotation matrix from the IMU coordinate system to the camera coordinate system. And install the lever arm vector, and complete the time alignment; Step S22, during the online operation phase, the deterministic error parameters of the accelerometer and gyroscope are used to compensate for the real-time acquired IMU data point by point, as shown below: ； ； in, Acceleration detected in real time; The acceleration after compensation; This refers to the actual angular velocity collected. The compensated angular velocity; and This is the compensation matrix that includes the scale coefficients and misalignment corrections; The random error parameters obtained from Allan calibration are used to set the bandpass filter parameters, the adaptive threshold coefficient for event detection, and the fusion noise covariance. Step S23, based on the compensated , The data is used to calculate the attitude, and the attitude from the IMU coordinate system to the world coordinate system is output in real time. ; First, the gravity is transformed to the IMU coordinate system, as shown below: ； in, This is the gravity vector in the world coordinate system; This is the gravity vector transformed into the IMU coordinate system; This represents the transformation matrix from the IMU coordinate system to the world coordinate system; its transpose is its inverse. Then, after subtracting the gravity component in the IMU coordinate system, it is then rotated using a rotation matrix. Transform to the camera coordinate system to obtain the dynamic acceleration, as shown below: ； in, It is dynamic acceleration; When the IMU's sensing center does not coincide with the camera's optical center, the machine's rotation will introduce additional acceleration at the camera. The complete form of the compensated dynamic acceleration is shown below: ； in, For the body's angular acceleration, Obtained by difference; For tangential terms; For the centripetal term; This indicates the installation arm.

4. The method for coordinated control of image stabilization and exposure of an airborne camera on a quadruped robot according to claim 3, characterized in that, In step S3, the impact fundamental frequency refers to the fundamental frequency of the foot impact event sequence. The quadruped robot's impact event sequence is related to gait: each trot gait cycle contains 2 diagonal foot impacts, and the impact fundamental frequency is twice the step frequency; each walk gait cycle contains 4 impact events; when the impact events are not uniformly distributed within the gait cycle, the time sequence of each impact event is directly used to replace the single fundamental frequency for impact prediction and exposure scheduling. Step S31: Based on the dynamic acceleration in the camera coordinate system Constructing an impact signal sequence As shown below: ； in, Indicates bandpass filtering; Step S32: Perform periodic analysis on the impact signal sequence to obtain the estimated fundamental frequency of the impact. ; First, the shock signal sequence Sliding analysis windows of varying lengths are applied, and these windows are updated as sampling progresses. Next, periodic analysis is performed on the sequences within each window, searching for the main peak within a preset frequency band and using parabolic interpolation to obtain the fundamental frequency estimate and main peak significance for each window. Then, the estimates from each window are weighted and fused according to their significance, and the fused fundamental frequency estimate is output. When the significance of the main peak in each window is below the preset threshold, freeze the output and retain the most recent valid value; estimate the fundamental frequency of the fused impulse. Apply a first-order low-pass filter and limit the rate of change; Step S33: Perform impact event detection on the impact signal sequence to obtain the impact event time; When the impact signal sequence When the adaptive threshold is exceeded and a local maximum is reached, a foot impact event is determined to have occurred; parabolic interpolation is performed on the sampling points near the peak to obtain the event time with sub-sampling precision. ; No new events will be responded to within a preset interval after an event is detected, in order to avoid repeated counting caused by the oscillation of a single impact; Step S34: Using the most recently confirmed impact event time Calculate the current impact phase using the phase zero point. As shown below: ； in, The smoothed fundamental frequency of the impact is estimated; the phase at future time Δ is obtained by linear extrapolation, as shown below: ； Wherein, Δ is the lead time for prediction, which is not greater than the estimated value of the current impact cycle; The predicted time of the next impact event is obtained by extrapolating the impact phase. As shown below: ； Step S35: Record the time of each detected impact event. Event timing predicted by phase extrapolation By comparison, the phase error is obtained. ;when Maintain phase continuity within the preset tolerance, and Proportional feedback correction of fundamental frequency estimation This constitutes a frequency-phase closed-loop tracking; when When the tolerance is exceeded, Reset the phase zero point; when If no new impact event is detected after a preset time period, the event is considered missed, and the current status is maintained. Continue extrapolating the phase; if there are no valid events for several consecutive impact cycles, set the unlock flag and freeze the output; Step S36: Take the moving average or envelope of the peak values ​​of the N impact cycles as the current intensity estimate, denoted as... And extrapolate the intensity of the next cycle according to the trend, and denot it as normalized. ; The reliability of the prediction is measured by the sliding statistics of the deviation between the predicted time and the actual time of the impact event over several periods, and is mapped to the prediction confidence level. .

5. The method for coordinated control of image stabilization and exposure of an airborne camera on a quadruped robot according to claim 4, characterized in that, In step S4, the gain prediction gating is corrected, and the specific process is as follows: An attitude calculation algorithm is used to estimate the real-time attitude. Its update equation is as follows: ； in, The gyroscope measurement value after compensation; Normalized accelerometer measurements; The gradient of the attitude error function is constructed from the accelerometer gravity reference; To correct the gain; Predictive gating is applied to the modified gain, as shown below: ； in, Correct the gain for normal operating conditions; The impact proximity function takes values ​​[0,1], with a value of 1 at the predicted impact time and approaching 0 during the calm phase. The formula is defined as follows: ； in, The angular distance from the impact phase to the nearest impact phase; It is a Gaussian window function; This refers to the gating depth.

6. The method for coordinated control of image stabilization and exposure of an airborne camera on a quadruped robot according to claim 5, characterized in that, In step S5, the smoothing coefficient undergoes double-layer amplitude limiting adaptive smoothing, the specific process of which is as follows: The calculated pose is smoothed to separate the intended motion from jitter. The basic form is quaternion spherical linear interpolation, as shown below: ； in, For smoothing coefficients; According to the predicted impact Implement a two-layer amplitude limiting adaptive adjustment, as shown below: ； ； ； in, and These are the maximum and minimum values ​​of the smoothing coefficient; The strength coefficient; The impact proximity function value is calculated based on the extrapolated impact phase. To predict lead time; This is the impact strength gain coefficient; This is the upper limit of the gain adjustment; The predicted impact strength; Basic upper limit; , These are the lower cutoff interval and the upper cutoff interval, respectively; The maximum change in the smoothing coefficient in a single step; In step S6, a correction transformation is calculated based on the difference between the pose and the reference pose to correct the image acquired by the camera. The specific process is as follows: The relative rotation is calculated based on the difference between the actual pose and the reference pose, and then the corrected transformation and affine image are calculated, as shown below: ； in, This represents the relative rotation between the actual attitude and the reference attitude. for The reference attitude rotation matrix at time t; for The actual attitude rotation matrix at any given time; This is the camera intrinsic parameter matrix.

7. The method for coordinated control of image stabilization and exposure of an airborne camera on a quadruped robot according to claim 6, characterized in that, In step S7, the specific process of triggering control is as follows: Step S71: Generate the nominal trigger time of the k-th frame recursively. As shown below: ； in, Set the output frame rate for the k-th frame; Impact avoidance zone and calm window: based on the predicted impact time Define the impact avoidance zone around the center as follows: ； in, , Set according to the statistical characteristics of the impact transient duration; The adjacent avoidance zones form a calm window, with the point furthest from the impact on either side, centered at the impact phase π. As shown below: ； ； in, Indicates the impact period at the foot; Step S72: Set the allowable adjustment range near the nominal trigger time, as shown below: ； in, The adjustment should not exceed one-quarter of the current estimated shock cycle to ensure that the adjustment does not cross adjacent calm windows. Within the allowable adjustment range Select the trigger time and align the exposure center with the center of the nearest calm window, as shown below: ； in, The time from the nominal trigger time The most recent calm window number; For trigger delay; For exposure duration; This indicates that the time within the parentheses will be projected onto the allowable adjustment range. The time in parentheses means that the trigger is activated earlier than expected after deducting the trigger delay and half the exposure time, so that the exposure center falls exactly in the center of the calm window. To trigger earlier after deducting the trigger delay and half the exposure time, so that the exposure center falls exactly in the center of the calm window; Step S73: Degradation of infeasible operating conditions; When the adjustment range is allowed If the avoidance zone is completely covered and no triggering time satisfying the avoidance constraint can be found, the nominal triggering time shall be used. Triggered, a degradation strategy is executed simultaneously: the exposure time is shortened, the ISO is increased accordingly, and the corresponding smoothing coefficient is increased. .

8. The method for coordinated control of image stabilization and exposure of an airborne camera on a quadruped robot according to claim 7, characterized in that, In step S8, the exposure time and ISO gain are adjusted based on the estimated fundamental frequency and impact intensity. The specific process is as follows: The number of pixels in motion blur is approximately proportional to the image plane angular displacement during the exposure time, as shown below: ； in, The number of pixels for motion blur; The expected angular velocity of the exposure window; The focal length is the pixel value. From normalized impact strength With impulse fundamental frequency estimation The calculation is as follows: ； Where A is the scaling factor for offline calibration; Given a fuzzy budget The upper limit of exposure time is solved using a closed-loop method, as shown below: ； The insufficient exposure caused by shortened exposure time is compensated by ISO, as shown below: ； in, The camera's baseline ISO. and These are the lower and upper limits for adjusting the camera's ISO sensitivity, respectively.

9. The method for coordinated control of image stabilization and exposure of an airborne camera on a quadruped robot according to claim 8, characterized in that, In step S9, when the prediction confidence level Below the preset threshold Time: Triggers switch to fixed frame rate free running, smoothing coefficient Restore fixed value Exposure parameters are restored to the default automatic exposure strategy; pending After several consecutive impact cycles exceeding the threshold, the system will automatically exit the rollback mode.

10. A quadruped robot airborne camera image stabilization and exposure collaborative control system, used to execute the quadruped robot airborne camera image stabilization and exposure collaborative control method according to any one of claims 1-9, characterized in that, include: A six-axis inertial measurement unit (IMU) rigidly mounted on the camera module, a global shutter external trigger camera supporting external trigger mode, and an onboard computing unit; The airborne computing unit includes an online impact prediction module, an image stabilization branch, and an exposure control branch. The IMU is installed as close as possible to the camera's optical center and rigidly connected to the camera; the trigger pulse is generated by an onboard computing unit or a dedicated timer; the IMU and the camera triggering system share the same hardware clock reference; the IMU-camera extrinsic parameter rotation matrix is ​​calibrated offline. With camera intrinsic parameter K.

Images