Active image stabilization cooperative control method, device and equipment of quadruped robot and medium

Abstract

This application provides a method, device, equipment, and medium for active image stabilization and cooperative control of a quadruped robot, including: determining gait phase variables based on the quadruped robot's gait parameters and foot force data; predicting body posture vibration data within future gait cycles using a constructed body posture vibration prediction model with the gait phase variables as independent variables; generating a feedforward compensation signal based on the body posture vibration data; performing image stabilization control on the gait prediction feedforward outer loop and the gimbal native feedback inner loop based on the feedforward compensation signal and real-time posture data; determining a stable shooting range, and automatically triggering the gimbal to perform image acquisition when the gait phase enters the stable shooting range. This achieves a paradigm breakthrough from passive hysteresis stabilization to active feedforward image stabilization, significantly improving image quality and upgrading the inspection mode from the inefficient "walk-stop-shoot" to intelligent shooting during continuous walking through deep coupling of gait, gimbal, and shooting.

Description

Technical Field

[0001] This application relates to the technical field of motion control and visual inspection of quadruped robots, and in particular to active image stabilization and cooperative control methods, devices, equipment and media for quadruped robots. Background Technology

[0002] Quadruped robots, with their all-terrain mobility, are increasingly being used for routine inspections of power lines, utility tunnels, and other similar environments. High-definition gimbal cameras are the core component for quadruped inspection robots to collect visualized data such as equipment defects and meter readings. Currently, the closest existing technologies in gimbal stabilization and quadruped inspection include: PID single-loop passive image stabilization based on real-time attitude feedback from the gimbal's built-in IMU; a basic attitude compensation scheme that synchronizes the quadruped's IMU attitude data to the gimbal; and a gimbal target tracking and attitude adjustment scheme based on visual feedback. However, existing technologies suffer from the following core shortcomings: 1. Existing passive compensation schemes inherently suffer from control lag, only able to offset high-frequency, small-amplitude random vibrations, unable to adapt to the low-frequency, large-amplitude, periodic gait vibrations unique to quadruped robots, and unable to solve the core problem of blurred images captured while walking; 2. There is a widespread technical bias in the industry, believing that quadruped robot gait vibrations are highly uncertain due to terrain, load, and walking speed, and can only be compensated for lag through real-time posture feedback, unable to achieve feedforward compensation through gait periodic characteristics, thus failing to break through the technical bottleneck of passive image stabilization; 3. In existing solutions, the quadruped gait control system and the gimbal stabilization control system are completely decoupled, failing to utilize the periodicity and predictability of quadruped gait for collaborative optimization, and unable to solve the lag problem of "compensating after vibration occurs"; 4. Existing quadruped inspection solutions require frequent stops for shooting, unable to achieve continuous inspection while walking, and unable to adapt to the routine inspection needs of long-distance routes. Therefore, how to achieve active image stabilization and collaborative control of quadruped robots has become a significant technical problem. Summary of the Invention

[0003] In view of this, the purpose of this application is to provide an active image stabilization cooperative control method, device, equipment and medium for quadruped robots, and to construct a full-link control closed loop of "gait phase variable recognition - vibration prediction - feedforward compensation - dual-loop coordination - phase triggering", which realizes a paradigm breakthrough from passive hysteresis stabilization to active feedforward image stabilization. This not only significantly improves the imaging quality, but also upgrades the inspection mode from the inefficient "walk-stop-shoot" to intelligent shooting during continuous walking through the deep coupling of gait-gimbal-photography.

[0004] This application provides an active image stabilization cooperative control method for a quadruped robot, the active image stabilization cooperative control method comprising: Based on real-time acquisition of gait parameters and foot force data of the quadruped robot, the gait phase variables are determined. Using the gait phase variable as the independent variable, the fuselage attitude vibration data in the future gait cycle is predicted by the constructed fuselage attitude vibration prediction model, and a feedforward compensation signal matching the vibration phase is generated based on the fuselage attitude vibration data. Based on the aforementioned feedforward compensation signal and real-time attitude data, image stabilization control is performed on the gait prediction feedforward outer loop and the gimbal native feedback inner loop. Once a stable shooting range is determined, the gimbal is automatically triggered to perform image acquisition when the gait phase enters the stable shooting range.

[0005] In one possible implementation, determining the gait phase variable based on real-time acquisition of gait parameters and foot force data of the quadruped robot includes: A single gait cycle is divided into a support phase, a swing phase, and a transition phase; Using the diagonal foot contact moment as the gait phase zero point, the gait phase zero point is calibrated in real time based on the foot contact / leave events identified by the foot force data; Based on the calibrated gait phase zero point, a gait phase variable that varies linearly with time is generated in combination with the gait parameters.

[0006] In one possible implementation, the fuselage attitude vibration prediction model is: ; in, Gait period t Lower body posture vibration data, For gait phase variables It is a periodic function of the independent variable, characterizing the inherent vibrational properties of gait. It is a random perturbation.

[0007] In one possible implementation, the prediction of fuselage attitude vibration data during future gait cycles using the constructed fuselage attitude vibration prediction model includes: Fourier series expansion was performed on the fuselage inertial measurement unit data of historical gait cycles, and the fundamental frequency was determined based on the real-time gait frequency in order to fit the fuselage attitude change curve of future gait cycles. A lightweight long short-term memory (LSTM) neural network is used, with gait parameters, foot force sensor data and in-body IMU data in the historical gait cycle as input sequence, and output the posture change prediction value of the future gait cycle. The fuselage attitude change curve of the future gait cycle and the predicted attitude change value of the future gait cycle are weighted and fused to generate the fuselage attitude vibration data.

[0008] In one possible implementation, the image stabilization control based on the feedforward compensation signal and real-time attitude data for the gait prediction feedforward outer loop and the gimbal native feedback inner loop includes: The gait prediction feedforward outer loop receives the feedforward compensation signal and converts the feedforward compensation signal into compensation angles for each axis of the gimbal, thereby actively canceling the low-frequency large-amplitude periodic vibrations caused by gait. The native feedback inner loop of the gimbal is based on the real-time attitude data of the built-in high-frequency inertial measurement unit of the gimbal, and compensates for high-frequency random vibrations and residual disturbances through proportional, integral and derivative control.

[0009] In one possible implementation, the step of determining a stable shooting range, and automatically triggering the gimbal to perform an image acquisition operation when the gait phase enters the stable shooting range, includes: Based on the body attitude vibration data, the phase range where the body vibration amplitude is lower than a preset threshold is determined as the stable shooting range; When the gait phase enters the stable shooting range, the gimbal is automatically triggered to perform a shooting operation; If a stable shooting window does not appear within several consecutive gait cycles, the quadruped robot will be controlled to switch to a stable gait mode and resume its original inspection gait after the photo is taken.

[0010] In one possible implementation, after the automatically triggered gimbal performs the image acquisition operation, the active image stabilization collaborative control method further includes: The sharpness evaluation index of the acquired images is fed back to the fuselage attitude vibration prediction model for online fine-tuning of the gain coefficient and phase offset of the feedforward compensation signal. Using real-time point cloud data output by a 3D spatial camera, the actual fuselage attitude vibration data is determined, and the parameters of the fuselage attitude vibration prediction model are updated based on the fuselage attitude vibration data and the error between the fuselage attitude vibration data.

[0011] This application embodiment also provides an active image stabilization cooperative control device for a quadruped robot, the active image stabilization cooperative control device comprising: The gait phase recognition module is used to determine the gait phase variables based on the real-time acquisition of gait parameters and foot force data of the quadruped robot. The vibration prediction module is used to predict the fuselage attitude vibration data in the future gait cycle by using the gait phase variable as the independent variable and constructing a fuselage attitude vibration prediction model, and to generate a feedforward compensation signal that matches the vibration phase based on the fuselage attitude vibration data. The dual closed-loop control module is used to perform image stabilization control on the gait prediction feedforward outer loop and the gimbal native feedback inner loop based on the feedforward compensation signal and real-time attitude data. The image acquisition module is used to determine the stable shooting range. When the gait phase enters the stable shooting range, the gimbal is automatically triggered to perform image acquisition operation.

[0012] This application also provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. When the machine-readable instructions are executed by the processor, the steps of the active image stabilization cooperative control method for a quadruped robot as described above are performed.

[0013] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, performs the steps of the above-described active image stabilization cooperative control method for a quadruped robot.

[0014] The active image stabilization cooperative control method, device, equipment, and medium for quadruped robots provided in this application embodiment include: determining gait phase variables based on real-time acquisition of gait parameters and foot force data of the quadruped robot; predicting body posture vibration data within future gait cycles using the gait phase variables as independent variables through a constructed body posture vibration prediction model; generating a feedforward compensation signal matching the vibration phase based on the body posture vibration data; performing image stabilization control on the gait prediction feedforward outer loop and the gimbal native feedback inner loop based on the feedforward compensation signal and real-time posture data; determining a stable shooting range; and automatically triggering the gimbal to perform image acquisition operation when the gait phase enters the stable shooting range. By constructing a full-link control closed loop of "gait phase variable recognition - vibration prediction - feedforward compensation - dual-loop collaboration - phase triggering", a paradigm breakthrough from passive hysteresis stabilization to active feedforward image stabilization has been achieved. This not only significantly improves image quality, but also upgrades the inspection mode from the inefficient "walk-stop-shoot" to intelligent shooting during continuous walking through deep coupling of gait, gimbal and shooting.

[0015] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1A flowchart illustrating an active image stabilization cooperative control method for a quadruped robot provided in an embodiment of this application; Figure 2 A schematic diagram illustrating an active image stabilization cooperative control method for a quadruped robot provided in an embodiment of this application; Figure 3 One of the structural schematic diagrams of an active image stabilization cooperative control device for a quadruped robot provided in an embodiment of this application; Figure 4 A second schematic diagram of the structure of an active image stabilization and cooperative control device for a quadruped robot provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. Based on the embodiments of this application, every other embodiment obtained by those skilled in the art without inventive effort falls within the scope of protection of this application.

[0019] First, the applicable application scenarios of this application are introduced. This application can be applied to the technical field of quadruped robot motion control and visual inspection.

[0020] Research has shown that quadruped robots, with their all-terrain mobility, are increasingly being used for routine inspections of power lines, utility tunnels, and other similar environments. High-definition gimbal cameras are the core component for quadruped inspection robots to collect visualized data such as equipment defects and meter readings. Among current gimbal stabilization and quadruped inspection technologies, the closest existing technologies include: PID single-loop passive image stabilization based on real-time attitude feedback from the gimbal's built-in IMU; a basic attitude compensation scheme that synchronizes the quadruped's IMU attitude data to the gimbal; and a gimbal target tracking and attitude adjustment scheme based on visual feedback. However, existing technologies suffer from the following core shortcomings: 1. Existing passive compensation schemes inherently suffer from control lag, only able to offset high-frequency, small-amplitude random vibrations, unable to adapt to the low-frequency, large-amplitude, periodic gait vibrations unique to quadruped robots, and unable to solve the core problem of blurred images captured while walking; 2. There is a widespread technical bias in the industry, believing that quadruped robot gait vibrations are highly uncertain due to terrain, load, and walking speed, and can only be compensated for lag through real-time posture feedback, unable to achieve feedforward compensation through gait periodic characteristics, thus failing to break through the technical bottleneck of passive image stabilization; 3. In existing solutions, the quadruped gait control system and the gimbal stabilization control system are completely decoupled, failing to utilize the periodicity and predictability of quadruped gait for collaborative optimization, and unable to solve the lag problem of "compensating after vibration occurs"; 4. Existing quadruped inspection solutions require frequent stops for shooting, unable to achieve continuous inspection while walking, and unable to adapt to the routine inspection needs of long-distance routes. Therefore, how to achieve active image stabilization and collaborative control of quadruped robots has become a significant technical problem.

[0021] Based on this, the embodiments of this application provide an active image stabilization cooperative control method for quadruped robots, which constructs a full-link control closed loop of "gait phase variable recognition - vibration prediction - feedforward compensation - dual-loop coordination - phase triggering", realizing a paradigm breakthrough from passive hysteresis stabilization to active feedforward image stabilization. This not only significantly improves the imaging quality, but also upgrades the inspection mode from the inefficient "walk-stop-shoot" to intelligent shooting during continuous walking through the deep coupling of gait-gimbal-photography.

[0022] Please see Figure 1 , Figure 1 This is a flowchart illustrating an active image stabilization cooperative control method for a quadruped robot provided in an embodiment of this application. Figure 1 As shown in the embodiments of this application, the active image stabilization cooperative control method includes: S101: Based on real-time acquisition of gait parameters and foot force data of the quadruped robot, the gait phase variable is determined.

[0023] In this step, the gait parameters and foot force data of the quadruped robot are collected in real time, and the gait phase variables are determined based on the gait parameters and foot force data.

[0024] It should be noted that the gait phase variable φ (with a value range of [0, 2π]) is defined as the core dimensionless driving parameter characterizing the instantaneous motion state of a quadruped robot within a single complete gait cycle. Its physical meaning is: taking the moment when the diagonal feet (e.g., the right forefoot and the left hindfoot) simultaneously touch the ground as the phase zero point (φ=0), it progresses linearly with the gait cycle, and completes a 2π cycle when the next same-phase ground touch event occurs.

[0025] Here, before collecting gait parameters and foot force data, homogeneous transformation matrices are established for the quadruped robot's body coordinate system, body IMU coordinate system, and gimbal coordinate system to complete the joint calibration of multiple coordinate systems. Real-time attitude data from the quadruped robot's body IMU and data from the gimbal's built-in attitude sensors are collected simultaneously.

[0026] In one possible implementation, determining the gait phase variable based on real-time acquisition of gait parameters and foot force data of the quadruped robot includes: A: A single gait cycle is divided into a support phase, a swing phase, and a transition phase.

[0027] Here, a single gait cycle is divided into three discrete states: the support phase, the swing phase, and the transition phase. The support phase is the stable range of fuselage vibration, the swing phase is the range of fluctuating fuselage vibration, and the transition phase is the impact range of foot contact with / leaving the ground.

[0028] B: Using the diagonal foot contact moment as the gait phase zero point, the gait phase zero point is calibrated in real time based on the foot contact / leave events identified by the foot force data.

[0029] Here, the moment the diagonal foot of the quadruped robot touches the ground is taken as the phase zero point, and the gait phase variable φ∈[0,2π] is defined, with the phase linearly synchronized with the gait cycle over time. Foot contact / lift events are identified based on foot force data, and the gait phase zero point is calibrated in real time.

[0030] The process involves real-time acquisition of the Z-axis (perpendicular to the ground) force signal Fz(t) output from the six-dimensional force sensors at each foot. Adaptive threshold detection is performed on each foot signal: a dynamic baseline Fbase is set as the moving average of Fz over the most recent 500ms. When Fz(t) > Fbase + ΔF (ΔF = 15N) and persists for ≥3 sampling periods (corresponding to 3ms at a sampling rate of 1kHz), it is considered a valid ground contact event. Further filtering is used to identify synchronous ground contact events at diagonal feet (e.g., RF and LH) based on the time difference |tRF between the two feet. If tLH|≤20ms, it is considered to be synchronized, and the time t0 is marked as the phase zero point of the current gait cycle.

[0031] C: Based on the calibrated gait phase zero point, a gait phase variable that varies linearly with time is generated in combination with the gait parameters.

[0032] Here, after the gait phase zero-point calibration is completed, the system takes time t0 as the starting point and, in conjunction with real-time gait parameters, calculates the gait phase variable at the current time t in real time according to a linear function.

[0033] To avoid phase accumulation errors caused by gait frequency fluctuations, the system actively recalibrates the gait phase zero point based on real-time foot force data at the end of each gait cycle, i.e., when φ(t) reaches 2π. If the expected foot contact event is not detected at the end of the expected cycle, the system corrects the phase zero point according to the actual detected event time, eliminating phase drift caused by terrain changes or gait adjustments.

[0034] S102: Using the gait phase variable as the independent variable, predict the fuselage attitude vibration data in the future gait cycle through the constructed fuselage attitude vibration prediction model, and generate a feedforward compensation signal that matches the vibration phase based on the fuselage attitude vibration data.

[0035] In this step, the gait phase variable is used as the independent variable, and the fuselage attitude vibration prediction model is constructed to predict the fuselage attitude vibration data in the future gait cycle. Based on the fuselage attitude vibration data, a feedforward compensation signal that matches the vibration phase is generated.

[0036] The feedforward compensation signal, which matches the vibration phase, is not a simple copy of the predicted attitude change. Instead, it is transformed into a command signal that is time-advanced and spatially precisely reversed, directly executable by the gimbal's three-axis controller, through deterministic coordinate transformation and physical mapping under four rigid constraints. This process strictly satisfies: ① Phase zero-point synchronization (the compensation start time corresponds to the minimum phase of vibration); ② Phase period consistency (the compensation signal period equals the gait period); ③ Phase alignment (the phase difference between the compensation signal and the vibration is always π, i.e., completely out of phase); ④ Linear amplitude scaling (the compensation gain is calibrated by the gimbal's mechanical transmission ratio and control accuracy).

[0037] Here, the fuselage attitude vibration prediction model outputs fuselage attitude vibration data within a future time window of Δt (Δt≥1 gait cycle), and calculates and generates a feedforward compensation signal for the gimbal that perfectly matches the vibration phase. The time advance of the compensation signal perfectly matches the gimbal control response delay.

[0038] In one possible implementation, the fuselage attitude vibration prediction model is: ; in, Gait period t Lower body posture vibration data, For gait phase variables It is a periodic function of the independent variable, characterizing the inherent vibrational properties of gait. It is a random perturbation.

[0039] In one possible implementation, the prediction of fuselage attitude vibration data during future gait cycles using the constructed fuselage attitude vibration prediction model includes: a: Perform Fourier series expansion on the fuselage inertial measurement unit data of historical gait cycles, and determine the fundamental frequency based on real-time gait frequency to fit the fuselage attitude change curve of future gait cycles.

[0040] Here, taking advantage of the strong periodicity of gait, Fourier series expansion is performed on IMU data from at least two historical gait cycles, retaining the fundamental frequency and 2nd-3rd harmonics. The fundamental frequency is directly determined by the real-time step frequency, and the attitude change curves for the next 1-2 gait cycles are extrapolated.

[0041] b: A lightweight long short-term memory (LSTM) neural network is used, which takes gait parameters, foot force sensor data and in-body IMU data in the historical gait cycle as input sequence and outputs the predicted value of posture change in the future gait cycle.

[0042] To achieve lightweight deployment, this application designs a compact Long Short-Term Memory (LSTM) neural network structure, balancing prediction accuracy and edge computing efficiency. A typical network structure is as follows: Input Layer: Dimensions are (batch_size, L, input_dim), where input_dim is the dimension of the input feature vector (e.g., step frequency, vertical force of each foot, IMU triaxial acceleration, and angular velocity, totaling approximately 30-50 dimensions). LSTM Layer: 1-2 LSTM layers are used, with 32-128 hidden units per layer. A unidirectional LSTM is chosen instead of a bidirectional one to reduce computational cost and inference latency. The LSTM layer outputs the hidden state at each time step, or retains only the hidden state of the last time step for subsequent decoding. Decoding Layer: Depending on the prediction task, two methods can be used: Direct Output: The hidden state of the last step of the LSTM is mapped to a sequence of pose angles for the next T_out time steps through a fully connected layer. The output dimension is T_out × 3. Autoregressive Output: An additional LSTM decoder is used, starting with the encoder's last state as the initial state, to progressively generate future sequences. This method offers higher accuracy but slightly increases computational cost. Output layer: A linear fully connected layer with no activation function, outputting predicted values ​​for each future time step.

[0043] c: The fuselage attitude change curve of the future gait cycle and the attitude change prediction value of the future gait cycle are weighted and fused to generate the fuselage attitude vibration data.

[0044] Here, the fuselage attitude change curves of future gait cycles and the predicted values ​​of attitude changes of future gait cycles are weighted and fused to generate fuselage attitude vibration data.

[0045] It should be noted that the fuselage attitude change curve of the future gait cycle or the predicted value of the attitude change of the future gait cycle can also be directly used as the fuselage attitude vibration data.

[0046] In this application, the phase zero point is identified in real time based on the gait state machine, and an advanced feedforward compensation signal is generated by combining the body attitude vibration prediction model, so that the gimbal can start the cancellation action before the body vibration occurs, thus solving the fundamental pain point of "blurring when walking" in line inspection.

[0047] S103: Based on the feedforward compensation signal and real-time attitude data, image stabilization control is performed on the gait prediction feedforward outer loop and the gimbal native feedback inner loop.

[0048] In this step, a dual closed-loop control structure is constructed, consisting of a gait prediction feedforward outer loop and a gimbal native feedback inner loop. The input to the gait prediction feedforward outer loop is the feedforward compensation signal, and the input to the gimbal native feedback inner loop is the real-time attitude data output by the gimbal's built-in IMU.

[0049] It should be noted that data consistency is guaranteed: the feedforward compensation signal and the real-time attitude data both originate from the same hardware synchronization system—through the main control IO trigger pulse, ensuring that the timestamp references of the two are consistent, with a clock deviation of ≤10μs.

[0050] In one possible implementation, the image stabilization control based on the feedforward compensation signal and real-time attitude data for the gait prediction feedforward outer loop and the gimbal native feedback inner loop includes: (1): The gait prediction feedforward outer loop receives the feedforward compensation signal and converts the feedforward compensation signal into the compensation angle of each axis of the gimbal, so as to actively cancel the low-frequency large-amplitude periodic vibration caused by gait.

[0051] Here, the gait prediction feedforward closed loop is the core control layer of this application. The input is the feedforward compensation signal, which is converted into the compensation angles of the three axes of the gimbal (Pan / Tilt / Roll) through the coordinate system transformation matrix. This is then input into the gimbal controller in advance to actively cancel the low-frequency large-amplitude periodic vibrations caused by gait.

[0052] (2): The gimbal native feedback inner loop is based on the real-time attitude data of the gimbal's built-in high-frequency inertial measurement unit, and compensates for high-frequency random vibration and residual disturbance through proportional, integral and derivative control.

[0053] Here, the gimbal native feedback closed-loop is based on the real-time attitude data of the gimbal's built-in high-frequency IMU, and compensates for random small disturbances and high-frequency small vibrations through PID control as a supplement to the feedforward closed-loop.

[0054] In this application, the inherent limitations of the single PID feedback closed-loop in the prior art are discarded, and a two-level double-closed-loop collaborative control architecture with functional decoupling, data homology, and strict time synchronization is constructed. Its essence is to physically separate the "vibration cause" (gait periodicity) and the "disturbance source" (random high-frequency noise) at the control level, and achieve collaborative gain through deterministic mathematical relationships. This architecture does not change the gimbal native control hardware, but only forms a new image stabilization paradigm of "feedforward dominant, feedback backup" by embedding a feedforward outer loop in its control link.

[0055] S104: Determine the stable shooting interval, and when the gait phase enters the stable shooting interval, automatically trigger the gimbal to perform an image acquisition operation.

[0056] In this step, determine the stable shooting interval, and when the gait phase enters the stable shooting interval, automatically trigger the gimbal to perform an image acquisition operation.

[0057] In a possible implementation manner, the determining the stable shooting interval and automatically triggering the gimbal to perform an image acquisition operation when the gait phase enters the stable shooting interval includes: i: Based on the body attitude vibration data, determine the phase interval where the body vibration amplitude is lower than the preset threshold as the stable shooting interval.

[0058] Here, based on the extreme value distribution of the body attitude vibration prediction model, determine the phase interval φ∈[φ1, φ2] as the stable shooting interval. The body vibration amplitude within this interval is lower than the preset threshold, and the threshold can be adaptively adjusted according to the gimbal zoom ratio.

[0059] Among them, the stable interval extraction algorithm: 1. Sample A(φ) at a step size of Δφ = 2π / 256 on φ∈[0, 2π] to obtain the discrete sequence {A(φk)}, k = 1, …, 256; 2. Set the dynamic threshold Ath = α·Amax, where Amax = max{A(φk)}, and the coefficient α is adaptively set according to the gimbal zoom magnification M: when M≤10×, α = 0.35 (allowing slightly larger vibrations); when 10× < M≤30×, α = 0.22 (higher stability is required at high magnification); when M>30×, α = 0.15 (extreme accuracy requirements); 3. Extract all continuous phase points that satisfy A(φk)≤Ath and merge them into the largest connected interval [φ1, φ2], which is the stable shooting interval within the current gait cycle; 4. If there are multiple non-connected intervals, select the one with the largest length (length Δφ = φ2 φ1); if Δφ < 0.1π (i.e. < 18°), then it is determined that there is no effective stationary window in this cycle.

[0060] ii: When the gait phase enters the stable shooting range, the gimbal is automatically triggered to perform the shooting operation.

[0061] Here, when the gait phase enters the stable shooting range, a hardware trigger signal is automatically sent to the gimbal, and the entire process of focusing, exposure, and shooting is completed within the phase range with minimal vibration.

[0062] iii: If a stable shooting window does not appear within several consecutive gait cycles, the quadruped robot will be controlled to switch to a stable gait mode and resume its original inspection gait after the photo is taken.

[0063] Here, if there is no stable shooting window that meets the requirements within two consecutive gait cycles, the quadruped robot gait control system will be activated one gait cycle in advance to automatically switch to the stable gait mode; after the photo is taken, the original inspection gait will be automatically restored.

[0064] In this application, a stable shooting window (such as φ∈[0.8π,1.2π]) is identified by phase interval, and the hardware shutter is triggered during the golden period when the body vibration amplitude is less than the threshold. When there is no effective window for two consecutive cycles, the system automatically switches to Crawl gait, truly realizing "continuous high-definition inspection while walking" and adapting to the needs of unattended operation and maintenance.

[0065] In one possible implementation, after the automatically triggered gimbal performs the image acquisition operation, the active image stabilization collaborative control method further includes: I: The sharpness evaluation index of the acquired image is fed back to the fuselage attitude vibration prediction model for online fine-tuning of the gain coefficient and phase offset of the feedforward compensation signal.

[0066] Here, the sharpness evaluation index of the acquired images is fed back to the fuselage attitude vibration prediction model to fine-tune the gain coefficient and phase offset of the feedforward compensation signal online.

[0067] II: Using real-time point cloud data output by a 3D spatial camera, determine the actual fuselage attitude vibration data, and update the parameters of the fuselage attitude vibration prediction model based on the fuselage attitude vibration data and the error between the fuselage attitude vibration data.

[0068] Here, real-time point cloud data output by a 3D spatial camera is used to determine the actual fuselage attitude vibration data, and the parameters of the fuselage attitude vibration prediction model are updated based on the fuselage attitude vibration data and the error between the fuselage attitude vibration data.

[0069] In addition, the actual vibration compensation error can be fed back to the fuselage attitude vibration prediction model in real time, and the harmonic model parameters can be updated online by recursive least squares method to adapt to the vibration characteristics changes under different terrains and different time states.

[0070] For further details, please refer to Figure 2 , Figure 2 This is a schematic diagram of an active image stabilization cooperative control method for a quadruped robot provided in an embodiment of this application. Figure 2 As shown, Step S1: System Initialization and Calibration: Joint calibration of the quadruped robot's body coordinate system, body IMU coordinate system, and gimbal coordinate system is completed, establishing a homogeneous transformation matrix; hardware-level time synchronization of multiple sensors is achieved through hardware I / O triggering; the quadruped robot is controlled to walk in a conventional inspection gait for three cycles, completing the initial calibration of the gait state machine and fitting of the vibration prediction model baseline parameters. Step S2: Real-time Gait Phase Recognition and Synchronization: Real-time gait parameters, foot force sensor data, and body IMU data of the quadruped robot are collected synchronously. Based on the foot force data, foot contact / lift-off events are identified, and the gait state machine and phase variables are updated in real time to complete gait cycle synchronization. Step S3: Proactive Prediction of Body Vibration: Based on the current gait phase and historical cycle data, the three-axis attitude changes of the body within the next 1-2 gait cycles are predicted using analytical modeling or data-driven modeling methods, and a feedforward compensation signal matching the vibration phase is calculated and generated. Step S4: Dual-Loop Active Image Stabilization Control: The feedforward compensation signal is input into the gait prediction feedforward outer loop, while the gimbal's native feedback inner loop executes the native passive feedback closed loop. The dual closed loops work together to control the gimbal's three-axis attitude, proactively offsetting gait periodic vibrations and ensuring stable gimbal shooting posture. Step S5: Linked Photography and Gait Coordination: Based on the gait phase recognition stable shooting window, the gimbal automatically triggers photography within the window; when no effective stable window exists, it switches to a stabilized gait mode to ensure image clarity; after photography is completed, the normal inspection gait mode is restored. Step S6: Online Adaptive Model Optimization: Based on image clarity evaluation results, 3D point cloud attitude correction data, and actual compensation errors, the vibration prediction model parameters are iteratively optimized online to continuously improve the image stabilization effect in different scenarios.

[0071] This application provides an active image stabilization cooperative control method for a quadruped robot. The method includes: determining gait phase variables based on real-time acquisition of gait parameters and foot force data; predicting body posture vibration data within future gait cycles using the gait phase variables as independent variables through a constructed body posture vibration prediction model; generating a feedforward compensation signal matching the vibration phase based on the body posture vibration data; performing image stabilization control on the gait prediction feedforward outer loop and the gimbal native feedback inner loop based on the feedforward compensation signal and real-time posture data; determining a stable shooting range; and automatically triggering the gimbal to perform image acquisition when the gait phase enters the stable shooting range. This constructs a full-link control closed loop of "gait phase variable recognition—vibration prediction—feedforward compensation—dual-loop coordination—phase triggering," achieving a paradigm shift from passive hysteresis stabilization to active feedforward image stabilization. This not only significantly improves image quality but also upgrades the inspection mode from the inefficient "walk-stop-shoot" to intelligent shooting during continuous walking through deep coupling of gait, gimbal, and shooting.

[0072] Please see Figure 3 , Figure 4 , Figure 3 One of the structural schematic diagrams of an active image stabilization cooperative control device for a quadruped robot provided in an embodiment of this application; Figure 4 This is a second schematic diagram of the structure of an active image stabilization and cooperative control device for a quadruped robot provided in an embodiment of this application. Figure 3 As shown, the active image stabilization cooperative control device 300 for the quadruped robot includes: The gait phase recognition module 310 is used to determine the gait phase variables based on the real-time acquisition of gait parameters and foot force data of the quadruped robot. The vibration prediction module 320 is used to predict the fuselage attitude vibration data in the future gait cycle by using the gait phase variable as the independent variable and constructing a fuselage attitude vibration prediction model, and to generate a feedforward compensation signal that matches the vibration phase based on the fuselage attitude vibration data. The dual closed-loop control module 330 is used to perform image stabilization control on the gait prediction feedforward outer loop and the gimbal native feedback inner loop based on the feedforward compensation signal and real-time attitude data. The image acquisition module 340 is used to determine the stable shooting range. When the gait phase enters the stable shooting range, the gimbal is automatically triggered to perform image acquisition operation.

[0073] Furthermore, the determination of gait phase variables based on real-time acquisition of gait parameters and foot force data of the quadruped robot includes: A single gait cycle is divided into a support phase, a swing phase, and a transition phase; Using the diagonal foot contact moment as the gait phase zero point, the gait phase zero point is calibrated in real time based on the foot contact / leave events identified by the foot force data; Based on the calibrated gait phase zero point, a gait phase variable that varies linearly with time is generated in combination with the gait parameters.

[0074] Furthermore, the vibration prediction module 320 is used for the fuselage attitude vibration prediction model as follows: ; in, Gait period t Lower body posture vibration data, For gait phase variables It is a periodic function of the independent variable, characterizing the inherent vibrational properties of gait. It is a random perturbation.

[0075] In one possible implementation, the vibration prediction module 320 is used to predict fuselage attitude vibration data during future gait cycles using the constructed fuselage attitude vibration prediction model. Fourier series expansion was performed on the fuselage inertial measurement unit data of historical gait cycles, and the fundamental frequency was determined based on the real-time gait frequency in order to fit the fuselage attitude change curve of future gait cycles. A lightweight long short-term memory (LSTM) neural network is used, with gait parameters, foot force sensor data and in-body IMU data in the historical gait cycle as input sequence, and output the posture change prediction value of the future gait cycle. The fuselage attitude change curve of the future gait cycle and the predicted attitude change value of the future gait cycle are weighted and fused to generate the fuselage attitude vibration data.

[0076] In one possible implementation, the dual closed-loop control module 330 is used to perform image stabilization control on the gait prediction feedforward outer loop and the gimbal native feedback inner loop based on the feedforward compensation signal and real-time attitude data. The gait prediction feedforward outer loop receives the feedforward compensation signal and converts the feedforward compensation signal into compensation angles for each axis of the gimbal, thereby actively canceling the low-frequency large-amplitude periodic vibrations caused by gait. The native feedback inner loop of the gimbal is based on the real-time attitude data of the built-in high-frequency inertial measurement unit of the gimbal, and compensates for high-frequency random vibrations and residual disturbances through proportional, integral and derivative control.

[0077] In one possible implementation, the image acquisition module 340 is used to determine the stable shooting range, and when the gait phase enters the stable shooting range, it automatically triggers the gimbal to perform an image acquisition operation: Based on the body attitude vibration data, the phase range where the body vibration amplitude is lower than a preset threshold is determined as the stable shooting range; When the gait phase enters the stable shooting range, the gimbal is automatically triggered to perform a shooting operation; If a stable shooting window does not appear within several consecutive gait cycles, the quadruped robot will be controlled to switch to a stable gait mode and resume its original inspection gait after the photo is taken.

[0078] Furthermore, such as Figure 4 As shown, the active image stabilization cooperative control device 300 for the quadruped robot also includes an online optimization module 350, which is used for: The sharpness evaluation index of the acquired images is fed back to the fuselage attitude vibration prediction model for online fine-tuning of the gain coefficient and phase offset of the feedforward compensation signal. Using real-time point cloud data output by a 3D spatial camera, the actual fuselage attitude vibration data is determined, and the parameters of the fuselage attitude vibration prediction model are updated based on the fuselage attitude vibration data and the error between the fuselage attitude vibration data.

[0079] This application provides an active image stabilization and cooperative control device for a quadruped robot. The active image stabilization and cooperative control device includes: a gait phase recognition module, used to determine the gait phase variable based on real-time acquisition of gait parameters and foot force data of the quadruped robot; a vibration prediction module, used to predict the body posture vibration data in the future gait cycle by using the gait phase variable as the independent variable and constructing a body posture vibration prediction model, and generating a feedforward compensation signal matching the vibration phase based on the body posture vibration data; a dual closed-loop control module, used to perform image stabilization control on the gait prediction feedforward outer loop and the gimbal native feedback inner loop based on the feedforward compensation signal and real-time posture data; and an image acquisition module, used to determine a stable shooting range, and automatically trigger the gimbal to perform image acquisition operation when the gait phase enters the stable shooting range. By constructing a full-link control closed loop of "gait phase variable recognition - vibration prediction - feedforward compensation - dual-loop collaboration - phase triggering", a paradigm breakthrough from passive hysteresis stabilization to active feedforward image stabilization has been achieved. This not only significantly improves image quality, but also upgrades the inspection mode from the inefficient "walk-stop-shoot" to intelligent shooting during continuous walking through deep coupling of gait, gimbal and shooting.

[0080] Please see Figure 5 , Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 5 As shown, the electronic device 500 includes a processor 510, a memory 520, and a bus 530.

[0081] The memory 520 stores machine-readable instructions executable by the processor 510. When the electronic device 500 is running, the processor 510 and the memory 520 communicate via the bus 530. When the machine-readable instructions are executed by the processor 510, they can perform the operations described above. Figure 1 as well as Figure 2 The steps of the active image stabilization cooperative control method for the quadruped robot in the method embodiment shown are described in detail in the method embodiment, and will not be repeated here.

[0082] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, can perform the above-described actions. Figure 1 as well as Figure 2 The steps of the active image stabilization cooperative control method for the quadruped robot in the method embodiment shown are described in detail in the method embodiment, and will not be repeated here.

[0083] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0084] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the shown or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0085] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0086] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0087] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a processor-executable, non-volatile, computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0088] Finally, it should be noted that the above-described embodiments are merely specific implementations of this application, used to illustrate the technical solutions of this application, and not to limit them. The scope of protection of this application is not limited thereto. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can still modify or easily conceive of changes to the technical solutions described in the foregoing embodiments, or make equivalent substitutions for some of the technical features, within the scope of the technology disclosed in this application. Such modifications, changes, or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application, and should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An active image stabilization cooperative control method for a quadruped robot, characterized in that, The active image stabilization collaborative control method includes: Based on real-time acquisition of gait parameters and foot force data of the quadruped robot, the gait phase variables are determined. Using the gait phase variable as the independent variable, the fuselage attitude vibration data in the future gait cycle is predicted by the constructed fuselage attitude vibration prediction model, and a feedforward compensation signal matching the vibration phase is generated based on the fuselage attitude vibration data. Based on the aforementioned feedforward compensation signal and real-time attitude data, image stabilization control is performed on the gait prediction feedforward outer loop and the gimbal native feedback inner loop. A stable shooting range is determined, and when the gait phase enters the stable shooting range, the gimbal is automatically triggered to perform image acquisition operation; The fuselage attitude vibration prediction model is as follows: ； in, Gait period t Lower body posture vibration data, For gait phase variables It is a periodic function of the independent variable, characterizing the inherent vibrational properties of gait. It is a random perturbation.

2. The active image stabilization cooperative control method according to claim 1, characterized in that, The determination of gait phase variables based on real-time acquisition of gait parameters and foot force data of the quadruped robot includes: A single gait cycle is divided into a support phase, a swing phase, and a transition phase; Using the diagonal foot contact moment as the gait phase zero point, the gait phase zero point is calibrated in real time based on the foot contact / leave events identified by the foot force data; Based on the calibrated gait phase zero point, a gait phase variable that varies linearly with time is generated in combination with the gait parameters.

3. The active image stabilization cooperative control method according to claim 1, characterized in that, The prediction of fuselage attitude vibration data during future gait cycles using the constructed fuselage attitude vibration prediction model includes: Fourier series expansion was performed on the fuselage inertial measurement unit data of historical gait cycles, and the fundamental frequency was determined based on the real-time gait frequency in order to fit the fuselage attitude change curve of future gait cycles. A lightweight long short-term memory (LSTM) neural network is used, with gait parameters, foot force sensor data and in-body IMU data in the historical gait cycle as input sequence, and output the posture change prediction value of the future gait cycle. The fuselage attitude change curve of the future gait cycle and the predicted attitude change value of the future gait cycle are weighted and fused to generate the fuselage attitude vibration data.

4. The active image stabilization cooperative control method according to claim 1, characterized in that, The image stabilization control based on the feedforward compensation signal and real-time attitude data for the gait prediction feedforward outer loop and the gimbal native feedback inner loop includes: The gait prediction feedforward outer loop receives the feedforward compensation signal and converts the feedforward compensation signal into compensation angles for each axis of the gimbal, thereby actively canceling the low-frequency large-amplitude periodic vibrations caused by gait. The native feedback inner loop of the gimbal is based on the real-time attitude data of the built-in high-frequency inertial measurement unit of the gimbal, and compensates for high-frequency random vibrations and residual disturbances through proportional, integral and derivative control.

5. The active image stabilization cooperative control method according to claim 1, characterized in that, The process involves determining a stable shooting range, and when the gait phase enters this range, automatically triggering the gimbal to perform image acquisition, including: Based on the body attitude vibration data, the phase range where the body vibration amplitude is lower than a preset threshold is determined as the stable shooting range; When the gait phase enters the stable shooting range, the gimbal is automatically triggered to perform a shooting operation; If a stable shooting window does not appear within several consecutive gait cycles, the quadruped robot will be controlled to switch to a stable gait mode and resume its original inspection gait after the photo is taken.

6. The active image stabilization cooperative control method according to claim 1, characterized in that, After the automatic triggering of the gimbal to perform image acquisition, the active image stabilization collaborative control method further includes: The sharpness evaluation index of the acquired images is fed back to the fuselage attitude vibration prediction model for online fine-tuning of the gain coefficient and phase offset of the feedforward compensation signal. Using real-time point cloud data output by a 3D spatial camera, the actual fuselage attitude vibration data is determined, and the parameters of the fuselage attitude vibration prediction model are updated based on the fuselage attitude vibration data and the error between the fuselage attitude vibration data.

7. An active image stabilization cooperative control device for a quadruped robot, characterized in that, The active image stabilization and collaborative control device includes: The gait phase recognition module is used to determine the gait phase variables based on the real-time acquisition of gait parameters and foot force data of the quadruped robot. The vibration prediction module is used to predict the fuselage attitude vibration data in the future gait cycle by using the gait phase variable as the independent variable and constructing a fuselage attitude vibration prediction model, and to generate a feedforward compensation signal that matches the vibration phase based on the fuselage attitude vibration data. The dual closed-loop control module is used to perform image stabilization control on the gait prediction feedforward outer loop and the gimbal native feedback inner loop based on the feedforward compensation signal and real-time attitude data. The image acquisition module is used to determine the stable shooting range. When the gait phase enters the stable shooting range, the gimbal is automatically triggered to perform image acquisition operation. The fuselage attitude vibration prediction model is as follows: ； in, Gait period t Lower body posture vibration data, For gait phase variables It is a periodic function of the independent variable, characterizing the inherent vibrational properties of gait. It is a random perturbation.

8. An electronic device, characterized in that, include: The device includes a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. When the electronic device is running, the processor communicates with the memory via the bus. The machine-readable instructions are executed by the processor to perform the steps of the active image stabilization cooperative control method for a quadruped robot as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, performs the steps of the active image stabilization cooperative control method for a quadruped robot as described in any one of claims 1 to 6.

Images