A complex terrain-oriented four-legged robot eye-leg cooperative obstacle crossing method and system

Abstract

The application belongs to the technical field of robot motion control, and provides a four-legged robot eye-leg cooperative obstacle crossing method and system for complex terrains, which comprises the following steps: acquiring an original depth image of a four-legged robot; repairing the original depth image, performing smoothing processing on the repaired original depth image by combining an adaptive smoothing model based on gradient constraints, and obtaining a depth information matrix of the four-legged robot; performing image cropping and standardization processing based on the depth information matrix, and obtaining visual depth information of the four-legged robot; constructing a robot body state vector according to the current motion state of the four-legged robot; performing mapping of the visual depth information of the four-legged robot and the robot body state vector in a low-dimensional feature space, generating a terrain latent feature vector; fusing the terrain latent feature vector and the robot body state vector, calculating the joint torque of the four-legged robot, generating a joint motion instruction, and completing the four-legged robot eye-leg cooperative obstacle crossing for complex terrains.

Description

Technical Field

[0001] This invention belongs to the field of robot motion control technology, specifically relating to a method and system for quadruped robots to overcome obstacles in complex terrain using eye-leg coordination. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] As quadruped robots are increasingly used in disaster relief, field exploration, military reconnaissance, and industrial inspection, their mobility in complex environments is becoming increasingly important. In challenging scenarios such as post-disaster ruins and mountainous terrain, the terrain is often rugged and full of unknown obstacles, requiring robots to efficiently meet mission requirements and possess high mobility and robust obstacle-crossing capabilities. Therefore, robots need to have efficient "eye-leg coordination" capabilities, that is, to achieve real-time coordination between high-precision environmental perception and high-dynamic motion control.

[0004] In recent years, significant progress has been made in the control of quadruped robots based on reinforcement learning. However, existing strategies mostly rely on proprioception, which can easily lead to missteps or instability in complex terrain due to a lack of environmental prediction. Although some studies have introduced visual sensors to enhance perception capabilities, in real-world environments, changes in lighting, occlusion, and material differences often cause noise enhancement or local missing data in depth data. This makes it difficult to reliably transfer simulation models to real-world environments, seriously affecting the robot's actual obstacle-crossing ability. Summary of the Invention

[0005] To address the aforementioned issues, this invention proposes a method and system for quadruped robots to overcome obstacles in complex terrain through eye-leg coordination. By constructing a deep fusion mechanism of depth visual perception and proprioception, the depth visual perception information is reconstructed and optimized, effectively suppressing problems such as depth information noise and local loss caused by environmental changes, thereby significantly improving the robot's obstacle-crossing stability under complex terrain and perceptual interference conditions.

[0006] According to some embodiments, the first aspect of the present invention provides a quadruped robot eye-leg coordinated obstacle crossing method for complex terrain, employing the following technical solution:

[0007] A method for quadruped robots to overcome obstacles in complex terrain using eye-leg coordination includes:

[0008] Obtain the raw depth image of the quadruped robot;

[0009] The original depth image is repaired, and the repaired original depth image is smoothed by combining an adaptive smoothing model based on gradient constraints to obtain the depth information matrix of the quadruped robot.

[0010] Based on the obtained depth information matrix, image cropping and standardization are performed to obtain the visual depth information of the quadruped robot;

[0011] Construct the robot's body state vector based on the current motion state of the quadruped robot;

[0012] The obtained visual depth information of the quadruped robot and the robot's body state vector are mapped into a low-dimensional feature space to generate a terrain potential feature vector.

[0013] By fusing the terrain potential feature vector and the robot's body state vector, a comprehensive input state vector for the quadruped robot is constructed. The joint torque of the quadruped robot is calculated, and joint motion commands for the quadruped robot are generated, enabling the quadruped robot to overcome obstacles in complex terrain with coordinated eye and leg movements.

[0014] As a further technical limitation, the gradient-constrained adaptive smoothing model adopts an adaptive edge adjustment function. The adaptive edge adjustment function monitors the local geometric changes of each pixel in the repaired original depth image in real time, thereby realizing differentiated processing of smooth areas and edge areas in the image.

[0015] As a further technical limitation, the obtained depth information matrix of the quadruped robot for

[0016] ;

[0017] in, This is the original depth value of the current pixel. For the set of neighboring pixels, The fill depth value of the neighboring pixels. The spatial distance between pixels. The Gaussian space decay coefficient, These are the normalized weighting coefficients.

[0018] As a further technical limitation, an airborne inertial measurement unit and joint encoder are used to acquire the current motion state information of the quadruped robot and construct the robot's body state vector. ,Right now

[0019] ;

[0020] in, The three-axis angular velocities representing the robot's base. This represents the two-dimensional fuselage attitude, which is composed of roll and pitch angles. and These refer to the current heading deviation and the predicted heading deviation for the next moment, respectively. Indicates three-dimensional velocity commands; This represents the deviation of each joint's real-time position from the default reference position. Represents the real-time angular velocity vector of the joint. This is the strategy output of the previous control cycle.

[0021] As a further technical limitation, the calculation of the joint torque of the quadruped robot incorporates the latent feature vector of the terrain. With the robot's body state vector Construct the integrated input state vector of the quadruped robot. ,Right now Based on the obtained integrated input state vector Perform real-time reasoning to calculate the quadruped robot's movements. Based on the quadruped robot's motion and the proportional-differential controller, the joint torque of the quadruped robot is calculated.

[0022] Furthermore, based on the obtained movements of the quadruped robot and preset motion adjustment factors The actual angle increment of the quadruped robot is obtained. ,Right now The actual angle increment obtained Superimposed on the default reference pose of the quadruped robot The final joint position tracking target is constructed; the required driving torque for each joint is calculated using a PD controller. ,Right now ;in, This is the joint stiffness coefficient. This is the joint damping coefficient. For real-time joint angles, This refers to the real-time angular velocity; the resulting torque command. This refers to the joint torque of a quadruped robot.

[0023] According to some embodiments, a second aspect of the present invention provides a quadruped robot eye-leg coordinated obstacle crossing system for complex terrain, employing the following technical solution:

[0024] A quadruped robot eye-leg coordinated obstacle crossing system for complex terrain includes:

[0025] The acquisition module is configured to acquire the raw depth image of the quadruped robot;

[0026] The processing module is configured to repair the acquired original depth image, and then smooth the repaired original depth image using an adaptive smoothing model based on gradient constraints to obtain the depth information matrix of the quadruped robot. Based on the obtained depth information matrix, image cropping and standardization are performed to obtain the visual depth information of the quadruped robot.

[0027] The generation module is configured to construct a robot body state vector based on the current motion state of the quadruped robot; and to map the obtained quadruped robot visual depth information and robot body state vector into a low-dimensional feature space to generate a terrain potential feature vector.

[0028] The obstacle crossing module is configured to fuse the terrain potential feature vector and the robot's body state vector to construct the quadruped robot's comprehensive input state vector, calculate the quadruped robot's joint torque, generate the quadruped robot's joint motion commands, and complete the quadruped robot's eye-leg coordinated obstacle crossing in complex terrain.

[0029] According to some embodiments, a third aspect of the present invention provides a computer-readable storage medium, employing the following technical solution:

[0030] A computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of the eye-leg cooperative obstacle-crossing method for quadruped robots oriented towards complex terrain as described in the first aspect of the present invention.

[0031] According to some embodiments, the fourth aspect of the present invention provides an electronic device, which adopts the following technical solution:

[0032] An electronic device includes a memory, a processor, and a program stored in the memory and running on the processor, wherein the processor executes the program to implement the steps in the eye-leg cooperative obstacle crossing method for quadruped robots facing complex terrain as described in the first aspect of the present invention.

[0033] According to some embodiments, the fifth aspect of the present invention provides a computer program product, which adopts the following technical solution:

[0034] A computer program product includes software code, wherein the program in the software code performs the steps of the eye-leg cooperative obstacle crossing method for quadruped robots facing complex terrain as described in the first aspect of the present invention.

[0035] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0036] In terms of perception, this invention adopts an adaptive smoothing and minimum deviation repair mechanism based on gradient constraints, which solves the problem of "noise reduction and fidelity preservation being mutually exclusive" in complex environments, provides high-precision environmental spatial data for subsequent motion decisions, and ensures the robot's accurate perception of terrain changes.

[0037] In terms of control, this invention constructs a motion control architecture based on deep fusion of multi-source perception and "eye-leg coordination". It adopts a hierarchical reinforcement learning strategy to build a deep fusion of visual latent features and ontological state information. It solves the problem of information separation between the perception layer and the control layer in traditional hierarchical control, enabling the robot to "understand" the dynamic relationship between visual information and body state. It breaks the limitation of response lag in traditional "blind" control that relies solely on ontological state feedback, and realizes true eye-leg coordination. Attached Figure Description

[0038] The accompanying drawings, which form part of this embodiment, are used to provide a further understanding of this embodiment. The illustrative embodiments and their descriptions are used to explain this embodiment and do not constitute an improper limitation of this embodiment.

[0039] Figure 1 This is a flowchart of the quadruped robot's eye-leg coordinated obstacle crossing method for complex terrain in Embodiment 1 of the present invention;

[0040] Figure 2 This is an architectural diagram of the quadruped robot's eye-leg coordinated obstacle-crossing method for complex terrain in Embodiment 1 of the present invention;

[0041] Figure 3 This is a schematic diagram of the quadruped robot's eye-leg coordinated obstacle-crossing method for complex terrain in Embodiment 1 of the present invention;

[0042] Figure 4 This is a flowchart of the reconstruction and optimization based on depth visual information in Embodiment 1 of the present invention;

[0043] Figure 5 This is a schematic diagram of the hierarchical reinforcement learning controller in Embodiment 1 of the present invention;

[0044] Figure 6 This is a schematic diagram illustrating the principle of the hierarchical reinforcement learning controller architecture in Embodiment 1 of the present invention;

[0045] Figure 7 This is a schematic diagram of the robot in Embodiment 1 of the present invention;

[0046] Figure 8 This is a schematic diagram of the obstacle-crossing effect in Embodiment 1 of the present invention;

[0047] Figure 9 This is a structural block diagram of the quadruped robot eye-leg coordinated obstacle-crossing system for complex terrain in Embodiment 2 of the present invention;

[0048] The components include: 1. Depth camera; 2. GPU computing unit; 3. Underlying development board. Detailed Implementation

[0049] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0050] It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0051] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0052] In this invention, terms such as "upper," "lower," "left," "right," "front," "back," "vertical," "horizontal," "side," and "bottom" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are used only to facilitate the description of the structural relationships of the various components or elements of this invention and do not specifically refer to any component or element in this invention. They should not be construed as limiting the invention.

[0053] In this invention, terms such as "fixed connection," "connected," and "linked" should be interpreted broadly, indicating a fixed connection, an integral connection, or a detachable connection; a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can determine the specific meaning of these terms in this invention based on the specific circumstances, and they should not be construed as limitations on the invention.

[0054] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0055] Example 1

[0056] Embodiment 1 of this invention introduces a method for quadruped robots to overcome obstacles in complex terrain using eye-leg coordination.

[0057] like Figure 1 The method shown is a quadruped robot eye-leg cooperative obstacle crossing method for complex terrain, including:

[0058] Obtain the raw depth image of the quadruped robot;

[0059] The original depth image is repaired, and the repaired original depth image is smoothed by combining an adaptive smoothing model based on gradient constraints to obtain the depth information matrix of the quadruped robot.

[0060] Based on the obtained depth information matrix, image cropping and standardization are performed to obtain the visual depth information of the quadruped robot;

[0061] Construct the robot's body state vector based on the current motion state of the quadruped robot;

[0062] The obtained visual depth information of the quadruped robot and the robot's body state vector are mapped into a low-dimensional feature space to generate a terrain potential feature vector.

[0063] By fusing the terrain potential feature vector and the robot's body state vector, a comprehensive input state vector for the quadruped robot is constructed. The joint torque of the quadruped robot is calculated, and joint motion commands for the quadruped robot are generated, enabling the quadruped robot to overcome obstacles in complex terrain with coordinated eye and leg movements.

[0064] Due to the highly complex and dynamically uncertain characteristics of the outdoor environment, such as rugged terrain and drastic changes in lighting, the perception system of quadruped robots is easily affected by noise interference during actual operation, which can affect the final motion stability. Therefore, improving the robot's environmental perception capability in such extreme scenarios is crucial for enhancing its obstacle-crossing ability. To this end, this embodiment proposes an eye-leg cooperative obstacle-crossing method for quadruped robots in complex terrain. This method can reconstruct environmental features in real time and generate precise control commands under strong interference conditions, thereby achieving highly robust obstacle crossing.

[0065] like Figure 2 and Figure 3 As shown, the quadruped robot's eye-leg coordinated obstacle-crossing method for complex terrain in this embodiment includes the following steps:

[0066] The original depth images of the quadruped robot are acquired in real time using an airborne depth camera.

[0067] The original depth image is repaired for invalid pixels, denoised, and its feature space is reconstructed to obtain visual depth information;

[0068] By combining the visual depth information of a quadruped robot, a multi-source perception fusion control architecture based on hierarchical reinforcement learning is constructed. This architecture deeply integrates environmental visual features with the robot's own state to generate joint motion control commands.

[0069] As one or more implementation methods, when acquiring the original depth information of the quadruped robot, such as Figure 7 As shown, a depth camera 1 is installed on the head of the quadruped robot; in order to cover the key obstacle crossing area from 0.1 meters to 2.0 meters in front of the robot and also take into account the nearby blind spot, the installation tilt angle of the camera is precisely set to 30 degrees downward.

[0070] Calibration is performed using a camera calibration tool to obtain the rotation matrix R and the offset T, i.e. To address the nonlinear measurement errors of depth cameras at different distances, a known-distance-plane method is used for accuracy calibration. The specific steps are as follows:

[0071] A standard plane (such as a high-precision checkerboard calibration plate) was selected, and multiple sets of samples were taken within a range of 0.5m to 4.0m using the standard checkerboard calibration plate. The measured depth values ​​of the camera were recorded. The actual depth value measured by a physical ruler The second-order polynomial error model is fitted using the least squares method:

[0072] ;

[0073] in, , , These are the fitting coefficients. During actual system operation, this model is used to correct the pixel values ​​of each frame of the original depth map in real time, resulting in the corrected depth values. ,Right now

[0074] ;

[0075] This step effectively reduces geometric errors caused by sensor hardware.

[0076] The depth image stream of the terrain ahead is acquired at a resolution of 640×480 and a frequency of 30 frames per second (FPS), and transmitted in real time to the GPU computing unit of the upper-layer perception network.

[0077] It should be noted that, as Figure 7 As shown, in this embodiment, the GPU computing unit 2 is located above the quadruped robot, and the underlying development board 3 is installed inside the quadruped robot's torso.

[0078] As one or more implementation methods, to address the image information loss and noise issues caused by "drastic changes in illumination" (such as entering and exiting forest shadows) and differences in surface material (such as light absorption by black rocks) in unstructured outdoor environments, the following processing is performed based on the original depth flow:

[0079] like Figure 4 As shown, the specific steps for depth visual information reconstruction and optimization include:

[0080] Invalid pixel repair based on local windows (in the depth image, there will be certain invalid pixels due to strong light overexposure, light absorption by black materials, or parallax blind spots); specifically, when dealing with depth loss caused by black light-absorbing rocks (basalt material), this embodiment fixes the size of the local search neighborhood to 3×3 pixels.

[0081] In this embodiment, when coordinates are detected When the depth value is invalid, the system does not simply take the neighborhood mean (which would blur the texture), but instead calculates the mean of all valid pixels in the neighborhood. Then search for a neighboring region that is similar to... The real pixel value with the smallest difference is used as the fill value. After this step, the depth faults caused by natural environmental interference are effectively repaired, preventing the robot from misjudging passable flat ground as a cliff or a deep pit.

[0082] In this embodiment, the depth image matrix is ​​defined as follows: Invalid pixels with a depth value of 0 were detected in the matrix. Construct a local search neighborhood centered on this point. To ensure coverage of effective depth information, extract the effective depth set from the neighborhood. Calculate the local mean of the neighborhood :

[0083] ;

[0084] in, This represents the depth value of the effective pixels within the neighborhood. This indicates the number of valid pixels in the neighborhood.

[0085] To avoid texture blurring caused by directly using mean fill, the repaired depth value... Select set The optimal observation is the one with the smallest deviation from the local mean, in order to maintain the consistency of local geometric features. The optimization criterion is as follows:

[0086] ;

[0087] This embodiment effectively fills in the depth information gaps caused by environmental interference, generating spatially continuous depth images. That is, the restored original depth image.

[0088] To address the issue that traditional Gaussian or median filtering, while suppressing noise, can easily blur obstacle boundaries (such as step edges and rock contours), leading to collision misjudgments by the robot, this embodiment introduces an adaptive smoothing model based on gradient constraints to smooth the restored original depth image. The core of this model lies in constructing an adaptive edge adjustment function. It monitors the local geometric changes of each pixel in the depth image in real time, thereby enabling differentiated processing of smooth and edge regions.

[0089] This embodiment distinguishes between regions with gradual depth changes (such as smooth surfaces) and regions with dramatic depth changes (such as object edges), thereby suppressing noise while preserving key edge information. The function is defined as follows:

[0090] ;

[0091] in, The preset depth gradient threshold, Represents the gradient of a depth image. This represents the gradient magnitude, approximated using the Manhattan distance.

[0092] Therefore, pixel Final depth output at the location It is not a simple neighborhood average, but is determined by the original depth value of the current pixel and the weighted contributions of its neighboring pixels. Its mathematical expression is as follows:

[0093] ;

[0094] in, For depth information matrix, This is the original depth value of the current pixel. For the set of neighboring pixels, The fill depth value of the neighboring pixels. The spatial distance between pixels. The Gaussian space decay coefficient, These are the normalized weighting coefficients.

[0095] This embodiment will determine the depth gradient threshold of the edge ( The threshold is precisely set to 50mm. This threshold is based on the fact that only steps or rock edges with a height exceeding 50mm will substantially hinder the robot's movement.

[0096] In this embodiment, pixel points are calculated in real time. The local gradient is determined by the adaptive function when a local depth change of less than 50 mm is detected (e.g., on a rough concrete surface or a small fluctuation caused by light and shadow). Outputting a logic value of 1 activates the Gaussian spatial smoothing mechanism, where the Gaussian kernel size is set to 5×5 to effectively smooth high-frequency noise; conversely, when a depth jump exceeding 50mm is detected (i.e., a real physical abrupt change is encountered), the function... Outputting a logic value of 0 forcibly blocks the weighted contribution of neighboring pixels, directly outputting the original depth value. This achieves absolute fidelity of the physical contour while filtering out spot noise.

[0097] To eliminate redundant information that is not only useless but may also interfere with the control strategy, this embodiment performs spatial cropping, invalid data filtering, and dimensional compression mapping on the smoothed depth image based on the camera's field of view characteristics and the robot's physical motion space. Because depth cameras exhibit significant distortion and a severe decrease in depth accuracy at the lens edges, the processed depth information matrix... Cut from the top, bottom, left, and right sides respectively. Pixels, generating a depth information matrix that retains only the central region of the image. .

[0098] To accommodate the low-dimensional input tensors required by the control network and reduce subsequent computational load, the cropped high-resolution image is compressed to the target dimension (e.g., 60×80). Region mean pooling is employed. The target output image is defined. coordinates in Corresponding cropped image A local window is defined, and the average value of the pixels within that window is calculated as the output. :

[0099] ;

[0100] in, , These represent the height and width of the pooling window, respectively. Indicates the offset in the height direction (row). Indicates the offset in the width direction (column).

[0101] This embodiment reduces dimensionality while simultaneously performing a second low-pass filter on the image, effectively smoothing out minor noise.

[0102] This embodiment sets the effective detection range based on the robot's maximum single-step traverse distance and minimum perception blind spot. For the cropped depth information matrix Threshold truncation is performed to obtain the regional interest depth matrix. :

[0103] ;

[0104] Therefore, this embodiment constructs an effective physical space that only contains terrain information of the area accessible to the robot in front, and eliminates interference from distant background light spots.

[0105] To ensure the processed perception data is adaptable to reinforcement learning-based policy networks and to guarantee robustness in virtual-to-real transfer, this embodiment requires standardization of the data distribution; the physical depth values ​​in meters are linearly mapped to the unit interval [0,1], and a zero-centered translation operation is performed, ultimately generating data distributed in […]. The standard feature tensor [0.5, 0.5] is used; this step ensures that the input data has a consistent and stable distribution, thereby significantly improving the inference stability of the policy network; specifically:

[0106] Mapping the truncated physical depth values ​​to a unit interval yields the normalized depth information matrix. ,Right now:

[0107] ;

[0108] Subsequently, a zero-centralized translation is performed to generate visual depth information. ,Right now

[0109] ;

[0110] As one or more implementation methods, this embodiment constructs as follows: Figure 5 The multi-source perception fusion control architecture based on hierarchical reinforcement learning, as shown, deeply integrates environmental visual features with the robot's own state to generate joint motion control commands; the specific process is as follows: Figure 6 As shown:

[0111] The robot's current motion state information is acquired using an onboard inertial measurement unit and joint encoders. A body state vector is then constructed. ,Right now

[0112] ;

[0113] in, The three-axis angular velocities representing the robot's base. This represents the two-dimensional fuselage attitude, which is composed of roll and pitch angles. and These refer to the current heading deviation and the predicted heading deviation for the next moment, respectively. Indicates three-dimensional velocity commands; This represents the deviation of each joint's real-time position from the default reference position. Represents the real-time angular velocity vector of the joint. This is the strategy output of the previous control cycle.

[0114] Construct a multi-source perception fusion module deployed on the GPU computing unit to perform perception fusion and perception strategies. Real-time inference. During robot operation, two data streams are received in real time: based on the processed visual depth information... and the constructed robot body state vector Simultaneously, inference is performed on the pre-trained perception strategy, mapping visual depth information and aircraft motion state to a low-dimensional feature space to generate terrain latent feature vectors. ,Right now

[0115] ;

[0116] It should be noted that the multi-source perception fusion module in this embodiment simultaneously receives a 30Hz depth image and a 50Hz ontology state information. The ontology perception information is received from the underlying control layer via UDP. The latent features are output at a frequency of 10Hz through the perception policy network, which means that the inference frequency of the perception policy is 10Hz.

[0117] This embodiment is based on an inference-based motion control strategy. The control layer receives latent features generated by the perception layer via UDP communication and fuses them with ontological perception information to construct a comprehensive input vector. Based on this input data, the motion control strategy performs real-time inference at a frequency of 50Hz, calculates and outputs joint control commands adapted to the current fuselage state.

[0118] This embodiment will generate the terrain potential feature vector. With the robot's body state vector By fusing the data, a comprehensive input state vector is constructed. As input to the motion control program strategy, i.e.

[0119] ;

[0120] This embodiment will Input to pre-trained motion control strategy Real-time reasoning is performed. The network calculates the appropriate actions based on the robot's current state and potential features. ,Right now:

[0121] ;

[0122] This implementation considers using a proportional-derivative (PD) controller to calculate the joint torque; and transforms the abstract output value into a concrete physical control signal.

[0123] First, output the action Multiplied by a preset motion adjustment factor Map it to the actual angle increment ,Right now

[0124] ;

[0125] The adjustment factor in this embodiment Used to constrain the output boundaries of the policy network, ensuring that the output actions are within the physical safety range of the mechanical structure.

[0126] Subsequently, this embodiment will increment the actual angle. Superimposed on the robot's default reference pose The final joint position tracking target is constructed; a PD controller is used to calculate the required driving torque for each joint. ,Right now

[0127] ;

[0128] in, This is the joint stiffness coefficient. This is the joint damping coefficient. For real-time joint angles, For real-time angular velocity; the calculated torque command Finally, the drive motor performs the action.

[0129] This embodiment introduces an adaptive edge adjustment function in the perception layer. As a logic switch, and in conjunction with a neighborhood filling strategy based on minimizing local mean deviation, a dynamic dual-mode processing logic is established by monitoring the local gradient changes of each pixel in the depth image in real time: in flat areas where the gradient is less than a preset threshold (e.g., 50mm), Gaussian spatial smoothing is activated to filter out high-frequency noise caused by light and shadow flicker; while in physical edge areas with drastic gradient changes (e.g., the corners of steps), the smooth collinearity of neighboring pixels is forcibly blocked, directly locking the original geometric features. Simultaneously, the principle of minimum deviation is used to repair depth holes caused by black light-absorbing materials, avoiding texture blurring caused by traditional interpolation. This design fundamentally solves the technical drawback of traditional global filtering algorithms that easily misprocess "vertical steps" as "gentle slopes," significantly improving the robot's perception robustness under extreme conditions such as direct sunlight, alternating shadows, and low-reflectivity materials.

[0130] This embodiment constructs a motion control architecture based on multi-source perception deep fusion, namely "eye-leg coordination," and the robot's motion pattern is as follows: Figure 8As shown, the optimized depth image is first mapped into low-dimensional latent features using a perception strategy. Then, these features are concatenated with the body state vector to form a comprehensive input tensor containing both environmental geometry and body state information. The motion strategy network then directly outputs the target position commands for each joint. This fundamentally changes the motion interaction logic of the quadruped robot, establishing a composite control loop of "visual feedforward + body feedback." The visual flow acts as the feedforward link, informing the motion control strategy network in advance of the geometric distribution of the terrain ahead (such as step height and ditch width) through latent features. This allows the robot to actively adjust the trajectory and landing point of its swinging leg several control cycles before contacting an obstacle. The body flow acts as the feedback link, monitoring the body's balance and contact force feedback in real time. This solves the problem of information separation between the perception and control layers in traditional hierarchical control, enabling the robot to "understand" the dynamic relationship between visual information and body state, such as pre-lifting its leg to overcome an obstacle upon detection. This technical solution breaks through the response lag limitations of traditional "blind" control that relies solely on body state feedback, achieving true eye-leg coordination.

[0131] Example 2

[0132] Embodiment 2 of the present invention introduces a quadruped robot eye-leg coordinated obstacle crossing system for complex terrain.

[0133] like Figure 9 The illustrated quadruped robot eye-leg coordinated obstacle crossing system for complex terrain includes:

[0134] The acquisition module is configured to acquire the raw depth image of the quadruped robot;

[0135] The processing module is configured to repair the acquired original depth image, and then smooth the repaired original depth image using an adaptive smoothing model based on gradient constraints to obtain the depth information matrix of the quadruped robot. Based on the obtained depth information matrix, image cropping and standardization are performed to obtain the visual depth information of the quadruped robot.

[0136] The generation module is configured to construct a robot body state vector based on the current motion state of the quadruped robot; and to map the obtained quadruped robot visual depth information and robot body state vector into a low-dimensional feature space to generate a terrain potential feature vector.

[0137] The obstacle crossing module is configured to fuse the terrain potential feature vector and the robot's body state vector to construct the quadruped robot's comprehensive input state vector, calculate the quadruped robot's joint torque, generate the quadruped robot's joint motion commands, and complete the quadruped robot's eye-leg coordinated obstacle crossing in complex terrain.

[0138] The detailed steps are the same as those of the quadruped robot eye-leg cooperative obstacle crossing method for complex terrain provided in Example 1, and will not be repeated here.

[0139] Example 3

[0140] Embodiment 3 of the present invention provides a computer-readable storage medium.

[0141] A computer-readable storage medium having a program stored thereon, which, when executed by a processor, implements the steps of the eye-leg cooperative obstacle-crossing method for quadruped robots oriented towards complex terrain as described in Embodiment 1 of the present invention.

[0142] The detailed steps are the same as those of the quadruped robot eye-leg cooperative obstacle crossing method for complex terrain provided in Example 1, and will not be repeated here.

[0143] Example 4

[0144] Embodiment 4 of the present invention provides an electronic device.

[0145] An electronic device includes a memory, a processor, and a program stored in the memory and running on the processor. When the processor executes the program, it implements the steps in the quadruped robot eye-leg cooperative obstacle crossing method for complex terrain as described in Embodiment 1 of the present invention.

[0146] The detailed steps are the same as those of the quadruped robot eye-leg cooperative obstacle crossing method for complex terrain provided in Example 1, and will not be repeated here.

[0147] Example 5

[0148] Embodiment 5 of the present invention provides a computer program product.

[0149] A computer program product includes software code, wherein the program in the software code performs the steps of the eye-leg coordinated obstacle crossing method for quadruped robots facing complex terrain as described in Embodiment 1 of the present invention.

[0150] The detailed steps are the same as those of the quadruped robot eye-leg cooperative obstacle crossing method for complex terrain provided in Example 1, and will not be repeated here.

[0151] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0152] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0153] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0154] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0155] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0156] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

[0157] The above description is merely a preferred embodiment of this practice and is not intended to limit the scope of this practice. Various modifications and variations can be made to this practice by those skilled in the art. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this practice should be included within the protection scope of this practice.

Claims

1. A method for quadruped robots to overcome obstacles in complex terrain using eye-leg coordination, characterized in that, include: Obtain the raw depth image of the quadruped robot; The original depth image is repaired, and the repaired original depth image is smoothed by combining an adaptive smoothing model based on gradient constraints to obtain the depth information matrix of the quadruped robot. Based on the obtained depth information matrix, image cropping and standardization are performed to obtain the visual depth information of the quadruped robot; Construct the robot's body state vector based on the current motion state of the quadruped robot; The obtained visual depth information of the quadruped robot and the robot's body state vector are mapped into a low-dimensional feature space to generate a terrain potential feature vector. By integrating the terrain potential feature vector and the robot's body state vector, a comprehensive input state vector of the quadruped robot is constructed, the joint torque of the quadruped robot is calculated, and the joint motion commands of the quadruped robot are generated, thus enabling the quadruped robot to overcome obstacles in complex terrain with eye-leg coordination. The gradient-constrained adaptive smoothing model employs an adaptive edge adjustment function, which monitors the local geometric changes of each pixel in the repaired original depth image in real time, thereby achieving differentiated processing between smooth and edge regions in the image. The obtained depth information matrix of the quadruped robot for ； in, This is the original depth value of the current pixel. For the set of neighboring pixels, The fill depth value of the neighboring pixels. The spatial distance between pixels. The Gaussian space decay coefficient, These are the normalized weighting coefficients; The adjustment function of the adaptive edge for ;in, The preset depth gradient threshold, Represents the gradient of a depth image. The gradient magnitude is represented by an approximation using the Manhattan distance; when When, it is in the smooth region; when At that time, it is in the edge region.

2. The method for quadruped robot eye-leg coordinated obstacle crossing in complex terrain as described in claim 1, characterized in that, An airborne inertial measurement unit and joint encoder are used to acquire the current motion state information of a quadruped robot, and a robot body state vector is constructed. ,Right now ； in, The three-axis angular velocities representing the robot's base. This represents the two-dimensional fuselage attitude, which is composed of the roll angle and the pitch angle. and These refer to the current heading deviation and the predicted heading deviation for the next moment, respectively. Indicates three-dimensional velocity commands; This represents the deviation of each joint's real-time position from the default reference position. Represents the real-time angular velocity vector of the joint. This is the strategy output of the previous control cycle.

3. The method for quadruped robot eye-leg coordinated obstacle crossing in complex terrain as described in claim 1, characterized in that, In calculating the joint torque of a quadruped robot, the latent feature vector of the terrain is fused. With the robot's body state vector Construct the integrated input state vector of the quadruped robot. ,Right now Based on the obtained integrated input state vector Perform real-time reasoning to calculate the quadruped robot's movements. Based on the quadruped robot's motion and the proportional-differential controller, the joint torque of the quadruped robot is calculated.

4. The method for quadruped robot eye-leg coordinated obstacle crossing in complex terrain as described in claim 3, characterized in that, Based on the obtained movements of the quadruped robot and preset motion adjustment factors The actual angle increment of the quadruped robot is obtained. ,Right now ; The actual angle increment obtained Superimposed on the default reference pose of the quadruped robot The final joint position tracking target is constructed; the required driving torque for each joint is calculated using a PD controller. ,Right now ;in, This is the joint stiffness coefficient. This is the joint damping coefficient. For real-time joint angles, This refers to the real-time angular velocity; the resulting torque command. This refers to the joint torque of a quadruped robot.

5. A quadruped robot eye-leg coordinated obstacle-crossing system for complex terrain, characterized in that, include: The acquisition module is configured to acquire the raw depth image of the quadruped robot; The processing module is configured to repair the acquired original depth image, and then smooth the repaired original depth image using an adaptive smoothing model based on gradient constraints to obtain the depth information matrix of the quadruped robot. Based on the obtained depth information matrix, image cropping and standardization are performed to obtain the visual depth information of the quadruped robot. The generation module is configured to construct a robot body state vector based on the current motion state of the quadruped robot; and to map the obtained quadruped robot visual depth information and robot body state vector into a low-dimensional feature space to generate a terrain potential feature vector. The obstacle crossing module is configured to fuse the terrain potential feature vector and the robot's body state vector to construct the quadruped robot's comprehensive input state vector, calculate the quadruped robot's joint torque, generate the quadruped robot's joint motion commands, and complete the quadruped robot's eye-leg coordinated obstacle crossing in complex terrain. The gradient-constrained adaptive smoothing model employs an adaptive edge adjustment function, which monitors the local geometric changes of each pixel in the repaired original depth image in real time, thereby achieving differentiated processing between smooth and edge regions in the image. The obtained depth information matrix of the quadruped robot for ； in, This is the original depth value of the current pixel. For the set of neighboring pixels, The fill depth value of the neighboring pixels. The spatial distance between pixels. The Gaussian space decay coefficient, These are the normalized weighting coefficients; The adjustment function of the adaptive edge for ;in, The preset depth gradient threshold, Represents the gradient of a depth image. The gradient magnitude is represented by an approximation using the Manhattan distance; when When, it is in the smooth region; when At that time, it is in the edge region.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the quadruped robot eye-leg cooperative obstacle crossing method for complex terrain as described in any one of claims 1-4.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the program, it implements the steps of the quadruped robot eye-leg cooperative obstacle crossing method for complex terrain as described in any one of claims 1-4.

8. A computer program product, comprising software code, characterized in that, The program in the software code performs the steps of the quadruped robot eye-leg cooperative obstacle crossing method for complex terrain as described in any one of claims 1-4.

Images