Four-legged robot multi-modal semantic navigation method and system for three-no environment

CN122613980BActive Publication Date: 2026-10-16NINGXIA LINGJI DIGITAL TECHNOLOGY CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202611086019.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2026-07-21
Publication Date
2026-10-16
Estimated Expiration
2046-07-21

AI Technical Summary

Technical Problem

[0004](1)无光照条件导致视觉传感器失效:激光雷达(LiDAR)和深度相机依赖主动或被动光学成像

Benefits of technology

[0070]As described above, the multimodal semantic navigation scheme for quadruped robots in unmanned environments provided by this solution first applies a sweep frequency excitation to the ground using an exciter integrated at the foot, and collects vibration response signals using an accelerometer. Then, spectral analysis is performed on the vibration response signals, and the physical semantic attributes of the foundation are identified based on the feature vectors extracted from the spectral analysis. Further, a physical semantic grid map labeled with physical semantics is constructed, with the robot's entry point as the origin, and the structural safety level of each grid is marked. Then, a gravitational potential field for signs of life is constructed based on the spatial concentration gradient determined by a carbon dioxide sensor array and the orientation of the heat source locked by an infrared pyroelectric array. Simultaneously, a repulsive potential field for danger is constructed based on the danger markings in the physical semantic grid map. Further, the desired movement direction of the quadruped robot is determined based on the gravitational potential field for signs of life and the repulsive potential field for danger. Finally, before executing the movement, the physical semantic grid map is queried, and only grids marked as safe are selected as landing points, mapping the desired movement direction to quadrupedal gait control commands. Therefore, this solution does not rely on visual sensors or wireless communication. It identifies the physical semantics of the foundation solely through foot excitation and spectral analysis, determines the gas concentration gradient through a carbon dioxide sensor array, and locates the heat source using an infrared pyroelectric array. This enables the quadruped robot to navigate and search autonomously even in extreme conditions with no light, no signal, and no map. This solution relies solely on physical contact and non-visual physical fields to autonomously assess structural safety, detect lifelines, and plan the lowest-risk path. It can be widely applied in dangerous environments that are difficult for humans to access, such as earthquake rubble search and rescue, mine collapse detection, and nuclear accident site investigation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122613980B_ABST
    Figure CN122613980B_ABST
Patent Text Reader

Abstract

The application provides a quadruped robot multimodal semantic navigation method and system for a three-nothing environment, and relates to the technical field of quadruped robots. The method comprises the following steps: applying a sweep excitation to the ground by using an exciter, and collecting a vibration response signal; performing frequency spectrum analysis on the vibration response signal to identify the physical semantic attributes of the foundation; constructing a physical semantic grid map with physical semantics as labels, and marking the structural safety level of each grid; constructing a life sign gravitational potential field based on a carbon dioxide sensor and an infrared pyroelectric; constructing a danger repulsive potential field based on the danger markings in the physical semantic grid map; determining the desired motion direction of the robot according to the life sign gravitational potential field and the danger repulsive potential field; querying the physical semantic grid map to select a safe grid marked as a landing point, and mapping the desired motion direction to a gait control instruction. The scheme can enable the quadruped robot to realize accurate and safe autonomous navigation under the three-nothing conditions of no light, no signal and no prior map.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of quadruped robot technology, and in particular to a multimodal semantic navigation method and system for quadruped robots in unmanned environments. Background Technology

[0002] Quadruped robots are a type of legged mobile robot designed to mimic the locomotion of quadrupedal animals. Compared to traditional wheeled or tracked robots, quadruped robots, with their discrete footholds and flexible gait adjustment capabilities, can move effectively in complex terrains inaccessible to wheeled robots. In recent years, quadruped robots have made remarkable progress in locomotion capabilities, and their applications have expanded from the laboratory to real-world working environments, finding wide application in disaster relief and emergency response, industrial inspection and security, logistics handling, and special operations.

[0003] Despite continuous improvements in the locomotion capabilities of quadruped robots, autonomous navigation and perception in complex and extreme environments remain the core bottlenecks hindering their large-scale practical application. For example, in extreme disaster environments such as earthquake rubble, mine collapses, or building implosions, current quadruped robot navigation systems still face the following technical bottlenecks:

[0004] (1) Lack of illumination causes visual sensor failure: LiDAR and depth cameras rely on active or passive optical imaging. In environments with high concentrations of dust, dense smoke, or complete darkness, optical signals may be completely absorbed or severely scattered, causing the sensor to be unable to generate effective point clouds and lose its ability to perceive the environment.

[0005] (2) Absolute positioning failure due to lack of GPS / public network conditions: GPS signals are completely lost in building shielding or underground spaces, and traditional GNSS-IMU fusion positioning solutions cannot work. At the same time, damage to public network infrastructure such as 4G / 5G / Wi-Fi prevents robots from relying on cloud computing power or centralized scheduling systems.

[0006] (3) The lack of prior map conditions poses a challenge to traditional SLAM: the terrain at the disaster site is severely deformed due to collapse, and the pre-stored high-precision map becomes completely invalid. In the absence of visual loop closure detection, the cumulative error of the inertial navigation unit in traditional SLAM schemes will lead to pose drift, making it impossible to build a usable map.

[0007] Therefore, existing quadruped robots struggle to achieve accurate and safe autonomous navigation in scenarios with no light, no signal, and no prior map. Summary of the Invention

[0008] In view of this, and to address the above shortcomings, it is necessary to propose a multimodal semantic navigation method and system for quadruped robots in a three-no environment, so that quadruped robots can achieve accurate and safe autonomous navigation under the three-no conditions of no light, no signal, and no prior map.

[0009] In a first aspect, the present invention provides a multimodal semantic navigation method for quadruped robots in unmanned environments, comprising:

[0010] A sweep frequency excitation was applied to the ground using an exciter integrated at the foot, and the vibration response signal was collected by an accelerometer.

[0011] The vibration response signal is subjected to spectral analysis, and the physical semantic attributes of the foundation are identified based on the feature vectors extracted from the spectral analysis.

[0012] Using the robot's entry point as the origin, construct a physical semantic grid map labeled with physical semantics, and mark the structural safety level of each grid.

[0013] Based on the spatial concentration gradient determined by the carbon dioxide sensor array and the location of the heat source locked by the infrared pyroelectric array, a gravitational potential field for signs of life is constructed.

[0014] Based on the hazard markers in the physical semantic raster map, a hazard repulsion potential field is constructed.

[0015] The desired direction of motion for the quadruped robot is determined based on the gravitational potential field of the life signs and the repulsive potential field of the danger.

[0016] Before executing the movement, the physical semantic grid map is queried, and only grids marked as safe are selected as landing points. The desired movement direction is then mapped to quadrupedal gait control commands.

[0017] Preferably, the step of performing spectral analysis on the vibration response signal and identifying the physical semantic attributes of the foundation based on the feature vector extracted from the spectral analysis includes:

[0018] The vibration response signal is subjected to a fast Fourier transform to obtain a frequency domain representation, and the power spectral density is calculated based on this frequency domain representation.

[0019] Extract frequency domain feature vectors from the power spectral density; wherein, the frequency domain feature vectors include peak frequency, stiffness estimate, damping ratio estimate, and band energy ratio;

[0020] The physical semantic attributes are determined based on the stiffness estimate, damping ratio estimate, and frequency band energy ratio; wherein the physical semantic attributes include at least: solid concrete, cracked structure, hollow cavity, and loose gravel.

[0021] Preferably, the power spectral density is calculated as follows:

[0022]

[0023] in, For power spectral density, Let T be the frequency domain representation, and T be the sampling time window;

[0024] The extraction methods for the frequency domain feature vector include:

[0025]

[0026] in, Peak frequency, This is an estimate of the stiffness. The equivalent mass of the foot participating in vibration. For half-power bandwidth, This refers to the frequency band energy ratio.

[0027] Preferably, the construction of a physical semantic grid map labeled with physical semantics, and the marking of the structural security level of each grid, includes:

[0028] When the robot starts, a two-dimensional grid map is created with the initial position as the origin;

[0029] Each time the robot completes a step, it marks the grid where the current landing point is located based on the recognized physical semantic attributes. The marking content includes: physical semantic attributes, security level, and timestamp.

[0030] Preferably, the construction of the gravitational potential field for signs of life based on the spatial concentration gradient determined by the carbon dioxide sensor array and the orientation of the heat source locked by the infrared pyroelectric array includes:

[0031] The gravitational potential field of carbon dioxide is calculated by a carbon dioxide sensor array mounted on the nose of a quadruped robot.

[0032] The infrared gravitational potential field is calculated using an infrared pyroelectric array sensor mounted on the head of the quadruped robot.

[0033] The gravitational potential field of carbon dioxide and the gravitational potential field of infrared are fused to obtain the gravitational potential field of life signs.

[0034] Preferably, the calculation method for the carbon dioxide gravitational potential field includes:

[0035]

[0036]

[0037]

[0038] in, Let be the gravitational potential field function of carbon dioxide. This represents the gravitational gain coefficient for carbon dioxide. The spatial gradient vector, Use the center as a reference for the sensor reading. For the reading of the i-th peripheral sensor, For sensor spacing, For environmental background concentration, The basic gravitational gain coefficient;

[0039] The calculation method for the infrared gravitational potential field includes:

[0040]

[0041] in, Let be the infrared gravitational potential field function. The gravitational gain coefficient of the heat source. The temperature difference between the heat source and the environment, The distance from the quadruped robot to the heat source. This refers to the range of influence of the heat source's gravitational pull.

[0042] The calculation methods for the gravitational potential field of the life signs include:

[0043]

[0044] in, For life signs, gravitational potential field and These are the weighting coefficients for the carbon dioxide gravitational potential field and the infrared gravitational potential field, respectively.

[0045] Preferably, the construction of the dangerous repulsive potential field includes calculations based on the following formula:

[0046]

[0047] in, A dangerous repulsive potential field. This is the dynamic adjustment coefficient for global risk. The basic rejection coefficient for the i-th type of hazard. This is the shortest distance from the current position of the quadruped robot to the boundary of the hazard source. The radius of influence of the hazard source;

[0048] Wherein, the global risk dynamic adjustment coefficient The methods for determining this include:

[0049] The energy integral extracted from the power spectral density within a preset frequency band is expressed as: ;in, and These are the lower and upper limits of the frequency band, respectively.

[0050] when When the preset threshold is exceeded, Dynamic adjustment is performed according to the following formula:

[0051]

[0052] in, The preset threshold, This is the maximum energy integral threshold.

[0053] Preferably, determining the desired direction of movement of the quadruped robot based on the gravitational potential field of the signs of life and the repulsive potential field of danger includes:

[0054] The desired direction of motion for the quadruped robot is calculated using the following formula:

[0055]

[0056]

[0057] in, Let be the total potential field function. Let be the value of the dangerous repulsive potential field function of the i-th grid. The desired direction of motion for the robot.

[0058] Preferably, selecting only the marked safe grid as the landing point includes:

[0059] Generate a set of candidate landing points along the direction of motion;

[0060] Query the physical semantic raster map and remove candidate points marked as hollow cavities, cracked structures, and loose gravel;

[0061] If no safe candidate point is found, a trial landing is performed. The trial landing includes: lowering the foot end at a preset low speed, monitoring force feedback and displacement in real time, and immediately withdrawing and replanning if an abnormality is detected.

[0062] Secondly, the present invention also provides a multimodal semantic navigation system for quadruped robots in a three-no environment, including: an excitation acquisition module, a spectrum analysis module, a grid map construction module, a life sign gravitational potential field determination module, a danger repulsion potential field determination module, a desired motion direction determination module, and an execution control module;

[0063] The excitation acquisition module is used to apply a sweep frequency excitation to the ground using an exciter integrated at the foot end, and to acquire the vibration response signal through an accelerometer;

[0064] The spectrum analysis module is used to perform spectrum analysis on the vibration response signal and identify the physical semantic attributes of the foundation based on the feature vector extracted by the spectrum analysis.

[0065] The grid map construction module is used to construct a physical semantic grid map with physical semantics as labels, taking the robot entry point as the origin, and marking the structural safety level of each grid.

[0066] The life sign gravitational potential field determination module is used to construct the life sign gravitational potential field based on the spatial concentration gradient determined by the carbon dioxide sensor array and the orientation of the heat source locked by the infrared pyroelectric array.

[0067] The danger repulsion potential field determination module is used to construct a danger repulsion potential field based on the danger markings in the physical semantic grid map;

[0068] The desired motion direction determination module is used to determine the desired motion direction of the quadruped robot based on the gravitational potential field of the life signs and the repulsive potential field of the danger.

[0069] The execution control module is used to query the physical semantic grid map before executing the movement, select only the grids marked as safe as the landing points, and map the desired movement direction into quadruped gait control commands.

[0070] As described above, the multimodal semantic navigation scheme for quadruped robots in unmanned environments provided by this solution first applies a sweep frequency excitation to the ground using an exciter integrated at the foot, and collects vibration response signals using an accelerometer. Then, spectral analysis is performed on the vibration response signals, and the physical semantic attributes of the foundation are identified based on the feature vectors extracted from the spectral analysis. Further, a physical semantic grid map labeled with physical semantics is constructed, with the robot's entry point as the origin, and the structural safety level of each grid is marked. Then, a gravitational potential field for signs of life is constructed based on the spatial concentration gradient determined by a carbon dioxide sensor array and the orientation of the heat source locked by an infrared pyroelectric array. Simultaneously, a repulsive potential field for danger is constructed based on the danger markings in the physical semantic grid map. Further, the desired movement direction of the quadruped robot is determined based on the gravitational potential field for signs of life and the repulsive potential field for danger. Finally, before executing the movement, the physical semantic grid map is queried, and only grids marked as safe are selected as landing points, mapping the desired movement direction to quadrupedal gait control commands. Therefore, this solution does not rely on visual sensors or wireless communication. It identifies the physical semantics of the foundation solely through foot excitation and spectral analysis, determines the gas concentration gradient through a carbon dioxide sensor array, and locates the heat source using an infrared pyroelectric array. This enables the quadruped robot to navigate and search autonomously even in extreme conditions with no light, no signal, and no map. This solution relies solely on physical contact and non-visual physical fields to autonomously assess structural safety, detect lifelines, and plan the lowest-risk path. It can be widely applied in dangerous environments that are difficult for humans to access, such as earthquake rubble search and rescue, mine collapse detection, and nuclear accident site investigation. Attached Figure Description

[0071] Figure 1 The flowchart illustrates a multimodal semantic navigation method for quadruped robots in a three-no environment, as provided in this embodiment of the invention. Detailed Implementation

[0072] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0073] like Figure 1 As shown, this invention provides a multimodal semantic navigation method for quadruped robots in unmanned environments, which may include the following steps:

[0074] Step 101: Apply a sweep frequency excitation to the ground using the exciter integrated at the foot end, and collect the vibration response signal using an accelerometer;

[0075] Step 102: Perform spectral analysis on the vibration response signal and identify the physical semantic attributes of the foundation based on the feature vector extracted from the spectral analysis;

[0076] Step 103: Using the robot's entry point as the origin, construct a physical semantic grid map labeled with physical semantics, and mark the structural safety level of each grid.

[0077] Step 104: Construct a gravitational potential field for signs of life based on the spatial concentration gradient determined by the carbon dioxide sensor array and the location of the heat source locked by the infrared pyroelectric array;

[0078] Step 105: Based on the hazard markers in the physical semantic grid map, construct a hazard repulsion potential field;

[0079] Step 106: Determine the desired direction of motion for the quadruped robot based on the gravitational potential field of the life signs and the repulsive potential field of the danger.

[0080] Step 107: Before executing the movement, query the physical semantic grid map, select only the grids marked as safe as the landing points, and map the desired movement direction into quadruped gait control commands.

[0081] In this embodiment, without relying on visual sensors and wireless communication, the quadruped robot identifies the physical semantics of the foundation solely through foot excitation and spectral analysis. A carbon dioxide sensor array determines the gas concentration gradient, and an infrared pyroelectric array locates the heat source. This enables the quadruped robot to achieve accurate and safe autonomous navigation and search even under extreme conditions of no light, no signal, and no map. This solution relies solely on physical contact and non-visual physical fields to autonomously assess structural safety, discover lifelines, and plan the lowest-risk path. It can be widely applied in dangerous environments that are difficult for humans to access, such as earthquake rubble search and rescue, mine collapse detection, and nuclear accident site investigation.

[0082] The following provides a more detailed explanation of each of the above steps.

[0083] For step 101, a sweep frequency excitation is applied to the ground using an exciter integrated at the foot end, and the vibration response signal is collected by an accelerometer.

[0084] In this step, we consider using an exciter integrated into the foot to apply a sweep frequency excitation to the ground, and then collecting the vibration response signal using an accelerometer. Specifically, we consider first integrating the following sensing and actuation elements into the foot of the quadruped robot:

[0085] (1) Electromagnetic excitation device: Specifically, it can be a miniature electromagnetic vibrator, embedded in the center area of ​​the sole of the foot, which can generate a sweep frequency excitation force of 10Hz~2kHz, with a maximum amplitude of ±0.5mm and a peak force of 50N.

[0086] (2) Triaxial MEMS accelerometer: installed near the exciter, with a sampling rate of not less than 4kHz and a range of ±16g, used to collect foot vibration response.

[0087] (3) Flexible tactile array: covering the pressure area of ​​the sole (forefoot and heel), composed of piezoresistive or capacitive sensing units, with a sampling rate of 1kHz, used to monitor the pressure distribution and contact status of the sole.

[0088] (4) Six-dimensional force / torque sensor: installed at the ankle joint, with a range F z ≥2000N, used for measuring ground reaction force and ZMP calculation.

[0089] In one embodiment, when applying sweep frequency excitation to the ground using a foot-integrated exciter, the contact dynamics system between the foot and the ground is first modeled as a Kelvin-Voigt viscoelastic model. This model consists of a parallel connection of an elastic element (stiffness k) and a damping element (damping c), and its differential equation of motion is:

[0090]

[0091] Where m is the equivalent mass (kg) of the foot participating in the vibration, which is a known design parameter; c is the equivalent ground damping coefficient (N·s / m); k is the equivalent ground stiffness coefficient (N / m); F(t) is the sweep frequency excitation force (N) applied by the exciter; and x(t) is the foot displacement response (m), which is obtained by the second integration of the accelerometer signal.

[0092] Furthermore, while the exciter applies excitation, a triaxial MEMS accelerometer installed near the exciter acquires the vibration response signal of the foot at a sampling rate of no less than 4kHz. The accelerometer and the exciter are mounted on the same aluminum alloy base to ensure direct transmission of vibration. The exciter's push rod extends when triggered, directly contacting the ground to transmit vibration, ensuring effective coupling of the excitation signal to the foundation. During normal walking, it retracts into the buffer layer, which alone bears the impact load, protecting the exciter from overload damage.

[0093] For step 102, the vibration response signal is subjected to spectral analysis, and the physical semantic attributes of the foundation are identified based on the feature vector extracted by the spectral analysis.

[0094] In this step, when performing spectrum analysis and identifying the physical semantic attributes of the foundation based on the feature vectors extracted from the spectrum analysis, the following approach can be considered:

[0095] S21: Perform a fast Fourier transform on the vibration response signal to obtain a frequency domain representation, and calculate the power spectral density based on the frequency domain representation;

[0096] In this step, the time-domain response signal acquired by the accelerometer is considered. Perform a windowed Fast Fourier Transform (FFT) to obtain the frequency domain representation X(f), and then calculate the one-sided power spectral density, as shown below:

[0097]

[0098] in, For power spectral density, Let T be the frequency domain representation, and T be the sampling time window.

[0099] S22: Extract frequency domain feature vectors from the power spectral density; wherein, the frequency domain feature vectors include peak frequency, stiffness estimate, damping ratio estimate, and band energy ratio;

[0100] In this step, we consider extracting the following four features from the power spectral density (PSD) as the feature vector f for ground-based semantic classification. PSD They are respectively:

[0101]

[0102] in, The peak frequency is the frequency corresponding to the maximum value of the PSD, reflecting the structural resonance characteristics. This is an estimate of the stiffness. The equivalent mass of the foot participating in vibration, i.e., the reduced mass of a single robot leg participating in vibration at the moment of ground contact, is determined based on the modal mass distribution coefficient and is approximately 1 / 6 to 1 / 4 of the robot's total mass. This is the estimated damping ratio. For half-power bandwidth, This refers to the frequency band energy ratio.

[0103] S23: Determine the physical semantic attributes based on the stiffness estimate, damping ratio estimate, and frequency band energy ratio; wherein the physical semantic attributes include at least: solid concrete, cracked structure, hollow cavity, and loose gravel.

[0104] In this step, the aforementioned feature vectors are input into a pre-trained classifier (Support Vector Machine or Lightweight Neural Network), which outputs the ground-based physical semantic attributes. Specifically, the physical semantic attributes are determined according to the semantic determination logic in the table below: solid concrete <![CDATA[High (>10 7 N / m)]]> Low (<0.5) The main peak is sharp and concentrated in the 100-500Hz range. Safety Cracked structure moderate reduction Increase (0.05-0.15) Second harmonic peaks appear Mild danger Hollow / Cavity <![CDATA[sharply reduce (<10 6 N / m)]]> abnormal A strong resonance peak appears in the low frequency range (10-50Hz). Severe danger Loose gravel Extremely low and unstable High (>0.2) High-energy noise across the entire frequency band, with no obvious main peak. Moderate risk

[0105] The classifier outputs the foundation type and its confidence level when the stiffness estimate is <10. 6 When the PSD shows a strong resonance peak at 10-50Hz, it is determined to be a hollow cavity. This result is used for subsequent repulsive field construction.

[0106] For step 103, a physical semantic grid map is constructed with the robot's entry point as the origin, and the structural safety level of each grid is marked.

[0107] In this step, the first consideration is map initialization and coordinate system establishment. When the robot starts, a two-dimensional grid map is created with the initial position as the origin (0,0), and each grid cell is set to 0.1m × 0.1m (adjustable). All grid cells are initially set to "unexplored" with an unknown safety level. Next, semantic annotation and probability updates are performed. Each time the robot completes a landing step, the system annotates the grid cell containing the current landing point based on the ground physical semantics and safety level identified in step 102. The annotation includes: physical semantic category (solid / crack / cavity / rubble), safety level (safe / slightly dangerous / moderately dangerous / severely dangerous), and timestamp. Simultaneously, to handle sensor noise, a Bayesian update method can be used to maintain the safety probability of each grid cell. Finally, map boundary management and update frequency are considered. The map range dynamically expands as the robot moves, maintaining a fixed-size local window (e.g., 10m × 10m) centered on the robot's current position. Older grid cells exceeding the window can be compressed and stored or discarded to ensure real-time computation.

[0108] For step 104, a gravitational potential field for signs of life is constructed based on the spatial concentration gradient determined by the carbon dioxide sensor array and the orientation of the heat source locked by the infrared pyroelectric array.

[0109] In this step, we consider using a fusion of two physical fields—the carbon dioxide plume gradient field and the infrared heat source field—to drive the search behavior, without relying on visual target detection. Specifically, step 104 can be implemented as follows:

[0110] S41: Calculate the gravitational potential field of carbon dioxide using a carbon dioxide sensor array mounted on the nose of the quadruped robot;

[0111] In this step, for the gravitational potential field of carbon dioxide, considering the high-concentration CO2 plume generated by human respiration, a concentration gradient is formed in the gaps of the rubble. The quadruped robot's nose is equipped with an NDIR (non-dispersive infrared) CO2 sensor array, with each sensor spatially separated to calculate the local concentration gradient. Thus, the gravitational potential field of carbon dioxide is obtained, as shown below:

[0112]

[0113]

[0114]

[0115] in, Let be the gravitational potential field function of carbon dioxide. This represents the gravitational gain coefficient for carbon dioxide. The spatial gradient vector is obtained by differential calculation of the sensor array. Use the center as a reference for the sensor reading. For the reading of the i-th peripheral sensor, For sensor spacing, For environmental background concentration, The base gravity gain coefficient; this design enhances gravity in regions where carbon dioxide concentration is significantly higher than the background, and makes gravity approach zero in regions at the background level.

[0116] S42: Calculate the infrared gravitational potential field using an infrared pyroelectric array sensor mounted on the head of the quadruped robot;

[0117] In this step, the quadruped robot's head is equipped with an infrared pyroelectric array sensor, which can detect the location of heat sources within its field of view. Let the coordinates of the detected heat source be (x... T ,y T (Local coordinate system), the temperature difference between the heat source and the environment is ΔT, and the infrared gravitational potential field is defined as:

[0118]

[0119] in, Let be the infrared gravitational potential field function. The gravitational gain coefficient of the heat source. The temperature difference between the heat source and the environment, The distance from the quadruped robot to the heat source. This is a parameter representing the range of influence of the heat source's gravitational pull.

[0120] S43: The carbon dioxide gravitational potential field and the infrared gravitational potential field are fused to obtain the life sign gravitational potential field.

[0121] In this step, we consider weighted superposition of the carbon dioxide gravitational potential field and the infrared gravitational potential field to obtain the gravitational potential field for signs of life. This is represented as follows:

[0122]

[0123] in, For life signs, gravitational potential field and These are the weighting coefficients for the carbon dioxide gravitational potential field and the infrared gravitational potential field, respectively.

[0124] Furthermore, the weighting coefficients of the carbon dioxide gravitational potential field and the infrared gravitational potential field can be dynamically adjusted. For example, when the CO2 sensor and the infrared thermal array are both pointing in the same direction, , When only CO2 signal is present, , When only infrared signals are available, , If the angle between the two directions exceeds the threshold, and All are reduced to avoid oscillations caused by conflicting information.

[0125] It is easy to understand that when calculating the gravitational potential field of carbon dioxide, the infrared gravitational potential field, and constructing the repulsive potential field of danger, it is necessary to normalize the physical sensing quantities such as carbon dioxide, infrared, and the distance to the danger source so as to unify them into dimensionless variables.

[0126] For step 105, a danger repulsion potential field is constructed based on the danger labels marked in the physical semantic raster map.

[0127] In this step, we first consider mapping the identified physical semantic hazards (hollow cavities, cracked areas, toxic gas zones, etc.) to a repulsive force field in space. An improved FIRAS potential field function is used to make the repulsive force increase sharply when approaching the hazard, which can be specifically expressed as follows:

[0128]

[0129] in, A dangerous repulsive potential field. This is the dynamic adjustment coefficient for global risk. The basic rejection coefficient for the i-th type of hazard. This is the shortest distance from the current position of the quadruped robot to the boundary of the hazard source. This refers to the radius of influence of the hazard source. For example, and The possible values ​​for are shown in the table below:

[0130] Furthermore, the global risk dynamic adjustment coefficient The methods for determining this include:

[0131] While the quadruped robot is stationary or moving slowly, the accelerometers on its feet continuously monitor ambient micro-vibration signals. A short-time Fourier transform (STFT) is performed on the signal, and the energy integral within the 0.5Hz–3Hz frequency band is extracted. The energy integral within the preset frequency band is extracted from the power spectral density and expressed as: ;in, and These are the lower and upper limits of the frequency band, which can be 0.5Hz and 3Hz respectively.

[0132] when Exceeding the preset threshold hour, Dynamic adjustment is performed according to the following formula:

[0133]

[0134] in, The preset threshold, This is the maximum energy integral threshold. As the force increases, all dangerous repulsive forces increase simultaneously, and the quadruped robot's navigation behavior switches from "search priority" to "safety priority," tending to retreat along known physical paths.

[0135] based on Dynamic adjustments can enable aftershock response. For example, when the robot is stationary or stationary (such as during on-site monitoring), the foot accelerometer switches to a high-sensitivity mode to continuously monitor micro-vibrations in the environment.

[0136] Sampling window: Vibration recordings are taken for 5 seconds every 30 seconds;

[0137] Analysis frequency band: Focus on 0.5Hz~3Hz (frequency band characteristic of aftershocks / structural loosening);

[0138] Decision logic: If the energy integral value of this frequency band If the value increases to more than three times that of the previous sampling period within a single sampling period, or exceeds a preset absolute threshold, the value may also increase. The system determined that there was a risk of secondary collapse.

[0139] Response Action: Global Risk Coefficient Within 1 second, the force increases from 1.0 to 3.0, all dangerous repulsive forces are amplified simultaneously, and the resultant force direction quickly shifts to the known safe path of the entity. The robot then retreats along this path at the highest safe speed.

[0140] For step 106, the desired direction of motion of the quadruped robot is determined based on the gravitational potential field of the life signs and the repulsive potential field of the danger.

[0141] In this step, the desired motion direction of the quadruped robot is calculated using the following formula:

[0142]

[0143]

[0144] in, Let be the total potential field function. Let be the value of the dangerous repulsive potential field function of the i-th grid. For the robot's desired direction of motion, when If the target is not reached, it is determined that the system is trapped in a local minimum. In this case, a random perturbation is applied to facilitate escape. The escape strategy can be expressed as follows: , Let r be the tangent direction along the contour line of the potential field, and r be a unit random vector. The disturbance amplitude is used to escape for 2 seconds before restoring standard potential field navigation.

[0145] For step 107, before executing the movement, the physical semantic grid map is queried, and only grids marked as safe are selected as landing points. The desired movement direction is then mapped to quadrupedal gait control commands.

[0146] This step can be implemented in the following way:

[0147] S71: Generate a set of candidate landing points in the direction of motion;

[0148] In this step, taking the robot's current supporting foot as the center, 3 to 5 candidate foot placement points are generated within a 30° range on both sides of the desired movement direction. The distance between the candidate points and the current foot is approximately equal to the reference stride length (dynamically adjusted according to the current speed, such as 0.2 to 0.4 m), and the diagonal gait constraints of the quadruped robot (such as alternating support from the opposite foot) are taken into account.

[0149] S72: Query the physical semantic raster map and remove candidate points marked as hollow cavities, cracked structures, and loose gravel;

[0150] In this step, for each candidate landing point, the physical semantic raster map is queried to check the safety status of the raster. If it is marked as "Entity-Safe", the candidate is retained. If it is marked as "Cavity", "Crack", or "Rock", the candidate is discarded. If it is "Unknown", an unknown penalty is applied, and known safe points are selected first.

[0151] S73: If there is no safe candidate point, perform a trial landing; the trial landing includes: lowering the foot end at a preset low speed, monitoring force feedback and displacement in real time, and immediately withdrawing and replanning when an abnormality is detected.

[0152] In this embodiment, if there is no safe candidate point, a tentative landing strategy is initiated: the foot is lowered at an extremely low speed (0.05 m / s) while monitoring the data from the six-dimensional force sensor and accelerometer. If abnormal settlement (vertical displacement > 5 mm and force does not increase) or slippage (sudden increase in tangential force) is detected, the foot is immediately withdrawn and the potential field planning is re-performed (returning to step 106).

[0153] Furthermore, consider selecting the point with the smallest angle to the desired direction among the safe candidate points as the actual landing point, and map the displacement of this landing point relative to the current position to the gait parameters of the quadruped robot:

[0154] Calculate stride length and direction;

[0155] Adjust gait cycle according to the current terrain safety level (slow down in danger zones);

[0156] Generate foot trajectory (such as cycloidal trajectory) to ensure that the foot lift height is sufficient to clear the obstacle (usually 0.05~0.1m).

[0157] The joint angle sequence is calculated using inverse kinematics and sent to the servo driver for execution.

[0158] During execution, the IMU and foot force sensor are continuously monitored. If unexpected instability or ground change occurs, the gait can be terminated in advance and the process can return to step 101 to re-sensing.

[0159] Furthermore, considering that pure inertial navigation would produce unacceptable cumulative drift in the absence of GPS and visual loop closure detection, this application also employs a "tactile anchor point" relocalization mechanism: when the foot's physical semantic recognition determines that the current footing area is high-stiffness solid concrete, the system marks this point as a "strong feature point"; utilizing the dynamic constraints of the quadruped robot during the support phase (multi-legged landing), Kalman filtering is applied to correct the IMU's angular velocity and acceleration drift; when stepping on an area with similar physical semantic characteristics (stiffness, PSD peak frequency matching) again, the system identifies it as a loop closure, triggering pose graph optimization to eliminate cumulative errors. The physical basis of this mechanism is that the dynamic response characteristics of structures of the same material are repeatable and can replace visual feature points as the basis for loop closure detection.

[0160] This invention also provides a multimodal semantic navigation system for quadruped robots in uninhabited environments, comprising: an excitation acquisition module, a spectrum analysis module, a grid map construction module, a gravitational potential field determination module for signs of life, a repulsive potential field determination module for danger, a desired motion direction determination module, and an execution control module;

[0161] The excitation acquisition module is used to apply a sweep frequency excitation to the ground using an exciter integrated at the foot end, and to acquire the vibration response signal through an accelerometer;

[0162] The spectrum analysis module is used to perform spectrum analysis on the vibration response signal and identify the physical semantic attributes of the foundation based on the feature vector extracted by the spectrum analysis.

[0163] The grid map construction module is used to construct a physical semantic grid map with physical semantics as labels, taking the robot entry point as the origin, and marking the structural safety level of each grid.

[0164] The life sign gravitational potential field determination module is used to construct the life sign gravitational potential field based on the spatial concentration gradient determined by the carbon dioxide sensor array and the orientation of the heat source locked by the infrared pyroelectric array.

[0165] The danger repulsion potential field determination module is used to construct a danger repulsion potential field based on the danger markings in the physical semantic grid map;

[0166] The desired motion direction determination module is used to determine the desired motion direction of the quadruped robot based on the gravitational potential field of the life signs and the repulsive potential field of the danger.

[0167] The execution control module is used to query the physical semantic grid map before executing the movement, select only the grids marked as safe as the landing points, and map the desired movement direction into quadruped gait control commands.

[0168] This specification also provides a computer-readable storage medium having a computer program stored thereon, which, when executed in a computer, causes the computer to perform the methods in any of the embodiments of the specification.

[0169] This specification also provides a computing device, including a memory and a processor, wherein the memory stores executable code, and when the processor executes the executable code, it implements the method in any of the embodiments of the specification.

[0170] Since the system embodiments and method embodiments of the present invention are based on the same inventive concept, detailed descriptions can be found in the method embodiments, and will not be repeated here.

[0171] In summary, the multimodal semantic navigation method and system for quadruped robots in unmanned environments provided by this invention have at least the following beneficial effects:

[0172] (1) Completely eliminates dependence on lighting, signals, and prior maps. Traditional quadruped robots rely on LiDAR, GPS, and pre-stored maps in disaster environments, and become completely ineffective under "three-no" conditions such as complete darkness, dense dust and smoke, GPS signal shielding, and terrain collapse and deformation. This invention does not rely on visual sensors and wireless communication, but only on foot excitation and spectrum analysis to identify the physical semantics of the foundation (solid / cavity / rubble). It calculates the gas concentration gradient through an NDIR CO2 sensor array and locks the location of the heat source through an infrared pyroelectric array, enabling the robot to navigate and search autonomously under extreme conditions of no light, no signal, and no map.

[0173] (2) Foundation structure safety is taken as the core constraint of navigation. Existing navigation methods take "shortest path" as the optimization goal and do not consider the foundation bearing safety, which may cause the robot to trigger secondary collapse during the crossing. This invention maps the physical semantic hazard sources (hollow cavities, crack areas, etc.) identified by the foot to a spatial repulsive potential field, so that "safety" becomes the primary constraint of path planning rather than an optional additional item.

[0174] (3) Introducing environmental micro-vibration monitoring to achieve dynamic risk adjustment. This invention utilizes a foot accelerometer to continuously monitor micro-vibration signals in the 0.5Hz~3Hz frequency band and calculates the energy integral. When an aftershock or precursor to a secondary collapse is detected, the global risk coefficient α dynamically increases from 1.0 to 3.0, and all dangerous repulsive forces are simultaneously enhanced. This causes the robot's navigation behavior to automatically switch from "search priority" to "safety priority," retreating along the already explored physical path. This is the first real-time response mechanism for time-varying risks (aftershocks) at disaster sites.

[0175] (4) CO2-infrared dual-physics field fusion drives life search. Traditional search and rescue relies on visible light or microphones, which are ineffective for trapped persons who are comatose, weak, or unable to speak. This invention weightedly fuses the CO2 plume gradient field with an infrared heat source, achieving complementary fusion of "olfaction" and "thermal sensation", which greatly improves the detection probability of trapped persons who are unable to move independently.

[0176] (6) Secondary safety verification mechanism for landing points. After the resultant force direction is determined, the system generates a set of candidate landing points and queries the physical semantic grid map to remove candidate points marked as "cavity", "crack" or "rubble". If there are no safe candidate points, a trial landing is performed: the foot end is lowered at an extremely low speed of 0.05m / s, and the force feedback and displacement are monitored in real time. If abnormal settlement (displacement > 5mm and force does not increase) or slippage (sudden increase in tangential force) is detected, the landing is immediately withdrawn and replanned, adding a hardware-in-the-loop safety defense line on top of the potential field planning.

[0177] The modules or units in the device of this invention can be merged, divided, and deleted according to actual needs. The above-disclosed embodiments are merely preferred embodiments of the present invention and should not be construed as limiting the scope of the invention. Those skilled in the art will understand that implementing all or part of the processes of the above embodiments and making equivalent changes according to the claims of this invention still fall within the scope of the invention.

Claims

1. A multimodal semantic navigation method for quadruped robots in unmanned environments, characterized in that, The "three-no" environment includes no light, no signal, and no prior map; the method includes: A sweep frequency excitation was applied to the ground using an exciter integrated at the foot, and the vibration response signal was collected by an accelerometer. The vibration response signal is subjected to spectral analysis, and the physical semantic attributes of the foundation are identified based on the feature vectors extracted from the spectral analysis. Using the robot's entry point as the origin, construct a physical semantic grid map labeled with physical semantics, and mark the structural safety level of each grid. Based on the spatial concentration gradient determined by the carbon dioxide sensor array and the location of the heat source locked by the infrared pyroelectric array, a gravitational potential field for signs of life is constructed. Based on the hazard markers in the physical semantic raster map, a hazard repulsion potential field is constructed. The desired direction of motion for the quadruped robot is determined based on the gravitational potential field of the life signs and the repulsive potential field of the danger. Before executing the movement, the physical semantic grid map is queried, and only grids marked as safe are selected as footholds. The desired movement direction is then mapped to quadrupedal gait control commands. The construction of the gravitational potential field for signs of life based on the spatial concentration gradient determined by the carbon dioxide sensor array and the orientation of the heat source locked by the infrared pyroelectric array includes: The gravitational potential field of carbon dioxide is calculated by a carbon dioxide sensor array mounted on the nose of a quadruped robot. The infrared gravitational potential field is calculated using an infrared pyroelectric array sensor mounted on the head of the quadruped robot. The gravitational potential field of carbon dioxide and the gravitational potential field of infrared are fused to obtain the gravitational potential field of life signs; The calculation method for the carbon dioxide gravitational potential field includes: , , , in, Let be the gravitational potential field function of carbon dioxide. This represents the gravitational gain coefficient for carbon dioxide. The spatial gradient vector, Use the center as a reference for the sensor reading. For the reading of the i-th peripheral sensor, For sensor spacing, For environmental background concentration, The basic gravitational gain coefficient; The calculation method for the infrared gravitational potential field includes: , in, Let be the infrared gravitational potential field function. The gravitational gain coefficient of the heat source. The temperature difference between the heat source and the environment, The distance from the quadruped robot to the heat source. This refers to the range of influence of the heat source's gravitational pull. The calculation methods for the gravitational potential field of the life signs include: , in, For life signs, gravitational potential field and These are the weighting coefficients for the carbon dioxide gravitational potential field and the infrared gravitational potential field, respectively. The construction of the dangerous repulsive potential field is based on the following calculation formula: , in, A dangerous repulsive potential field. This is the dynamic adjustment coefficient for global risk. The basic rejection coefficient for the i-th type of hazard. This is the shortest distance from the current position of the quadruped robot to the boundary of the hazard source. The radius of influence of the hazard source; Wherein, the global risk dynamic adjustment coefficient The methods for determining this include: The energy integral extracted from the power spectral density within a preset frequency band is expressed as: ;in, and These are the lower and upper limits of the frequency band, respectively. Power spectral density; when When the preset threshold is exceeded, Dynamic adjustment is performed according to the following formula: , in, The preset threshold, This is the maximum energy integral threshold.

2. The multimodal semantic navigation method for quadruped robots in a three-no environment according to claim 1, characterized in that, The step of performing spectral analysis on the vibration response signal and identifying the physical semantic attributes of the foundation based on the feature vector extracted from the spectral analysis includes: The vibration response signal is subjected to a fast Fourier transform to obtain a frequency domain representation, and the power spectral density is calculated based on this frequency domain representation. Extract frequency domain feature vectors from the power spectral density; wherein, the frequency domain feature vectors include peak frequency, stiffness estimate, damping ratio estimate, and band energy ratio; The physical semantic attributes are determined based on the stiffness estimate, damping ratio estimate, and frequency band energy ratio; wherein the physical semantic attributes include at least: solid concrete, cracked structure, hollow cavity, and loose gravel.

3. The multimodal semantic navigation method for quadruped robots in a three-no environment according to claim 2, characterized in that, The power spectral density is calculated as follows: , in, For power spectral density, Let T be the frequency domain representation, and T be the sampling time window; The extraction methods for the frequency domain feature vector include: , in, Peak frequency, This is an estimate of the stiffness. The equivalent mass of the foot participating in vibration. For half-power bandwidth, The frequency band energy ratio, This is the estimated damping ratio.

4. The multimodal semantic navigation method for quadruped robots in a three-no environment according to claim 1, characterized in that, The construction of a physical semantic raster map labeled with physical semantics, and the marking of the structural security level of each raster, includes: When the robot starts, a two-dimensional grid map is created with the initial position as the origin; Each time the robot completes a step, it marks the grid where the current landing point is located based on the recognized physical semantic attributes. The marking content includes: physical semantic attributes, security level, and timestamp.

5. The multimodal semantic navigation method for quadruped robots in a three-no environment according to claim 4, characterized in that, Determining the desired direction of motion for the quadruped robot based on the gravitational potential field of the signs of life and the repulsive potential field of danger includes: The desired direction of motion for the quadruped robot is calculated using the following formula: , , in, Let be the total potential field function. Let be the value of the dangerous repulsive potential field function of the i-th grid. The desired direction of motion for the robot.

6. The multimodal semantic navigation method for quadruped robots in a three-no environment according to claim 1, characterized in that, The step of selecting only the safe grid cells as landing points includes: Generate a set of candidate landing points along the direction of motion; Query the physical semantic raster map and remove candidate points marked as hollow cavities, cracked structures, and loose gravel; If no safe candidate point is found, a trial landing is performed. The trial landing includes: lowering the foot end at a preset low speed, monitoring force feedback and displacement in real time, and immediately withdrawing and replanning if an abnormality is detected.

7. A multimodal semantic navigation system for quadruped robots in unmanned environments, characterized in that, The three-no environment includes no light, no signal, and no prior map; the system includes: an excitation acquisition module, a spectrum analysis module, a grid map construction module, a life sign gravitational potential field determination module, a danger repulsive potential field determination module, a desired motion direction determination module, and an execution control module; The excitation acquisition module is used to apply a sweep frequency excitation to the ground using an exciter integrated at the foot end, and to acquire the vibration response signal through an accelerometer; The spectrum analysis module is used to perform spectrum analysis on the vibration response signal and identify the physical semantic attributes of the foundation based on the feature vector extracted by the spectrum analysis. The grid map construction module is used to construct a physical semantic grid map with physical semantics as labels, taking the robot entry point as the origin, and marking the structural safety level of each grid. The life sign gravitational potential field determination module is used to construct the life sign gravitational potential field based on the spatial concentration gradient determined by the carbon dioxide sensor array and the orientation of the heat source locked by the infrared pyroelectric array. The danger repulsion potential field determination module is used to construct a danger repulsion potential field based on the danger markings in the physical semantic grid map; The desired motion direction determination module is used to determine the desired motion direction of the quadruped robot based on the gravitational potential field of the life signs and the repulsive potential field of the danger. The execution control module is used to query the physical semantic grid map before executing the movement, select only the grid marked as safe as the landing point, and map the desired movement direction into a quadruped gait control command. The construction of the gravitational potential field for signs of life based on the spatial concentration gradient determined by the carbon dioxide sensor array and the orientation of the heat source locked by the infrared pyroelectric array includes: The gravitational potential field of carbon dioxide is calculated by a carbon dioxide sensor array mounted on the nose of a quadruped robot. The infrared gravitational potential field is calculated using an infrared pyroelectric array sensor mounted on the head of the quadruped robot. The gravitational potential field of carbon dioxide and the gravitational potential field of infrared are fused to obtain the gravitational potential field of life signs; The calculation method for the carbon dioxide gravitational potential field includes: , , , in, Let be the gravitational potential field function of carbon dioxide. This represents the gravitational gain coefficient for carbon dioxide. The spatial gradient vector, Use the center as a reference for the sensor reading. For the reading of the i-th peripheral sensor, For sensor spacing, For environmental background concentration, The basic gravitational gain coefficient; The calculation method for the infrared gravitational potential field includes: , in, Let be the infrared gravitational potential field function. The gravitational gain coefficient of the heat source. The temperature difference between the heat source and the environment, The distance from the quadruped robot to the heat source. This refers to the range of influence of the heat source's gravitational pull. The calculation methods for the gravitational potential field of the life signs include: , in, For life signs, gravitational potential field and These are the weighting coefficients for the carbon dioxide gravitational potential field and the infrared gravitational potential field, respectively. The construction of the dangerous repulsive potential field is based on the following calculation formula: , in, A dangerous repulsive potential field. This is the dynamic adjustment coefficient for global risk. The basic rejection coefficient for the i-th type of hazard. This is the shortest distance from the current position of the quadruped robot to the boundary of the hazard source. The radius of influence of the hazard source; Wherein, the global risk dynamic adjustment coefficient The methods for determining this include: The energy integral extracted from the power spectral density within a preset frequency band is expressed as: ;in, and These are the lower and upper limits of the frequency band, respectively. Power spectral density; when When the preset threshold is exceeded, Dynamic adjustment is performed according to the following formula: , in, The preset threshold, This is the maximum energy integral threshold.

Citation Information

Patent Citations

  • Terrain semantic perception method based on vision and vibration tactile fusion

    CN110956651A

  • Life characteristic detection and identification method for rescue robot

    CN112248032A