Method and system for quadruped robot dog to climb industrial stairs

Abstract

The invention discloses a method and system for a quadruped robot dog to climb industrial stairs, and relates to the technical field of intelligent robots. Starting the quadruped robot dog, loading the deep learning model and activating the multi-mode sensor, performing self-inspection on a hardware state to complete initialization, and waiting for receiving a climbing instruction; according to the three-dimensional map, after it is confirmed that a stair exists in front, the height, width and gradient parameters of steps in a stair image obtained by the camera are analyzed through an image processing algorithm, and geometric features of the steps are obtained; dynamically evaluating risk values of all paths by adopting a risk scoring function based on geometric features of steps, selecting the path with the lowest risk to climb, adjusting the posture of the robot dog according to data of the paths, and stepping out front legs to contact a first step; and after reaching a specified position, sending a completion signal to an operator, recording task data, and entering a sleep mode to wait for a subsequent instruction. According to the invention, the autonomous climbing of the quadruped robot dog on the industrial stairway is realized by intelligently identifying the geometrical characteristics of the industrial stairway and dynamically evaluating the risk.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent robots, and particularly to a method and system for a quadruped robot dog to climb industrial stairs. Background Art

[0002] With the rapid development of robot technology, quadruped robot dogs have received extensive attention due to their superior mobility and ability to adapt to complex environments. Especially in the industrial field, the application of quadruped robot dogs is becoming more and more common. In recent years, the progress of deep learning and multi-modal sensor technologies has greatly improved the robot's perception ability of the environment. Existing quadruped robot dogs perform outstandingly on flat ground operations, but in complex environments, especially the climbing ability on vertical structures such as stairs is still insufficient.

[0003] The existing technology lacks a dynamic risk assessment mechanism during the stair climbing process and fails to calculate the risk values of different paths in real time. Many systems fail to effectively integrate multi-modal sensor data, resulting in low environmental perception accuracy and inability to accurately identify the geometric features of steps. In addition, existing methods often rely on static image processing algorithms and fail to make full use of deep learning technology for intelligent analysis. Therefore, aiming at the deficiencies of quadruped robot dogs in complex environments, especially during the industrial stair climbing process, a new method is proposed, which significantly improves the climbing ability and safety of the robot dog. Summary of the Invention

[0004] In view of the above existing problems, the present invention is proposed.

[0005] Therefore, the present invention provides a method for a quadruped robot dog to climb industrial stairs to solve the deficiencies of the quadruped robot dog stair climbing technology in a dynamic environment, especially the technical problems in environmental perception, path planning, and dynamic risk assessment.

[0006] To solve the above technical problems, the present invention provides the following technical solutions:

[0007] In a first aspect, the present invention provides a method for a quadruped robot dog to climb industrial stairs, which includes starting the quadruped robot dog, loading a deep learning model and activating multi-modal sensors, self-checking the hardware status to complete initialization, and waiting to receive a climbing instruction;

[0008] After receiving the climbing instruction, use multi-modal sensors to sense and scan the surrounding environment, and generate a three-dimensional map through edge computing;

[0009] According to the three-dimensional map, after confirming that there is a staircase ahead, analyze the height, width, and inclination parameters of the steps in the staircase image obtained by the camera through an image processing algorithm to obtain the geometric features of the steps;

[0010] Based on the geometric features of the steps, a risk scoring function is used to dynamically evaluate the risk values of all paths, and the path with the lowest risk is selected for climbing. The robotic dog adjusts its posture according to the path data and extends its front leg to touch the first step.

[0011] When the front leg lands, the stability of the foot is judged through the pressure sensor and tactile feedback, and the center of gravity is gradually transferred to the front leg. Then the hind legs follow in turn to climb each step until the designated position is reached.

[0012] After reaching the designated position, a completion signal is sent to the operator and the task data is recorded, and then it enters the sleep mode waiting for subsequent instructions.

[0013] As a preferred solution of the method for a quadruped robotic dog to climb industrial stairs according to the present invention, wherein: the quadruped robotic dog is started, the deep learning model is loaded and the multi-modal sensors are activated, the hardware status is self-checked to complete the initialization, and it waits to receive the climbing instruction. The specific steps are as follows.

[0014] Activate the robotic dog, perform a comprehensive hardware diagnosis through the built-in startup process, and obtain the self-check result.

[0015] Based on the self-check result, load and automatically update the deep learning model to optimize the resource configuration.

[0016] Use the loaded model to activate and calibrate the multi-modal sensors to provide real-time feedback of the environmental information.

[0017] According to the feedback environmental information, dynamically adjust the limb joints and enter the standby mode, waiting to receive the climbing instruction.

[0018] As a preferred solution of the method for a quadruped robotic dog to climb industrial stairs according to the present invention, wherein: after receiving the climbing instruction, use the multi-modal sensors to sense and scan the surrounding environment, and generate a three-dimensional map through edge computing. The specific steps are as follows.

[0019] After receiving the climbing instruction, the multi-modal sensors synchronously collect environmental data to generate the raw point cloud data.

[0020] Align the time stamps of the raw point cloud data, and use the Kalman filter to fuse the inertial measurement unit data and LiDAR data to obtain the fused point cloud data.

[0021] Process the fused point cloud data using the deep learning model to identify the key features of the stairs and obstacles in the environment and perform semantic annotation.

[0022] Based on the semantic annotation results, apply SLAM to stitch the point cloud data into a complete three-dimensional map, forming a three-dimensional map with detailed information about the stair structure.

[0023] Use the graphics processing unit to accelerate the calculation of the three-dimensional map for optimization and simplification.

[0024] As a preferred solution of the method for a quadruped robot dog to climb industrial stairs according to the present invention, wherein: after confirming that there is a staircase ahead based on the three-dimensional map, the height, width, and inclination parameters of the steps in the staircase image obtained by the camera are analyzed through an image processing algorithm to obtain the geometric features of the steps. The specific steps are as follows:

[0025] Identify the staircase image based on the three-dimensional map, and obtain the depth image through the depth sensor;

[0026] Use the adaptive depth map fusion algorithm for the depth image to obtain the depth information of the steps;

[0027] Combine the depth information and the depth image, perform adaptive adjustment using the integral method, introduce the adaptive integral filtering algorithm based on deep learning, and extract the geometric data of the height, width, and inclination of the steps. The expression is as follows:

[0028]

[0029] Where P is the geometric data of the steps extracted, I(x, y) is the pixel intensity value in the staircase image, D(x, y) is the depth value in the depth image, is the x-axis coordinate range of the integral region on the image, is the y-axis coordinate range of the integral region on the image, λ is the adaptive parameter, α is the attenuation factor, is the Gaussian function, dx is the tiny increment along the x-axis during the integral process, and dy is the tiny increment along the y-axis during the integral process;

[0030] Through the adaptive integral filtering algorithm based on deep learning, jointly analyze the depth image and the depth information to obtain the geometric features of the steps.

[0031] As a preferred solution of the method for a quadruped robot dog to climb industrial stairs according to the present invention, wherein: based on the geometric features of the steps, use the risk scoring function to dynamically evaluate the risk values of all paths, select the path with the lowest risk to climb, and the robot dog adjusts its posture according to the data of the path and steps forward the front leg to touch the first step. The specific steps are as follows:

[0032] Introduce the risk scoring function, and the expression is:

[0033]

[0034] Where R is the risk value of the candidate path, T is the time range, t is the time point from the start to any moment, F(t) is the complex summation function related to time, N is the number of steps, and G i is about the height H of the i-th step i and the width W of the i-th stepi , the inclination θ of the i-th step i a complex information filtering function, where i is the step index variable, f(β) is the normalization function, β is the scaling factor, M is the number of time-related factors affecting risk, w j is the importance coefficient assigned to the j-th risk factor, h j (t) is the degree of influence of the j-th risk factor at time point t, and dt is the infinitesimal change in time;

[0035] Calculate the risk values of all paths based on the risk assessment function and set the safety threshold δ;

[0036] If the risk value of the candidate path is greater than the safety threshold, the robotic dog does not continue to move forward. If the risk value R of the candidate path satisfies being less than or equal to δ, select the path with the theoretically minimum risk as the optimal path;

[0037] According to the optimal path, plan the action sequence, adjust its own posture to prepare for climbing, and stretch out the front leg to touch the first step.

[0038] As a preferred solution of the method for a quadruped robotic dog to climb an industrial staircase according to the present invention, wherein: while the front leg lands, judge the foot stability through the pressure sensor and tactile feedback, gradually transfer the center of gravity to the front leg, and the hind legs then follow to climb each step in turn until reaching the designated position. The specific steps are as follows.

[0039] While the front leg lands, activate the pressure sensor and tactile feedback, analyze the frequency-domain distribution of the ground reaction force through Fourier transform, and evaluate the foot stability;

[0040] Based on the foot stability, use the inertial measurement unit and PID controller to gradually move the center of gravity forward to the front leg, and coordinate the hind leg movements through the inverse kinematics algorithm for continuous climbing;

[0041] During continuous climbing, loop through the landing, center-of-gravity transfer, and hind-leg following actions for each step, climb each step in turn until reaching the designated position.

[0042] As a preferred solution of the method for a quadruped robotic dog to climb an industrial staircase according to the present invention, wherein: after reaching the designated position, send a completion signal to the operator and record the task data, and enter the sleep mode to wait for subsequent instructions. The specific steps are as follows.

[0043] After reaching the designated position, activate the wireless communication module and send a completion signal of the position information, timestamp, and task status to the operator through the MQTT protocol;

[0044] While sending the completion signal, record the task data of path planning, pressure distribution, and center-of-gravity transfer trajectory, and analyze the data results using a machine learning model;

[0045] According to the data results, automatically adjust the internal settings through a machine learning model and set up an automatic wake-up mechanism for receiving a new instruction signal;

[0046] Configure the automatic wake-up mechanism, enter the sleep mode, and prepare to receive new instructions.

[0047] In a second aspect, the present invention provides a system for a quadruped robot dog to climb industrial stairs, including an initialization module, an image generation module, an image analysis module, a risk assessment module, a posture adjustment module, and a task completion module;

[0048] The initialization module is used to start the quadruped robot dog, load a deep learning model and activate multimodal sensors, self-check the hardware status to complete initialization, and wait to receive a climbing instruction;

[0049] The image generation module is used to, after receiving the climbing instruction, use multimodal sensors to sense and scan the surrounding environment, and generate a three-dimensional map through edge computing;

[0050] The image analysis module is used to, according to the three-dimensional map, after confirming that there are stairs ahead, analyze the height, width, and inclination parameters of the steps in the stair image obtained by the camera through an image processing algorithm to obtain the geometric features of the steps;

[0051] The risk assessment module is used to dynamically evaluate the risk values of all paths based on the geometric features of the steps by using a risk scoring function, select the path with the lowest risk to climb, and the robot dog adjusts its posture according to the data of the path and steps forward its front legs to touch the first step;

[0052] The posture adjustment module is used to, when the front legs land, judge the stability of the feet through pressure sensors and tactile feedback, gradually transfer the center of gravity to the front legs, and then the hind legs follow to climb each step in turn until reaching the designated position;

[0053] The task completion module is used to, after reaching the designated position, send a completion signal to the operator and record the task data, and enter the sleep mode to wait for subsequent instructions.

[0054] In a third aspect, the present invention provides a computer device, including a memory and a processor, where the memory stores a computer program, and: when the computer program is executed by the processor, any step of the method for a quadruped robot dog to climb industrial stairs as described in the first aspect of the present invention is implemented.

[0055] In a fourth aspect, the present invention provides a computer-readable storage medium, on which a computer program is stored, and: when the computer program is executed by the processor, any step of the method for a quadruped robot dog to climb industrial stairs as described in the first aspect of the present invention is implemented.

[0056] The beneficial effects of the present invention are as follows: By combining a deep learning model with multi-modal sensors, the present invention can perceive and generate a three-dimensional map of the environment in real time, and then accurately identify the geometric data of the steps. An innovative risk scoring function is used to dynamically evaluate all possible paths, and a path with the theoretically lowest risk is selected for the quadruped robot to climb. During the climbing process, the quadruped robot can judge the stability of its feet based on the pressure sensing information and tactile feedback in real time, and reasonably distribute the center of gravity to ensure the safety of each step. After reaching the designated position, the quadruped robot will automatically record the relevant data of this task and enter the sleep mode to save energy, waiting for the next instruction. The whole process not only improves the reliability and efficiency of the quadruped robot working in an industrial environment, but also enhances its ability of autonomous decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0058] Figure 1 It is a flowchart of the method for a quadruped robot to climb an industrial staircase in Embodiment 1.

[0059] Figure 2 It is a module diagram of the system for a quadruped robot to climb an industrial staircase in Embodiment 1. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0060] In order to make the above objects, features, and advantages of the present invention more obvious and understandable, the following will make a detailed description of the specific embodiments of the present invention with reference to the accompanying drawings of the specification.

[0061] Many specific details are set forth in the following description in order to provide a thorough understanding of the present invention. However, the present invention may be implemented in other ways different from those described herein. Those skilled in the art can make similar extensions without departing from the connotation of the present invention. Therefore, the present invention is not limited by the specific embodiments disclosed below.

[0062] Secondly, the so-called "one embodiment" or "embodiment" herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation manner of the present invention. The appearances of "in one embodiment" in different places in this specification do not all refer to the same embodiment, nor are they separate or alternative embodiments that exclude other embodiments.

[0063] Embodiment 1, refer to Figure 1 and Figure 2, which is the first embodiment of the present invention. This embodiment provides a method for a quadruped robot dog to climb industrial stairs, including the following steps:

[0064] S1. Start the quadruped robot dog, load the deep learning model and activate the multi-modal sensors, complete the initialization by self-checking the hardware status, and wait for the climbing instruction to be received;

[0065] Furthermore, activate the robot dog, perform a comprehensive hardware diagnosis through the built-in startup process, and obtain the self-check result;

[0066] The built-in startup process is a self-check mechanism that will sequentially perform hardware diagnosis. First, it performs a self-check on the power management unit, tests the connection between the wireless communication module and the control system, checks the read and write functions of the built-in memory, performs a self-check on the embedded software, confirms that each sensor can work normally, and ensures that all moving parts can operate normally. Through this step, a comprehensive inspection of the internal hardware status of the robot dog is carried out, potential hardware failures and performance bottlenecks are identified, a status baseline is provided for subsequent operations, operation failures and inefficiencies caused by hardware problems are prevented, reliability and stability are improved, the occurrence probability of on-site failures is reduced, and the service life of the robot dog is extended.

[0067] Based on the self-check result, load and automatically update the deep learning model to optimize resource allocation;

[0068] Among them, according to the data results obtained from the self-check, select the ResNet50 deep learning model adapted to the hardware condition and perform automatic update when necessary, so as to keep the learning ability and adaptability of the robot dog always at the forefront level, and at the same time reasonably allocate computing resources to support efficient operation, improving the task execution accuracy and response speed of the robot dog.

[0069] Utilize the loaded model to activate and calibrate the multi-modal sensors to provide real-time feedback of environmental information;

[0070] Specifically, through the loaded deep learning model, accurately control and calibrate the multi-modal sensors. The multi-modal sensors can capture various environmental data including vision, hearing, and touch, providing the robot dog with all-round perception ability, enabling the robot dog to understand and adapt to the surrounding environment in real time, and thus making more informed decisions. The environmental adaptability is enhanced, enabling the robot dog to autonomously navigate and perform tasks in complex and dynamic environments, improving the success rate of task completion.

[0071] According to the feedback environmental information, dynamically adjust the limb joints and enter the standby mode, waiting for the climbing instruction to be received.

[0072] S2. After receiving the climbing instruction, use the multi-modal sensors to sense and scan the surrounding environment, and generate a three-dimensional map through edge computing;

[0073] Furthermore, after receiving the climbing instruction, the multi-modal sensor synchronously collects environmental data to generate raw point cloud data;

[0074] Trigger the synchronous operation of the multi-modal sensor by receiving the climbing instruction for immediate perception and data collection of the surrounding environment. Through the sensor's inertial measurement unit and LiDAR, the surrounding environment can be comprehensively perceived, ensuring the accuracy, integrity, and real-time nature of the data. It also guarantees the time consistency of data from different sensors, providing accurate and comprehensive basic data for subsequent data processing and enabling quick response to user commands.

[0075] Align the timestamps of the raw point cloud data, and use the Kalman filter to fuse the data from the inertial measurement unit and LiDAR data to obtain the fused point cloud data;

[0076] Among them, timestamp alignment ensures the consistency of data from different sensors in the same time dimension, improves the accuracy of data fusion, and avoids errors caused by time delay. The Kalman filter can effectively reduce noise when processing dynamic data, especially when fusing data from the inertial measurement unit and LiDAR, achieving high-precision data fusion. The fused point cloud data integrates information from multiple sensors, enhancing the reliability and information content of the data and providing a richer input for subsequent processing.

[0077] Process the fused point cloud data using a deep learning model to identify the key features of stairs and obstacles in the environment and perform semantic annotation;

[0078] Specifically, by applying deep learning algorithms to the fused point cloud data, accurately identify and classify the key features of stairs and potential obstacles in the environmental structure, endowing the robot dog with the ability to understand the operating environment, distinguish safe paths and dangerous areas. Semantic annotation gives a high-level understanding to the environmental data, enabling the robot dog to make intelligent decisions in complex environments. In a complex environment, the ResNet50 deep learning model can accurately identify stairs and obstacles, significantly improving the navigation ability of the robot dog in a dynamic environment.

[0079] Based on the semantic annotation results, apply SLAM to stitch the point cloud data into a complete three-dimensional map, forming a three-dimensional map with detailed information about the stair structure;

[0080] It should be noted that SLAM improves the positioning accuracy based on the semantic annotation results, uses data from inertial measurement units, LiDAR, and other sensors to estimate the movement trajectory of the robot dog itself, and aligns the newly acquired point cloud data with the existing map to ensure the consistency and accuracy of the map. By continuously updating and expanding the map, a three-dimensional map of the detailed information of the staircase structure is finally formed. The three-dimensional map not only includes the geometric shape of the object but also contains semantic labels. The detailed three-dimensional map helps to predict the possible risk areas in advance, enabling the robot dog to take preventive measures before the danger occurs.

[0081] The three-dimensional map is accelerated and calculated using a graphics processing unit for optimization and simplification.

[0082] S3. After confirming that there is a staircase ahead according to the three-dimensional map, analyze the height, width, and inclination parameters of the steps in the staircase image obtained by the camera through an image processing algorithm to obtain the geometric features of the steps;

[0083] Furthermore, identify the staircase image according to the three-dimensional map and obtain the depth image through a depth sensor;

[0084] Preferably, the depth sensor is a device used to measure the distance between an object and the sensor, usually using laser, ultrasonic, or infrared technology. The depth sensor is used here to generate a real-time depth image. The use of the depth sensor improves the robot's ability to identify and locate in complex environments, especially in insufficient light, where the depth sensor can provide stable environmental information.

[0085] Use an adaptive depth map fusion algorithm for the depth image to obtain the depth information of the steps;

[0086] Among them, the adaptive depth map fusion algorithm dynamically adjusts the fusion strategy through the characteristics of the depth image, integrates the depth data from different sensors, improves the quality of the depth image, can effectively eliminate the differences and noises between sensors, and makes the obtained depth information of the steps more reliable. Especially in the case of inconsistent sensor performance, it significantly improves the accuracy of the depth information of the steps.

[0087] Combine the depth information and the depth image, perform adaptive adjustment using the integral method, introduce an adaptive integral filtering algorithm based on deep learning, and extract the height, width, and inclination geometric data of the steps. The expression is as follows:

[0088]

[0089] Among them, P is the extracted geometric data of the steps, I(x, y) is the pixel intensity value in the staircase image, D(x, y) is the depth value in the depth image, is the x-axis coordinate range of the integration region on the image, is the y-axis coordinate range of the integration region on the image, λ is the adaptive parameter, and α is the attenuation factor. is the Gaussian function, dx is the tiny increment along the x-axis during the integration process, and dy is the tiny increment along the y-axis during the integration process;

[0090] It should be noted that the adaptive integral filtering algorithm combines the integral method and deep learning, and can adaptively adjust the filtering process according to the characteristics of the input data. By introducing deep learning, the adaptive integral filtering algorithm can more effectively identify and extract the height, width, and inclination of the steps when processing complex data, improving the adaptability to changes in the dynamic environment, so that the robot dog can perform tasks more accurately in practical applications.

[0091] Specifically, the introduction of the adaptive parameter λ is used to adjust the weight balance between the depth values in the depth image and the pixel intensity values in the staircase image. Through machine learning, the value of λ is adjusted in real time according to the environmental conditions. The introduction of the attenuation factor α is mainly used to control the spatial distribution characteristics of the Gaussian function, defining the attenuation rate of the influence from the center to the edge. Combining environmental perception information, the value of α can be dynamically adjusted. By carefully designing and constructing a method for extracting the geometric features of the steps, this method improves the intelligent level and provides strong technical support for the navigation of the robot dog in complex environments.

[0092] Through the adaptive integral filtering algorithm based on deep learning, the depth image and depth information are jointly analyzed to obtain the geometric features of the steps.

[0093] Among them, the joint analysis is to integrate and compare the depth image data and depth information data to obtain more geometric features of the steps. By comprehensively applying the deep learning model and the adaptive integral filtering algorithm, the joint analysis of the depth image and the corresponding depth information effectively integrates the information from different data sources, enables the robot dog to intelligently understand the environment, reduces the dependence on a single data source, and improves the accuracy and real-time performance of decision-making.

[0094] S4. Based on the geometric features of the steps, use the risk scoring function to dynamically evaluate the risk values of all paths, select the path with the lowest risk to climb, and the robot dog adjusts its posture according to the data of the path and steps forward with its front leg to touch the first step;

[0095] Furthermore, introduce the risk scoring function, and the expression is:

[0096]

[0097] Among them, R is the risk value of the candidate path, T is the time range, t is the time point from the start to any moment, F(t) is a time-related complex summation function, N is the number of steps, and G i is related to the height H of the i-th stepi , the width W of the i-th step i , the inclination θ of the i-th step i A complex information filtering function, where i is the step index variable, f(β) is the normalization function, β is the scaling factor, M is the number of time-related factors affecting risk, and w j is the importance coefficient assigned to the j-th risk factor, and h j (t) is the degree of influence of the j-th risk factor at time point t, and dt is the infinitesimal change in time;

[0098] It should be noted that the scaling factor is set according to the characteristics of the specific environment and is automatically learned from historical data through machine learning algorithms. According to the characteristics of indoor and outdoor stairs, a set of β value ranges is preset. The normalization function f(β) is used to ensure that after all step geometric features are adjusted by β, their contributions are on the same order of magnitude, avoiding unreasonable weight distribution caused by unit and scale differences. The function G i indicates that the risk value of each step is the result of the combined action of the step geometric features and the scaling factor.

[0099] Specifically, a risk scoring function is introduced to quantify the potential risks of height, width, and inclination factors, the higher-order integral function is combined to integrate time-related risk factors, the cumulative effect of different risk factors over time is calculated through a complex summation function, and the complex information filtering function is used to screen and process the step geometric features. Through these methods, the risk values of each candidate path are measured, a reliable risk assessment is obtained, enabling the robotic dog to make more intelligent and safe choices in complex dynamic environments and enhancing its ability to cope with unknown environments.

[0100] Based on the risk assessment function, the risk values of all paths are calculated, and a safety threshold δ is set;

[0101] Preferably, first collect historical data on ground types, obstacle distributions, slopes, and wetness levels under different environmental conditions. By analyzing these historical data, identify the key factors affecting path safety. Then, predict the safety of different paths through the risk scoring function to form a quantification of the path risk values. According to the risk value results calculated by the risk assessment function, set a safety threshold lower than the risk values of all known safe paths to ensure that any path with a risk value higher than the safety threshold is considered to have a greater risk and should be avoided. The safety threshold provides an objective criterion for judging which paths are safe. The quantitative assessment of all candidate paths based on the risk assessment function simplifies the path selection process, ensures the safety of the selected paths, reduces the accident rate, and at the same time improves the speed and accuracy of path planning.

[0102] The risk value of the candidate path is greater than the safety threshold, and the robotic dog does not move forward. When the risk value R of the candidate path satisfies being less than or equal to δ, the path with the theoretically lowest risk is selected as the optimal path;

[0103] Among them, by comparing the risk values of each candidate path with the preset safety threshold, the safety of the candidate paths is verified, and the path with the lowest risk is selected as the optimal path. This strategy selects the one with the theoretically lowest risk from multiple low-risk paths, which can prevent the robotic dog from entering high-risk areas and ensure the effect of task execution.

[0104] According to the optimal path, plan the action sequence, adjust its own posture to prepare for climbing, and step forward the front leg to touch the first step.

[0105] Preferably, analyze the data of the optimal path, identify the step edges, height changes and their precise positions on the optimal path, formulate the action sequence according to the path analysis results, adjust the body posture of the robotic dog, as well as the positions and angles of the four limbs, and step forward the front leg to touch and support on the step surface. Through effective action planning and posture adjustment, the robotic dog can maintain stability in a dynamic step environment and reduce the risk of falling and collision.

[0106] S5. While the front leg lands, judge the stability of the foot through the pressure sensor and tactile feedback, gradually transfer the center of gravity to the front leg, and then the hind legs follow to climb each step in turn until the designated position is reached;

[0107] Furthermore, while the front leg lands, activate the pressure sensor and tactile feedback, analyze the frequency-domain distribution of the ground reaction force through Fourier transform, and evaluate the stability of the foot;

[0108] Among them, the pressure sensor is a device for measuring the force applied to its surface, which can provide real-time data of the ground reaction force. The tactile feedback is combined with the pressure sensor to provide the pressure distribution of the instant feedback on the ground state, enabling the robotic dog to perceive minute surface changes and make rapid responses in a dynamic environment, improving its adaptability in complex terrains. Through Fourier transform, the robotic dog identifies the reaction force patterns at different frequencies, realizes the measurement of the mechanical characteristics at the moment when the foot contacts the ground, especially identifies different frequency components and their intensities through frequency-domain analysis, so as to judge the ground stability.

[0109] Based on the stability of the foot, use the inertial measurement unit and PID controller to gradually move the center of gravity forward to the front leg, and coordinate the actions of the hind legs through the inverse kinematics algorithm for continuous climbing;

[0110] Specifically, the inertial measurement unit is a sensor combination used to measure the acceleration and angular velocity of an object, which is used to obtain the posture information of the robot dog. The PID controller is a feedback controller that uses three control methods: proportional, integral and differential to adjust the center of gravity transfer of the robot dog. The combination of the inertial measurement unit and the PID controller enables the robot dog to accurately track the position and posture, ensuring that it can quickly respond to environmental changes during the climbing process. The inverse kinematics algorithm is used to calculate the angles of each joint required to reach the target position and coordinate the movement of the hind legs, so that the robot dog can flexibly adjust the movement strategy of the hind legs in a complex movement environment, achieve natural climbing movements, and enhance flexibility and adaptability in practical applications.

[0111] During continuous climbing, cycle through the steps of landing, weight transfer, and following with the back leg, climbing each step in turn until you reach the designated position.

[0112] S6. After reaching the designated position, send a completion signal to the operator and record the task data, then enter sleep mode and wait for subsequent instructions.

[0113] Furthermore, after reaching the designated location, the wireless communication module is activated to send the location information, timestamp and completion signal of the task status to the operator through the MQTT protocol;

[0114] The MQTT protocol is a lightweight message transmission protocol suitable for low-bandwidth, high-latency or unreliable network scenarios. Using the MQTT protocol to efficiently transmit the robot dog's location information and status signals makes data transmission more efficient, especially in poor network conditions, and can ensure reliable information transmission, thereby improving the robot dog's adaptability in dynamic environments.

[0115] While sending the completion signal, the task data of path planning, pressure distribution and center of gravity transfer trajectory are recorded, and the data results are analyzed using machine learning models;

[0116] It should be noted that machine learning models are tools that use algorithms to learn from data and make predictions. Machine learning models are used to analyze the path planning, pressure distribution, and center of gravity transfer trajectory task data of the robot dog during task execution. Machine learning models are used to discover potential problems in the data and optimize the robot dog's internal settings and decision-making mechanisms. This self-optimization capability is particularly prominent in complex and rapidly changing tasks.

[0117] Based on the data results, the machine learning model automatically adjusts the internal settings and sets an automatic wake-up mechanism when receiving new command signals;

[0118] Among them, the automatic wake-up mechanism is a technology that enables the device to resume operation from the sleep mode under specific conditions. By intelligently adjusting the internal settings, it is possible to save energy and other resources without affecting performance. Set the automatic wake-up mechanism to allow the robotic dog to maintain a certain level of alertness in the sleep state, ensuring that it can start working immediately once a new task instruction arrives.

[0119] Configure the automatic wake-up mechanism, enter the sleep mode, and prepare to receive new instructions.

[0120] This embodiment also provides a system for a quadruped robotic dog to climb industrial stairs, including: an initialization module, an image generation module, an image analysis module, a risk assessment module, a posture adjustment module, and a task completion module; the initialization module is used to start the quadruped robotic dog, load the deep learning model and activate the multi-modal sensors, self-check the hardware status to complete the initialization, and wait to receive the climbing instruction; the image generation module is used to, after receiving the climbing instruction, use the multi-modal sensors to sense and scan the surrounding environment, and generate a three-dimensional map through edge computing; the image analysis module is used to, based on the three-dimensional map, after confirming that there are stairs ahead, analyze the height, width, and inclination parameters of the steps in the stair image obtained by the camera through an image processing algorithm to obtain the geometric features of the steps; the risk assessment module is used to, based on the geometric features of the steps, dynamically evaluate the risk values of all paths using a risk scoring function, select the path with the lowest risk to climb, and the robotic dog adjusts its posture according to the data of the path and steps forward with its front legs to touch the first step; the posture adjustment module is used to, while the front legs land, judge the stability of the feet through the pressure sensors and tactile feedback, gradually transfer the center of gravity to the front legs, and then the hind legs follow to climb each step in turn until the designated position is reached; the task completion module is used to, after reaching the designated position, send a completion signal to the operator and record the task data, and enter the sleep mode to wait for subsequent instructions.

[0121] This embodiment also provides a computer device applicable to the case of the method for a quadruped robotic dog to climb industrial stairs, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to implement the method for a quadruped robotic dog to climb industrial stairs as proposed in the above embodiment.

[0122] The computer device can be a terminal, which includes a processor, a memory, a communication interface, a display screen, and an input device connected via a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The communication interface of the computer device is used to communicate with external terminals in a wired or wireless manner, and the wireless manner can be implemented through WIFI, carrier network, NFC (Near Field Communication), or other technologies. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads provided on the housing of the computer device, or an external keyboard, touchpad, or mouse, etc.

[0123] This embodiment also provides a storage medium, on which a computer program is stored. When the program is executed by a processor, it implements the method for a quadruped robot dog to climb an industrial staircase as proposed in the above embodiment; the storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (Static Random Access Memory, abbreviated as SRAM), electrically erasable programmable read-only memory (Electrically Erasable Programmable Read-Only Memory, abbreviated as EEPROM), erasable programmable read-only memory (Erasable Programmable Read Only Memory, abbreviated as EPROM), programmable read-only memory (Programmable Red-Only Memory, abbreviated as PROM), read-only memory (Read-Only Memory, abbreviated as ROM), magnetic memory, flash memory, a magnetic disk, or an optical disc.

[0124] In summary, in the present invention: By combining a deep learning model with multi-modal sensors, it can sense the environment in real time and generate a three-dimensional map of the environment, and then accurately identify the geometric data of the steps. An innovative risk scoring function is used to dynamically evaluate all possible paths, and a path with the theoretically lowest risk is selected for the robot dog to climb. During the climbing process, the robot dog can judge the stability of the feet based on the pressure sensing information and tactile feedback in real time, and reasonably distribute the center of gravity to ensure the safety of each step. After reaching the designated position, the robot dog will automatically record the relevant data of this task and enter the sleep mode to save energy and wait for the next instruction. The whole process not only improves the reliability and efficiency of the robot dog working in an industrial environment, but also enhances its ability of autonomous decision-making.

[0125] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the technical solutions of the present invention, and they should all be covered by the scope of the claims of the present invention.

Claims

1. A method for a quadruped robot dog to climb an industrial staircase, characterized in that: including, Start the quadruped robot dog, load the deep learning model and activate the multi-modal sensors, self-check the hardware status to complete the initialization, and wait for the climbing instruction to be received; After receiving the climbing instruction, use the multi-modal sensors to sense and scan the surrounding environment, and generate a 3D map through edge computing; According to the 3D map, after confirming that there is a staircase ahead, analyze the height, width, and inclination parameters of the steps in the staircase image obtained by the camera through the image processing algorithm to obtain the geometric features of the steps; Based on the geometric features of the steps, use the risk scoring function to dynamically evaluate the risk values of all paths, select the path with the lowest risk to climb, and the robot dog adjusts its posture according to the data of the path and steps forward the front leg to touch the first step; At the same time as the front leg lands, judge the stability of the foot through the pressure sensor and tactile feedback, gradually transfer the center of gravity to the front leg, and the hind legs then follow to climb each step in turn until the specified position is reached; After reaching the specified position, send a completion signal to the operator and record the task data, and enter the sleep mode to wait for subsequent instructions.

2. The method for a quadruped robot dog to climb an industrial staircase according to claim 1, wherein: The steps of starting the quadruped robot dog, loading the deep learning model and activating the multi-modal sensors, self-checking the hardware status to complete the initialization, and waiting for the climbing instruction are as follows: Activate the robot dog, perform hardware diagnosis through the built-in startup process, and obtain the self-check result; Based on the self-check result, load and automatically update the deep learning model to optimize the resource configuration; Use the loaded model to activate and calibrate the multi-modal sensors to provide real-time feedback of environmental information; According to the feedback environmental information, dynamically adjust the limb joints, enter the standby mode, and wait for the climbing instruction to be received.

3. The method for a quadruped robot dog to climb an industrial staircase according to claim 2, characterized in that: The steps of using the multi-modal sensors to sense and scan the surrounding environment and generate a 3D map after receiving the climbing instruction are as follows: After receiving the climbing instruction, the multi-modal sensors synchronously collect environmental data to generate raw point cloud data; Align the timestamps of the raw point cloud data, and fuse the inertial measurement unit data and LiDAR data using the Kalman filter to obtain the fused point cloud data; Process the fused point cloud data using the deep learning model to identify the key features of the stairs and obstacles in the environment and perform semantic annotation; Based on the semantic annotation results, apply SLAM to stitch the point cloud data into a complete 3D map to form a 3D map with detailed information about the staircase structure; Use the graphics processing unit to accelerate the calculation of the 3D map for optimization and simplification.

4. The method for a quadruped robot dog to climb an industrial staircase according to claim 3, characterized in that: The steps of identifying the staircase image according to the 3D map, analyzing the height, width, and inclination parameters of the steps in the staircase image obtained by the camera through the image processing algorithm to obtain the geometric features of the steps are as follows: Identify the staircase image according to the 3D map and obtain the depth image through the depth sensor; Use the adaptive depth map fusion algorithm for the depth image to obtain the depth information of the steps; Combine the depth information with the depth image, perform adaptive adjustment using the integral method, introduce the adaptive integral filtering algorithm based on deep learning, and extract the geometric data of the height, width, and inclination of the steps. The expression is as follows: Where P is the extracted step geometric data, I(x, y) is the pixel intensity value in the staircase image, and D(x, y) is the depth value in the depth image. is the x-axis coordinate range of the integration region on the image. is the y-axis coordinate range of the integration region on the image, λ is the adaptive parameter, and α is the attenuation factor. is the Gaussian function, dx is the tiny increment along the x-axis during the integration process, and dy is the tiny increment along the y-axis during the integration process. Through the adaptive integral filtering algorithm based on deep learning, jointly analyze the depth image and depth information to obtain the geometric features of the steps.

5. The method for a quadruped robot dog to climb an industrial staircase according to claim 4, characterized in that: Based on the geometric features of the steps, a risk scoring function is used to dynamically evaluate the risk values of all paths, and the path with the lowest risk is selected for climbing. The robotic dog adjusts its posture according to the risk value data of the path and extends its front leg to touch the first step. The specific steps are as follows: Introduce a risk scoring function, and the expression is: Where, R is the risk value of the candidate path, T is the time range, t is the time point from the start to any moment, F(t) is a time-related complex summation function, N is the number of steps, G i is the complex information filtering function regarding the height H i of the i-th step, the width W i of the i-th step, and the inclination θ i of the i-th step, i is the step index variable, f(β) is the normalization function, β is the scale factor, M is the number of time-related factors affecting the risk, w j is the importance coefficient assigned to the j-th risk factor, h j (t) is the influence degree of the j-th risk factor at the time point t, and dt is the infinitesimal change of time; Calculate the risk values of all paths based on the risk assessment function, and set a safety threshold δ; If the risk value of the candidate path is greater than the safety threshold, the robotic dog does not move forward. When the risk value R of the candidate path satisfies less than or equal to δ, select the candidate path with the minimum risk value as the optimal path; According to the optimal path, plan the action sequence, adjust its own posture to prepare for climbing, and extend the front leg to touch the first step.

6. The method for a quadruped robot dog to climb an industrial staircase according to claim 5, characterized in that: While the front leg lands, judge the stability of the foot through the pressure sensor and tactile feedback, gradually transfer the center of gravity to the front leg, and then the hind legs follow to climb each step in turn until the designated position is reached. The specific steps are as follows: While the front leg lands, activate the pressure sensor and tactile feedback, analyze the frequency domain distribution of the ground reaction force through Fourier transform, and evaluate the stability of the foot; Based on the stability of the foot, use the inertial measurement unit and PID controller to gradually move the center of gravity forward to the front leg, and coordinate the actions of the hind legs through the inverse kinematics algorithm for continuous climbing; During the continuous climbing process, loop through the landing, center of gravity transfer, and hind leg following actions of each step, and climb each step in turn until the designated position is reached.

7. The method for a quadruped robot dog to climb an industrial staircase according to claim 6, characterized in that: After reaching the designated position, send a completion signal to the operator and record the task data, and enter the sleep mode to wait for subsequent instructions. The specific steps are as follows: After reaching the designated position, activate the wireless communication module, and send a completion signal of the position information, timestamp, and task status to the operator through the MQTT protocol; While sending the completion signal, record the task data of path planning, pressure distribution, and center of gravity transfer trajectory, and analyze the data results using a machine learning model; According to the data results, automatically adjust the internal settings through the machine learning model and set an automatic wake-up mechanism for receiving a new instruction signal; Configure the automatic wake-up mechanism, enter the sleep mode, and prepare to receive new instructions.

8. A system for a quadruped robot dog to climb industrial stairs, based on the method for a quadruped robot dog to climb industrial stairs according to any one of claims 1 to 7, characterized in that: Including an initialization module, an image generation module, an image analysis module, a risk assessment module, a posture adjustment module, and a task completion module; The initialization module is used to start the quadruped robotic dog, load the deep learning model and activate the multi-modal sensors, self-check the hardware status to complete the initialization, and wait for the climbing instruction; The image generation module is used to sense and scan the surrounding environment using the multi-modal sensors after receiving the climbing instruction, and generate a 3D map through edge computing; The image analysis module is used to confirm that there is a staircase ahead according to the 3D map, and analyze the height, width, and inclination parameters of the steps in the staircase image obtained by the camera through the image processing algorithm to obtain the geometric features of the steps; The risk assessment module is used to dynamically evaluate the risk values of all paths based on the geometric features of the steps, select the path with the lowest risk for climbing, and the robotic dog adjusts its posture according to the data of the path and extends its front leg to touch the first step; The posture adjustment module is used to, while the front legs land, judge the stability of the feet through pressure sensors and tactile feedback, gradually transfer the center of gravity to the front legs, and then the hind legs follow to climb each step in turn until the designated position is reached; The task completion module is used to, after reaching the designated position, send a completion signal to the operator, record the task data, and enter the sleep mode to wait for subsequent instructions.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the method for a quadruped robot dog to climb an industrial staircase according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the method for a quadruped robot dog to climb an industrial staircase according to any one of claims 1 to 7.

Images