Anti-falling control method and device for quadruped robot in tunnel scene

Abstract

The invention provides an anti-falling control method and device for a quadruped robot in a tunnel scene. According to the method, based on point cloud depth data acquired by a laser radar, an advancing strategy of the quadruped robot is obtained by inputting a first motion strategy model trained by adopting a DQN algorithm and a second motion strategy model comprising an SLAM positioning and navigation module. The problem that in the prior art, in a power cable tunnel scene, due to a tunnel and a high platform, a quadruped robot cannot work after falling off is solved.

Description

Technical Field

[0001] The present invention relates to the field of robot control, and in particular to an anti-fall control method for a quadruped robot in a tunnel scene, an anti-fall control device for a quadruped robot in a tunnel scene, a computer-readable storage medium, a processor and an electronic device. Background Art

[0002] Power equipment is an important pillar of modern society. The maintenance technology that ensures the safe operation of power equipment is an important key technology in my country's power industry. In the complex and hazardous operation scenarios of the power industry, new robot technology has huge application potential and demand. As my country's power system becomes more and more complex, the maintenance technology, safety assurance and intelligent requirements of power equipment are gradually improved, and the application of new robot technology in this field will be more extensive and in-depth. However, this field often faces harsh operating environments and a variety of potential dangerous factors. Equipment maintenance and overhaul work requires highly specialized personnel in a high-risk environment. There are a series of technical and operational challenges such as personal safety hazards and low efficiency. These have increased the urgency and importance of the research and development of key technologies for the maintenance of new robots for power equipment. Therefore, key technologies such as new robot operation, maintenance and overhaul of power equipment will be the focus of my country's power industry.

[0003] Power cables and tunnels are the main carriers of power transmission and power supply security in the capital's power grid. Due to historical reasons, some tunnels are aging and in disrepair. In addition, due to the harsh underground environment, personnel inspections are difficult and risky, and traditional inspection methods require a lot of manpower and material resources.

[0004] Quadruped robots have superior obstacle-crossing and adaptive capabilities, and their requirements for ground conditions are much lower than those for wheeled robots. They have broad application prospects in cable tunnel environments. However, in narrow power cable tunnels, especially in some tunnels, the presence of tunnel factors will cause the quadruped robot to be unable to work autonomously once it falls, and can only be intervened manually, which is time-consuming and labor-intensive, greatly affecting the stability and efficiency of the robot's autonomous inspection work. Therefore, under uncertain conditions, how to reduce the probability of a quadruped robot falling from a high platform into a tunnel and improve the working reliability of the quadruped robot is one of the keys to improving the inspection efficiency in cable tunnels. Summary of the invention

[0005] The main purpose of the present application is to provide a method for preventing a quadruped robot from falling in a tunnel scenario, an anti-fall control device for a quadruped robot in a tunnel scenario, a computer-readable storage medium and a quadruped robot, so as to at least solve the problem in the prior art that in a power cable tunnel scenario, the quadruped robot cannot work after falling due to tunnels and platforms.

[0006] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a fall prevention control method for a quadruped robot in a tunnel scene is provided, including: acquiring point cloud data centered on the quadruped robot through a laser radar to obtain point cloud depth data; inputting the point cloud depth data into a first motion strategy model to obtain a first moving strategy, the first moving strategy at least includes the angular velocity and linear velocity of the quadruped robot, and the first motion strategy model is a model trained based on a DQN algorithm and generates a moving mode based on the surrounding environment of the quadruped robot; inputting the point cloud depth data into a second motion strategy model to obtain a second moving strategy, the second moving strategy at least includes the angular velocity and linear velocity of the quadruped robot, and the second motion strategy model includes a SLAM positioning module and a navigation module; performing a weighted average operation based on the first moving strategy and the second moving strategy to obtain a target control strategy, and controlling the movement of the quadruped robot according to the target control strategy.

[0007] Optionally, before inputting the point cloud depth data into the first motion strategy model, the method further includes: obtaining the tunnel width, the pit width and the distance between the quadruped robot and the tunnel wall on one side of the pit to obtain a first target distance, a second target distance and a third target distance, obtaining the initial position of the quadruped robot to obtain a first position, obtaining the terminal position of the quadruped robot to obtain a second position; determining the first target distance, the second target distance, the third target distance, the first position and the second position as the initial state of the model; determining the angular velocity and the linear velocity of the quadruped robot as behaviors, and randomly taking values ​​for the angular velocity and the linear velocity to obtain multiple first target behaviors; determining multiple update states based on the initial state and each of the first target behaviors, and updating each of the update states based on preset rules. The new state is scored to obtain a state reward; the initial state, the first target behavior, the state reward, the updated state, and whether the first position coincides with the second position are determined as an experience group, and stored in an experience pool; the neural network is trained based on the samples in the experience pool, and the Q value is calculated based on the target Q network, the maximum Q value corresponding to the updated state is determined as the target Q value, the neural network is updated based on the target Q value, and the target Q network is updated based on the neural network after a first preset number of intervals; the updated state is determined as the initial state, and the first target behavior and the updated state and the experience group corresponding to the updated state are regenerated based on the iterated initial state, and the neural network is trained according to the iterated updated state and the experience group until the number of iterations reaches a second preset number of times, so as to obtain the first motion strategy model.

[0008] Optionally, before scoring each of the update states based on preset rules to obtain a state reward, the method further includes: calculating the product of the first target distance and a first preset coefficient to obtain a fourth target distance, and calculating the product of the second target distance and a second preset coefficient to obtain a fifth target distance; determining the first preset rule as follows: when the third target distance in the update state is less than or equal to the fifth target distance, determining the first preset score as the state reward; determining the second preset rule as follows: when the third target distance in the update state is greater than the fifth target distance and less than the fourth target distance, determining the second preset score as the state reward; determining the third preset rule as follows: when the third target distance in the update state is greater than or equal to the fourth target distance, determining the second preset score as the state reward. The fourth preset rule is determined as follows: when the distance between the first position and the second position decreases in the updated state, the fourth preset score is determined as the state reward; the fifth preset rule is determined as follows: when the distance between the first position and the second position increases in the updated state, the fifth preset score is determined as the state reward; the sixth preset rule is determined as follows: when the quadruped robot collides in the updated state, the first preset score is determined as the state reward, the first preset score is less than the fifth preset score, the fifth preset score is less than the third preset score, the third preset score is less than the fourth preset score, and the fourth preset score is less than the second preset score.

[0009] Optionally, the point cloud depth data is input into a first motion strategy model to obtain a first moving strategy, including: updating the first target distance, the second target distance and the third target distance based on the point cloud depth data; determining the first position and the second position based on the point cloud depth data; determining the first target distance, the second target distance, the third target distance, the first position and the second position as the current state; randomly generating the angular velocity and the linear velocity of the quadruped robot based on the current state to obtain multiple alternative moving strategies; determining the corresponding predicted state based on the current state and each of the alternative moving strategies, and determining the state reward corresponding to the predicted state to obtain an alternative strategy score; determining the alternative moving strategy corresponding to the maximum value of the alternative strategy score as the first moving strategy.

[0010] Optionally, the point cloud depth data is input into a second motion strategy model to obtain a second travel strategy, including: inputting the point cloud depth data into the SLAM positioning module to obtain the current posture of the quadruped robot and a target map constructed based on the point cloud depth data; inputting the current posture and the target map into the navigation module to predict the angular velocity and the linear velocity of the quadruped robot within a preset time length to obtain the second travel strategy.

[0011] Optionally, the point cloud depth data is input into the SLAM positioning module to obtain the current posture of the quadruped robot and a target map constructed based on the point cloud depth data, including: performing feature extraction on the point cloud depth data based on the SLAM algorithm, determining the points therein that match the preset objects, and obtaining key feature points; determining the key feature points obtained when the feature extraction on the point cloud depth data was performed last time as comparison feature points, and determining the travel trajectory of the quadruped robot based on the key feature points and the comparison feature points; performing map reconstruction based on the key feature points and the travel trajectory to obtain the target map; determining the current position and travel direction of the quadruped robot based on the target map and the key feature points to obtain the current posture.

[0012] Optionally, after controlling the movement of the quadruped robot according to the target control strategy, the method further includes: re-collecting the point cloud depth data at every preset time interval, and updating the target control strategy through the first motion strategy model and the second motion strategy model based on the point cloud depth data; and controlling the movement of the quadruped robot according to the updated target control strategy.

[0013] According to another aspect of the present application, an anti-fall control device for a quadruped robot in a tunnel scene is provided, the device comprising: a first acquisition unit, used to acquire point cloud data centered on the quadruped robot through a laser radar to obtain point cloud depth data; a first input unit, used to input the point cloud depth data into a first motion strategy model to obtain a first moving strategy, the first moving strategy at least includes the angular velocity and linear velocity of the quadruped robot, the first motion strategy model is a model trained based on a DQN algorithm and generates a moving mode based on the surrounding environment of the quadruped robot; a second input unit, used to input the point cloud depth data into a second motion strategy model to obtain a second moving strategy, the second moving strategy at least includes the angular velocity and linear velocity of the quadruped robot, the second motion strategy model includes a SLAM positioning module and a navigation module; a first calculation unit, used to perform a weighted average operation based on the first moving strategy and the second moving strategy to obtain a target control strategy, and control the movement of the quadruped robot according to the target control strategy.

[0014] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute any one of the methods described.

[0015] According to another aspect of the present application, a quadruped robot is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include methods for executing any one of the described methods.

[0016] Applying the technical solution of the present application, firstly, the point cloud data centered on the quadruped robot is obtained by laser radar to obtain the point cloud depth data; then the point cloud depth data is input into the first motion strategy model to obtain the first travel strategy, the first travel strategy at least includes the angular velocity and linear velocity of the quadruped robot, and the first motion strategy model is a model based on the DQN algorithm training to generate the travel mode based on the surrounding environment of the quadruped robot; then the point cloud depth data is input into the second motion strategy model to obtain the second travel strategy, the second travel strategy at least includes the angular velocity and linear velocity of the quadruped robot, and the second motion strategy model includes a SLAM positioning module and a navigation module; finally, a weighted average operation is performed based on the first travel strategy and the second travel strategy to obtain the target control strategy, and the movement of the quadruped robot is controlled according to the target control strategy. The present application is based on the point cloud depth data obtained by laser radar, and then by inputting the first motion strategy model trained by the DQN algorithm and the second motion strategy model including the SLAM positioning and navigation modules, the travel strategy of the quadruped robot is obtained. The present invention solves the problem in the prior art that in a power cable tunnel scenario, a quadruped robot cannot work after falling due to a pit or a high platform. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A hardware structure block diagram of a mobile terminal for anti-fall control of a quadruped robot in a tunnel scene provided in an embodiment of the present application is shown;

[0018] Figure 2 A schematic flow chart of a method for preventing a quadruped robot from falling in a tunnel scenario according to an embodiment of the present application is shown;

[0019] Figure 3 A flowchart of a method for preventing a quadruped robot from falling in a tunnel scenario according to an embodiment of the present application is shown;

[0020] Figure 4A structural block diagram of an anti-fall control device for a quadruped robot in a tunnel scenario provided according to an embodiment of the present application is shown.

[0021] The above drawings include the following reference numerals:

[0022] 102, processor; 104, memory; 106, transmission device; 108, input and output devices. DETAILED DESCRIPTION

[0023] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0024] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0025] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0026] As introduced in the background technology, in the prior art, in some tunnel scenarios, the existence of tunnel factors will cause the quadruped robot to be unable to work autonomously once it falls, and can only be manually intervened, which is time-consuming and labor-intensive. In order to solve the problem in the prior art that in power cable tunnel scenarios, the quadruped robot cannot work after falling due to tunnels and platforms, the embodiments of the present application provide a quadruped robot anti-fall control method in a tunnel scenario, a quadruped robot anti-fall control device in a tunnel scenario, a computer-readable storage medium and a quadruped robot.

[0027] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0028] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 1 is a hardware structure block diagram of a mobile terminal of a method for preventing a quadruped robot from falling in a tunnel scenario according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0029] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the display method of device information in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The transmission device 106 is used to receive or send data via a network. The above-mentioned specific examples of the network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0030] In this embodiment, a method for preventing a quadruped robot from falling in a tunnel scene running on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in an order different from that shown here.

[0031] Figure 2 It is a flow chart of the anti-fall control method of a quadruped robot in a tunnel scene according to an embodiment of the present application.

[0032] like Figure 2 As shown, the method comprises the following steps:

[0033] Step S201, acquiring point cloud data centered on the quadruped robot through a laser radar to obtain point cloud depth data;

[0034] Specifically, the laser radar is installed on the body of the quadruped robot to measure the distance to the surrounding obstacles. The point cloud data is obtained through the laser radar. After preprocessing, the depth data on both sides of the robot is obtained. The point cloud depth data is published through the ROS node. The relevant nodes can obtain the above point cloud depth data by subscribing to the ROS node. The point cloud data centered on the quadruped robot is obtained through the laser radar, and then the depth data of the surrounding environment is obtained.

[0035] Step S202, inputting the point cloud depth data into a first motion strategy model to obtain a first moving strategy, wherein the first moving strategy at least includes an angular velocity and a linear velocity of the quadruped robot, and the first motion strategy model is a model for generating a moving mode based on the surrounding environment of the quadruped robot and trained based on a DQN algorithm;

[0036] Specifically, the first motion strategy model that generates the travel mode based on the surrounding environment of the quadruped robot is trained by the DQN algorithm, and the first travel strategy including the travel angular velocity and the travel linear velocity is obtained after inputting the point cloud depth data. This strategy can quickly generate the travel mode according to the real-time environment, improving the response speed of the quadruped robot in a dynamic environment.

[0037] Step S203, inputting the point cloud depth data into a second motion strategy model to obtain a second moving strategy, wherein the second moving strategy at least includes the moving angular velocity and the moving linear velocity of the quadruped robot, and the second motion strategy model includes a SLAM positioning module and a navigation module;

[0038] Specifically, the second motion strategy model contains a SLAM positioning module and a navigation module, and generates a second movement strategy including angular velocity and linear velocity after inputting the same point cloud depth data. This strategy has higher accuracy in positioning and navigation, ensuring the path planning ability of the quadruped robot in complex environments.

[0039] Step S204, performing a weighted average operation based on the first moving strategy and the second moving strategy to obtain a target control strategy, and controlling the movement of the quadruped robot according to the target control strategy.

[0040] Specifically, the first and second travel strategies are fused by weighted average to obtain the target control strategy. This method optimizes the complementary advantages and disadvantages between strategies, improves the synergy of the quadruped robot in environmental perception, path planning and motion control, and further enhances its autonomy and adaptability in complex environments. According to the target control strategy, the movement of the quadruped robot is precisely controlled.

[0041] It can be seen that the embodiment of the present application provides an anti-fall control method for a quadruped robot in a tunnel scene. First, the point cloud data centered on the quadruped robot is obtained through a laser radar to obtain point cloud depth data; then the point cloud depth data is input into the first motion strategy model to obtain a first travel strategy, the first travel strategy at least includes the angular velocity and linear velocity of the quadruped robot, and the first motion strategy model is a model based on the DQN algorithm training based on the surrounding environment of the quadruped robot to generate a travel mode; then the point cloud depth data is input into the second motion strategy model to obtain a second travel strategy, the second travel strategy at least includes the angular velocity and linear velocity of the quadruped robot, and the second motion strategy model includes a SLAM positioning module and a navigation module; finally, a weighted average operation is performed based on the first travel strategy and the second travel strategy to obtain a target control strategy, and the quadruped robot movement is controlled according to the target control strategy. This application is based on the point cloud depth data obtained by the laser radar, and then by inputting the first motion strategy model trained by the DQN algorithm and the second motion strategy model including the SLAM positioning and navigation modules, the travel strategy of the quadruped robot is obtained. The present invention solves the problem in the prior art that in a power cable tunnel scenario, a quadruped robot cannot work after falling due to a pit or a high platform.

[0042] As a possible implementation manner, before inputting the point cloud depth data into the first motion strategy model, the method further includes the following steps:

[0043] Step S301, obtaining the tunnel width, the pit width, and the distance between the quadruped robot and the tunnel wall on one side of the pit, obtaining the first target distance, the second target distance, and the third target distance, obtaining the initial position of the quadruped robot, obtaining the first position, obtaining the terminal position of the quadruped robot, and obtaining the second position;

[0044] Specifically, this process involves data collection and processing by sensors, obtaining information about the surrounding environment through devices such as lidar, and providing a basis for subsequent data analysis and model training.

[0045] Step S302, determining the first target distance, the second target distance, the third target distance, the first position and the second position as the initial state of the model;

[0046] Specifically, the initial state is set as the operating benchmark for the robot in a specific environment, laying the foundation for subsequent motion strategy generation.

[0047] Step S303, determining the angular velocity and linear velocity of the quadruped robot as behaviors, and randomly taking values ​​of the angular velocity and linear velocity to obtain a plurality of first target behaviors;

[0048] Specifically, this step generates multiple possible movement strategies through random sampling, providing data support for the model to explore different movement schemes.

[0049] Step S304, determining a plurality of update states based on the initial state and each first target behavior, scoring each update state based on a preset rule, and obtaining a state reward;

[0050] Specifically, by defining scoring rules, the effect of each behavior can be quantitatively evaluated so as to optimize the model in subsequent training.

[0051] Step S305, determining the initial state, the first target behavior, the state reward, the updated state, and whether the first position and the second position overlap as an experience group, and storing it in the experience pool;

[0052] Specifically, the experience groups saved in the experience pool will provide a rich data basis for subsequent neural network training and promote knowledge accumulation in the learning process.

[0053] Step S306, training a neural network based on samples in the experience pool, and calculating a Q value based on a target Q network, determining a maximum Q value corresponding to the updated state as a target Q value, updating the neural network based on the target Q value, and updating the target Q network based on the neural network at intervals of a first preset number of times;

[0054] Specifically, this process involves the Q-learning mechanism in deep reinforcement learning, which improves the decision-making ability of the model by continuously updating network parameters.

[0055] Step S307, determine the updated state as the initial state, and regenerate the first target behavior and the updated state and the experience group corresponding to the updated state based on the iterated initial state, and train the neural network according to the iterated updated state and the experience group until the number of iterations reaches a second preset number, so as to obtain the first motion strategy model.

[0056] Specifically, through state updates and behavior adjustments in the iterative process, the output of the model is continuously optimized so that the generated first motion strategy can better adapt to actual needs in complex environments.

[0057] Through the above steps, the first motion strategy model finally obtained not only takes environmental factors into account, but also can dynamically adjust the movement strategy to adapt to different obstacles and space constraints, thereby improving the flexibility and stability of the quadruped robot in different walking environments.

[0058] As a possible implementation, before scoring each update status based on a preset rule and obtaining a status reward, the method further includes the following steps:

[0059] Step S401, calculating the product of the first target distance and the first preset coefficient to obtain the fourth target distance, and calculating the product of the second target distance and the second preset coefficient to obtain the fifth target distance;

[0060] Specifically, by setting different preset coefficients, the target distance can be weighted, thereby introducing adaptability to different situations. For example, if the environment is complex, the first preset coefficient can be set to a smaller value so that the fourth target distance allows a larger safety space. In this way, the dynamic adjustment of the target distance can better reflect the actual environmental obstacles encountered, ensuring that the robot's path planning has flexibility and responsiveness.

[0061] Step S402, determining the first preset rule is that when the third target distance in the update state is less than or equal to the fifth target distance, the first preset score is determined as the state reward;

[0062] Specifically, this means that when the distance between the robot and the obstacle is less than the safe distance, the lowest score is given to encourage the robot to maintain the safe interval and avoid potential collisions. This rule ensures that the robot will try to maintain a safe distance when approaching obstacles, thereby improving operational safety.

[0063] Step S403, determining the second preset rule is that, in the case where the third target distance is greater than the fifth target distance and less than the fourth target distance in the update state, the second preset score is determined as the state reward;

[0064] Specifically, when the robot walks within a relatively safe boundary, a certain reward is given. At this time, although it is far away from obstacles, it is still within the safe range. This rule encourages the robot to maintain a stable speed and direction, and enhances the flexibility of path planning. At the same time, this scoring mechanism improves the robot's ability to self-adjust its behavior, making it more adaptable in complex environments.

[0065] Step S404, determining the third preset rule is that when the third target distance in the update state is greater than or equal to the fourth target distance, the third preset score is determined as the state reward;

[0066] Specifically, this rule is designed to encourage the robot to stay away from potentially dangerous areas, effectively manage the distance between existing obstacles and targets, gain more rewards by moving to safer areas, and ensure that the robot can avoid sharp turns and other unsafe behaviors as much as possible, thereby enhancing the robot's stability.

[0067] Step S405, determining the fourth preset rule is that when the distance between the first position and the second position decreases in the update state, the fourth preset score is determined as the state reward;

[0068] Specifically, this rule ensures that sufficient rewards are given when the robot approaches the target position, encouraging the robot to move toward the target. Reducing the distance not only means improving the travel efficiency, but also promotes the robot's optimal decision in path selection, thereby reducing travel time and energy consumption and increasing the speed of task completion.

[0069] Step S406, determining the fifth preset rule is that when the distance between the first position and the second position increases in the update state, the fifth preset score is determined as the state reward;

[0070] Specifically, the introduction of this rule prevents the robot from deviating from the target point during its movement, reducing the risk of unexpected situations. If the distance continues to increase, it indicates that the robot's path selection is inappropriate, which will have a negative impact on the completion of the overall task. Therefore, this reward system is specially set to warn the robot to adjust its direction in time.

[0071] Step S407, determining the sixth preset rule is that when the quadruped robot collides in the update state, the first preset score is determined as the state reward, the first preset score is less than the fifth preset score, the fifth preset score is less than the third preset score, the third preset score is less than the fourth preset score, and the fourth preset score is less than the second preset score.

[0072] Specifically, by setting the penalty for collision to the lowest score, the important warning when planning the path is emphasized. This rule not only helps the robot recognize the severity of the collision, but also prompts it to reflect on and adjust its travel strategy to avoid similar situations. This forms a positive feedback mechanism, allowing the robot to gradually learn and optimize its behavior strategy and interact with the surrounding environment more intelligently.

[0073] Through the above steps, the preset rules can effectively manage the safety and efficiency of the robot in a complex environment, enabling it to make autonomous decisions in a dynamically changing environment, reduce collision risks, and improve travel efficiency. At the same time, such a detailed evaluation system also provides reliable feedback for subsequent action optimization, which is of great benefit to the training of deep learning models.

[0074] As a possible implementation method, the point cloud depth data is input into the first motion strategy model to obtain the first moving strategy, including the following steps:

[0075] Step S501, updating the first target distance, the second target distance and the third target distance based on the point cloud depth data;

[0076] Specifically, the point cloud depth data obtained from the current scan is used to update the distance parameters in front of the robot and on the left and right sides (i.e., the first, second, and third target distances) in real time. By updating these distance parameters, the robot can quickly respond to dynamic changes in the surrounding environment, such as adjusting the distance in time when an obstacle is detected approaching, preventing collisions and ensuring travel safety.

[0077] Step S502, determining a first position and a second position based on the point cloud depth data;

[0078] Specifically, the current position (first position) and target position (second position) of the robot are accurately determined by combining point cloud data with SLAM technology. This step provides accurate information about the current position and target point of the robot, which is used to guide the robot's path planning, enabling the robot to select the optimal path based on the current state and target position, thereby improving the accuracy and efficiency of navigation.

[0079] Step S503, determining the first target distance, the second target distance, the third target distance, the first position and the second position as the current state;

[0080] Specifically, all key environmental information and robot states are integrated into an overall state description. The purpose of this step is to provide a comprehensive, instantly updated current state that includes environmental perception and pose information. The model relies on these detailed information for strategy generation and decision-making, thus ensuring that the path planning strategy is not only fast, but also robust and reliable.

[0081] Step S504, randomly generating the angular velocity and linear velocity of the quadruped robot based on the current state, and obtaining a plurality of candidate moving strategies;

[0082] Specifically, multiple sets of candidate angular velocities and linear velocities are randomly generated based on the current state information. These alternative strategies ensure the diversity and exploration of the travel path, prevent the robot from falling into the local optimum, and improve the adaptability and robustness of the entire system. Different speed combinations ensure that the robot can try multiple possible travel modes in a changing environment, so as to find the optimal strategy.

[0083] Step S505, determining the corresponding predicted state based on the current state and each alternative driving strategy, and determining the state reward corresponding to the predicted state, and obtaining the alternative strategy score;

[0084] Specifically, the model predicts future states based on the current state and each set of candidate strategies, and scores these states. The scoring is based on the preset rules of state rewards, and comprehensively evaluates factors such as robot safety, path efficiency, and target proximity. This process helps the robot weigh the pros and cons of each alternative strategy by simulating various possible future states, thereby improving the rationality and effectiveness of strategy selection.

[0085] Step S506: determine the alternative traveling strategy corresponding to the maximum value of the alternative strategy score as the first traveling strategy.

[0086] Specifically, the strategy with the highest score is selected from all the candidate strategies as the first travel strategy. This decision-making process ensures that the robot will select the travel strategy from the safest and most efficient travel plan, and by selecting the best from the best, the success rate of path planning and travel efficiency are significantly improved. This strategy optimization process enables the robot to find the optimal travel path in a complex and changing environment when performing tasks.

[0087] Through the above detailed steps, the first motion strategy model can not only respond quickly based on real-time point cloud data, but also generate efficient and safe movement strategies based on accurate environmental perception. This approach provides strong technical support for the autonomous navigation of quadruped robots in complex environments, ensuring that they can demonstrate excellent adaptability and reliability in various tasks.

[0088] As a possible implementation method, the point cloud depth data is input into the second motion strategy model to obtain the second moving strategy, including the following steps:

[0089] Step S601, inputting the point cloud depth data into the SLAM positioning module to obtain the current posture of the quadruped robot and a target map constructed based on the point cloud depth data;

[0090] Specifically, the real-time point cloud depth data is processed through SLAM (Simultaneous Localization and Mapping) technology to obtain the robot's current accurate posture (including position and orientation) and environmental map. This step ensures that the robot can accurately perceive its current environment and its own accurate position. SLAM not only builds a detailed map of the surrounding environment, but also improves the accuracy of posture estimation and the reliability of the map through loop detection and posture graph optimization, thus providing a solid basic data for the navigation module.

[0091] Step S602: input the current position and the target map into the navigation module to predict the angular velocity and linear velocity of the quadruped robot within a preset time period to obtain a second travel strategy.

[0092] Specifically, the navigation module receives the current position and the constructed target map, and uses this information to plan the path. By predicting the angular velocity and linear velocity in the future preset time period, the model can evaluate the possible results of different strategies. As a result, the navigation module can comprehensively consider factors such as obstacles, feasible paths, and robot motion performance in the actual environment to generate the optimal travel strategy.

[0093] Through the above steps, the second motion strategy model can make full use of real-time point cloud data and SLAM technology to generate an efficient and stable path planning strategy. This method performs well in the autonomous navigation task of the quadruped robot, can cope with complex and changing environments, and ensure safe and efficient completion of tasks.

[0094] As a possible implementation method, the point cloud depth data is input into the SLAM positioning module to obtain the current position and posture of the quadruped robot and the target map constructed based on the point cloud depth data, including the following steps:

[0095] Step S701, extracting features from the point cloud depth data based on the SLAM algorithm, determining points therein that match the preset objects, and obtaining key feature points;

[0096] Specifically, at this stage, the SLAM algorithm first processes the input point cloud data and extracts key points that can represent the characteristics of the environment. These feature points are the basis for the robot to identify and locate in the environment, and usually include the edges of walls, corners of obstacles, specific patterns, etc. These key feature points can not only effectively describe different environmental areas, but also help the robot understand its location by matching with known features. The feature extraction process can use various techniques, such as the RANSAC algorithm to remove noise points, or use feature descriptors such as PFH (Point Feature Histograms) to accurately identify object features.

[0097] Step S702, determining the key feature points obtained when the feature extraction of the point cloud depth data was performed last time as comparison feature points, and determining the moving trajectory of the quadruped robot based on the key feature points and the comparison feature points;

[0098] Specifically, at this stage, the system matches the currently extracted key feature points with the previously extracted comparison feature points. By calculating the relative position changes between the current feature points and the comparison feature points, the SLAM algorithm can infer the robot's trajectory over a period of time. This trajectory information is crucial for subsequent map reconstruction and pose estimation, helping the robot understand its movement in the environment, thereby effectively updating the map and pose information.

[0099] Step S703, reconstructing a map based on key feature points and the travel trajectory to obtain a target map;

[0100] Specifically, in this step, the key feature points and the trajectory obtained after matching are combined to reconstruct the target map. By performing three-dimensional geometric reconstruction of each feature point in the environment, the SLAM module can construct an environmental map of the robot. The reconstructed map is usually a sparse or dense point cloud model that can more comprehensively reflect the robot's surrounding environment at a specific moment. The map reconstruction process may use weighted averaging, B-spline and other methods to improve the precision and accuracy of the reconstructed map.

[0101] Step S704, determine the current position and direction of travel of the robot based on the target map and key feature points to obtain the current posture.

[0102] Specifically, by comprehensively analyzing the reconstructed target map and the currently extracted key feature points, the SLAM algorithm can accurately infer the robot's current position information and direction of movement. This process usually involves matching with feature points on the map, optimizing the position estimate through the least squares method, and ensuring the accuracy and reliability of the current position. Ultimately, the algorithm will output the robot's current position coordinates (x, y, z) and direction angles (such as heading angle, pitch angle, etc.) to provide support for subsequent navigation and movement decisions.

[0103] Through the above steps, the SLAM positioning module can efficiently and accurately process point cloud depth data, helping the quadruped robot to continuously update the environment map and posture information. This process ensures the robot's autonomous navigation in complex environments and improves its adaptability and intelligence. This method not only improves the reliability of path planning, but also provides important guarantees for the robot to perform various tasks.

[0104] As a possible implementation, after controlling the movement of the quadruped robot according to the target control strategy, the method further includes the following steps:

[0105] Step S801, re-collecting point cloud depth data at every preset time interval, and updating the target control strategy through the first motion strategy model and the second motion strategy model based on the point cloud depth data;

[0106] Specifically, in this step, the system periodically re-collects point cloud depth data, which reflects the real-time dynamic changes of the environment and the new state of the robot. Using the collected point cloud depth data, the first motion strategy model and the second motion strategy model are called again to generate new control strategies. These strategy models consider the current state and environmental information in different ways to accurately adjust the robot's movement direction and speed in a changing environment. This regular update method ensures that the strategy is always generated based on the latest data, so as to better cope with environmental changes and emergencies.

[0107] Step S802, controlling the movement of the quadruped robot according to the updated target control strategy.

[0108] Specifically, once new target control policies are generated, the system immediately applies them to control the motion of the quadruped robot. These policies take into account new data and the latest environmental perception, and are designed to guide the robot to continue moving in the best way. This avoids the problem of policy obsolescence, maintains the robot's ability to respond to the environment in real time, and ensures that it can continue to perform tasks safely and efficiently.

[0109] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the implementation process of the anti-fall control method of the quadruped robot in the tunnel scene of the present application will be described in detail below in combination with specific embodiments.

[0110] This embodiment relates to a specific anti-fall control method for a quadruped robot in a tunnel scenario. Figure 3 As shown, the following steps are included:

[0111] Step S1: Install a laser radar on the quadruped robot to measure the distance to surrounding obstacles;

[0112] Step S1 includes the following steps:

[0113] Step S11, obtaining point cloud data through laser radar, and obtaining depth data of both sides of the robot after preprocessing;

[0114] Step S12, publishing the point cloud depth data through the ROS node;

[0115] Step S13: The relevant nodes can obtain the above point cloud depth data by subscribing to the ROS node.

[0116] Step S2: Based on the reinforcement learning algorithm (DQN), the interaction between the agent and the environment is continuously trial-and-error, and more intelligent decision-making control of the system is achieved through environmental evaluation feedback;

[0117] Step S2 includes the following steps:

[0118] S21, through pre-measurement, the average width of the tunnel is d, and the average width of the tunnel is s;

[0119] S22, the reinforcement learning algorithm module (DQN) subscribes to the above node, obtains environmental information for training, traverses the point cloud depth data, and obtains the minimum value x; the specific process of DQN (supplement to point 2) is as follows: DQN is a deep reinforcement learning algorithm that uses a neural network to learn the Q value function. The Q value function is a function that maps states and actions to Q values, which represents the expected return obtained by performing the action in a specific state. The steps are roughly as follows:

[0120] S221, initialize the experience pool and neural network parameters.

[0121] S222, reset the environment and observe the initial state.

[0122] S223, select an action at each time step and observe the next state and reward returned by the environment.

[0123] S224, store this information in the experience pool, that is, store the tuple (state, action, reward, next_state, done) in the experience pool.

[0124] S225, train the neural network using a randomly selected batch of experiences.

[0125] S226, use the target network to update the policy network. Specific method: Before updating the policy network, first use the target network to calculate the target Q value. Then, take the maximum Q value of the next state as the target Q value and use it to update the policy network. In addition, after a certain number of iterations, it is also necessary to update the parameters of the target network to make it consistent with the policy network. Updating the parameters of the target network can be achieved by assigning the parameters of the policy network to the parameters of the target network.

[0126] S227, repeat steps S222-S226 until convergence or the maximum number of training times is reached.

[0127] S23, obtain the control strategy through the reward and penalty values ​​obtained through training, so as to determine the next action of the robot;

[0128] S24, the above-mentioned reward and penalty values ​​are used to make a final reward or penalty determination through rules. The specific rewards and penalties are shown in Table 1.

[0129] Table 1

[0130] state Reward / Penalty Value x<1.1s -20 1.1s<x<d / 2 20 x>d / 2 0 The distance to the target point decreases 5 Increased distance to target point -5 Collision -20

[0131] Step S3: Through iterative training of the SLAM+DQN algorithm, the quadruped robot can be controlled to always move along the center of the tunnel, reducing the risk of the quadruped robot falling from the platform into the tunnel and improving the reliability of inspection.

[0132] Step S3 includes the following steps:

[0133] Step S31, by using SLAM and navigation algorithms, by solving sensor information, odometer information, map information, etc., the angular velocity and linear velocity of the robot are analyzed to obtain the underlying control instruction M1;

[0134] Step S32: Through the reinforcement learning module, the robot can obtain the underlying control instruction M2. The specific process is shown in the figure below. The robot dog transmits the laser radar data through the scan topic, and then performs deep reinforcement learning training on the collected data to obtain the underlying motion control command of the robot dog.

[0135] Step S33, performing weighted average of the above two control instructions through a decision module to obtain a final control instruction M3 = β*(M1 + M2);

[0136] Step S34, the weighting coefficient β is in the range of 0.1-1;

[0137] Step S35, through the above steps, the robot's motion trajectory is controlled to always be near the tunnel centerline.

[0138] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0139] The embodiment of the present application also provides an anti-fall control device for a quadruped robot in a tunnel scene. It should be noted that the anti-fall control device for a quadruped robot in a tunnel scene in the embodiment of the present application can be used to execute the anti-fall control method for a quadruped robot in a tunnel scene provided in the embodiment of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and those that have been explained will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0140] The following is an introduction to the anti-fall control device of the quadruped robot in the tunnel scene provided by the embodiment of the present application.

[0141] Figure 4 is a structural block diagram of an anti-fall control device for a quadruped robot in a tunnel scene according to an embodiment of the present application. Figure 4 As shown, the device includes: a first acquisition unit 10, a first input unit 20, a second input unit 30 and a first calculation unit 40.

[0142] A first acquisition unit 10 is used to acquire point cloud data centered on the quadruped robot through a laser radar to obtain point cloud depth data;

[0143] Specifically, the laser radar is installed on the body of the quadruped robot to measure the distance to the surrounding obstacles. The point cloud data is obtained through the laser radar. After preprocessing, the depth data on both sides of the robot is obtained. The point cloud depth data is published through the ROS node. The relevant nodes can obtain the above point cloud depth data by subscribing to the ROS node. The point cloud data centered on the quadruped robot is obtained through the laser radar, and then the depth data of the surrounding environment is obtained.

[0144] A first input unit 20 is used to input the point cloud depth data into a first motion strategy model to obtain a first moving strategy, wherein the first moving strategy at least includes an angular velocity and a linear velocity of the quadruped robot, and the first motion strategy model is a model for generating a moving mode based on the surrounding environment of the quadruped robot and trained based on a DQN algorithm;

[0145] Specifically, the first motion strategy model that generates the travel mode based on the surrounding environment of the quadruped robot is trained by the DQN algorithm, and the first travel strategy including the travel angular velocity and the travel linear velocity is obtained after inputting the point cloud depth data. This strategy can quickly generate the travel mode according to the real-time environment, improving the response speed of the quadruped robot in a dynamic environment.

[0146] A second input unit 30 is used to input the point cloud depth data into a second motion strategy model to obtain a second travel strategy, wherein the second travel strategy at least includes a travel angular velocity and a travel linear velocity of the quadruped robot, and the second motion strategy model includes a SLAM positioning module and a navigation module;

[0147] Specifically, the second motion strategy model contains a SLAM positioning module and a navigation module, and generates a second movement strategy including angular velocity and linear velocity after inputting the same point cloud depth data. This strategy has higher accuracy in positioning and navigation, ensuring the path planning ability of the quadruped robot in complex environments.

[0148] The first calculation unit 40 is used to perform a weighted average operation based on the first moving strategy and the second moving strategy to obtain a target control strategy, and control the movement of the quadruped robot according to the target control strategy.

[0149] Specifically, the first and second travel strategies are fused by weighted average to obtain the target control strategy. This method optimizes the complementary advantages and disadvantages between strategies, improves the synergy of the quadruped robot in environmental perception, path planning and motion control, and further enhances its autonomy and adaptability in complex environments. According to the target control strategy, the movement of the quadruped robot is precisely controlled.

[0150] The embodiment of the present application provides an anti-fall control device for a quadruped robot in a tunnel scene, the device comprising: a first acquisition unit, a first input unit, a second input unit and a first calculation unit. The device is based on the point cloud depth data acquired by the laser radar, and then obtains the travel strategy of the quadruped robot by inputting the first motion strategy model trained by the DQN algorithm and the second motion strategy model including the SLAM positioning and navigation module. The problem in the prior art that the quadruped robot cannot work after falling due to the pit and the platform in the power cable tunnel scene is solved.

[0151] As a possible implementation, the device further includes: an acquisition unit, an initial state determination unit, a behavior value acquisition unit, a scoring unit, a storage unit, a network update unit and a first model unit.

[0152] An acquisition unit is used to acquire the tunnel width, the pit width, and the distance between the quadruped robot and the tunnel wall on one side of the pit, obtain the first target distance, the second target distance, and the third target distance, obtain the initial position of the quadruped robot, obtain the first position, obtain the terminal position of the quadruped robot, and obtain the second position;

[0153] Specifically, this process involves data collection and processing by sensors, obtaining information about the surrounding environment through devices such as lidar, and providing a basis for subsequent data analysis and model training.

[0154] an initial state determining unit, configured to determine the first target distance, the second target distance, the third target distance, the first position, and the second position as the initial state of the model;

[0155] Specifically, the initial state is set as the operating benchmark for the robot in a specific environment, laying the foundation for subsequent motion strategy generation.

[0156] A behavior value taking unit, used for determining the angular velocity and linear velocity of the quadruped robot as behaviors, and randomly taking values ​​of the angular velocity and linear velocity to obtain a plurality of first target behaviors;

[0157] Specifically, this step generates multiple possible movement strategies through random sampling, providing data support for the model to explore different movement schemes.

[0158] A scoring unit, used to determine a plurality of update states based on the initial state and each first target behavior, and score each update state based on a preset rule to obtain a state reward;

[0159] Specifically, by defining scoring rules, the effect of each behavior can be quantitatively evaluated so as to optimize the model in subsequent training.

[0160] A storage unit, used to determine the initial state, the first target behavior, the state reward, the updated state, and whether the first position and the second position overlap as an experience group, and store it in the experience pool;

[0161] Specifically, the experience groups saved in the experience pool will provide a rich data basis for subsequent neural network training and promote knowledge accumulation in the learning process.

[0162] A network updating unit, configured to train a neural network based on samples in an experience pool, calculate a Q value based on a target Q network, determine a maximum Q value corresponding to an updated state as a target Q value, update the neural network based on the target Q value, and update the target Q network based on the neural network at intervals of a first preset number of times;

[0163] Specifically, this process involves the Q-learning mechanism in deep reinforcement learning, which improves the decision-making ability of the model by continuously updating network parameters.

[0164] The first model unit is used to determine the updated state as the initial state, and regenerate the first target behavior and the updated state and the experience group corresponding to the updated state based on the iterated initial state, and train the neural network according to the iterated updated state and the experience group until the number of iterations reaches a second preset number, so as to obtain the first motion strategy model.

[0165] Specifically, through state updates and behavior adjustments in the iterative process, the output of the model is continuously optimized so that the generated first motion strategy can better adapt to actual needs in complex environments.

[0166] Through the above steps, the first motion strategy model finally obtained not only takes environmental factors into account, but also can dynamically adjust the movement strategy to adapt to different obstacles and space constraints, thereby improving the flexibility and stability of the quadruped robot in different walking environments.

[0167] As a possible implementation manner, the device further includes: a distance calculation unit, a first preset unit, a second preset unit, a third preset unit, a fourth preset unit, a fifth preset unit and a sixth preset unit.

[0168] a distance calculation unit, configured to calculate the product of the first target distance and the first preset coefficient to obtain the fourth target distance, and calculate the product of the second target distance and the second preset coefficient to obtain the fifth target distance;

[0169] Specifically, by setting different preset coefficients, the target distance can be weighted, thereby introducing adaptability to different situations. For example, if the environment is complex, the first preset coefficient can be set to a smaller value so that the fourth target distance allows a larger safety space. In this way, the dynamic adjustment of the target distance can better reflect the actual environmental obstacles encountered, ensuring that the robot's path planning has flexibility and responsiveness.

[0170] A first preset unit, configured to determine a first preset rule that, in a case where the third target distance in the update state is less than or equal to the fifth target distance, the first preset score is determined as a state reward;

[0171] Specifically, this means that when the distance between the robot and the obstacle is less than the safe distance, the lowest score is given to encourage the robot to maintain the safe interval and avoid potential collisions. This rule ensures that the robot will try to maintain a safe distance when approaching obstacles, thereby improving operational safety.

[0172] A second preset unit is used to determine that the second preset rule is that when the third target distance is greater than the fifth target distance and less than the fourth target distance in the update state, the second preset score is determined as the state reward;

[0173] Specifically, when the robot walks within a relatively safe boundary, a certain reward is given. At this time, although it is far away from obstacles, it is still within the safe range. This rule encourages the robot to maintain a stable speed and direction, and enhances the flexibility of path planning. At the same time, this scoring mechanism improves the robot's ability to self-adjust its behavior, making it more adaptable in complex environments.

[0174] A third preset unit, configured to determine a third preset rule that, when the third target distance in the update state is greater than or equal to the fourth target distance, a third preset score is determined as a state reward;

[0175] Specifically, this rule is designed to encourage the robot to stay away from potentially dangerous areas, effectively manage the distance between existing obstacles and targets, gain more rewards by moving to safer areas, and ensure that the robot can avoid sharp turns and other unsafe behaviors as much as possible, thereby enhancing the robot's stability.

[0176] A fourth preset unit, configured to determine a fourth preset rule that, when the distance between the first position and the second position decreases in the update state, a fourth preset score is determined as a state reward;

[0177] Specifically, this rule ensures that sufficient rewards are given when the robot approaches the target position, encouraging the robot to move toward the target. Reducing the distance not only means improving the travel efficiency, but also promotes the robot's optimal decision in path selection, thereby reducing travel time and energy consumption and increasing the speed of task completion.

[0178] a fifth preset unit, configured to determine a fifth preset rule that, when the distance between the first position and the second position increases in the update state, a fifth preset score is determined as a state reward;

[0179] Specifically, the introduction of this rule prevents the robot from deviating from the target point during its movement, reducing the risk of unexpected situations. If the distance continues to increase, it indicates that the robot's path selection is inappropriate, which will have a negative impact on the completion of the overall task. Therefore, this reward system is specially set to warn the robot to adjust its direction in time.

[0180] The sixth preset unit is used to determine the sixth preset rule: when the quadruped robot collides during the update state, the first preset score is determined as the state reward, the first preset score is less than the fifth preset score, the fifth preset score is less than the third preset score, the third preset score is less than the fourth preset score, and the fourth preset score is less than the second preset score.

[0181] Specifically, by setting the penalty for collision to the lowest score, the important warning when planning the path is emphasized. This rule not only helps the robot recognize the severity of the collision, but also prompts it to reflect on and adjust its travel strategy to avoid similar situations. This forms a positive feedback mechanism, allowing the robot to gradually learn and optimize its behavior strategy and interact with the surrounding environment more intelligently.

[0182] Through the above steps, the preset rules can effectively manage the safety and efficiency of the robot in a complex environment, enabling it to make autonomous decisions in a dynamically changing environment, reduce collision risks, and improve travel efficiency. At the same time, such a detailed evaluation system also provides reliable feedback for subsequent action optimization, which is of great benefit to the training of deep learning models.

[0183] As a possible implementation manner, the first input unit includes: a data updating module, a position determining module, a state determining module, a strategy generating module, a strategy scoring module and a first traveling strategy module.

[0184] A data updating module, used for updating the first target distance, the second target distance and the third target distance based on the point cloud depth data;

[0185] Specifically, the point cloud depth data obtained from the current scan is used to update the distance parameters in front of the robot and on the left and right sides (i.e., the first, second, and third target distances) in real time. By updating these distance parameters, the robot can quickly respond to dynamic changes in the surrounding environment, such as adjusting the distance in time when an obstacle is detected approaching, preventing collisions and ensuring travel safety.

[0186] a position determination module, configured to determine a first position and a second position based on the point cloud depth data;

[0187] Specifically, the current position (first position) and target position (second position) of the robot are accurately determined by combining point cloud data with SLAM technology. This step provides accurate information about the current position and target point of the robot, which is used to guide the robot's path planning, enabling the robot to select the optimal path based on the current state and target position, thereby improving the accuracy and efficiency of navigation.

[0188] a state determination module, configured to determine the first target distance, the second target distance, the third target distance, the first position, and the second position as current states;

[0189] Specifically, all key environmental information and robot states are integrated into an overall state description. The purpose of this step is to provide a comprehensive, instantly updated current state that includes environmental perception and pose information. The model relies on these detailed information for strategy generation and decision-making, thus ensuring that the path planning strategy is not only fast, but also robust and reliable.

[0190] A strategy generation module is used to randomly generate the angular velocity and linear velocity of the quadruped robot based on the current state to obtain multiple candidate travel strategies;

[0191] Specifically, multiple sets of candidate angular velocities and linear velocities are randomly generated based on the current state information. These alternative strategies ensure the diversity and exploration of the travel path, prevent the robot from falling into the local optimum, and improve the adaptability and robustness of the entire system. Different speed combinations ensure that the robot can try multiple possible travel modes in a changing environment, so as to find the optimal strategy.

[0192] A strategy scoring module is used to determine the corresponding predicted state based on the current state and each alternative driving strategy, and determine the state reward corresponding to the predicted state to obtain the alternative strategy score;

[0193] Specifically, the model predicts future states based on the current state and each set of candidate strategies, and scores these states. The scoring is based on the preset rules of state rewards, and comprehensively evaluates factors such as robot safety, path efficiency, and target proximity. This process helps the robot weigh the pros and cons of each alternative strategy by simulating various possible future states, thereby improving the rationality and effectiveness of strategy selection.

[0194] The first traveling strategy module is used to determine the alternative traveling strategy corresponding to the maximum value of the alternative strategy score as the first traveling strategy.

[0195] Specifically, the strategy with the highest score is selected from all the candidate strategies as the first travel strategy. This decision-making process ensures that the robot will select the travel strategy from the safest and most efficient travel plan, and by selecting the best from the best, the success rate of path planning and travel efficiency are significantly improved. This strategy optimization process enables the robot to find the optimal travel path in a complex and changing environment when performing tasks.

[0196] Through the above detailed steps, the first motion strategy model can not only respond quickly based on real-time point cloud data, but also generate efficient and safe movement strategies based on accurate environmental perception. This approach provides strong technical support for the autonomous navigation of quadruped robots in complex environments, ensuring that they can demonstrate excellent adaptability and reliability in various tasks.

[0197] As a possible implementation manner, the second input unit includes: a map construction module and a second strategy module.

[0198] A map construction module is used to input the point cloud depth data into the SLAM positioning module to obtain the current position and posture of the quadruped robot and the target map constructed based on the point cloud depth data;

[0199] Specifically, the real-time point cloud depth data is processed through SLAM (Simultaneous Localization and Mapping) technology to obtain the robot's current accurate posture (including position and orientation) and environmental map. This step ensures that the robot can accurately perceive its current environment and its own accurate position. SLAM not only builds a detailed map of the surrounding environment, but also improves the accuracy of posture estimation and the reliability of the map through loop detection and posture graph optimization, thus providing a solid basic data for the navigation module.

[0200] The second strategy module is used to input the current posture and target map into the navigation module to predict the angular velocity and linear velocity of the quadruped robot within a preset time period to obtain a second travel strategy.

[0201] Specifically, the navigation module receives the current position and the constructed target map, and uses this information to plan the path. By predicting the angular velocity and linear velocity in the future preset time period, the model can evaluate the possible results of different strategies. As a result, the navigation module can comprehensively consider factors such as obstacles, feasible paths, and robot motion performance in the actual environment to generate the optimal travel strategy.

[0202] Through the above steps, the second motion strategy model can make full use of real-time point cloud data and SLAM technology to generate an efficient and stable path planning strategy. This method performs well in the autonomous navigation task of the quadruped robot, can cope with complex and changing environments, and ensure safe and efficient completion of tasks.

[0203] As a possible implementation method, the map construction module includes: a feature extraction submodule, a feature comparison submodule, a map reconstruction submodule and a posture module.

[0204] The feature extraction submodule is used to extract features from point cloud depth data based on the SLAM algorithm, determine the points that match the preset objects, and obtain key feature points;

[0205] Specifically, at this stage, the SLAM algorithm first processes the input point cloud data and extracts key points that can represent the characteristics of the environment. These feature points are the basis for the robot to identify and locate in the environment, and usually include the edges of walls, corners of obstacles, specific patterns, etc. These key feature points can not only effectively describe different environmental areas, but also help the robot understand its location by matching with known features. The feature extraction process can use various techniques, such as the RANSAC algorithm to remove noise points, or use feature descriptors such as PFH (Point Feature Histograms) to accurately identify object features.

[0206] The feature comparison submodule is used to determine the key feature points obtained when the feature extraction of the point cloud depth data was performed last time as comparison feature points, and determine the moving trajectory of the quadruped robot based on the key feature points and the comparison feature points;

[0207] Specifically, at this stage, the system matches the currently extracted key feature points with the previously extracted comparison feature points. By calculating the relative position changes between the current feature points and the comparison feature points, the SLAM algorithm can infer the robot's trajectory over a period of time. This trajectory information is crucial for subsequent map reconstruction and pose estimation, helping the robot understand its movement in the environment, thereby effectively updating the map and pose information.

[0208] The map reconstruction submodule is used to reconstruct the map based on key feature points and travel trajectories to obtain the target map;

[0209] Specifically, in this step, the key feature points and the trajectory obtained after matching are combined to reconstruct the target map. By performing three-dimensional geometric reconstruction of each feature point in the environment, the SLAM module can construct an environmental map of the robot. The reconstructed map is usually a sparse or dense point cloud model that can more comprehensively reflect the robot's surrounding environment at a specific moment. The map reconstruction process may use weighted averaging, B-spline and other methods to improve the precision and accuracy of the reconstructed map.

[0210] The posture module is used to determine the current position and direction of the robot based on the target map and key feature points to obtain the current posture.

[0211] Specifically, by comprehensively analyzing the reconstructed target map and the currently extracted key feature points, the SLAM algorithm can accurately infer the robot's current position information and direction of movement. This process usually involves matching with feature points on the map, optimizing the position estimate through the least squares method, and ensuring the accuracy and reliability of the current position. Ultimately, the algorithm will output the robot's current position coordinates (x, y, z) and direction angles (such as heading angle, pitch angle, etc.) to provide support for subsequent navigation and movement decisions.

[0212] Through the above steps, the SLAM positioning module can efficiently and accurately process point cloud depth data, helping the quadruped robot to continuously update the environment map and posture information. This process ensures the robot's autonomous navigation in complex environments and improves its adaptability and intelligence. This method not only improves the reliability of path planning, but also provides important guarantees for the robot to perform various tasks.

[0213] As a possible implementation manner, the device further includes: a policy updating unit and a control unit.

[0214] A strategy updating unit, configured to re-collect the point cloud depth data at every preset time interval, and update the target control strategy through the first motion strategy model and the second motion strategy model based on the point cloud depth data;

[0215] Specifically, in this step, the system periodically re-collects point cloud depth data, which reflects the real-time dynamic changes of the environment and the new state of the robot. Using the collected point cloud depth data, the first motion strategy model and the second motion strategy model are called again to generate new control strategies. These strategy models consider the current state and environmental information in different ways to accurately adjust the robot's movement direction and speed in a changing environment. This regular update method ensures that the strategy is always generated based on the latest data, so as to better cope with environmental changes and emergencies.

[0216] The control unit is used for controlling the movement of the quadruped robot according to the updated target control strategy.

[0217] Specifically, once new target control policies are generated, the system immediately applies them to control the motion of the quadruped robot. These policies take into account new data and the latest environmental perception, and are designed to guide the robot to continue moving in the best way. This avoids the problem of policy obsolescence, maintains the robot's ability to respond to the environment in real time, and ensures that it can continue to perform tasks safely and efficiently.

[0218] The anti-fall control device of the quadruped robot in the tunnel scene includes a processor and a memory. The first acquisition unit, the first input unit, the second input unit and the first calculation unit are all stored in the memory as program units, and the processor executes the program units stored in the memory to implement corresponding functions. The above modules are all located in the same processor; or, the above modules are located in different processors in any combination.

[0219] The processor includes a kernel, which calls the corresponding program unit from the memory. One or more kernels can be set, and the communication efficiency can be improved by adjusting the kernel parameters.

[0220] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0221] An embodiment of the present invention provides a computer-readable storage medium, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the anti-fall control method of the quadruped robot in the tunnel scene.

[0222] An embodiment of the present invention provides a quadruped robot, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a method for executing an anti-fall control method of the quadruped robot in the above-mentioned tunnel scenario.

[0223] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0224] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0225] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0226] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1A function specified in one or more boxes.

[0227] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0228] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0229] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0230] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0231] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0232] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0233] 1) The anti-fall control method of the quadruped robot in the tunnel scene of the present application first obtains point cloud data centered on the quadruped robot through a laser radar to obtain point cloud depth data; then the point cloud depth data is input into the first motion strategy model to obtain the first travel strategy, the first travel strategy at least includes the angular velocity and linear velocity of the quadruped robot, and the first motion strategy model is a model based on the DQN algorithm training based on the surrounding environment of the quadruped robot to generate a travel mode; then the point cloud depth data is input into the second motion strategy model to obtain the second travel strategy, the second travel strategy at least includes the angular velocity and linear velocity of the quadruped robot, and the second motion strategy model includes a SLAM positioning module and a navigation module; finally, a weighted average operation is performed based on the first travel strategy and the second travel strategy to obtain a target control strategy, and the quadruped robot movement is controlled according to the target control strategy. The present application is based on the point cloud depth data obtained by the laser radar, and then the travel strategy of the quadruped robot is obtained by inputting the first motion strategy model trained with the DQN algorithm and the second motion strategy model including the SLAM positioning and navigation modules. The present invention solves the problem in the prior art that in a power cable tunnel scenario, a quadruped robot cannot work after falling due to a pit or a high platform.

[0234] 2) The anti-fall control device of the quadruped robot in the tunnel scene of the present application includes: a first acquisition unit, a first input unit, a second input unit and a first calculation unit. The device is based on the point cloud depth data acquired by the laser radar, and then obtains the travel strategy of the quadruped robot by inputting the first motion strategy model trained by the DQN algorithm and the second motion strategy model including the SLAM positioning and navigation module. The problem that the quadruped robot cannot work after falling due to the pit and the platform in the power cable tunnel scene in the prior art is solved.

[0235] The above are only preferred embodiments of the present application and are not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for preventing a quadruped robot from falling in a tunnel scene, characterized in that: include: The point cloud data centered on the quadruped robot is acquired through the laser radar to obtain the point cloud depth data; Inputting the point cloud depth data into a first motion strategy model to obtain a first moving strategy, wherein the first moving strategy at least includes an angular velocity and a linear velocity of the quadruped robot, and the first motion strategy model is a model trained based on a DQN algorithm and generates a moving mode based on the surrounding environment of the quadruped robot; Inputting the point cloud depth data into a second motion strategy model to obtain a second travel strategy, wherein the second travel strategy at least includes a travel angular velocity and a travel linear velocity of the quadruped robot, and the second motion strategy model includes a SLAM positioning module and a navigation module; A weighted average operation is performed based on the first moving strategy and the second moving strategy to obtain a target control strategy, and the movement of the quadruped robot is controlled according to the target control strategy.

2. The method according to claim 1, characterized in that Before inputting the point cloud depth data into the first motion strategy model, the method further includes: Obtaining the tunnel width, the pit width, and the distance between the quadruped robot and the tunnel wall on one side of the pit, obtaining a first target distance, a second target distance, and a third target distance, obtaining an initial position of the quadruped robot, obtaining a first position, obtaining an end position of the quadruped robot, and obtaining a second position; determining the first target distance, the second target distance, the third target distance, the first position, and the second position as an initial state of a model; Determining the angular velocity and the linear velocity of the quadruped robot as behaviors, and taking random values ​​of the angular velocity and the linear velocity to obtain a plurality of first target behaviors; Determine a plurality of update states based on the initial state and each of the first target behaviors, score each of the update states based on a preset rule, and obtain a state reward; Determine the initial state, the first target behavior, the state reward, the updated state, and whether the first position and the second position overlap as an experience group, and store it in an experience pool; Training a neural network based on samples in the experience pool, calculating a Q value based on a target Q network, determining a maximum Q value corresponding to the updated state as a target Q value, updating the neural network based on the target Q value, and updating the target Q network based on the neural network at intervals of a first preset number of times; The updated state is determined as the initial state, and the first target behavior and the updated state and the experience group corresponding to the updated state are regenerated based on the iterated initial state, and the neural network is trained according to the iterated updated state and the experience group until the number of iterations reaches a second preset number, so as to obtain the first motion strategy model.

3. The method according to claim 2, characterized in that Before scoring each of the update states based on a preset rule to obtain a state reward, the method further includes: Calculating the product of the first target distance and a first preset coefficient to obtain a fourth target distance, and calculating the product of the second target distance and a second preset coefficient to obtain a fifth target distance; Determining a first preset rule is that, when the third target distance in the updated state is less than or equal to the fifth target distance, a first preset score is determined as the state reward; Determining the second preset rule is that, when the third target distance is greater than the fifth target distance and less than the fourth target distance in the update state, determining the second preset score as the state reward; Determining a third preset rule is that, when the third target distance in the updated state is greater than or equal to the fourth target distance, a third preset score is determined as the state reward; Determining a fourth preset rule is that, when the distance between the first position and the second position decreases in the update state, a fourth preset score is determined as the state reward; Determining a fifth preset rule is that, when the distance between the first position and the second position increases in the update state, a fifth preset score is determined as the state reward; The sixth preset rule is determined as follows: when the quadruped robot collides in the update state, the first preset score is determined as the state reward, the first preset score is less than the fifth preset score, the fifth preset score is less than the third preset score, the third preset score is less than the fourth preset score, and the fourth preset score is less than the second preset score.

4. The method according to claim 2, characterized in that: Inputting the point cloud depth data into a first motion strategy model to obtain a first moving strategy includes: Update the first target distance, the second target distance, and the third target distance based on the point cloud depth data; determining the first position and the second position based on the point cloud depth data; determining the first target distance, the second target distance, the third target distance, the first position, and the second position as current states; Randomly generate the angular velocity and linear velocity of the quadruped robot based on the current state to obtain multiple candidate travel strategies; Determine a corresponding predicted state based on the current state and each of the alternative travel strategies, and determine the state reward corresponding to the predicted state to obtain an alternative strategy score; The alternative driving strategy corresponding to the maximum value of the alternative strategy score is determined as the first driving strategy.

5. The method according to claim 1, characterized in that Inputting the point cloud depth data into a second motion strategy model to obtain a second moving strategy, including: Inputting the point cloud depth data into the SLAM positioning module to obtain the current posture of the quadruped robot and a target map constructed based on the point cloud depth data; The current position and the target map are input into the navigation module to predict the angular velocity and the linear velocity of the quadruped robot within a preset time period to obtain the second travel strategy.

6. The method according to claim 5, characterized in that Inputting the point cloud depth data into the SLAM positioning module to obtain the current posture of the quadruped robot and a target map constructed based on the point cloud depth data, including: Based on the SLAM algorithm, feature extraction is performed on the point cloud depth data to determine points therein that match the preset objects and obtain key feature points; Determine the key feature points obtained when the feature extraction of the point cloud depth data was performed last time as comparison feature points, and determine the moving trajectory of the quadruped robot based on the key feature points and the comparison feature points; Reconstructing a map based on the key feature points and the travel trajectory to obtain the target map; The current position and direction of travel of the quadruped robot are determined based on the target map and the key feature points to obtain the current posture.

7. The method according to claim 1, characterized in that After controlling the movement of the quadruped robot according to the target control strategy, the method further includes: Recollecting the point cloud depth data at a preset time interval, and updating the target control strategy through the first motion strategy model and the second motion strategy model based on the point cloud depth data; The movement of the quadruped robot is controlled according to the updated target control strategy.

8. An anti-fall control device for a quadruped robot in a tunnel scene, characterized in that: The device comprises: A first acquisition unit is used to acquire point cloud data centered on the quadruped robot through a laser radar to obtain point cloud depth data; A first input unit is used to input the point cloud depth data into a first motion strategy model to obtain a first travel strategy, wherein the first travel strategy at least includes a travel angular velocity and a travel linear velocity of the quadruped robot, and the first motion strategy model is a model trained based on a DQN algorithm and generates a travel mode based on the surrounding environment of the quadruped robot; A second input unit is used to input the point cloud depth data into a second motion strategy model to obtain a second travel strategy, wherein the second travel strategy at least includes a travel angular velocity and a travel linear velocity of the quadruped robot, and the second motion strategy model includes a SLAM positioning module and a navigation module; The first calculation unit is used to perform a weighted average operation based on the first moving strategy and the second moving strategy to obtain a target control strategy, and control the movement of the quadruped robot according to the target control strategy.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method according to any one of claims 1 to 7.

10. A quadruped robot, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include methods for executing any one of claims 1 to 7.

Images