Energy-Efficient Motion Control Method and System Based on Deep Reinforcement Learning

Abstract

This invention discloses an energy-efficient motion control method and system based on deep reinforcement learning, relating to the field of robot motion control technology. The method includes: receiving load data, operating status data, and terrain data of a robot in wheeled mode in the current area; generating a slip probability map through a slip prediction model; calculating the energy consumption difference between switching to legged mode and maintaining wheeled mode by combining a preset terrain-energy consumption mapping table; and generating control commands to maintain wheeled mode or switch to legged mode by judging whether the maximum slip probability in the slip probability map exceeds a preset probability threshold, or by weighing slip risk and energy consumption cost based on a preset scoring function. The beneficial effects are: it can identify the real slip risk, avoid misjudging the "complex appearance" of terrain as "inevitable slip", thereby reducing unnecessary legged switching actions.

Description

Technical Field

[0001] This invention relates to the field of robot motion control technology, and in particular to an energy-efficient motion control method and system based on deep reinforcement learning. Background Technology

[0002] With the widespread application of mobile robots in industrial inspection, disaster relief, and field exploration, robots need to operate autonomously for a long time in complex terrain environments that are unknown or changing rapidly. Traditional wheeled robots, due to their advantages such as simple structure, low energy consumption, and high speed, became the mainstream platform as early as the mid-20th century. In the past decade or so, multi-degree-of-freedom bionic legged robots have gradually emerged. Their flexible maneuverability and strong adaptability have prompted researchers to organically integrate wheeled and legged mechanisms, proposing a wheel-legged composite mobile platform.

[0003] In existing technologies, wheeled-legged hybrid robots typically rely on machine vision to perceive the terrain ahead and proactively switch between wheeled and legged modes accordingly. This is also true for typical scenarios where tires are prone to slippage, such as steep slopes, loose soil, or slippery surfaces. The robot will switch to legged mode in advance when it senses such terrain. This strategy can improve its terrain adaptability and task completion rate. However, legged movement is usually accompanied by higher energy consumption, with energy consumption per unit distance typically several times that of wheeled movement. Furthermore, frequent movements of leg joints, servo motors, or hydraulic components can easily lead to structural fatigue, thereby affecting the overall reliability of the machine. In reality, a "complex appearance" of terrain does not necessarily mean "uncontrollable slippage." On some surfaces that seem to cause tire slippage, the tires can still maintain good friction, without needing to switch to legged mode. In fact, the degree of slippage is often determined by a combination of factors, including terrain material and slope changes. Simply relying on visual perception to trigger mode switching can easily lead to unnecessary legged intervention, resulting in redundant switching, decreased system energy efficiency, and weakened control stability.

[0004] Therefore, an energy-efficient motion control method and system based on deep reinforcement learning are proposed. Summary of the Invention

[0005] In view of the above-mentioned prior art, this application is hereby filed. Embodiments of this application provide an energy-efficient motion control method and system based on deep reinforcement learning, which can balance safety risks and energy consumption costs, and improve the energy efficiency performance of wheeled-legged hybrid robots in complex terrain.

[0006] According to one aspect of this application, an energy-efficient motion control method based on deep reinforcement learning is provided, comprising: receiving load data, operating status data, and terrain data of a robot in a wheeled mode in a current area, wherein the current area is an area where the robot may slip; generating a slip probability map characterizing the slip risk at different positions on the robot's current travel path using a slip prediction model based on deep reinforcement learning, according to the load data, operating status data, and terrain data; and calculating the robot's speed and energy consumption based on a preset terrain-energy consumption mapping table and the slip probability map. The energy consumption difference between the first expected energy consumption for switching to legged mode and the second expected energy consumption for maintaining wheeled mode on the current travel path; determining whether the maximum slip probability in the slip probability map exceeds a preset probability threshold; if so, generating a command to switch to legged mode; otherwise, calculating a switching score that balances safety risk and energy cost based on the maximum slip probability and the energy consumption difference using a preset scoring function; determining whether the switching score is greater than a preset scoring threshold; if so, generating a command to maintain wheeled mode; otherwise, generating a command to switch to legged mode; and sending the command to the robot's command execution terminal.

[0007] According to another aspect of this application, an energy-efficient motion control system based on deep reinforcement learning is provided, comprising: a data receiving module for receiving load data, operating status data, and terrain data of the robot in a wheeled mode in a current area, wherein the current area is an area where the robot may slip; a slip prediction module for generating a slip probability map characterizing the slip risk at different positions on the robot's current path, based on the load data, operating status data, and terrain data, using a slip prediction model based on deep reinforcement learning; and an energy consumption calculation module for calculating the robot's energy consumption based on a preset terrain-energy consumption mapping table and the slip probability map. The energy consumption difference between the first expected energy consumption for switching to legged mode and the second expected energy consumption for maintaining wheeled mode on the current travel path; the risk decision module is used to determine whether the maximum slip probability in the slip probability map exceeds a preset probability threshold. If so, it generates an instruction to switch to legged mode; otherwise, it calculates a switching score that balances safety risk and energy cost based on the maximum slip probability and the energy consumption difference using a preset scoring function; the mode decision module is used to determine whether the switching score is greater than a preset scoring threshold. If so, it generates an instruction to maintain wheeled mode; otherwise, it generates an instruction to switch to legged mode; the instruction sending module is used to send the instruction to the robot's instruction execution terminal.

[0008] According to another aspect of this application, an electronic device is provided, including a memory and a processor, the memory being used to store computer-executable instructions, and the processor being used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the method described above.

[0009] According to another aspect of this application, a computer storage medium is provided that stores computer-executable instructions thereon, which, when executed by a processor, implement the steps of the method described above.

[0010] Compared with existing technologies, the energy-efficient motion control method and system based on deep reinforcement learning according to the embodiments of this application can identify real slip risks and avoid misjudging the "complex appearance" of terrain as "inevitable slip", thereby reducing unnecessary leg switching actions. At the same time, by combining energy consumption difference and scoring function for comprehensive decision-making, the targeting and energy efficiency of robot motion mode switching are improved, avoiding problems such as redundant switching and high energy consumption caused by relying on visual judgment. Attached Figure Description

[0011] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0012] Figure 1 This is a flowchart of the energy efficiency optimization motion control method based on deep reinforcement learning according to the present invention.

[0013] Figure 2 This is a block diagram of the energy efficiency optimization motion control system based on deep reinforcement learning according to the present invention.

[0014] Figure 3 This is a timing diagram of the energy efficiency optimization motion control system based on deep reinforcement learning of this invention.

[0015] Figure 4 This is a block diagram of an electronic device according to the present invention. Detailed Implementation

[0016] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0017] Application Overview

[0018] In existing technologies, wheel-leg hybrid robots typically rely on visual sensors to identify terrain types and trigger mode switching in real time when they detect terrains that may cause tire slippage, such as steep slopes, loose soil, or slippery surfaces. However, the complexity of terrain appearance is not entirely positively correlated with the risk of slippage. Some seemingly complex surfaces can still maintain effective tire adhesion. Relying solely on visual perception to trigger mode switching can easily lead to unnecessary leg movements, resulting in redundant switching, reduced system energy efficiency, and weakened control stability.

[0019] To address the aforementioned issues, the inventors recognized that slip risk is determined by multiple factors, including terrain, robot load, and operating status, requiring dynamic prediction. Furthermore, they realized that robot mode switching decisions need to balance safety risks and energy costs. Simply relying on vision cannot quantify the correlation between slip probability and energy consumption. Therefore, they proposed combining a deep reinforcement learning model to predict slip risk distribution and using energy consumption difference calculation and scoring functions to achieve multi-objective optimization decisions.

[0020] Exemplary methods

[0021] Figure 1 The illustration shows an energy-efficient motion control method based on deep reinforcement learning according to an embodiment of this application, including: receiving load data, operating status data, and terrain data of a robot in wheeled mode in a current area, wherein the current area is a region where the robot may slip; generating a slip probability map representing the slip risk at different positions on the robot's current path using a slip prediction model based on deep reinforcement learning, according to the load data, operating status data, and terrain data; calculating the energy consumption difference between a first expected energy consumption for switching to legged mode and a second expected energy consumption for maintaining wheeled mode on the current path, based on a preset terrain-energy consumption mapping table and the slip probability map; determining whether the maximum slip probability in the slip probability map exceeds a preset probability threshold, and if so, generating an instruction to switch to legged mode; otherwise, calculating a switching score that balances safety risk and energy cost based on the maximum slip probability and the energy consumption difference using a preset scoring function; determining whether the switching score is greater than a preset scoring threshold, and if so, generating an instruction to maintain wheeled mode; otherwise, generating an instruction to switch to legged mode; and sending the instruction to the robot's instruction execution terminal.

[0022] The terrain data consists of point cloud data collected by LiDAR. The operational status data includes the robot's motor speed, tilt angle, and other operational parameters. The load data represents the robot's current load distribution. The slip probability map is a visual representation of the slip risk at each point along the robot's current path; it can be a heatmap or a two-dimensional matrix. The terrain-energy consumption mapping table is a database storing the unit energy consumption of different terrain types corresponding to different motion modes; for example, energy consumption benchmarks for terrains such as gravel and mud can be established through experimental measurements.

[0023] Specifically, when the robot enters a potential slip zone in wheeled mode, it collects load data and operating status data in real time. Combined with terrain point cloud data obtained by LiDAR, the data is input into a pre-trained slip prediction model. The model outputs the slip probability of each location point on the path. Then, it queries a preset database to obtain the unit energy consumption of wheeled mode and legged mode corresponding to the current terrain. Combined with the slip probability, it calculates the total energy consumption difference of the entire path. If the maximum probability output by the slip prediction model exceeds the threshold, it directly switches to legged mode. Otherwise, it calculates a balance between risk and energy consumption switching score through a preset scoring function. Based on the calculation result, it continues to determine whether to switch to legged mode.

[0024] Through the above technical solution, this application realizes the quantitative assessment of slippage risk and the dynamic trade-off of energy consumption cost. Compared with relying on machine vision for mode switching, it can reduce unnecessary leg movements. Under the premise of ensuring movement safety, it reduces energy consumption by maintaining wheel mode, extends robot endurance, avoids mechanical wear caused by frequent switching, and improves system reliability and terrain adaptability.

[0025] This application further proposes that before determining whether the maximum slip probability in the slip probability map exceeds a preset probability threshold, the following steps are included: obtaining the ambient temperature data of the current area; extracting the corresponding probability adjustment function based on the preset terrain-temperature range-probability adjustment function mapping table and the terrain data and ambient temperature data; and adjusting the preset probability threshold based on the ambient temperature and the probability adjustment function.

[0026] The ambient temperature data refers to the thermodynamic parameters of the surface area where the robot is located, collected by temperature sensors. This can be achieved using infrared thermal imaging modules or contact temperature probes, reflecting the physical state of surface friction characteristics as temperature changes. The terrain-temperature range-probability adjustment function mapping table is a database storing probability adjustment rules corresponding to different terrain types and temperature range combinations. This can be implemented using a multi-dimensional lookup table structure, used to establish a quantitative model of the influence of temperature on slip probability. The probability adjustment function is a mathematical formula that dynamically adjusts a preset probability threshold based on temperature parameters. This can be implemented using piecewise linear interpolation or nonlinear regression models, used to eliminate the interference of temperature changes on the slip determination benchmark value.

[0027] Specifically, when the robot enters a potentially slippery area, it simultaneously collects surface temperature data for that area. By querying a terrain-temperature range-probability adjustment function mapping table, it matches the current terrain type with the temperature range and extracts the corresponding probability adjustment function. For example, in sandy terrain, when the temperature is in the 30-50℃ range, the probability adjustment function can be set to reduce the original preset probability threshold by 5% to 15%. The adjusted probability threshold will serve as a dynamic benchmark for slip risk assessment, preventing misjudgments of slip risk caused by using the judgment standard under normal temperature conditions even when the surface friction coefficient decreases due to temperature increases.

[0028] Through the above technical solution, this application realizes the adaptive adjustment of the slip probability threshold under different temperature environments, avoids the slip risk omission caused by the decrease in friction coefficient under high temperature environment, and prevents redundant mode switching caused by excessive friction coefficient under low temperature environment, thereby improving the reliability of motion control and energy efficiency optimization accuracy of wheel-leg composite robot.

[0029] This application further proposes that determining whether the switching score is greater than the preset score threshold also includes: extracting the corresponding score adjustment function based on the preset terrain-temperature range-score adjustment function mapping table and terrain data and ambient temperature data; and adjusting the preset score threshold based on the ambient temperature data and the score adjustment function.

[0030] The terrain-temperature range-score adjustment function mapping table is a data structure that stores the score adjustment functions corresponding to different combinations of terrain types and temperature ranges. It can be implemented using a database or hash table and is used to dynamically adjust the score judgment criteria based on real-time ambient temperature. The score adjustment function is a mathematical expression defining the relationship between temperature and the score threshold adjustment amount. It can be implemented using linear interpolation or piecewise functions and is used to quantify the impact of temperature changes on slip risk as a threshold adjustment parameter.

[0031] Specifically, based on the terrain type and the current temperature range, the corresponding scoring adjustment function is matched from the terrain-temperature range-scoring adjustment function mapping table. For example, if the current terrain is loose sand and the temperature is in the 30-40℃ range, the scoring adjustment function corresponding to this combination is selected. The temperature value is substituted into the calculation to obtain the scoring threshold adjustment amount. The adjusted scoring threshold will be used for subsequent switching scoring judgment, so that the change in slip risk caused by tire material softening under high temperature environment is incorporated into the decision-making logic.

[0032] Through the above technical solution, this application can avoid misjudgment of mode switching caused by fixed scoring thresholds in high or low temperature environments, achieve a balance between energy consumption and safety in complex temperature scenarios, and further reduce mechanical structure damage caused by redundant mode switching.

[0033] This application further proposes a method for generating a slip probability map, which includes: dividing the current travel path into multiple location points for slip prediction; extracting terrain sub-region data corresponding to each location point based on terrain data; outputting the slip probability of each location point using a slip prediction model based on terrain sub-region data, load data, and operating status data; and constructing a slip probability map based on multiple location points and their corresponding slip probabilities, wherein the slip probability map is a two-dimensional matrix representing the spatial distribution of slip probabilities on the current travel path.

[0034] Location point segmentation refers to discretizing a continuous travel path into a finite number of prediction units. This can be achieved through equal-interval sampling or dynamic segmentation based on terrain abrupt changes. Discretization reduces model computational complexity and improves prediction resolution. Topographic sub-region data refers to the set of local terrain feature parameters corresponding to each location point. Specifically, this can be achieved by acquiring 3D point cloud data using LiDAR or depth cameras, and then extracting parameters such as slope, friction coefficient, and surface material type using a grid segmentation algorithm. These parameters characterize the impact of local terrain on slippage.

[0035] Specifically, the current travel path is discretized into multiple location points, for example, a prediction point is set at every 0.5 meters. For each location point, terrain data such as slope and surface roughness of the sub-region are collected and input into the slip prediction model along with the robot's current load weight, drive wheel speed and other operating status parameters. The model processes the input data through a multi-layer neural network and outputs the probability value of slippage at each location point. Finally, the slippage probabilities of all location points on the path are arranged in spatial order into a two-dimensional matrix. The row and column indices of the matrix correspond to the path coordinates, and the matrix element values ​​represent the slippage risk intensity, thus forming a slippage probability distribution map covering the entire path.

[0036] Through the above technical solution, this application can achieve a refined assessment of the sliding risk of the robot's movement path, avoid the omission or misjudgment of local risks caused by the global unified judgment, thereby reducing unnecessary mode switching actions, reducing system energy consumption and improving motion control stability.

[0037] This application further proposes the following steps for calculating the energy consumption difference: obtaining the path length of the current travel path within the current region. ; path length Discretized Path units of equal length, each path unit having a length of [length missing]. ; Calculate the average slip probability of the robot's wheeled pattern in each path unit based on the slip probability diagram. Based on the preset terrain-energy consumption mapping table, the energy consumption per unit distance for the robot to traverse the current area in wheeled mode is obtained. And energy consumption per unit distance through the current area in legged mode. ,in, Indicate the index of the terrain in the terrain-energy map table; calculate the first expected energy consumption:

[0038] ;

[0039] Calculate the second expected energy consumption:

[0040] ;

[0041] in, Indicates the first An index of the terrain type for each path unit. Indicates the first The average slip probability of each path unit The preset slip compensation coefficient, The preset energy consumption for the robot switching from wheeled mode to legged mode; the energy consumption difference is calculated based on the first and second expected energy consumption. .

[0042] The path length can be calculated using coordinate data output from the robot's path planning module and the boundaries of the current area. This determines the spatial range for energy consumption calculations. The current area can be acquired using LiDAR or a depth camera. Perception boundary limits can be set to prevent the current area from becoming too large, thus increasing perception costs. Average slip probability. This refers to the average slip probability within each path unit, which can be obtained from the slip probability data of the corresponding point in the slip probability map. It is used to quantify the slip risk of wheeled mode on that path segment. (Switching energy consumption) This refers to the fixed energy consumption generated by the mechanical structure and control system during mode switching. It can be obtained through experimental measurement or historical data statistics and is used to correct the total energy consumption prediction value of the leg mode.

[0043] Specifically, path discretization divides a continuous travel path into multiple independently computed units, indexed according to the terrain type of each unit. Matching the corresponding unit energy consumption parameters from the terrain-energy consumption map, in the first expected energy consumption calculation of the wheel mode, the energy consumption of each path unit is determined by the base energy consumption. Superimposed slip compensation term Composition, slip probability The higher the value, the greater the energy consumption compensation. The second expected energy consumption of the legged mode is directly added to the base energy consumption of each unit. In addition, the fixed energy consumption during mode switching is also included. By calculating the energy consumption of the two motion modes in segments, the difference in energy consumption can be more accurately reflected by the cumulative calculation of the segments, thus obtaining a reliable energy consumption difference assessment result.

[0044] Through the above technical solution, this application can accurately quantify the incremental impact of slip risk on wheel mode energy consumption, and accurately calculate the fixed cost of mode switching, providing reliable energy consumption difference data support for robot mode switching decisions, thereby avoiding redundant switching or delayed switching problems caused by energy consumption prediction errors, and improving the robustness of system energy efficiency optimization control.

[0045] This application further proposes a pre-defined scoring function, which is as follows:

[0046] ;

[0047] in, Indicates the maximum slip probability. This is the energy consumption difference. and This is a preset normalized reference constant. The preset risk weight adjustment coefficient, This is the preset energy consumption weighting adjustment coefficient. and Characterize the importance of slip risk and the importance of energy consumption difference respectively, satisfying .

[0048] in, and Specifically, historical data-based baseline values ​​for slip probability and energy consumption are used to eliminate the influence of different dimensions on the scoring calculation. and Specifically, this can be achieved using dynamic adjustment algorithms or empirical value settings, which are used to adjust the priority of safety and economy according to task requirements.

[0049] Specifically, the scoring function constructs a quantitative evaluation index by normalizing and weighting the difference between the maximum slip probability and energy consumption. When the maximum slip probability exceeds a preset threshold, a mode switch is directly triggered; otherwise, the slip risk and energy cost are converted into comparable numerical values. For example, when... Set to 0.7 and When the value is 0.3, there is a greater tendency to avoid slippage risk; when It is 0.4 and When the value is 0.6, more attention is paid to energy conservation.

[0050] This application further proposes: obtaining the actual energy consumption data of the robot traveling along the current path in wheeled or legged mode; calculating the energy consumption deviation value based on the actual energy consumption data and the corresponding first or second expected energy consumption; and adjusting the energy consumption mapping parameters of the robot in wheeled or legged mode under the corresponding terrain in the terrain-energy consumption mapping table based on the energy consumption deviation value.

[0051] The actual energy consumption data refers to the energy consumption collected in real time by sensors when the robot performs wheeled or legged movements. This can be achieved using current sensors, voltage sensors, or power meters, reflecting the energy consumption level under real-world operating conditions. The energy consumption deviation value refers to the difference between expected and actual energy consumption, which can be calculated through subtraction or percentage error calculations, used to quantify the accuracy of the model's predictions. The terrain-energy consumption mapping table is a database storing energy consumption parameters per unit distance for different terrain types and corresponding movement modes. This can be implemented using hash tables or matrix structures, providing fundamental parameters for energy consumption prediction.

[0052] Specifically, after the robot completes its current path, the system collects energy consumption data during actual movement using embedded sensors. This includes data such as the number of current pulses from the motor in wheeled mode or the pressure change curve of the hydraulic pump in legged mode. The actual energy consumption data is then compared with the first or second expected energy consumption calculated during the mode switching decision phase. If the deviation exceeds a preset threshold, a parameter update mechanism is triggered. For example, in sandy terrain, if the actual energy consumption in wheeled mode is 15% lower than the predicted value in the mapping table, the gradient descent algorithm iteratively optimizes the wheeled energy consumption parameters for sandy terrain in the mapping table. This process is implemented through online learning, automatically calibrating parameters after each path execution.

[0053] Through the above technical solution, this application solves the problem of mismatch between the terrain-energy consumption mapping table and the real environment, and improves the energy efficiency optimization accuracy of mode switching decisions. For example, in long-term field operation scenarios, as tire wear intensifies, the actual energy consumption of wheeled mode may gradually deviate from the initial parameters. In this case, real-time calibration can avoid overly conservative switching strategies caused by model aging, thereby maintaining better energy utilization while ensuring safety.

[0054] Exemplary System

[0055] Figures 2-3The figure illustrates an energy-efficient motion control system based on deep reinforcement learning according to an embodiment of this application, comprising: a data receiving module for receiving load data, running status data, and terrain data of a robot in a wheeled mode in a current area, wherein the current area is a region where the robot may slip; a slip prediction module for generating a slip probability map characterizing the slip risk at different positions on the robot's current path based on the load data, running status data, and terrain data using a slip prediction model based on deep reinforcement learning; and an energy consumption calculation module for calculating the robot's energy consumption risk at the current path based on a preset terrain-energy consumption mapping table and the slip probability map. The system calculates the energy difference between the first expected energy consumption for switching to legged mode and the second expected energy consumption for maintaining wheeled mode on the path; the risk decision module determines whether the maximum slip probability in the slip probability map exceeds a preset probability threshold. If so, it generates a command to switch to legged mode; otherwise, it calculates a switching score that balances safety risk and energy cost based on the maximum slip probability and the energy consumption difference using a preset scoring function; the mode decision module determines whether the switching score is greater than a preset scoring threshold. If so, it generates a command to maintain wheeled mode; otherwise, it generates a command to switch to legged mode; and the command sending module sends commands to the robot's command execution end.

[0056] In one example, before the risk decision module determines whether the maximum slip probability in the slip probability map exceeds the preset probability threshold, it also includes: obtaining the ambient temperature data of the current area; extracting the corresponding probability adjustment function based on the preset terrain-temperature range-probability adjustment function mapping table and the terrain data and ambient temperature data; and adjusting the preset probability threshold based on the ambient temperature and the probability adjustment function.

[0057] In one example, the mode decision module's determination of whether the switching score is greater than the preset score threshold also includes: extracting the corresponding score adjustment function based on the preset terrain-temperature range-score adjustment function mapping table and terrain data and ambient temperature data; and adjusting the preset score threshold based on the ambient temperature data and the score adjustment function.

[0058] In one example, the slip prediction module generates a slip probability map by: dividing the current travel path into multiple location points for slip prediction; extracting terrain sub-region data corresponding to each location point based on terrain data; outputting the slip probability of each location point based on the terrain sub-region data, load data, and operating status data using the slip prediction model; and constructing a slip probability map based on multiple location points and their corresponding slip probabilities. The slip probability map is a two-dimensional matrix representing the spatial distribution of slip probabilities on the current travel path.

[0059] In one example, the steps for the energy consumption calculation module to calculate the energy consumption difference are as follows: obtain the path length of the current travel path within the current region. ; path length Discretized Path units of equal length, each path unit having a length of [length missing]. ; Calculate the average slip probability of the robot's wheeled pattern in each path unit based on the slip probability diagram. Based on the preset terrain-energy consumption mapping table, the energy consumption per unit distance for the robot to traverse the current area in wheeled mode is obtained. And energy consumption per unit distance through the current area in legged mode. ,in, Indicate the index of the terrain in the terrain-energy map table; calculate the first expected energy consumption:

[0060] ;

[0061] Calculate the second expected energy consumption:

[0062] ;

[0063] in, Indicates the first An index of the terrain type for each path unit. Indicates the first The average slip probability of each path unit The preset slip compensation coefficient, The preset energy consumption for the robot switching from wheeled mode to legged mode; the energy consumption difference is calculated based on the first and second expected energy consumption. .

[0064] In one example, the preset scoring function in the risk decision-making module is as follows:

[0065] ;

[0066] in, Indicates the maximum slip probability. This is the energy consumption difference. and This is a preset normalized reference constant. The preset risk weight adjustment coefficient, This is the preset energy consumption weighting adjustment coefficient. and Characterize the importance of slip risk and the importance of energy consumption difference respectively, satisfying .

[0067] In one example, the system also includes a feedback optimization module: acquiring the actual energy consumption data of the robot traveling along the current path in wheeled or legged mode; calculating the energy consumption deviation value based on the actual energy consumption data and the corresponding first or second expected energy consumption; and adjusting the energy consumption mapping parameters of the robot in wheeled or legged mode under the corresponding terrain in the terrain-energy consumption mapping table based on the energy consumption deviation value.

[0068] Exemplary electronic devices

[0069] Figure 4 An electronic device according to an embodiment of this application is illustrated. The electronic device may be the mobile device itself, or a standalone device independent of it, which may communicate with the mobile device to receive collected input signals from it and send selected target driving behaviors to it.

[0070] Figure 4 A block diagram of an electronic device according to an embodiment of this application is illustrated.

[0071] like Figure 4 As shown, the electronic device includes one or more processors and memory.

[0072] A processor can be a central processing unit (CPU) or other form of processing unit with data processing and / or instruction execution capabilities, and can control other components in an electronic device to perform desired functions.

[0073] The memory may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and a processor may execute the program instructions to implement the driving behavior decision-making methods of the various embodiments of this application described above, and / or other desired functions.

[0074] In one example, the electronic device may also include input devices and output devices, which are interconnected via a bus system and / or other forms of connection mechanism (not shown).

[0075] Of course, for the sake of simplicity, Figure 4 Only some of the components of the electronic device relevant to this application are shown in this illustration; components such as buses, input / output interfaces, etc., are omitted. In addition, the electronic device may include any other suitable components depending on the specific application.

[0076] Exemplary computer-readable media

[0077] Embodiments of this application may also be computer-readable storage media storing computer program instructions thereon, which, when executed by a processor, cause the processor to perform the steps in the driving behavior decision-making methods according to various embodiments of this application described in the "Exemplary Methods" section above.

[0078] Computer-readable storage media may take the form of any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may, for example, include, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, or devices, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0079] The basic principles of this application have been described above with reference to specific embodiments. However, it should be noted that the advantages, benefits, and effects mentioned in this application are merely examples and not limitations, and should not be considered as essential features of each embodiment of this application. Furthermore, the specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the application to the necessity of employing the aforementioned specific details for implementation.

[0080] The block diagrams of devices, apparatuses, devices, and systems involved in this application are merely illustrative examples and are not intended to require or imply that they must be connected, arranged, or configured in the manner shown in the block diagrams. As those skilled in the art will recognize, these devices, apparatuses, devices, and systems can be connected, arranged, and configured in any manner. Words such as “comprising,” “including,” “having,” etc., are open-ended terms meaning “including but not limited to,” and are used interchangeably with them. The terms “or” and “and” as used herein refer to the terms “and / or,” and are used interchangeably with them unless the context clearly indicates otherwise. The term “such as” as used herein refers to the phrase “such as but not limited to,” and is used interchangeably with it.

[0081] It should also be noted that in the apparatus, equipment, and methods of this application, the components or steps can be disassembled and / or recombined. These disassemblies and / or recombinations should be considered as equivalent solutions of this application.

[0082] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use this application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein can be applied to other aspects without departing from the scope of this application. Therefore, this application is not intended to be limited to the aspects shown herein, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0083] The above description has been given for purposes of illustration and description. Furthermore, this description is not intended to limit the embodiments of this application to the forms disclosed herein. Although numerous exemplary aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, alterations, additions, and sub-combinations thereof.

Claims

1. An energy-efficient motion control method based on deep reinforcement learning, applied to mode switching control of a wheel-legged hybrid robot, characterized in that... include: Receive the robot's load data, operating status data, and terrain data of the current area in wheel mode, where the current area is the region where the robot may slip. Using a slip prediction model based on deep reinforcement learning, a slip probability map is generated based on the load data, operating status data, and terrain data to characterize the slip risk at different positions on the robot's current travel path. Based on the preset terrain-energy consumption mapping table and the slip probability map, calculate the energy consumption difference between the first expected energy consumption of the robot switching to legged mode and the second expected energy consumption of maintaining wheeled mode on the current travel path; If the maximum slip probability in the slip probability graph exceeds a preset probability threshold, a command to switch to leg mode is generated; otherwise, a switching score that balances safety risk and energy cost is calculated based on the difference between the maximum slip probability and the energy consumption using a preset scoring function. Determine whether the switching score is greater than a preset score threshold. If yes, generate an instruction to maintain wheel mode; otherwise, generate an instruction to switch to leg mode. Send the instruction to the robot's instruction execution terminal; The preset scoring function is specifically as follows: ； Among them, the This represents the maximum slip probability. The energy consumption difference is... and This is a preset normalized reference constant. The preset risk weight adjustment coefficient, This is the preset energy consumption weighting adjustment coefficient. and Characterize the importance of slip risk and the importance of energy consumption difference respectively, satisfying .

2. The energy efficiency optimization motion control method based on deep reinforcement learning according to claim 1, characterized in that, Before determining whether the maximum slip probability in the slip probability graph exceeds a preset probability threshold, the following steps are also included: Obtain the ambient temperature data of the current area; Based on the preset terrain-temperature range-probability adjustment function mapping table and the terrain data and ambient temperature data, the corresponding probability adjustment function is extracted; The preset probability threshold is adjusted based on the ambient temperature and the probability adjustment function.

3. The energy efficiency optimization motion control method based on deep reinforcement learning according to claim 2, characterized in that, The step of determining whether the switching score is greater than a preset score threshold also includes: Based on the preset terrain-temperature range-score adjustment function mapping table and the terrain data and ambient temperature data, the corresponding score adjustment function is extracted; The preset scoring threshold is adjusted based on the ambient temperature data and the scoring adjustment function.

4. The energy efficiency optimization motion control method based on deep reinforcement learning according to claim 1, characterized in that, The generation of the slip probability map includes: The current travel path is divided into multiple location points for slip prediction; Based on the terrain data, extract the terrain sub-region data corresponding to each of the location points; The slip prediction model outputs the slip probability of each location point based on the terrain sub-region data, load data, and operating status data. The slip probability map is constructed based on multiple location points and their corresponding slip probabilities. The slip probability map is a two-dimensional matrix representing the spatial distribution of the slip probabilities on the current travel path.

5. The energy efficiency optimization motion control method based on deep reinforcement learning according to claim 1, characterized in that, The calculation steps for the energy consumption difference are as follows: Obtain the path length of the current travel path within the current region. ; The path length Discretized Path units of equal length, each path unit having a length of [length missing]. ; The average slip probability of the robot's wheeled mode in each path unit is calculated based on the slip probability map. ; The energy consumption per unit distance of the robot traversing the current area in wheeled mode is obtained based on the preset terrain-energy consumption mapping table. And energy consumption per unit distance through the current area in legged mode. ,in, This represents the index of the terrain in the terrain-energy consumption mapping table; Calculate the first expected energy consumption: ； Calculate the second expected energy consumption: ； in, Indicates the first An index of the terrain type for each path unit. Indicates the first The average slip probability of each path unit The preset slip compensation coefficient, The preset energy consumption of the robot when switching from the wheeled mode to the legged mode; The energy consumption difference is calculated based on the first expected energy consumption and the second expected energy consumption: .

6. The energy efficiency optimization motion control method based on deep reinforcement learning according to claim 1, characterized in that, Also includes: Obtain the actual energy consumption data of the robot as it travels along the current path in either wheeled or legged mode; Calculate the energy consumption deviation value based on the actual energy consumption data and the corresponding first expected energy consumption or second expected energy consumption; Based on the energy consumption deviation value, adjust the energy consumption mapping parameters of the robot in wheeled or legged mode under the corresponding terrain in the terrain-energy consumption mapping table.

7. An energy-efficient motion control system based on deep reinforcement learning, characterized in that, include: The data receiving module is used to receive the robot's load data, running status data, and terrain data of the current area in wheel mode in the current area, where the current area is the area where the robot may slip. The slip prediction module is used to generate a slip probability map that characterizes the slip risk at different positions on the robot's current travel path, based on the load data, operating status data, and terrain data, using a slip prediction model based on deep reinforcement learning. The energy consumption calculation module is used to calculate the energy consumption difference between the first expected energy consumption of the robot switching to legged mode and the second expected energy consumption of maintaining wheeled mode on the current travel path, based on the preset terrain-energy consumption mapping table and the slip probability map. The risk decision module is used to determine whether the maximum slip probability in the slip probability graph exceeds a preset probability threshold. If it does, an instruction to switch to leg mode is generated. Otherwise, a switching score that balances safety risk and energy cost is calculated based on the difference between the maximum slip probability and the energy consumption using a preset scoring function. The mode decision module is used to determine whether the switching score is greater than a preset score threshold. If it is, an instruction to maintain the wheel mode is generated; otherwise, an instruction to switch to the leg mode is generated. The instruction sending module is used to send the instruction to the instruction execution terminal of the robot; The preset scoring function is specifically as follows: ； Among them, the This represents the maximum slip probability. The energy consumption difference is... and This is a preset normalized reference constant. The preset risk weight adjustment coefficient, This is the preset energy consumption weighting adjustment coefficient. and Characterize the importance of slip risk and the importance of energy consumption difference respectively, satisfying .

8. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the method as described in any one of claims 1 to 7.

9. A computer storage medium storing computer-executable instructions thereon, characterized in that: When the computer-executable instructions are executed by a processor, they implement the steps of the method as described in any one of claims 1 to 7.

Images